Testing Web Applications and Filing Bugs
For the past 6 months or so, I’ve been a casual contributor to a number of Mozilla related projects.
When I signed up to be a contributor, I joined the Web Quality Assurance team which is responsible for ensuring that all Mozilla web properties are secure, error free and of the highest quality.
To achieve this, a lot of testing has to be done. Having a working site isn’t enough, you need to be sure that it works the way the developers intended it to. This is done by testing the website or application under different environments to which it may be exposed to. This includes the different operating systems, browsers, internet connection speeds, machine architecture, language, time zone etc the users may be using.
This sounds like a lot of work, trust me it is, but the rewards are worth it. There’s a question I used to ask when I was starting out.Quality Assurance? What for? My logic was, if an application is coded well, why should there be a need to test it. The answer I got from a fellow contributor was that we don’t know that it’s coded well. True, the devs that work on the applications are experienced and good at what they do but, as good as they may be, they are not perfect. Testing therefore is necessary to ensure that the app is of the highest standard possible, plus it helps in developing new features or modifying already existing ones.
What do we test?
This all depends on whether you’re performing an automated test, manual test or doing exploratory testing.
Below are links to pages containing more info on these types of tests
A great article on what to consider when testing a site by Stephen Donner can be found here and here
For this post, I’ll use the Mozillians Community Directory Site as an example.
First, load the site, which in my case is the staging site.
The staging site is exactly similar to the Live site, used for testing purposes.
The site is a community directory for Mozilla employees and volunteer contributors. It makes it easy for them to find each other.
When you load the site for the first time, you have to either sign in or sign up. For the tests I was running, I created a new account using a disposable email address (got one at mailinator.com).
Testing means applying a critical eye to every aspect of the page and doing the same thing many times over just to be sure that it works ok ok.
The sign up process should be simple, login, update your profile and save. I did this on three different browsers and I found 4 bugs, First, the skills and languages form did not allow non latin characters(@#$%^&* etc) to be used anywhere in the string.This meant that one could not put up C++ or C# as skills as they contain special characters.
The second bug was an error I got when using Chromium browser for Ubuntu 12.04. Moving from one section to the next was not possible using the previous and next buttons. Nothing showed up in the HTTP tools window whenever i clicked these buttons. The console window did show a number of errors. Could not reproduce this in any other browser I used for the tests.
Third bug, was an unclear error message.
After filling in all except one field(my full name) I clicked on the update button, which is meant to create the profile for you. The validation check took me back to the name field to notify me that that field was necessary. The error message however, wasn’t very clear.
Fourth bug, or a suggestion in all fairness.
I felt that it would be better to validate after each field loses focus, instead of validating at the end of filling in the form.
So that’s the first part, finding the bugs. Next step is to file them in Bugzilla. A word definition from Wikipedia:
Bugzilla is a Web-based general-purpose bugtracker and testing tool originally developed and used by the Mozilla project, and licensed under the Mozilla Public License.
To prevent duplicate bugs from being filled in, Bugzilla requires that you give a short description of the bug before you file it.
this in turn is used to search the database for similar bugs. When i did this for the 4 bugs i found, I found matches for two of the bugs. This meant that the bugs I’d found had already been discovered by someone else. This is a good thing, but it can be demotivating if you, like me had already added four bugs to your bug count :(, guess you really shouldn’t count your bugs before they hatch.
That said, I filled in the one bug( the issue with the Chromium Browser) and am still to file the suggestion concerning the validation.The bug can be found here.
This post only focused on the first part of my testing; the signup process. I’m still to use the site as a potential user and hopefully find me some bugs.
A brief run down on how to test a site:
1) Explore the site.
This is the fun part, especially if it’s a site still in development.
2) Find an issue, reproduce it
If it’s reproduceable you’ve found yourself a bug.
3) Get as much detail as you can.
A good place to start are HTTP codes, console messages, error messages, dialog boxes, urls, etc
4) File the bug!
Provide as much detail as possible, screenshots/casts are great too.
5) Change your browser/operating system and jump to 1)
Get a coffee while you’re at it.
I’ll put up more blog posts about my testing efforts.