Testing in the open source world. Part 2

Intro
In my last post I discussed the gathering of requirements, and the organization of those requirements. In this post I will be moving to the next step, case creation based on those requirements.

Getting Started
Now that we have requirements collected and numbered, we can begin creating cases to exercise the features described. So where to begin? Let's take a simple requirement from the example we used in the last post here

1.1 Select "Featured Content" to see content we highlight that changes on a regular basis

So to best exercise this feature there are several things to consider. The first is the name of the section where the feature exists. In this example that is "Featured Content". So our first test case would be to confirm that this area is named correctly:

1) Launch Vuze
2) Confirm main application window contains "Featured Content" area
3) Confirm spelling and grammar of "Featured Content" area is correct

The next section of the requirement is "to see content we highlight". This means there should be something there by default. So, next case:

1) Launch Vuze
2) Open Featured Content area
3) Confirm Featured Content area is populated with links to various resources by default

Along with having content available, we also need to check that the content is functioning as expected. To do this, we need to explore each of the links provided in the content area.

1) Launch Vuze
2) Open Featured Content area
3) Click each content link
4) Confirm that each link launches a service

Lastly, there is the closing part of the requirement "that changes on a regular basis". For this one we may need to contact the developer to find out what regular basis means, or if he/she can trigger a change so we can test that feature. Either way, our case will be the same.

1) Launch Vuze
2) Open Featured Content area
3) Click each content link
4) Confirm that each link launches a service
5) Close Vuze
6) Trigger content shift or wait X hours for normal system cycle
7) Launch Vuze
8) Open Featured Content area
9) Click each content link
10) Confirm that each link launches a service

Now, I did something a little different here. In the previous cases I did one feature at a time. This gets you the best results and makes tracking cases much easier. However, it is also very time consuming. To save a bit, you can often combine cases in a logical way. For instance in this one, I combined the cycling of links and the case previous to it where we checked the links.

Be Careful here. You really only want to combine a new case with ones already run. If you start combining too many unique steps you risk losing focus on what you are actually testing, and the tracking of which cases have passed and which have failed becomes a nasty situation. Any time you have saved writing the cases, quickly evaporates when you try to collect statistics on your testing results.

Mapping
Ok, So we have some cases now. The next step is to ensure that all requirements are covered by cases. We do this by mapping cases to the requirement they were written to explore. This is accomplished by simply referencing the requirement number next to the case. So for the cases we wrote above, each would reference the 1.1 requirement. Having multiple cases for each requirement is very common. If you have several requirements with only a single cases, spend a little more time thinking about how the user might interact with the feature, and I'm sure some more cases will come to mind.

Pulling It All Together
We now have requirements, numbered and organized, and cases created for each or our requirements. The last stage is organizing these into a format that is easy to read, follow, and track. Most QA houses use a table to to this. An example of the cases created in this post can be found here

You'll notice that there are several additional columns in the table to help with tracking. These include a case number, description, and date/build. These help the tester easily find the cases that need to be executed. The pass/fail, bug, and date/build columns help with tracking the state of each feature. These give the reader an idea which areas of the program need the most work and what are the major issues with those areas.

Put all of these together and you have a very efficient way of organizing your cases into a simple format that can be used by both technical and non-technical readers.

Next: Execution
Join me next time for the fun part! Executing the cases, finding bugs, and tracking the fixes through the software development process. Thanks for reading and Happy Hunting.