The hunt for Test Case Management Systems

Whats out there?

For a start, there are plenty of tools out there. TestDirector by Mercury, SilkTest from Segue, QA Director from Compuware, to name but a few. However, there is a common understanding that these tools are not suitable for smaller projects, there are a number of reasons, but the main driving factors are: cost and complexity.

How do small to medium size projects differ?

DrivenQA work with a number of clients and consultancies who have important clients (blue chips, government, etc). The projects worked on usually last anywhere between 1 to 12 months and they can be working on a number of projects at any one time.

For testing in this scenario the ultimate requirement is flexibility and usability. Mentioning the costly licensing figures for the test management tools above is out of bounds. In addition to this is the learning curve with projects that dont last that long, is it worth learning a system that can only be used for one project? And not necessarily be used on all projects that are going on?

In the real world

Many of us work with several clients, on small to medium size projects. Each client has different demands and requests. We therefore require different solutions to different projects. One size does not fit all.

The results

Things are starting to move in the right direction, however the provision for a test management solution is still somewhat lacking. Following are DrivenQAs top choices:

Test Run

Test Run is available online as a subscription service or is available for download. The product is currently in Beta, but I believe this is due to change soon.

The attractive features of this are its simplicity. With the ability to have multiple users, it has the nice features such as tags, reporting capabilities and the ability to import/export test cases rapidly. The fact that it is available online is super feature for those who do not want or have the time to install any software. The fact that it is available separately for download is even better for those who need the confidence of being able to secure their own data.

In its final stages of development, Test Run appears to be on the right track in terms of solutions to small to mid sized projects. The fact that there is a forum, (though still in its early days), a blog and support is a definite plus for anyone considering any IT product.

Testopia (formerly Test Runner)

A recent name change from 'Test Runner', Testopia is open source software that is an add on to the popular Bugzilla. Main downside to it is that you have to have Bugzilla installed, not a problem if you use Bugzilla, but it is a big problem if you don't.

Consideration should be given to the fact that it is open source software. Whilst everyone likes the idea of free, consideration should be given to the fact that nothing in life is free. The advantage of Testopia is the fact that it has a large back from the open source community, giving it better chance to succeed and develop into a mature product.

This is probably the most comprehensive open source test management system and once up and running provides everything needed of a test management system. Testopia is a rewrite of Test Runner and is Web 2.0 enabled (using Dojo and Ajax). In addition to this, it contains features, such as tags, ability to cc people into test cases, ability to add Bugzilla bugs to test case results, full history of changes and a very powerful search capability.

Fitnesse

Being a wiki, Fitnesse is a collaboration tool. Though it is not just any kind of wiki, it is also an acceptance test framework.

The first point to get across is that being a web server, it takes minutes to download and get running. It really is very simple.

Secondly, being a wiki, as a user you can take this as far as you like. Complete documentation can be written, managed, updated and accessed immediately. Test cases can be written and updated easily, though it would be a considerable manual process. (As is using excel spreadsheets, which is still often used in todays world). It has all the powers of a wiki, creating and editing pages instantly is simple. It is down to the user how far they want to take it, as it may only be in their interest to use it as an acceptance testing tool.

Being a test tool, it can perform tests using tables. The advantage of the acceptance testing framework is that anyone can easily (with guidance) either understand or implement tests themselves. It does not require programming knowledge because of its use of tables for testing.

Take this as far as you like, the choice is yours.

Tesly

Tesly is a basic web based test case management system, and in many aspects is quite similar to Test Run, though Test Run is more advanced. On offer is a free limited version or a subscription based model with added features, which makes this option very affordable.

This may be ideal for small projects, however, I believe it is worth a mention if it expands on its current functionality. Worth keeping an eye on future developments.