May 5th, 2020 by inflectra
Often we get asked the question, how do you determine the Return on Investment (ROI) of purchasing and using SpiraTest to manage a team's requirements, test cases and defects. If the organization in question is using another tool (for example Micro Focus / HP ALM) it is easy, you simply compare features and license cost. However what is the ROI and business case for teams that are using manual methods, using MS-Word and MS-Excel instead of a tool? This is a bit trickier to calculate, so don't worry - we've put together a handy spreadsheet to help you out!
As described in our introductory article - What is Test Management Software? - test management is the process of taking your project's requirements, building a test plan, writing the tests, planning the test activities and capturing the results. Those test cases, test plans, and test activities can consist of manual tests, automated functional tests, API tests, load/performance tests, security tests, and frankly any other kind of test and experimentation activity that improves the quality of the final product.
Whether you choose to formally use a test management tool or not, you are in effect performing the functions of test management, namely:
(note that I did not perpetuate the fallacy that it will be defect free).
Now if you don't have a test management tool, then you are probably using:
Plus... Lots of people to review these documents, determine what tests need to be run, and see how close you are to being ready to release.
Whereas if you are using an integrated test management tool (like SpiraTest) you have all of this information in one place, up to date in real-time:
So it clearly makes sense to invest in an easy to use requirements and test management tool that can manage your entire software quality assurance process. If you are interested in knowing what features we think you should look for, here's a good whitepaper - How To Choose A Test Management Tool. However you probably are wondering, how do I justify the cost of such a tool, what is the Return on Investment (ROI) over my existing manual methods?
We have helped several of our customers put together business cases over the years to help them justify the cost of a tool like SpiraTest, and usually what we do is create two scenarios:
In the current state, we need to determine which roles are performing each of these tasks:
Writing requirements in MS-Word
Managing documents in SharePoint, etc.
Test Case Editing
Writing test cases in Excel
Reviewing test cases
Updating and distributing test cases
Executing tests and recording results in Excel
Manually updating the test results into a central sheet
Creating new defects in bug-tracker and linking to results
Reviewing bugs in old Bug Tracker
Reviewing fixed bugs in old bug tracker
Bug triaging and prioritization
Compiling requirements coverage metrics
Compiling testing metrics and making graphs
Compiling defect metrics and making graphs
For each activity you need to assign the number and types of role that perform that activity, and in a given month, how much time they spend on it.
For the future state, we do a similar activity, but now we consider the test management tool to be in place, and estimate (for the same number of requirements) how much time each of the roles would spend on the new activities:
Finally, you need to add in the cost of the tool (software, hosting and any training) and add that to the total for the future state.
That gives you the total future and current states, from which you can determine the cost savings (assuming the future state is cheaper!!) and ultimately the ROI.
To make this process a bit easier, we have created this handy SpiraTest Business Case Estimation spreadsheet. This spreadsheet should not be used 'as-is', but should be customized to meet the specifics of your organization.
In the first Metrics tab, you need to specify how many months you need to demonstrate the ROI for. The longer the time, the greater the ROI, but you'll need to figure out what is a reasonable investment horizon for your organization. You might want to use 3-months for a Proof of Concept (POC) project, or 6-months to a year for a real implementation.
You should also define the various roles you have, the number of people in those roles, and an approximate, notional salary. It doesn't have to be 100% accurate (or include payroll tax costs, etc.), it's just used for comparison purposes.
Then in the current state tab, you will assign the different roles to the various activities, and determine how many hours they would spend on a project of the specified length.
If your team works differently, feel free to adjust then names, number and types of activities, the ones in the spreadsheet are just a guide.
The final row will contain the total cost for the project if you were to do it using your current team and tools.
Then in the future state tab, estimate how many people and how many hours they would need to perform the test management tasks illustrated, for a project of the same length. Unlike the current state, for the most part these activities are pretty standard when using SpiraTest, so you won't need to add/change the activities themselves, just the actors, quantity, and # hours.
Finally, the spreadsheet includes built-in formulas for determining the number of SpiraTest licenses you will need, the duration of the subscription, and the commensurate cost. It will add on the future state personnel cost and use that to determine the % ROI of the software investment.