Easy Way to Assess Release Test Coverage from Test Set Level?

Thursday, December 13, 2012 1:51:46 AM
Wondering if there is an easy way to base the Release Test Coverage (#passing, #failing, etc) on instances of test cases - i.e. from test set level - versus based on actual # of associated test cases.

We generally use highly parameterized Test Cases, which can be run several times within a Test Set(s) for the same release (and with different results).  For example, TX 1 contains two instances of TC001, where one is passing and the other failing.

We want to be able to assess Release Test Coverage based on all test cases for the Release (where 'all' accounts for TC001 twice, because each instance has different parameters) .  Currently, Release Test Coverage is based off of test case only - so that if the same TC is run multiple times for the Release, only the most recent test run status is recorded in the Release Test Coverage status bar/#count.   It would nice to be able to generate a count of how many instances of TC001 passed/failed/etc for the Release.

Any suggestions for how do to handle monitoring the test run statuses of parameterized TCs?
2 Replies
Friday, December 14, 2012 2:45:35 PM
Avatar
re: vcrandall on Thursday, December 13, 2012 1:51:46 AM

Hi Tony

Have you tried the following:

1) Go to Testing > Test Sets

2) Click Show/Hide Columns and make sure that the Release # column is displayed

3) Filter your test sets by Release. You'll see the coverage of tests for each set.

Regards

Adam

Friday, December 14, 2012 7:20:54 PM
Avatar
re: inflectra.adam on Friday, December 14, 2012
Yes, this works...was just hoping there was a way to view it from the Releases page using the Test Coverage Column since that column makes it so easy to see total #s of passed/failed etc test cases. 

I guess really what I'm looking for is a Test Set Coverage column on that page (which would imply tying test sets to releases much like you tie test cases to releases, and then reporting the test set execution status from the Releases page).  ?   This would also seemingly allow us to use the Release Reports for reporting purposes (currently not of use to us since they only report last run status of a test case, regardless of parameterization).

Thanks for your help.
Statistics
  • Started: Thursday, December 13, 2012 1:51:46 AM
  • Last Reply: Friday, December 14, 2012 7:20:54 PM
  • Replies: 2
  • Views: 808