test execution with many variables: how to do this efficiently

Wednesday, September 4, 2013

we currently have about 1000 testcases in our project.
20 testcases of them can be executed with 4 different variables (each having their values):
- variable A with values YES/NO
- variable B with values 1,2,3
- variable C with values 4,5,6
- variable D with values YES/NO/Planned
(resulting in a total of 20x54=1080 combinations)
options that I have:
- create 1 testcase for each combination, but that is not maintainable
- work with a custom field for each variable, but I only have 20 tests out of the 1000 that use these fields, so gives too much clutter
- use parameters in the test. Problem here is that I am not triggered for this parameter until actual execution. Ideally I am triggered about this parameter the moment that I add the test in a test set. On that moment, I remember that I should add multiple instances to guarantee that at least some of the combinations are executed.

Anybody had a similar case and found an elegant solution?

Ideally, I would like to set a parameter on test set, where the tests in the test set having a parameter defined in the test case with the same name, take the value of the test set upon execution.
In that way, I create test sets with parameters for variables A to D with specific values, and in that way, different combinations are executed in a controlled way.

thx for some help!
1 Replies
Thursday, September 5, 2013
re: FilipBaert Wednesday, September 4, 2013

Hi Filip

We're looking to add better support for this in v4.2 where you can define data-sets to pass to a test set.

For now, you could try adding the same test cases to a test set multiple times. and set different parameter values for each instance.



  • Started: Wednesday, September 4, 2013
  • Last Reply: Thursday, September 5, 2013
  • Replies: 1
  • Views: 5151