Automated software testing is the ability to have a software tool or suite of software tools test your applications directly without human intervention. Generally test automation involves the testing tool send data to the application being tested and then compare the results with those that were expected when the test was created.
The average test plan for a commercial grade application will have between 2,000 and 10,000 test cases. Your test team of five must manually execute and document results for between 400 and 2,000 test cases. And the scheduled release date of your product is fast approaching. No worries; clone your team and work around the clock. Or perhaps there’s a better way.
As the graph above illustrates, there is an upfront cost to automated testing (as opposed to purely manual testing), but as the number of test cases and builds increases, the cost per test decreases.
This might be a good time to add automated testing to your test plan. The first step in this direction is realizing that no test plan can be executed completely with automated methods. The challenge is determining which components of the test plan belong in the manual test bucket and which components are better served in the automated bucket.
This is about setting realistic expectations; automation cannot do it all. You should not automate everything. Humans are smarter than machines (at least currently) and we can see patterns and detect failures intuitively in ways that computers are not able to.
Let’s begin by setting the expectations at a reasonable level. Let’s say we’ll automate 20% of test cases. Too small, management cries! Let’s address those concerns by describing what automating 20% means.
So, we have decided to automate 20% of our test cases, great! There is only one problem - 20% means 20% of your test cases. How about the 20% of test cases used most often, that have the most impact on customer satisfaction, and that chew up around 70% of the test team’s time? The 20% of test cases that will reduce overall test time by the greatest factor, freeing the team for other tasks. That might be a good place to start.
These are the test cases that you dedicate many hours performing, every day, every release, every build. These are the test cases that you dread. It is like slamming your head into a brick wall – the outcome never seems to change. It is monotonous, it is boring, but, yes, it is very necessary. These test cases are critical because most of your clients use these paths to successfully complete tasks. Therefore, these are the tasks that pay the company and the test team to exist. These test cases are tedious but important.
Now that you have decided to add automated testing to your test plan, you need to consider which types of testing you will need:
The Unit testing part of a testing methodology is the testing of individual software modules or components that make up an application or system. These tests are usually written by the developers of the module and in a test-driven-development methodology (such as Agile, Scrum or XP) they are actually written before the module is created as part of the specification. Each module function is tested by a specific unit test fixture written in the same programming language as the module.
In an ideal world, the presentation layer would be very simple and with sufficient unit tests and other code-level tests (e.g. API testing if there are external application program interfaces (APIs)) you would have complete code coverage by just testing the business and data layers. Unfortunately, reality is never quite that simple and you often will need to test the Graphic User Interface (GUI) to cover all of the functionality and have complete test coverage. That is where GUI testing comes in.
Testing of the user interface (called a GUI when it’s graphics based vs. a simple text interface) is called GUI testing and allows you to test the functionality from a user’s perspective. Sometimes the internal functions of the system work correctly but the user interface doesn’t let a user perform the actions. Other types of testing would miss this failure, so GUI testing is good to have in addition to the other types.
API testing involves testing the application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer. API testing is critical for automating testing because APIs now serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps.
When you release a new version of the system (e.g. changing some of the business components or internal data structures) you need to have a fast, easy to run set of API regression tests that verify that those internal changes did not break the API interfaces and therefore the client applications that rely on the APIs will continue to function as before.
There are several different types of performance testing in most testing methodologies, for example: performance testing is measuring how a system behaves under an increasing load (both numbers of users and data volumes), load testing is verifying that the system can operate at the required response times when subjected to its expected load, and stress testing is finding the failure point(s) in the system when the tested load exceeds that which it can support.
You need a test automation solution that can be integrated fully into your development process and that be adapted to your changing needs:
Rapise is the most powerful and flexible automated testing tool on the market. With support for testing desktop, web and mobile applications, Rapise can be used by testers, developers and business users to rapidly and easily create automated acceptance tests that integrate with your requirements and other project artifacts in SpiraTeam.
One of the obstacles to implementing test automation on projects is that the application’s user interface may be changing frequently, with new pushes to production daily. Therefore, it is critical that tests created one day, continue to work seamlessly the next.
Rapise comes with a built-in machine learning engine that records multiple different ways to locate each UI object and measures that against user feedback to determine the probabilistic likelihood of a match. This means that even when you change the UI, Rapise can still execute the test and determine if there is a failure or not.
Many test automation products are only able to test one type of platform, with Rapise your teams can learn a single tool and use it to test web, mobile, desktop, and legacy mainframe applications from the same unified tool:
Rapise is unique in offering both API and UI testing from within the same application. Rapise can handle the testing of both REST and SOAP APIs, with a powerful and user to use web service request editor:
One of the benefits of using Rapise is that you can have an integrated test scenario that combines both API and UI testing in the same script. For example, you may load a list of orders through a REST API, and then want to verify in the UI that the orders grid was correctly populated.
Inflectra has partnered with Neotys, the leader in performance testing and monitoring solutions. Our partnership allows you to seamlessly integrate Rapise and NeoLoad to get an integrated function and performance testing solution.
The integration between Rapise and NeoLoad allows you to take an existing test script written in Rapise and convert it seamlessly into a performance scenario in the NeoLoad load testing system.
This feature allows you to convert Rapise tests for HTTP/HTTPS based applications into protocol-based NeoLoad scripts that can be executed by a large number of virtual users (VUs) that simulate a load on the application being tested.
Rapise comes with a special extension for NUnit and Visual Studio’s MS-Test that facilitates the calling of Rapise tests from within unit test fixtures.
In addition, Rapise includes pre-built Visual Studio templates for NUnit and MS-Test that allows you to quickly and easily write GUI-based unit test scripts in a fraction of the time it would otherwise take.
To learn more about Rapise and how it can help you get started with automated software testing please:
Questions? email or call us at +1 (202) 558-6885