Spotlight on Rapise 6.2 - Machine Learning & Self-healing Web Testing

The Moz 500 Research Study

During our observations of testers writing automated tests for modern web applications and native mobile apps, we noticed how much easier in many ways the creation and maintenance of mobile tests was. In fact mobile apps and traditional desktop applications (such as Microsoft Dynamics AX and NAV) were generally much easier to write tests for, and significantly more reliable to execute those same tests months later for regression. So we wondered, what were the unique factors that made web testing so "tricksy" as a hobbit might have said. We decided that the best thing to do was perform some "big data" analysis of popular websites to see what could be the cause. This was our Moz 500 research study.

In this study, we analyzed the 500 most popular websites as ranked by the Search Engine Optimization (SEO) tool MOZ. This Moz 500 list of websites let us analyze not only the websites statically, but also by taking repeated snapshots over time, how frequently they changed and how the changes would impact the ease of test automation.

What we found was that:

1. Modern applications increasingly use dynamically generated IDs and Class Names that change on every page reload. This breaks most XPATH and CSS locators used in tools like Rapise or hand written using frameworks like Selenium WebDriver.
2. Developers refactor applications with increasing frequency and often inadvertently break Web element locators without even realizing it.

Rapise makes it easier to write tests because it includes multiple strategies and tools for creating the XPATH used to locate the objects on the web page (full, text, indexed, IDs, ARIA, etc.). However it still often results in flaky tests that fail when the application changes, and the need to frequently re-learn objects or manually "tweak" the XPATH or CSS to make the tests more reliable. We realized during this study that this is the Achilles heel of web testing and there had to be a better way...

Introducing Self-Healing Web Tests

With the new version of Rapise (6.2) that we shall be releasing shortly, we have added a completely new approach to locating objects on a web page that based on our research and development will yield significantly better results in creating tests that work without alteration, even when the application being tested changes, or when the application uses dynamic IDs and attributes.

How Does it Work?

With this new version, Rapise generates a full-path locator for each recorded element. The full-path locator includes information about all the attributes of an element and attributes of its ancestors in the DOM tree.

When you run a previously recorded test and the element is found with a traditional XPath locator then this new full-path locator is not used (there is nothing to repair).

But if the standard XPath generated for an element during recording or manually created by a tester is not valid then the new full-path locator is used instead to find the most suitable element on the page (using the fruits of our big data analysis and machine learning algorithms).

When the full-path locator is used you get the following healing notification in the report:

The healing notification indicates that an element was found with some confidence, it is a number from zero to 100%. In the notification details you can find the difference between recorded and matched elements. You can also see the images of the recorded and found elements. This information is a clue to either accept the found element or manually fix the test.

When you come across such a situation, you can then do one of three things:

1. Take no Action - This is the simplest yet powerful approach. If a test passes and all elements are found with pretty good confidence then there may be no reason to "fix" the test.

2. Build a WebAppProfile - When you analyze differences between the expected and found elements you use that to create or update your WebAppProfile.

3. Use the Object Manager - During playback, Rapise takes snapshots of elements found by the full-path locator. So after playback you can use the Object Manager to replace the old locators with new ones. If there are healing notifications after test playback Rapise automatically shows the prompt to open the Object Manager.

Benefits of Self-Healing Tests

So to summarize, the main benefits of the new self-healing tests are:

• Increases the stability of tests created using the Rapise automated recording, making it a viable technique for complex, modern web applications
• Reduces the maintenance cost of having to manually update the CSS or XPATH locators whenever a web application is changed by the developers during an agile sprint.
• Provides deeper insights to changes in tested application which aid the developers and testers in diagnosing issues introduced in application updates.