How AI helps QA Shift Left

As the focus on quality becomes more mainstream, QA teams are under intense pressure to deploy bug-free... but we all know that’s nearly impossible. Lacking test coverage, lacking test automation, lacking…simply enough testers – QA teams are never 100% confident their release will be bug-free.

And that’s just something every expert tester will tell you (whether you like it or not), they can only be 99% confident that the deployment will be clean…you’ll never hear them say 100%.

In all my time working with QA teams, I’ve never heard a Tester say they are 100% confident their release will be bug-free. And that’s ok. There’s an important tradeoff every digital leader must make, either release now with 99% confidence or delay the release by a week to achieve 99.1% confidence.

Pressure to get that release out is fierce, and so getting it out on time usually outweighs delay for further testing.  

Although there seems to be an abundant amount of tooling in the market, few exist that help streamline their team’s workflow to achieve greater confidence in release.

Cutting through the noise, how can Digital Leaders and Testers make things easier on themselves and release with more confidence?

We can’t write anything these days without mentioning AI, can we? Apparently not. Because it really does help QA Shift Left and deploy with more confidence…maybe even go from 99% to 99.9% confidence.

Never entirely perfect, but closer!

Here’s the areas I see AI really helping QA Shift Left and achieve higher Confidence in their deployments.

AI Risk Based Test Selection

There’s some real clout behind this technology. Test only in the areas where developers are making changes, thereby allowing teams to test earlier and more frequently.

There’s some early movement in this space, such as early mover Appsurify. Their AI Risk Based Testing solution is patent-pending and allows teams to reduce test cycles by 90% to get feedback to developers at time of change. Rather than running 100% of tests after each change, their AI-Model auto selects and executes just the few tests associated with areas Developer made changes.

By only running 10% or 20% of the tests while still catching the bugs, this approach saves significant time, reduces Infrastructure resources, and optimizes CI pipelines for faster feedback and allows for continuous testing.

For example:

A 1-hour Rapise test run may only be executed 2 or 3 times in a day. If the team leveraged AI Risk based testing, they could shorten this run down to under 5 minutes and execute tests on a per PR basis.

So instead of running their tests 2-3 times a day, they can run their tests 10+ times a day and find bugs while the developers are still focused on the task at hand. The optimal thread of work is 1, so when developers get feedback before they context switch - that helps them stay focused, find bugs in real-time, and ship with higher confidence.

Prevent Flaky Tests from Distracting the Team

AI can recognize trends in test results that pick up whether a test failed due to a real failure versus a false flag. When a run is executed, blocking out flaky signals helps give developers cleaner feedback to focus their debugging on actual bugs.

CI pipelines frequently fail when there’s a flaky test and cause unnecessary time waste. One way to optimize CI pipeline is to leverage AI to block out flaky tests from breaking builds. Only having CI builds fail due to real failures gives developers clean signals and direction for debugging so they only focus their time on real bugs.

AI helps QA Shift Left

Give QA clean signals whether good or bad. And not bad in the traditional sense, because catching a real bug early is an optimal outcome. It’s about cutting through the noise and distraction to focus where time is best spent.

For example:

A Jenkins build fails with 100 test failures. 2 bugs caused 60 tests to fail, and flakiness caused 40 tests to fail. A significant amount of time will be wasted looking into those Flaky Tests with no value. When there's an inherent amount of flakiness and builds are constantly failing, developers start to ignore the results which can be a dangerous outcome.

AI removes that 40% flaky distraction. Rather than failing the build and labeling everything as failed. AI says these 60 test failures are caused by real defects, whereas; these other 40 failures are caused by flakiness. And so, it’s probably in your team’s best interest to focus on the 60 test failures caused by real defects.

That saves the team's time while also building confidence into test results.

Conclusion

Shipping with confidence is hard.

Even leveraging all best practices, Digital Leaders and Testers may only feel 99% confident their release will be bug-free. Getting that additional 0.9% is painstakingly time-consuming. However, testers can leverage AI tools, such as Appsurify, to significantly increase confidence without additional effort so QA can ship earlier and faster confidently.