Test Automation vs Exploratory Testing - In Agile World

Many people are confused when they have to choose between these two types of testing in an agile environment. I also received some similar questions on my blog and questions on Quora for the same, so thought to share some information and my experience here.

I would say essentially both are important and it's a difficult situation for QA to choose out of these two as both require significant time and effort to produce a quality outcome.

The Agile methodology revolves around adaptive planning, early delivery, and continuous improvement, with the aim of being able to respond to change quickly and easily. Every change to the application must be tested, whether it’s a new feature, improvement, or a hotfix. To lower the risk of late fixes and increased costs, testing must be performed as closely as possible to the point when the development team made the change.

The critical role of exploratory testing

Exploratory testing is the simultaneous process of test design and test execution. Unlike automated testing, it doesn’t restrict the tester to a predefined set of instructions. This shouldn’t be seen as a lack of preparation but rather as a method of not constraining the tester. When executed well, exploratory testing is quick to implement and fluid in its approach.

1. Find bugs that automated tests miss

By exploring the application, you will uncover the unique, hard-to-find bugs you didn’t know existed. Scripted testing such as automated testing cannot do this because it follows predefined instructions. Anything outside of this scope will be missed.

Exploratory testing is certainly not a substitute for automated testing. However, it is a great way of uncovering critical bugs that would be missed with automated testing alone, as well as when an automated approach is not appropriate.

2. Diversity means discovering more bugs

The collective power of more heads is better than one. Diversity is vital for exploratory testing to succeed. The more diverse the group of exploratory testers – from developers to product owners, UAT testers, UI/UX team, etc.. the better the results will be.

In Agile teams, there is never enough time to test everything. However, exploratory testing will reduce the risk of bugs slipping through which impact the end-user, whilst maximizing the QA team’s time.

3. Speed of test cycles

Another reason why exploratory testing is so efficient in Agile is the speed of its test cycles. A common misconception about exploratory testing is that it slows down the development cycle. This is untrue. Unlike other testing methods, there is no need to invest time in authoring, preparing, and approving test cases for exploratory testing.

One of the main principles of exploratory testing is that it reduces unnecessary documentation, in line with the second value of the Agile Manifesto: "working software over comprehensive documentation".

In addition, exploratory testing feedback is very quick. It allows the QA team to adjust strategies and refocus swiftly.

Where does automation fit in with Agile teams?

Test automation fits with the principles of Agile teams. Communication and feedback from the QA team should decipher when automation should be used. For example, if a tester is repeating the same regression test on multiple occasions in a short period of time, it should be a candidate for automation. Highly repetitive, laborious testing is where automation works best. As an Agile team, it’s important to know when not to use automated testing too. One-time tasks generally shouldn't be automated. Given how long it takes and the effort required, this typically isn't an efficient use of time, nor is it cost-effective.

Conclusion

Software development is a constantly evolving environment. Change is inevitable. A key element of successful Agile teams is the ability to respond to change rather than stick to the same processes. The best Agile teams constantly re-evaluate their priorities and shift their approaches accordingly.