Exploratory Testing – An Agile Approach?
In a previous post I touched upon the notion of developers helping with the testing bottleneck, by executing scripts predefined by the tester. However, this does come with a risk because one test approach that is difficult to handover is the exploratory side of things. Let me explain...
Since moving into agile development where the requirements are typically lightweight and evolve during the coding phase, its difficult to plan all test cases upfront. More often I'm finding myself having to use an exploratory approach to ensure as much coverage as possible.
Give a non tester a script to execute, and they're only going to cover the predefined tests and not be thinking outside of the box. Hence the increased risk.
So what's the answer to this? Ensure your test script coverage is sufficient? This would require your requirements to be a really low level and set in stone (which goes against the whole agile approach).
To be honest, I don't think there's an answer to this (apart from the tester executing all tests). However, improving your team's understanding is a starting point.
What is Exploratory Testing?
Exploratory testing is simultaneous test design, execution and learning.
How is that different from structured testing?
Structured testing, is designed to verify that the software under test meets the specified requirements. This is not the same as verifying the quality of the software. Testing is directed and constrained by the design specification and test plan, both of which may have been created before a line of code was written.
Exploratory testing is not constrained in this way. The objective is to expose information that is of value to stakeholders and assess the quality of the software, of which compliance with the specification is only one factor.
Exploratory testers are always looking for emergent behaviours, which are behaviours that could not have been predicted prior to the start of testing.
Structured testing focuses on expected behaviours, so emergent behaviours remain entirely untested. It should not be a surprise that this is often where the biggest bugs are found.
Run high level tests allows testers to learn about an aspect of the application's behaviour, and each test provides information that allows ever more powerful tests to be devised. This knowledge is invariably lost in structured testing projects.
Rather than ask does the application meet this requirement, ask questions like what happens if...
The exploratory tester starts with an outline test plan in mind, but can adapt it in response to changing contexts. If one part of an application is not yielding bugs, it may make sense to test in other areas instead. Alternatively the tester may consider using different test techniques in the same area.
Exploratory testing is an efficient and effective approach to testing software. It produces useful results from the first hours of testing, and finds the biggest bugs faster than any other approach. Furthermore, it finds bugs that other approaches won't find at all.