Here’s Why You Still Need Exploratory Testing Documentation

Although exploratory testing is, by definition, unscripted, that doesn’t mean it should be undocumented.

Exploratory testing documentation differs from its scripted testing counterpart in several ways, including the point in the test cycle at which it occurs. Additionally, documentation for exploratory testing differs in detail, format, and general best practices.

Understanding Exploratory Testing

To understand the documentation of exploratory testing, you must first understand exploratory testing itself.

In their research paper, “Exploring Exploratory Testing”, Cem Kaner and Andy Tinkham define exploratory testing as “any test in which the tester actively controls the design of the tests as these tests are performed and uses the information gained from testing to design new and better tests.”

This includes test design, which usually involves documentation.

However, exploratory testing is not a type of testing; test types are used to achieve specific purposes. For example, security testing finds vulnerabilities in software. Nor is exploratory testing a testing technique. Testing techniques eliminate specific types of defects. For example, negative testing techniques ensure that the software doesn’t do something it’s not supposed to.

Instead, exploratory testing is an approach to testing; it’s a way for development teams to perform a test. A testing approach should consider the overall purpose of the test. If the objective, for example, is to verify that the software operates according to the acceptance criteria, the usual approach is to use scripted test cases directly linked to the user stories. However, this approach does not necessarily include all extreme cases.

If teams want full testing coverage, they should combine testing approaches and include exploratory testing to find defects that might be missed by testing only software requirements.

Since exploratory testing is often part of a comprehensive test plan, documentation is required.

Documentation of scripted or exploratory testing

The three main differences between documentation for exploratory testing and scripted testing are when it happens, how it’s designed, and how it’s captured.

1. When Documentation Happens

In traditional scripted testing, teams complete documentation prior to test execution. Test cases are designed, written and then executed. They can be used to develop automated test scripts and reused in regression testing.

With exploratory testing, test design occurs simultaneously with test execution. While some components of exploratory testing – such as charters and mind maps – involve planning before execution, most documentation takes place during execution.

2. How the tests are designed

Test design is one of the most important documentation activities in exploratory and scripted testing, but it is accomplished in very different ways. In scripted testing, developers design test cases in a specific format, detailing each step and its expected outcome. In exploratory testing, the test design involves providing parameters, but the specific steps and flow are not defined. This allows the tester to more effectively control the direction of test execution.

3. How the tests are documented

In scripted testing, teams typically document test cases using a test lifecycle management tool or in an Excel sheet or Word document. Test results and any defects are also documented in these tools. In some scenarios, such as regulated test projects, teams need to collect test evidence and include it in documentation.

In exploratory testing, the tests are documented using mind maps, decision tables, flowcharts, and even notes. Additionally, some test management tools – such as Test IO, Testpad, and others – have exploratory testing features that allow testers to record their actions through the system. No-code test automation tools can be used to record actions and therefore create documentation as well.

Guidelines for Documenting Exploratory Testing

One of the best approaches to documenting exploratory testing is to start with a session-based test charter. Then use a record and playback tool to capture actions and results.

A session-based charter describes the test mission or scenario, gives general instructions on how to test, and notes any issues that arise. It is usually time-limited. The record and playback tool then records the exploration of the chart. Teams still need to document flaws, barriers, and learnings.

Mind maps are another format that can be useful for documenting exploratory testing. Although they can be used during test execution, it is easier to schedule them before the test begins. With this approach, the tester documents all predetermined high-level scenarios, but continues to document throughout the test.

The main advantage of these tools is that the results are reviewable and reproducible. Repeatability is especially important for defect management and for planning a test suite for regression testing.

However, find the right balance between repeatability – especially when defects are detected – and freedom of execution. One of the purposes of exploratory testing is to learn more about the application. As such, the documentation should not interfere with the learning and querying processes.

Sam D. Gomez