/ Exploratory

Exploratory Testing

What is Exploratory Testing?

Exploratory software testing is an approach that can help uncover bugs quickly. It is described as simultaneously learning, test design, analysing and executing tests.

Cem Kaner, who coined the term in 1993, now defines exploratory testing as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.“ Exploratory testing can be seen as a structured approach that promotes lean documentation.

Misconceptions about Exploratory Testing

There is a misconception that exploratory testing is an undocumented approach, this type of testing would be more Ad Hoc. Another misconception is that exploratory testing is a test technique. Exploratory testing is an approach that can be applied to any test technique, for example, exploratory testing can be applied to boundary value analysis, decision tables, personas, mnemonics and so on.

Elements of Exploratory Testing

  • **Test Design: **Use test techniques to help with test design and to identify areas that are important to investigate further.

  • **Test Executing: **An approach to exploratory testing is to start execution right away, this helps guide further areas requiring testing and also helps gather questions which need to be asked. Executing straight away helps drive your investigation to the most interesting information.

  • **Learning: **As you explore the application you gain further information and learn how the application works.  The more you explore the more you discover, finding out information about its quirks and areas where bugs might be hiding.  “You have to look past what you expect or hope to see in order to see what’s really happening” Explore It, 2015

Structured Exploratory Testing Efforts

You can easily get lost, end up missing things, or testing the same stuff multiple times.  Structuring your testing efforts, to keep a focus on certain areas of the application without getting lost, is key to making exploratory testing effective and successful.  You need time management to ensure all testing has been completed by the required deadline. Session-Based testing helps manage these concerns.

What is Session-based Testing?

Session-based testing looks to structure exploratory testing, including different elements that make it easier to document and structure your exploratory testing efforts. Session-based testing allows you to be more accountable for what you are testing.

Wikipeida – Session-based testing** is a software test method that aims to combine accountability and exploratory testing to provide rapid defect discovery, creative on-the-fly test design, management control and metrics reporting. The method can also be used in conjunction with scenario testing. Session-based testing was developed in 2000 by Jonathan and James Bach.

Elements of Session-Based Testing

  • **Mission: **The purpose of the mission is to help focus the session.The mission tells us what we are going to be testing and what problems we might be looking for.
  • **Charter: **A charter is a goal for the test session. You create charters with your team and these can be created at any time. You can use requirements, acceptance criteria to help create charters.
  • **Session: **You allocate an uninterrupted period of time that you spend testing. Ensuring you are uninterrupted really helps keep the focus and makes it less likely to miss anything. During the session you execute and take rough notes, once finished your session you can then fill in the notes in more detail.
  • **Session Report: **The session report records the test session. Usually this includes: - Charter
  • Area tested
  • Detailed notes on how testing was conducted
  • A list of any bugs found
  • A list of issues (open questions, product or project concerns)
  • Any files the tester used or created to support their testing
  • Percentage of the session spent on the charter vs investigating new opportunities
  • Percentage of the session spent on:- Testing – creating and executing tests
  • Bug investigation / reporting
  • Session setup or other non-testing activities
  • Session Start time and duration

Reference from Wikipedia: https://en.wikipedia.org/wiki/Session-based_testing

More on Exploratory Charters

When using non-scripted testing such as exploratory testing it can be easy to get lost if you have no structure setup. While clicking and following wherever the application takes you, can sometimes be useful, at other times it can remove your focus from what is important.

If you were walking along a marked bush walk track, and you walked off the track, it would be easy to get lost if you haven’t prepared. Exploratory testing can be the same, if you haven’t prepared what you are testing, or looking for then you can easily get lost and lose focus on what areas are hiding bugs.

Focus on one area at a time. If you see something of interest, write it down and add it to an existing charter or create a new one.

Charter Template

Below is an example of a charter template from “Explore It by Elisabeth Hendrickson”

Explore (target)

with (resources)

to discover (information) 

**Target:** Where are you exploring? It could be a feature, a requirement, or a module.

**Resources: **What resources will you bring with you? Resources can be anything: a tool, a data set, a technique, a configuration, or perhaps an interdependent feature.

Information: What kind of information are you hoping to find? Are you characterizing the security, performance, reliability, capability, usability, or some other aspect of the system? Are you looking for consistency of design or violations of a standard?

Hendrickson, Elisabeth (2013-02-21). Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing (Kindle Locations 404-408). Pragmatic Bookshelf. Kindle Edition.

Timeboxed Charter

When exploring each charter, it can be open ended. You need to ensure you are giving adequate time to each charter, so you don’t spend all your time on the first charter then have to rush the rest of them.

Charter Timeboxing allows you to allocate a period of time for your characters, during this time you explore, design and execute tests, moving from one experiment to the next without pausing. You might find setting up a stopwatch for a period of time helps, during this time you would not allow anything else to distract you.

You take notes so you know what you have explored, noting down anything of interest and any areas that might be worth setting up more charters to investigate later.

At the end of each session, you can always write more detailed notes.  Use the notes you have created to communicate with the stakeholders about what areas have been tested and any questions that might have come up during the session.

Article Links

What is Exploratory Testing? – James Bach

Exploratory Testing Explained – James Bach

A tutorial in Exploratory Testing – Cam Kaner

Exploratory Testing – Ministry of Testing

Exploratory Mindmaps – Ministry of Testing

Benefits of Session-based Testing – Katrina Clokie

A Heuristic Approach to Test Charters – Adam Howard

You Tube Links

Understanding Exploratory Testing – James Bach

Large-scale Exploratory Testing – James Whittaker

Book Links

Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing – Elisabeth Hendrickson