Pair Testing

I'm currently working on defining a pair testing experiment to share testing knowledge across the agile teams within my organisation. What follows is my aggregated research on pair testing, which may be useful to others who are looking to implement pairing in their workplace.

Approach to pairing

Pair testing is a way of approaching a test design process by having two people test the same thing at the same time and place, continuously exchanging ideas. [1]

When paired, two people use a single machine or device. One has the keyboard, though it may pass back and forth in a session, while the other suggests ideas or tests, pays attention and takes notes, listens, asks questions, grabs reference material, etc. [2]

The pair should tackle a single testing task, so that they have a shared and specific goal in mind. Though the pair will work together, one person must own the responsibility for getting the task done. The person with ownership of the task may do some preparation, but pairing will be more successful if there is flexibility. A simple checklist or set of test ideas is likely to be a good starting point. [3]

During a pairing session the testers should talk at least as much as they test so that there is shared understanding of what they're doing and, more importantly, why they are doing it. [4]

Benefits of pairing

These benefits have been taken from the listed references and grouped into three themes:

High creativity

Working in a pair forces each person to explain their ideas and react to the ideas of others. The simple process of phrasing ideas seems to bring them into better focus and naturally triggers more ideas. 

Applying the information and insight of two people to a problem can lead to the discovery of how easily a person working alone can be a victim of tunnel vision.

Pairing brings people into close enough contact to learn about each other and practice communicating and resolving problems.

The camaraderie and the running commentary about the process, necessarily maintained by the pair in order to coordinate their efforts, tends to increase the positive energy in the process. 

High productivity

Each person must stay focused on the task or risk letting their partner down. 

Pairing allows the person at the keyboard to follow their train of thought without pausing to take notes or locate reference information. It encourages dogged pursuit of insights.

Two people working together limits the willingness of others to interrupt them.

Training Technique 

A strong pairing is one where people are grouped so that their strengths will mutually complement their weaknesses. This presents an opportunity for people to learn from one another.

Pairing is a good way for novices to keep learning by testing with others. It's also useful for experienced testers when they are new to a domain to quickly pick up business knowledge.

Experience Reports

A selection of referenced extracts from other blogs about pair testing that I found useful:

How do I know if I'm pairing or doing a demo? This is an important distinction to be aware of. If you're sitting with someone, and one of you is controlling all the conversation for the whole session, then you are not pairing.

Pairing is an interactive partnership. There is a certain level of inquiry and challenge, without feeling of threat or accusation. This is important - another party is essentially reviewing your testing as you perform it, and it's important to respond to these comments. Some of this feedback may suggest other tests, which if feasible you should attempt to include. Sometimes if your session has a goal, and the suggestion would take you off topic, you might want to leave it until the end or schedule another session.

The Power of Pairing Mike Talks

Remember to narrate as you code. What are you thinking? Are you hunting for a file? What’s the test you’re writing now? Why that test? As I was coding, I was often silent. I knew what I was trying to do, but since the code was unfamiliar, I was spending a lot of time hunting. What I discovered was that my partner was feeling a bit useless because he felt he couldn’t contribute. As soon as he told me this, I started describing what I was trying to do and he was immediately able to start pointing me to sections of the code that he had fresh in his mind. 

As a tester, be sure to ask questions. It can be hard to ask questions that you think are dumb – especially when starting out. When I first started pairing as a tester, I felt reluctant to speak up because I didn’t want the programmer to feel like I was telling them how to do their job. I also didn’t want them to think I was stupid. I’ve not had any of the programmers I’ve worked with get defensive or treat me like an idiot. In fact, many things that I thought were stupid questions led to a discussion where we decided to use a different strategy than the one the programmer initially chose. 

Pairing - seeing the flipside of the coin Ben Kelly

If one or the other goes in with the idea that it is a one-way learning experience, the experience will fail.” Pair testing is only effective in an environment of mutual respect and trust. 

Whoever is “driving” during pair testing must ensure that the other party is actively participating and understands what is going on. Encourage thinking and talking aloud, keeping the other person informed on the motivation behind your actions.

Pair Testing: How I bought developers into the test lab Jonathan Kohl

You have to trust them to light the way and they have to trust you to send them a signal the moment you are aware that you are over your head. A good pair will tell you that it’s ok and you will get back on track together. 

Trust, vulnerability and communication in this moment is the bedrock of pairing. It is also the bedrock of building great software. 

The Moment Marlena Compton

I do think there are some times when it does make sense to pair test. For example, if you have a new hire who just doesn't know the system or how to test it, you might have him ride along to learn the system by testing it with a buddy. Likewise, if you are coaching someone new to testing, (or teaching an old dog new tricks), it might make sense to sit down and do real, serious, mission-important test work with two people at one keyboard for an extended period of time, say over an hour. Third, if you notice that you and a peer are finding different kinds of bugs, you might pair test just to learn about each other's testing styles -- to see how the other works, and to gain the skills to put on the 'hat' that finds that other category of defect.

Notice all of these situations are about learning.

Pair Testing or something like it Matt Heusser

... it removes so much fear of failure, which removes a blame culture ... If something gets missed, it’s not one person’s fault.

Communication avenues to pairing opportunities Sean Cresswell

... testing together you will hit issues neither of you have hit [alone]. This is because you are both different testers and will test differently and so together you will try things neither have thought of. Also you will be able to track down more detail since you both have different ways to figure out the issue.

Pair Testing QA Hipster

Increasingly, organizations are bringing people with visual challenges or other disabilities into their accessibility test effort, but these testers still work in silos. Pairing testers with disabilities with non-disabled testers yields valuable results.

Pair testing - A best practice to enhance accessibility test coverage Sunil Dangwal

Do you know of any other resources that might be useful to add to this list?

References

1. Pair Testing
2. Exploratory Testing in Pairs Cem Kaner & James Bach
3. Boost acceptance testing with pair testing Mark de Bono
4. Testing Dojos
5. The Power of Pairing Mike Talks
6. Pairing - seeing the flipside of the coin Ben Kelly
7. Pair Testing: How I bought developers into the test lab Jonathan Kohl
8. The Moment Marlena Compton
9. Pair Testing or something like it Matt Heusser
10. Communication avenues to pairing opportunities Sean Cresswell
11. Pair Testing QA Hipster
12. Pair testing - A best practice to enhance accessibility test coverage Sunil Dangwal