Launch Wrangling

Imagine being one of five testers in an organisation with over 400 developers. Picture a pace of 1,251 production deploys in a week. Now throw in a distributed workforce that communicates almost exclusively online.

Last night I attended Sheri Bigelow's talk at WeTest. Sheri works as a tester at Automattic in what I believe to be quite a unique environment. She shared some fascinating insights into building a testing culture in continuous delivery, in a team where testers are vastly outnumbered and testing is an optional activity.

Of all the things that Sheri spoke about there was one in particular that resonated. It's something that I can imagine applying in my own organisation, despite our vastly different contexts in developer:tester ratio, rate of release, and risk profile.

Launch Wrangling

Many people who develop software consider a release to be the end of their process. The idea that once a feature is in production, being used by the hands of customers, the development team can move on to their next piece of work.

When Sheri talked about deploying to production she said that it's "not the end of the game, it's kind of the middle". At Automattic the developers have work to do beyond creating the code itself. They are expected to monitor their changes in production and help to provide support to users when required. A true DevOps culture.

In the middle of a sports game, the players will usually take a half-time break. They'll have some refreshments, reflect on the game so far, take inspiration from their coach, then return to the competition.

In teams where deploying to production is halfway in the process there'll be a similar lull in activity. That little bit of time between something being finished and being released is an opportunity for refreshment, reflection, then a return to action. A half-time break.

In this relatively empty time, Sheri saw an opportunity. She started asking delivery teams to use the space around their releases to participate in, essentially, a bug bash. People put aside their day-to-day duties and those from every type of role worked together for a short period of time to test the product.

At Automattic this activity is called launch wrangling. Apparently when your Company Founder is from Texas there's a strong cowboy influence in naming things!

Sheri has used launch wrangling as an opportunity to introduce testing as an activity to developers who may not have tested before. She also talked about getting a lot of eyes across the application to improve the chances for important problems to be discovered prior to release. This means that launch wrangling is both a coaching tool and a way to improve test coverage.

No matter what type of delivery schedule you adopt, breathing space around deployments is likely to exist in some capacity. In my experience the amount of time available will correlate to the size of the changes being made to production. Big changes create a bigger pause. Utilising this gap seems like a sensible way to appropriately time-box a bug bash activity.

I like the idea of launch wrangling to foster testing across disciplines and improve the scope of testing where resources are particularly limited.

Benefits of cross-team pair testing in agile

One year ago, in June 2015, I launched a pairing experiment in my organisation. The primary purpose of this experiment was to share knowledge between testers who were working in different agile teams, testing different applications and platforms.

I shared the results of our experiment in my talk at TestBash in Brighton earlier in the year. For those who missed this presentation, this is a short written summary of the four main benefits that we observed from cross-team pair testing.

Visibility of other teams

Before we began the experiment I had received feedback from the testers that they felt siloed from their testing peers. At that stage we had 20 testers spread across 15 different agile teams, which meant that many were working as the only specialist tester in a cross-functional delivery team. 

This isolation was starting to seed imposter syndrome. The testers were beginning to doubt their own abilities and feel uncertain about whether they were doing things right.

Happily, one of the strongest themes in the feedback about cross-team pairing was that it increased visibility of what was happening in other teams. The opportunity to understand how another team operated was described as interesting, eye opening and awesome. Seeing other practices and processes gave a degree of comfort to each tester that their own approach was appropriate.

Broader Scope

One of the challenges in being the only test specialist in a team is in generating testing ideas. It can be difficult to consider different perspectives when brainstorming as an individual.

Through pairing, the testers were able to see their application through fresh eyes by exploring with a tester from outside of their product. A different mindset helped them to identify gaps in the application and think of creative ideas to explore functionality. The opportunity to have deep discussions about testing led to the discovery of interesting problems on unexpected pathways.

The broader thinking demonstrated within a pairing session was then carried into future testing as each individual tester started to augment their own planning with ideas they had seen demonstrated by their peers.

Improve communication

Make fewer assumptions. Ask more questions. These are two central tenets of testing that most testers will believe they do. When compared to other disciplines, its often true that testers are asking more questions. Pairing highlighted situations where testers had started to relax these instincts.

The tester who was hosting a session would often make incorrect assumptions about the depth of their visitors knowledge. Their "simple" explanations were difficult for someone from outside of their delivery team to understand. 

The presence of an outsider exposed the amount of assumed institutional knowledge in the business stories, test planning and informal communication of a team. The tester who was visiting a peer would have to ask a lot of questions in order to understand the application and how it would be tested.

Pairing caused the testers to question their own expectations of knowledge in the team. They started to make fewer assumptions about what had been left unstated in team documentation. By increasing the number of questions they asked, the testers began to interrogate whether there was truly shared understanding or instead shallow agreement.

New Approach

Every person will have a unique way of working. Not just in their thinking, but in the particular combination of tools that they use to capture, process and report information. 

Pairing gave the testers the opportunity to observe and experience the work environment of a colleague. In many cases this first-hand experience led to the discovery of a new tool that could be adopted by the visiting tester in their own work. Through pairing we saw chrome extensions, excel macros and screenshot tools propagate across the department.

The proliferation of these tools meant that the testers were more productive. They were able to reduce the repetitive tasks in their workflow and use appropriate tools to support their test approach.

Benefits of cross-team pair testing in agile

For more information about pairing, please refer to my previous posts:

• Information about pair testing
• Details of the pairing experiment