Saturday 22 April 2017

Introducing testers to developers

While completing my Computer Science degree, I created a lot of software without testers. At the end of my qualification, I searched for my first role as a developer with a limited understanding of the other roles in IT that I would work alongside. I didn't know what a tester was, what they did, or how they might help me. I don't think this is an unusual position for a graduate developer to be in. 

At my first job we didn't have testers. Had I spent a longer period in that company, I may have become an experienced developer who still didn't understand what testers were. There are a lot of organisations that create software without employing dedicated testers. I don't think that a poor understanding of testers is an unusual position for an experienced developer to be in.

Developers who have never worked with testers are likely to have an understanding of testing, but as an activity rather than a role. Testing happens as part of their work rather than being lead by a separate person. Why would testing be delegated when a developer can successfully write and release quality software on their own?

If you are a developer with this mindset or history, it can be really challenging to encounter a tester for the first time. Allow me to introduce you through analogy.

No Tester

A restaurant chef is a creator. They make a meal that is delivered to a customer. Quality is determined by the skill of the chef, the quality of their ingredients, and the practices that they follow.

In many restaurants the food is made, delivered to the table, and consumed. The earliest feedback that the chef receives is from the customer. In the event that they have a negative opinion of the meal, it can be difficult to fix the problem. The damage is often done.

There are parallels to software development without testers. Quality is determined by the skill of the developers, the quality of their platform, and the practices that they follow. Feedback comes directly from the customer, who can be unforgiving.

Restaurants can succeed in this model, as can software development.

Waterfall Tester

As a student, I worked part-time as a waitress. In one of the restaurants where I worked the restaurant manager used a small workstation that was located directly outside the double doors to the kitchen.

The chef in this restaurant was temperamental and territorial, so the restaurant manager would rarely venture into the kitchen. However, during meal service, she would intercept the wait staff as we moved plates from the kitchen to our customers.

If the meals for a table were ready at different times, she would hold service until they could be delivered together. If the presentation of a dish was sloppy, she would tidy it up. In the rare event that she was unhappy with portion size or the quality of cooking, she would return the plate to the chef. 

The chef didn't like having his food returned. He would often over-correct in spite. You think the sweet and sour pork portion was too small?! Then you'll get double-sized dishes for the next hour! However he would ultimately settle into delivering a more appropriate size of meal.

On reflection, this restaurant manager was my first experience with a separate tester. Once development was complete she examined the final product. With a fresh perspective, she could identify problems that the kitchen had missed. 

The restaurant manager contributed to the quality of the product through small actions of her own or by reporting problems back to the chef so that he could make improvements. He may not have always responded well to these reports, but he did alter the meals, which improved what the customer received.

Testers bring the same contributions to software development - a fresh perspective to identify problems, fast feedback from someone with a customer focus, and the ability to make their own small contributions to the overall quality of the customer experience.

Agile Tester

The restaurant manager was improving quality of a finished dish. What if there was a way to improve the meal as it was being prepared? Then a chef could adapt as they worked, rather than trying to alter their end result.

Sometimes there are multiple elements of a meal cooking at the same time. As a chef, it can be difficult to keep track and sometimes things get burned. What if someone else had set a timer, noticed when it had lapsed, and could intervene?

Sometimes there ingredients in a dish that look similar and might be easily confused. This week I prepared a meal with two spices: tagine spice mix and couscous spice mix. What if someone else had noticed when I opened the wrong packet?

Sometimes there is a requirement for consistency between meals. How can a chef be sure that the crème brûlée they create today is the same as yesterday? What if someone else could check that the recipe, portion size, and presentation met a minimum standard?

Hypothetically, this all sounds reasonable. Realistically, if you're a chef who likes to cook alone, this might sound like a nightmare. Perhaps you would rather have to throw away the occasional plate of food that was burned or tasted strange than accommodate another person in your creative space.

There is a parallel to the role of a tester in agile development. Success relies on collaborative working relationships that can be difficult to negotiate. Dealing with feedback from a tester can be frustrating, particularly when a developer is not used to receiving it. The tester will need to explain and demonstrate how their perspective can contribute to an improved product. 

It can be difficult for the developer to adapt their approach and accommodate feedback. If they find that the information from a tester is not relevant, timely, or delivered in a constructive manner, then they should let them know. It takes time to form a useful working relationship and investment is required from both sides.

Every Tester

Whether a tester is contributing in agile or waterfall, they want to help deliver the best possible product to customers. If you are a developer who is encountering a tester for the first time, the first thing to assume is positive intent.

Testers bring a fresh perspective to identify problems, fast feedback from someone with a customer focus, and the ability to make their own small contributions to the overall quality of the customer experience. Allow them to influence the product. Ultimately they help to create a better outcome than what you could achieve alone.

Wednesday 5 April 2017

Test Coaching Competency Framework

A few weeks ago I was wondering how to explain the skills required for a Test Coach role. Toby Sinclair introduced me to Lyssa Adkin's Agile Coaching Competency Framework. It's a simple diagram of the core experience and skills that an agile coach might have:

Ref: Hiring or Growing Right Agile Coach by Lyssa Adkins and Michael Spayd

An individual can take this diagram and complete a relative assessment of themselves or others by shading in each slice, for example:

Ref: Hiring or Growing Right Agile Coach by Lyssa Adkins and Michael Spayd

In a presentation about the framework, Lyssa Adkins and Michael Spayd explain how it can be used as a self-assessment tool, for hiring and developing agile coaches, or to help communities of agile coaches to grow.

I like the simplicity of the framework and could see an opportunity to apply it to the testing domain to explain test coaching. As a small disclaimer, I have never heard Lyssa and Michael explain their framework in person, so I'm not entirely sure how true my interpretation is to their original intent.

Test Coaching Competency Framework

The test version of the framework is almost identical to the original:


Starting at the top, testing practitioner reflects the relevant experience that a person has. Different people will have different interpretations of what this means. Shading of this slice might reflect number of years in industry, number of organisations, variety of roles, etc.

If occupying a role is reflected at the top of the framework, the knowledge that you build through that experience is reflected at the bottom as technical, business, and transformation mastery. In simple terms, shading of the practitioner slice shows what you've done, shading in the mastery slices shows what you've learned as a result.

Technical mastery in testing might include:
  • test automation frameworks
  • coding skills
  • continuous integration and delivery tools
  • development practices e.g. source control, code review
  • basic understanding of solution architecture
Business mastery in testing might include:
  • shift-left testing skills e.g. Lean UX, BDD
  • user experience, usability, and accessibility testing
  • targeted exploratory testing and development of test strategy
  • business intelligence and analytics
  • relevant domain knowledge
Transformation mastery in testing might include being a leader or participant in:
  • switching test approach e.g. waterfall to agile
  • continuous improvement in development practices
  • engaging beyond the development team e.g. DevOps
  • change management

On the left of the framework are the skills that are important to communicate mastery to others: training and mentoring. The distinction is in the size of the audience. Training is prepared material that is delivered to a group of people. Mentoring is transfer of knowledge from one individual to another. Both training and mentoring require the coach to have knowledge of the topic that they're helping someone with.

On the right of the framework are the skills that are important to enable others to find their own solutions: facilitating and coaching. The same distinction in size is present: facilitating is to a group of people and coaching is between individuals. A person conducting these activities is guiding a group or an individual towards the discovery of their own solutions. Skill in facilitation and coaching does not come from personal mastery of a topic, but instead in developing the ability to draw the best from people through questioning.

Using the framework

I needed to be able to explain the skills of a Test Coach in order to recruit. I have used my version of this framework as an activity in eight Test Coach interviews so far. I draw the framework on a piece of paper, explain what is included in each slice as above, then ask the candidate to assess themselves and talk about their decisions.

Here's an example of how one candidate responded:



I've found the test coaching competency framework is an excellent alternative to the general prompt of "tell us a bit about yourself". I use it at the start of the interview to give more structure to the opening conversation and clearly illustrate what I'm interested in hearing from the candidate. 

I've found that it empowers people to talk more freely about both their achievements and opportunities for growth. I like that it gives control of the conversation to the candidate for a period of time rather than asking them to respond to specific questions for the entire interview.

What do I do with the results? I haven't been seeking an individual who is strong in every area, but I am looking to build a team of coaches with complementary skills and experiences. The shaded frameworks are helpful in determining how a group of individuals might become a team. Once a team is formed, I imagine that this information could help guide initial allocation of work and allow me to identify some opportunities for cross-training.

I have found the test coaching competency framework useful and would recommend it to others who are looking to build test coaching teams.