Wednesday, 27 June 2018

3 ways to define your role without a RACI matrix

I lead a team of Test Coaches, which is still a relatively unusual role in our industry. As a Test Coach I was part of a number of conversations about my role. As we scaled the team of coaches and I took a leadership position, these conversations increased in frequency and audience.

Uncertainty about roles can happen when there are:
  • new roles created, or 
  • multiple people performing the same role in different areas of an organisation, or 
  • a number of roles that interact closely with a shared purpose.

In all three situations there are alternatives to drawing up a traditional RACI matrix, which can feel like marking territory when executed poorly. I prefer collaborative approaches to role definition. This article explains how to use an interactive activity with practical scenarios, a role expectation mapping canvas, and a rainbow roles overview.

Interactive activity with practical scenarios

When our test coaching function grew there were questions about how we would operate. Some people had preconceptions about our role being similar to test management. Others had concerns that we might overstep boundaries toward people leadership or setting technical direction.

To kick-off our expanded services we scheduled a session for all the testers in the organisation, along with their line managers and delivery leadership, to explain test coaching. Attended by over 120 people, part of the agenda was an interactive activity to clarify and define our role.

We asked the audience to pair up with someone that they wouldn't normally work with, preferably someone from another department. We gave each pair a questionnaire that listed 13 scenarios and asked them to consider whether or not they would involve a Test Coach in each situation.

The scenarios in the activity were descriptive and practical. They were framed in the first person to encourage people to imagine themselves in each situation. Not just "bugs are being ignored", but "my bugs are being ignored".

Questionnaire for Test Coach role

Where a pair disagreed, they were allowed to record a two responses, however we encouraged discussion of differing opinions in the hope that a single answer could be found.

Once this task was complete, we handed out sets of coloured paper squares to each pair. Each set included green, yellow, and pink sheets to represent yes, maybe, and no respectively.

We used a set of slides to work through the questionnaire. Each question was displayed on a slide with a blue background, a neutral colour. I read out the question and each pair in the audience had to hold up a coloured card that corresponded to their answer. The card was green if they would engage a Test Coach, pink if they would not, and yellow if they were unsure.

Asking each pair to vote kept the audience engaged, and helped us to understand where there was confusion about our role. Where the responses were mixed, or where they consistently differed to what we expected the response to be, we gained insight into situations where there might be overlap with other roles or where we might have to clarify the intent of our involvement.

After each vote, the next slide had the same question with a background colour that revealed the answer: green, yellow or pink. I provided a short explanation of our rationale for involvement in each situation and gave the audience the opportunity to ask any questions that they had.

Slides that explain when to ask a Test Coach

This approach to defining our role was well received and created a clear set of expectations about how to interact with the new Test Coaches.

I have since applied the same type of exercise in a delivery tribe with five leadership roles. Both the team and the leaders were uncertain of who should be driving the outcome in some instances. I adapted the activity so that the columns of the questionnaire represented each role.


Questionnaire for support roles in agile delivery

The audience split into pairs who discussed a set of scenarios, choosing which role they would expect an action from in each situation. We used five different coloured cards for the audience to share their thoughts, one colour per role. With more roles in scope and a larger audience, there was robust conversation about some of these scenarios and the activity was longer than when it was focused on a single role.

Role expectation mapping canvas

The test coaches in my organisation are part of a wider team that includes technical and practices coaching. As the team grew there was confusion, particularly from newer coaches, about our responsibilities. During a team offsite day, we split into our specialist coaching disciplines for an activity to discuss this.

My manager facilitated the activity using the role expectation mapping canvas developed by Tony O'Halloran based on ideas from Jimmy Janlen's description of role expectation mapping. Each subset of coaches in the team completed the activity together i.e. all of the Test Coaches worked on a single canvas.

The canvas is separated into five parts:
  1. Roles that you work with
  2. Externally visible signs of success
  3. Things that you are responsible for
  4. Expectations of the people that you interact with
  5. Challenges that you face


Role expectation mapping canvas

As the manager of the test coaching function and the longest serving test coach, I decided to act as the scribe for this activity. I refrained from contributing to the conversation and instead focused on recording it. This was difficult, but my silence created space for the other test coaches to express their ideas, opinions and doubts.

As each test coach worked in a different neighbourhood the purpose of the activity was to understand what was common. Across each department in our organisation there are consistent stakeholder roles, indicators of success, responsibilities, expectations and challenges. The canvas was useful to focus conversation and capture these, to create clarity and alignment in our function.

This canvas was designed to be applied in a delivery team to clarify relationships between different roles. Though I am yet to use it in that context, I can see how it might be successful.

Rainbow roles

A tribe that I interact with as a Test Coach has other test leadership roles at a tribe and team level, along with delivery managers who have an interest in quality. This environment had created confusion in the test chapter of the tribe and conflict in direction for testing.

To resolve this, a meeting was scheduled for the people directly involved in test leadership. The facilitator drove the discussion towards a RACI matrix, which was distributed to the participants of the meeting after the session.

In this context the RACI matrix was usefully divisive for the leadership audience. It clearly showed which role was responsible for which task. However I felt that we needed to present something else to the rest of the tribe. Something that illustrated how we had agreed to work collaboratively towards a shared purpose, rather than defining our separate territories.

I converted the information from the RACI matrix into a one-page overview with five streams that showed how the test leadership team would work together on common testing goals: strategy, standards, day-to-day support, recruitment, and personal development.

Rainbow Roles

It isn't miles away from the original, but the information is presented in simple language and intentionally illustrates our different contributions towards a single result. The rainbow colours started as a joke to further emphasize our aspirations of harmony, but became core to the identity of this diagram: rainbow roles.

It is not unusual for roles to become murky as the organisation evolves. In each example we found clarity through collaboration using tools that emphasized how different people work together. I encourage you to treat confusion or conflict as an opportunity to positively engage with others about your role and theirs.

For another take on this topic, you might also want to read Riot's agile team leadership model: A story of challenging convention by Ahmed Sidky and Tricia Broderick.

Thursday, 7 June 2018

The world of test automation capability

Imagine a traditional capability matrix. A spreadsheet where each column is skill, each row is the name of an individual, and then the map is populated by people who rate their competence. These spreadsheets are useful for seeing trends within a team, both strengths and opportunities for improvement, but there are limitations.

A matrix captures state, but not journeys. It shows skills, but not always how the skills are applied. It lists what is important in a specific domain, but not how other people are using different skills to solve the same problem. The limitations of the matrix can stifle thinking about capability where evolution, application, and diversity of skill are important.

We could try to capture this information in our spreadsheet. Telling these stories by colour-coding or cross-referencing multiple tabs. Or we could think about capability from another angle.

The world of test automation capability is a model that illustrates the skills and experience of a test team using layers of the earth: core, mantle and crust. At the core are the testers with the least knowledge about coding and test automation frameworks. Paths to the surface travel through the mantle of programming languages towards the crust of applying coding skills.



Divisions exist within the layers. The mantle of programming languages might include Java, JavaScript, Visual Basic, Ruby, and Swift. The crust of test automation frameworks might include suites for web-based user interfaces, iOS and Android mobile applications, desktop applications, and APIs. Splitting each layer of the model proportionally allows us to see relative strengths and weaknesses in capability and tooling.

Divisions within the mantle and crust can be shared by different teams. In my organisation testers are split across four different departments. There is some overlap in the programming languages and automation frameworks in use across these different areas. In forming a strategy for training and framework evolution, we want to consider the distribution of testers within this model.

Part of the test coaching function in my organisation is to formalise paths between layers of the world, to support testers to learn a programming language and apply this knowledge to implementation of a test automation framework. These paths are created through training courses, coding dojos and knowledge sharing opportunities. Informal paths will exist where testers are self-taught or prototyping their own frameworks.

To illustrate, our current world of test automation capability can be visualised as:



The model shows testers at the core who have no coding or framework experience. In the mantle, the strongest programming language is Java, though capability also exists in four other languages. Our division in programming language is usually by department, with different areas adopting different solutions. At the crust, the largest population are working on WebDriver Frameworks, which includes testers who work on web-based applications across all areas of the organisation. The crust also includes API frameworks as an emerging area, UFT as a legacy framework choice, and the mobile teams who have specialised tooling in xCode and Espresso.

The world of test automation capability is a snapshot at a given point in time. Divisions within the model will shift as programming languages and automation frameworks evolve. People move through the model as their knowledge and experience develops. Paths will change as learning needs transform.

It is quite a different snapshot to a matrix though. It is one that includes relationships between training and skill level, between skills and frameworks, and between people across different departments. The evolution, application, and diversity of skill is easier to see.

Our Test Automation Strategy is formed within the context of this world, describing the forces that the Test Coaches will apply to the model to facilitate and direct movement. Our strategy differs by department, applying a different lens on the overarching visual.



Though how we rate our skill relative to others is useful, there is rich narrative in mapping skills within an environment then talking about how that environment is changing. The world of test automation capability is a new perspective that creates an opportunity for a new story.