Three examples of context-driven testing using visual test coverage modelling

I talk a lot about visual test coverage modelling. Sometimes I worry that people will think I am advocating it as a testing practice that can be used in the same way across any scenario; a "best practice". This is not the case.

A tester may apply the same practice in different contexts, yet still be context-driven. The crux is whether they have the ability to decompose and recompose the practice to fit the situation at hand [1].

To help illustrate this, here are three examples of how I have adapted visual test coverage modelling to my context, and one example where I chose not to adopt this practice at all.

Integrated with Automation

I was the only tester in an agile team within a large organisation. There were two agile teams operating in this organisation, everybody else followed a waterfall process.

The other agile team had a strong focus on test automation. When I joined, there was an expectation that I would follow the process pioneered in that team. I was asked to create an automated test suite on a shared continuous integration server.

Having previously worked in an organisation that put a high value on continuous integration, I wanted to be sure that its worth was properly understood. I decided to integrate visual test coverage models into the reports generated by the automated suites to illustrate the difference between testing and checking.

At this client I used FreeMind to create mind maps that showed the scope of testing. Icons were used against branches in the map to show where automated checks had been implemented, and where they had not. They also showed problems and where testing had been descoped.

Source: Mind Maps and Automation

The primary purpose of the visual test coverage model in this context was to provide a high-level electronic testing dashboard for the Product Owner that was integrated into the reporting from our automation.

Paper-based Thinking

I was a test consultant in an agile team where my role was to teach the developers and business analysts how to test. This team had no specialist testers. The existing team members wanted to become cross-functional to cover testing tasks.

The team had been working together for a while, with an attitude that testing and checking were synonymous. They were attempting "100% automation" and had started to require test sprints in their scrum process to achieve this.

At this client I wanted to separate the thinking about testing from the limits of what is possible with an automation tool. I wanted to encourage people in the team to think broadly about test ideas before they identified which were candidates for automation.

I asked the team to create their visual test coverage models on large pieces of paper; one per story. They would use one colour pen to note down the functionality, then another colour for test ideas. There was a process for review at several stages within the construction of the model.

Source: How to create a visual test coverage model

The primary purpose of the visual test coverage model in this context was to encourage collaborative lateral thinking about testing. By using paper-based models, each person was forced to think without first considering the capabilities of a tool.

Reporting Session Based Testing

I was one of two testers in an agile team, working in a department of many agile teams. I was involved in the project part time, with the other tester working full time.

The team was very small as most of the work in this particular project was testing. We had a lot of freedom in determining our test approach and the other tester was interested in implementing session based testing.

We used xMind to create a mind map that captured our test ideas and showed how the scope of our testing was separated into sessions. We updated the model to reflect progress of testing and issues discovered. It was also clear where there were still sessions to be executed, so the model also helped us to divide up the work.

Source: Evolution of a Model

The primary purpose of the visual test coverage model in this context was for quick communication of session based test progress between two testers in a team. As someone who was only involved part time, the model served as a simple way to rapidly identify how the status of testing had changed while I was away from client site.

A Quiet Team

I was a test consultant in an organisation where my role was to coach the existing testers to improve their confidence. The testers worked together in a single agile team. One of their doubts was that they thought production issues were due to tests that they had missed. They weren't sure about their test coverage.

The testers were very quiet. The team in general conversed at a low volume and with reluctance. There were only a couple of people who were confident communicators and, as a result, they tended to dominate conversation, both informal and in meetings.

I wanted to shift ownership of testing from the testers to the team. I felt that scoping of test ideas and decisions about priority should be shared. It seemed that the lack of confidence was due, in part, to the testers often working in isolation.

Though I wanted to implement a collaborative practice for brainstorming, I felt that visual test coverage models wouldn't work in this team dynamic. I could imagine the dominant team members would have disproportionate input, while the testers may have ideas that were never voiced.

Instead I thought that the team could adopt a time boxed, silent brainstorm where test ideas were written out on post-it notes. This allowed every person to share their ideas. Decisions about priority of tasks, and candidates for automated checks, could be discussed once everyone had contributed by using the collective output of the group to guide conversation.

*****

Before anything else, ask yourself why the practice you'd like to implement is a good fit for your situation. Being able to articulate the underlying purpose is important, both for communicating to the wider team and for knowing what aspects of the practice must remain to meet that need.

I have found visual test coverage modelling useful in many scenarios, but have yet to implement it in exactly the same way twice. I hope that this illustrates how a tester who aims to be context-driven might adapt a practice they know to suit the specific environment in which they are operating.