Sunday 20 May 2018

9 quick ideas for flexible testing

When you're a tester in an agile team, it can be easy to fall into a comfortable testing pattern. The transparency of daily stand-up and reflection of retrospectives can create an illusion of continuous improvement. These routines make us feel that we work in a flexible way but, if we dig a little deeper, we may not be as adaptable as we think.

If you think back to the last time that you felt uncomfortable at work, there's a strong probability that this feeling was associated with a change that you were experiencing. A flexible approach means that you are willing to accept and adopt change regularly, which means that you routinely experience discomfort.

When was the last time that you were surprised by the outcome of your retrospective or quizzed by a colleague in your stand-up? When was the last time that a stakeholder asked questions about your test artifacts? If you can't remember being challenged recently then you, and your team, might be stuck.

Being flexible is not just about activities or outcomes. Imagine that you used to plan your testing in an Excel spreadsheet and now you capture test ideas in a mind map. Does this make you a flexible tester? Not necessarily.

To be a versatile thinker you need to regularly inspect your own habits and create opportunities to collaborate with different people. If you cultivate flexibility as an attitude and make it part of the way that you work, you'll become more aware of how you think. You can change what you deliver, but a flexible tester will also challenge how they deliver.

How can you do that?

Here are nine quick, practical ideas that may help you develop your flexibility as a tester:

  1. Change the order of your test approach to break a routine.
  2. Ask for advice from a non-tester on how to diagnose a bug.
  3. Actively seek test ideas from non-testers outside your agile team e.g. UX, Ops.
  4. Copy the format of a colleague's test report to see your own results in a new light.
  5. Pair with a tester in another team to see a different test approach first-hand.
  6. Invite someone else to test your product, then debrief about what they found.
  7. Experiment with a tool that you haven't tried before.
  8. Take a second look at something that you thought was a bad suggestion.
  9. Ask for constructive feedback about your testing.

Being in an agile team does not guarantee that you are behaving in an agile way. Try to develop the habits that cultivate flexibility, so that you continue to learn and your testing continues to evolve.

Saturday 12 May 2018

No unit tests? No problem!

A couple of weeks ago I created a Twitter poll about unit tests that asked:

"Is code without unit tests inherently bad code?" 
The conversations that emerged covered a number of interesting points, which challenged some of my assumptions about unit tests and how we evaluate code.

What is bad code?

When I framed my original question, I deliberately chose the phrase "inherently bad code". I was trying to emphasize that the code would be objectively bad. That the absence of unit tests would be a definitive sign, one of a set of impartial measures for assessing code.

In my organisation, most of our agile development teams include unit tests in their Definition of Done. Agile practitioners define a Definition of Done to understand what is required for a piece of work to be completed to an acceptable level of quality. In this context, the absence of unit tests is something that the agile development team have agreed would be bad.

A Definition of Done may seem like an unbiased measure, but it is still a list that is collectively agreed by a team of people. The code that they create isn't good or bad in isolation. It is labeled as good or bad based on the criteria that this group have agreed will define good or bad for them. The bad code of one team may be the good code of another, where the Definition of Done criteria differs between each.

Is the code inherently bad when it doesn't do what the end user wanted? Not necessarily. What if it the unexpected is still useful? There are a number of famous products that were originally intended for a completely different purpose e.g. bubble wrap was originally marketed as wallpaper [Ref].

I believe there is no such thing as inherently bad code. It is important to understand how the people who are interacting with your code will judge its value.

Why choose to unit test?

Many people include unit testing in their test strategy as a default, without thinking much about what type of information the tests provide, the practices used to create them, or risks that they mitigate.

Unit tests are usually written by the same developer who is writing the code. They may be written prior to the code, in a test driven development approach, or after the code. Unit tests define how the developer expects the code to behave by coding the "known knowns" or "things we are aware of and understand" [Ref].

By writing unit tests the developer has to think carefully about what their code should do. Unit tests catch obvious problems with an immediate feedback loop to the developer, by running the tests locally and through build pipelines. If the developer discovers issues and resolves them as the code is being created, this offers opportunities for other people to discover unexpected or interesting problems via other forms of testing.

Where there are different types of automated testing, across integration points or through the user interface, unit tests offer an opportunity to exercise a piece of functionality at the source. This is especially useful when testing a function that behaves differently as data varies. Rather than running all of these variations through the larger tests, you may be able to implement these checks at a unit level.

Unit tests require the developer to structure their code so that it is testable. These implementation patterns create code that is more robust and easier to maintain. Where a production problem requires refactoring of existing code, the presence of unit tests can make this a much quicker process by providing feedback that the code is still behaving as expected.

The existence of unit tests does not guarantee these benefits. It is entirely possible to have a lot of unit tests that add little value. The developer may have misunderstood how to implement the tests, worked in isolation, or designed their test coverage poorly. The merit of unit tests is often dependent on team culture and other collaborative development practices.

No unit tests? No problem!

Though there are some solid arguments for writing unit tests, their absence isn't always a red flag. In some situations we can realise the benefits of unit testing through other tools.

Clean implementation patterns that make code easier to maintain may be enforced by static analysis tools. These require code to follow a particular format and set of conventions, rejecting anything that deviates from the agreed norm before it is committed to the code base. These tools can even detect some of the same functional issues as unit tests.

Rather than writing unit tests to capture known behaviour, you may choose to push this testing up into an integration layer. Where the data between dependent systems includes a lot of variation, shifting the tests can help to examine that the relationship is correct rather than focusing on the individual components. There is a trade-off in complexity and time to execution, but simple integrated tests can still provide fast feedback to the developers in a similar fashion to unit testing.

When dealing with legacy code that doesn't include unit tests, trying to retrofit this type of testing may not be worth the effort. Similarly if the code is unlikely to change in the future, the effort to implement unit tests might not provide a return through easy maintainability, as maintenance will not be required.

There may be a correlation between unit tests and code quality, but one doesn't cause the other. "Just because two trends seem to fluctuate in tandem ... that doesn’t prove that they are meaningfully related to one another" [Ref].