Saturday 9 November 2013

Evolution of Testing

I've been involved in a twitter discussion about the evolution of testing. Aleksis has been pushing me to think more about the comments I've made and now I find I need a whole blog post to gather my thoughts together.

Scary thoughts

The discussion began with me claiming it was a little scary that a 2008 Elizabeth Hendrickson presentation is so completely relevant to my job in 2013. Aleksis countered with a 1982 Gerald Weinberg extract that was still relevant to him. This caused me to rue the continued applicability of these articles. "I think testing is especially stuck. Other thinking professions must have a faster progression?"

It feels to me that innovative ideas in testing gain very little traction. Thoughts that are years or decades old are still referred to as new technique. The dominant practice in the industry appears to be one that has changed very little; a practice of writing manual test cases prior to interaction with the application then reporting progress as a percentage of test cases executed.

Changing programming

But we're not alone, right? Aleksis said "I don't know what thinking professions have progressed better. Programming?"

Yes, I think programming is evolving faster than testing. I'm not just talking about the tools or programming languages that are in use. I believe there has also been a shift in how code is written that represents a fundamental change in thinking about a problem.

Programming began in machine language at a very granular level, or even prior with punch cards. When I worked as a developer 10 years ago the abstraction had reached a point where there was focus on object oriented programming techniques that created siloed chunks that interacted with each other. Now I see a step beyond that in the extensive use of third party libraries (Boost C++) and frameworks (Spring). The level of abstraction has altered such that the way you approach solving a programming problem must fundamentally change.

Why do I call this evolution? Because I don't believe people are standing up and talking about object oriented technique at programming conferences anymore. There are swathes of literature about this, and many people have reached achieved mastery of these skills. The problem space has evolved and the content of programming conferences has changed to reflect this. The output from thought leaders has moved on.

Dogma Danger

I think "evolution is when the advice of 30 years ago is redundant because it has become practice and new ideas are required." Aleksis countered "That's how you end up growing up with dogma."

I don't want people to take ideas on as an incontrovertible truth. But I do want people to change how they go about testing because they think an idea is better than what they currently do and they are willing to try it. I am frustrated because it feels that in testing we aren't getting anywhere.

Why aren't we seeing the death of terrible testing practice? Why is a model of thinking conceived in 1996 still seen as new and innovative? Why is the best paper from EuroStar and StarWest in 2002 still describing a a test management technique that isn't in wide use?

Moving the minority

James Bach chimed in on Twitter saying "Testing has evolved in the same way as programming: a small minority learns". Fair, but then everyone else needs to catch up, so that the minority can continue their journey before we're worlds apart. I haven't been in testing long, and I'm already tired of having the same conversations over and over again. I want to move on.

Is anyone else frustrated about this? How can we create sweeping change? How can we be sure the test profession will be genuinely different in another 10 years?


  1. Hmmm - reading this today, and building on from some conversations yesterday, my initial reaction is this - that testing is like Christmas.

    Oh, everyone complains about Christmas, about how expensive it is. But when you really analyse it, Christmas was about something once (originally pre-Christianity, celebrating being half way through winter, knowing it got easier from here).

    But people have made it all about a whole fanfare of other things - the presents, the turkey, the brussel sprouts, the movie your old people fall asleep to after lunch, having alcohol on hand in different flavours so no-one's offended, mega boxes of chocolates blah blah blah ...

    But for some reason the people who complain most about the cost, are the ones who get DEEPLY offended if you say "well, do we really need this" and try and take a piece away, because you're "ruining Christmas". Nothing says more of the ritual of Christmas than Brits who insist on having brussel sprouts on Christmas despite hating them (and paying a small fortune for them in NZ).

    Testing has developed a lot of trappings that a few of us have developed the notion are "best practice". Painfully that was probably me in the 20th Century. And indeed certain of those trappings did seem to work in the context I originally encountered them in.

    But as I learned more about the nature of how testing really worked, the more I wondered about those trappings, and indeed if we needed different ways of doing things.

    Christmas dinner in the UK - a turkey roast meal - makes a lot of sense in the winter. Down in the Northern hemisphere in NZ, my business owner (AKA Mrs Talks) insists on tradition. In the middle of a bright summer day, messing around in a hot kitchen whilst your neighbours are kicking back on their patio with a BBQ - it's more than a bit crazy ...

    But this is what many testers are sticking to.

  2. Meh - we're in the Southern Hemisphere even.

  3. What I try to do: stop talking/listening/reading about what you don't like and start talking/listening/reading about what you do like or want to know more about.