Evolution of Testing

I've been involved in a twitter discussion about the evolution of testing. Aleksis has been pushing me to think more about the comments I've made and now I find I need a whole blog post to gather my thoughts together.

Scary thoughts

The discussion began with me claiming it was a little scary that a 2008 Elizabeth Hendrickson presentation is so completely relevant to my job in 2013. Aleksis countered with a 1982 Gerald Weinberg extract that was still relevant to him. This caused me to rue the continued applicability of these articles. "I think testing is especially stuck. Other thinking professions must have a faster progression?"

It feels to me that innovative ideas in testing gain very little traction. Thoughts that are years or decades old are still referred to as new technique. The dominant practice in the industry appears to be one that has changed very little; a practice of writing manual test cases prior to interaction with the application then reporting progress as a percentage of test cases executed.

Changing programming

But we're not alone, right? Aleksis said "I don't know what thinking professions have progressed better. Programming?"

Yes, I think programming is evolving faster than testing. I'm not just talking about the tools or programming languages that are in use. I believe there has also been a shift in how code is written that represents a fundamental change in thinking about a problem.

Programming began in machine language at a very granular level, or even prior with punch cards. When I worked as a developer 10 years ago the abstraction had reached a point where there was focus on object oriented programming techniques that created siloed chunks that interacted with each other. Now I see a step beyond that in the extensive use of third party libraries (Boost C++) and frameworks (Spring). The level of abstraction has altered such that the way you approach solving a programming problem must fundamentally change.

Why do I call this evolution? Because I don't believe people are standing up and talking about object oriented technique at programming conferences anymore. There are swathes of literature about this, and many people have reached achieved mastery of these skills. The problem space has evolved and the content of programming conferences has changed to reflect this. The output from thought leaders has moved on.

Dogma Danger

I think "evolution is when the advice of 30 years ago is redundant because it has become practice and new ideas are required." Aleksis countered "That's how you end up growing up with dogma."

I don't want people to take ideas on as an incontrovertible truth. But I do want people to change how they go about testing because they think an idea is better than what they currently do and they are willing to try it. I am frustrated because it feels that in testing we aren't getting anywhere.

Why aren't we seeing the death of terrible testing practice? Why is a model of thinking conceived in 1996 still seen as new and innovative? Why is the best paper from EuroStar and StarWest in 2002 still describing a a test management technique that isn't in wide use?

Moving the minority

James Bach chimed in on Twitter saying "Testing has evolved in the same way as programming: a small minority learns". Fair, but then everyone else needs to catch up, so that the minority can continue their journey before we're worlds apart. I haven't been in testing long, and I'm already tired of having the same conversations over and over again. I want to move on.

Is anyone else frustrated about this? How can we create sweeping change? How can we be sure the test profession will be genuinely different in another 10 years?

Catalyst for Curiosity

Yesterday I was asked to speak to a room of testers who all work at the same organisation. They were on a training course, and I was asked to visit for half an hour to speak about "Testing Mind Maps".

The room was varied. Four people did not know what a mind map was. One guy claimed to have been using mind maps for over 10 years and expressed some bitterness that the Bach brothers were internationally recognised for doing things that he had done first. With 30 minutes to speak and such a wide breadth of skill, it was difficult to say something valuable for everyone. So, I spoke for a bit, with a handful of people nodding along and others looked stricken.

Then my colleague posed a question to the group. "How is it that there's such a wide variety of knowledge in this room when you all work in the same place? What's your internal process for knowledge sharing?"

There was a moment of silence, followed by a flood of excuses.

"We used to do lunch time sessions, but then people got too busy and no one came"
"We have a wiki, but no one really uses it"
"We used to pair up junior and senior testers, but now our focus is delivery"
"We sit in project teams instead of a test team, so we don't see each other often"

This made me question my own experiences with knowledge sharing; pockets of knowledge spread across multiple repositories, accompanied by individual processes for access and sharing. Though rarely ignored entirely, this scatter-gun approach means that knowledge transfer is usually left to those who are eager to learn. A pull rather than a push. If we want to see better transfer of knowledge, we need to create curiosity.

When we talked about change at KWST3, there was a common experience among attendees of a catalyst that set them on a path of education. Curiosity comes from someone or something that makes you want to learn more. I like to imagine that a half an hour chat about mind maps will be a catalyst for some of my audience yesterday.

Who is creating the catalyst for curiosity where you work? How is your knowledge being shared?

TDD on Twitter

Sometimes people ask me things on Twitter and it's too tricky to answer in 140 characters. Even though I'm not entirely sure how I ended up in this particular discussion thread, nor am I an expert in any of the areas I'm being asked about, I do have thoughts to share (as I often do). So lets talk TDD, or test driven development.


Question for all. I have been hearing about TDD and unit testing a lot these days. Has the bug count reduced? I have to stream line the application I am working on. I am finding ways to do test automation. I am working on Selenium. But from the application design perspective I am also looking ways with TDD. So the question, do you see any change in the issue count, if you have worked with a team following TDD?

My first thought is to wonder what we're comparing the issue count to? Presumably this is the first time we've written the software that's being tested, so it's difficult to know for sure whether a practice that includes TDD has certainly resulted in fewer issues.

TDD is a practice that mandates developers writing tests before they write code. It is a fundamental mindset shift in the way someone goes about their development. I don't believe testers can practice TDD unless they are writing the actual application code.

TDD and unit testing are not the same thing. Unit tests are a result of TDD, but the practice of TDD is not the only way to get unit tests. Unit tests can be written after the code too. 

As testers, we should know, to a certain extent, what unit testing is in place, and what coverage these tests provide. And, generally, a good base of unit tests will give us a better starting point. The issues found in testing are fewer, because the developers have thought more about the code themselves. Generally.

I am implementing the best I can do regards to design and testing. Testing is worthless, if design is not correct. So design is what I am looking at.

I'm not quite sure why TDD is being cited as a design practice, unless specifically speaking to the design of code? If you want to look at "are we building it right?" then TDD is the practice for you. If you want to look at "are we building the right thing?" then it is not (verification and validation).


I have reservations with #TDD. Implementing TDD, the project deadlines also needs to be stretched. Right?

I think this depends on what your current development practices are. If unit tests are already in place, but they're being written after the code rather than before it, then implementing this practice shouldn't stretch out deadlines. If there's no unit testing, then development may start to take longer, but in theory at least you should see some corresponding benefits in testing as the code is delivered in a much better state.


Finally, if you're really curious about TDD, the Wikipedia page looks pretty comprehensive.

Experts of TDD, please voice any disagreement below...

Speak easy

A junior tester at work asked if I would mentor her. It's pretty exciting to be on the experienced side of a mentoring relationship and, simultaneously, terrifying that someone trusts your opinion. It was a little bizarre the first time she took notes as I was speaking...

Our session this month took an interesting turn when she asked for some tips on communicating in her team. It's her first placement in a team environment, as opposed to working as an individual or in a pair. The team she has gone in to is primarily populated with men, all of whom are significantly older than her. As a recently graduated woman, she's a little uncertain. I remember the feeling well.

As I started to suggest strategies she could use in her attempts to communicate more often and with value, I realised how many processes I have for dealing with similar situations in my own environment. Here are some of my suggestions for communicating with people who have more experience, more confidence and a louder voice.

Post It Note Questions

Someone just said something I don't understand. I don't want to interrupt them immediately to ask what it means, but I also don't want to forget to ask. What to do?

I like to write the word or phrase on a post-it note. At the conclusion of the conversation, or meeting, I approach the individual who used the term and ask them to explain it to me. I then re-frame what they said, by writing my understanding of the term on a second post-it note, and have this checked by the person who delivered the explanation. If I've got it, I stick the definition on my monitor.

I find at the start of a project, my monitor gets pretty full. Sometimes the area between my keyboard and the monitor fills up too. But after a week or two, the terms have become familiar enough that I can start to weed out post-it notes; things I now know can go in the bin.

I like this as a process for discovering what things mean, and not having to ask twice.

Stupid Questions

No one likes asking a stupid question. "There's no such thing!" you cry. Sometimes it feels like there is.

One way I give myself confidence to start asking questions, particularly in meetings with a number of people in attendance, is to monitor how many of the questions that I wanted to ask end up being raised by someone else. I note down questions as they occur to me, and then listen as the topic evolves. Any of my questions that are asked by someone else, I put a star next to.

At the end of my first meeting I attend on a project, I may have 20 questions and only two have stars. This indicates to me that the majority of my questions are learning that can happen outside of a meeting environment.

At the end of my fourth meeting, I may have 30 questions and 20 have stars. I feel this is positive feedback from the team (though they don't know they're providing it!) that I'm ready to start asking questions that will assist understanding and provoke discussion, rather than frustrate attendees and derail the agenda. Sure, I still send things off-track every now and then, but I have more confidence at this point that asking immediately is the best option.

Dealing with Dismissal

It can be easy to dismiss those who are young and new to the industry. Dismissal can also feel like the default reaction when ideas are confronting or uncomfortable for people to consider. I can get quite worked up when I feel that my voice isn't heard, but can be quite terrible at thinking on my feet under pressure. There's nothing worse than thinking of the comeback two hours later.

One strategy I use to combat dismissal is to prepare my arguments in advance. I like to float an idea to one person first, and note down their criticisms. Then I retreat to my desk and think about how I can address those criticisms, either in the initial idea or in rebuttal, and go try out my refined spiel on someone else. By the time I'm voicing an opinion in a meeting, I've run it past several individuals and prepared responses to a variety of reactions. I've got the tools to express myself confidently.

After a few iterations of this, I can start to imagine the criticisms without visiting people at their desks. The way I present my ideas become more compelling as I learn how the people in my team are most receptive to hearing ideas.


Would you have any other tips?

Process vs Purpose

I have just returned from a two day training experience that focused on the people management skills I will require in my new role. One of the interesting exercises within this was completing a Lominger sort; taking a set of 67 leadership competencies and sorting them in to categories based on what skills the attendees felt were required to succeed in our positions.

The process

We completed the activity in groups of three. Each group was asked to sort the Lominger card deck in to three categories; essential, very important, nice to have. Our 67 cards had to be split as 22 cards in the first category, 23 cards in the second category and 22 cards in the third category.

Upon completing the sort, each group was given 22 red and 22 blue dots. These were used for dot voting on a wall chart against the full list of key competencies. Blue marked what was essential and red what was nice to have (the top and bottom categories). When all six groups had completed this task, we gathered around this chart to talk through what was discovered.

The results

When looking at what people considered as essential, there were a number of competencies that everyone agreed upon; listening, understanding others, approachability, motivating others, managing diversity, integrity and trust. Our facilitator picked these from the chart and it was clear that with six votes against each there would be no argument.

The next item that the facilitator selected only had three votes. This caused a fair bit of confusion, as there were a number of other things that had five or four votes, yet we appeared to have jumped to discussing a much lower ranked competency. Why?

The purpose

Prior to this activity, the group had identified a set of key challenges in the role. A strong theme that emerged was the lack of time available to deliver on both our management responsibilities and our delivery responsibilities to clients. The competency with three votes was time management. Our facilitator argued that although the skill appeared to have been rated lower, the absence of any red dots meant that everybody considered the skill to be either essential or very important, plus it had been identified as a key challenge and was worthy of consideration.

It was interesting to me to observe how thrown everybody was by this shift. Our focus as a group was on identifying the competencies that had the most votes, but the purpose of the exercise was to identify the skills we felt were required to succeed in our positions. In executing the process we had lost our purpose.

In testing

When I test, I can find myself getting caught up in reporting things that I perceive as a problem, but the client sees as an enhancement. I do this because one outcome of my test process is logging defects and I want to record that these things were discussed. When I think about the wider purpose of my testing, I'm not sure whether this activity adds value. If the client accepts the behaviour, is this just noise?

When examining the outcomes of our test process, it's important that we remember to take a step back from what we have produced, or are expected to produce, and think about the purpose of what we're doing. What was the process put in place to achieve? Does the outcome meet the goal?

A Simple Service

Today I had the pleasure of speaking with Betty Zakheim for the first time, my mentor from Line at the Ladies Room. As we are both very busy women, working in entirely different time zones, today was the first opportunity we found to chat.

I wasn't sure what to expect from a mentor. I had a some things in mind that Betty might be able to help me with, but I didn't know how she wanted to approach our session. We started the call with introductions and Betty's friendly American accent calmed my nerves, then Betty asked what I'd like to talk about.

Service Offering

As I knew that Betty had a background in both Computer Science and Marketing, the first thing I wanted to ask her about was writing a service offering. I have recently been asked to lead an initiative in my organisation that means I have to define a new service and then write presentation material to share my vision with others. I've never had to do that before, so I asked Betty for some advice on how I could approach this. She gave me the following tips.

Elevator Pitch

Start by creating a crisp definition for the new service. What is it? How is it different to other testing? Why does it give a better outcome? Keep this definition short, quick and simple, so that a salesperson could hear it, remember it and repeat it without fault. This definition gets your foot in the door.

Speak Plainly

Write your pitch the same way you'd speak to somebody. Keep the language clear and avoid too much technical jargon. As you write, in cases where you can't think of an appropriate word, mark the point with ??? and move on. You can return to these points when the passage is complete, often the right word will appear given time.

Senior Management

When presenting to senior management, it's important to frame your argument in terms of cost and risk. This audience wants to know whether what you're offering is valuable to them. Often salespeople argue cost alone, but a proposition that reduces risk too is even more powerful.

Presentations

A good pitch caters to oral, visual and written learners, using PowerPoint slides and a strong script. As a rule of thumb, for a 1 hour meeting bring 30 minutes of prepared material that explain your services then be ready to answer questions for the remainder of the appointment. When you are selling professional services the product is you; be ready to prove your expertise.


Betty also gave me some great feedback on the content of my material, which was a real eye-opener for me. It made me realise that I need to differentiate between what I create to sell this service to testers and what I create to sell this service to managers. Betty really got my brain buzzing on how I can speak to the latter category successfully.

It was an incredibly helpful 45 minutes and I'm tremendously grateful to Betty for giving up her time on a Friday evening. I'm looking forward to our next session in a few weeks.

Community Question

Last night I slammed my finger in a door. It really hurt and it made me pretty grumpy. This happened as I was setting up a room for a WeTest Workshop that I was supposed to be facilitating. I decided to delegate facilitation to my friend Damian, who I was confident could do an excellent job of it. He did. 

A year ago, Damian had never participated in a LAWST style conference. Last night was the second time he facilitated one of our workshop events in Wellington. I believe this happened because he was asked; to attend, to present, to facilitate.

I am of the opinion that there are a number of people in software testing who are waiting for an opportunity. I hear people lamenting how difficult it is to create a community and this always makes me wonder, who have you asked?

When Aaron and I kicked off WeTest, we felt that there wasn't a strong community in Wellington. The opportunities available for testers didn't allow for passionate discussion on our industry; they were occasions that allowed presenters to escape unchallenged. We knew very few people to approach to change this situation. We started with the Wellington based folk who attended KWST2.

It took weeks for our first event to fill. We asked everyone we knew. Then we asked them to ask people. Then we asked on mailing lists and twitter. I remember how excited we were to finally find 20 people who wanted to talk about testing. 

After our first workshop the group started to grow. Those who attended recommended the event to others. A recommendation is not dissimilar to an invitation. In saying "you should come along to one of these" you're letting someone know that you believe they have the skills and potential to participate in, and enjoy, a challenging workshop environment. 

We now have over 100 members and our workshop events can fill within hours.

We're still asking, but now the questions have changed. Do you want to start a WeTest in Auckland? Do you want to lead a community of new testers? Do you want to console a community of battered warriors? Would you be willing to sponsor a new testing conference?

You can create a community. Ask someone to help you. Ask people to join you. Ask people to share their ideas.