Friday, 20 December 2013

Presentation Purpose

As speakers, we get varying periods of time in which to make an impression. Over the past three months I have often been standing at the front of the room. When I speak I am usually talking about an approach to testing that most in the audience do not follow. I've been thinking about how to best present a new idea within a given time and what outcome I should try to achieve.

10 minutes - Doubt

This week I was invited to speak at a mini-conference that was run within a large organisation. Each speaker had 20 minutes; we were requested to present for 10 minutes then lead an open discussion on our chosen topic. I decided to frame my slides within the context of a problem that I was fairly confident at least some within the audience would be facing. My presentation included functional coverage models using mind maps, session based test management, automated checking, and how to combine the three effectively in a scrum development framework to pop the testing bubble.

On the day I was nervous. I wasn't sure that I had pitched the material at the right level to accommodate the varied audience. As I watched the first presenter I realised it wouldn't matter what I said. We didn't have enough time to explain a concept properly. My goal changed. It wouldn't be what I said, it would be how I said it.

People remember passion. It makes them wonder why they don't feel that way about their work. It makes them wonder what they're missing. This is one reason that trainers like James Bach and Brian Osman are often cited in the journey of software testers in New Zealand. A presenter who speaks with passion sows the seeds of doubt. As I stood up to present, I decided that was how I could win the room. Speak memorably and create uncertainty. It worked.

"Wow, I had no idea you were so passionate about testing".

"You gave me a lot to think about... and I'm glad my boss was here for that".

"We have that exact problem in our team right now. I'll be bringing this up in our next retrospective."

When you don't have long to make an impression, you want to give some key information and terminology that people can latch on to and research. But more importantly, be excited about what you are saying. Engage the audience and make them question what they are doing. If you only have 10 minutes, the best thing you can do is create doubt that leads people on their own journey of discovery.

60 minutes - Curiosity

During December, I have been teaching one hour testing sessions with a practical focus. These have included:

  • Hendrickson variables and combinatorial testing using Hexawise
  • Scenario touring and flow testing using state transition modelling
  • Using oracles to identify bugs

When you have an hour, you don't focus on what you'll present or even how you say it. When you have an hour, you get your audience to do. You want people to leave the room eager to repeat the activity in their own role. 'I want to do that again'.

I had an attendee from my first class show up in the second week with a Hexawise screenshot. His testing nightmare had been simplified from several thousand possible variable combinations to less than 50. "Everyone should use this tool!" he said. I was a little concerned that he'd latched on to the tool alone. However he had done the reading too and had a good understanding of pairwise testing and what benefits it could offer in order to speak about the practice to his boss.

In an hour, you can run a hands on activity. You can demonstrate a practice. You can make people think and discuss. But, most importantly, you can make people curious enough to repeat what they've been shown, to continue to discover and learn on their own.

A day - Comfort

A day is a gift (or a crushing responsibility). I try to use a day to take people on a journey. I still want to sow a seed of doubt and create curiosity, but I want to go further. I try to anticipate what people will want to know next so that I can lead the expedition of discovery. I answer a lot of questions, we work until the class feel comfortable with a new idea.

Yesterday morning I ran a session about test cases. We started with test case execution, 6 pairs of testers simultaneously executing an identical set of 8 test cases against an online auction site (Adam wrote about this). The test results were incredibly varied, coverage was intentionally patchy and a number of bugs were missed. We talked about inattentional blindness, losing sight of our mission and the limitations of test cases.

"But those test cases weren't detailed enough!" claimed one student. "If the test cases were better, then we would have been fine". Excellent. I was hoping you'd claim that.

The next exercise asked each student to write a test case to verify one particular function of our national weather forecasting website, which they could interact with as they wrote. Handwriting a single test case took a remarkably long time. We then rotated the test cases through the group. Each tester had 2 minutes to complete the test case they held and record whether it was passed, failed, or unable to be executed. As the test cases circled the room, the mood went from frenetic energy to barely contained boredom. The results were as baffling as in the first exercise. We talked about procedural memory and change blindness, but I could see that some were still not convinced.

I had one more trick up my sleeve. An alien meets you and asks how to brush its teeth. Hilarity ensued as people attempted to brush their teeth following the instructions of their peers. I wanted to re-iterate the message that better test cases would not solve the problems we were seeing. I talked about having an external locus of control and how we absolve ourselves of personal responsibility when we see incredibly detailed instructions.

When you have time, you can take your audience on a journey and solidify your point of view. You can present an idea from multiple angles and address concerns of those in the audience. You can give enough information to make people comfortable with a different opinion, creating a base from which they can action change.

Doubt, curiosity and comfort. Are your goals the same?

Friday, 13 December 2013

Challenges, Puzzles & Games

It seems to me that one of the most difficult things about preparing training material is finding applications that students can explore; a set of public applications that can help us to teach software testing. These may be applications based on traditional testing exercises, applications written specifically to challenge or puzzle testers, games that highlight a particular aspect of software testing or an application with unintended and interesting behaviours.

Here are some of the things I have found or been pointed towards:

A simple parking calculator hosted by Adam Goucher for testing based on the original from Gerald R Ford Airport. The challenge I was given was to find the maximum dollar figure I could be charged for parking.

The horse lunge game, which is a nice application for state transition modeling.

The Triangle Tester exercise from Elisabeth Hendrickson, with background information that can point you to specific bugs for investigation.

The Black Box Software Testing machines from James Lyndsay has a number of flash games to practice exploratory testing.

A simple Javascript implementation of Escapa, linked from a (now closed) testing challenge to find the shortest and longest game times.

James Bach recommends the White Box exercise and Lye Calculator.

I'm very interested to learn what else is out there, please leave a comment.

Monday, 9 December 2013

Call for Proposals

It seems to be conference proposal season. I decided to submit for three conferences last week; Agile 2014, CAST 2014 and Let's Test Oz. Each had a very different process for submission.

Agile 2014

Agile 2014 is a large conference and the call for proposals is designed to support a large response. There are clear guidelines as to what a submission should include, with an explanatory paragraph against each of:
  • Title
  • Introduction
  • Audience
  • Presentation format
  • History of the presentation
  • Experience of the speaker
  • Learning outcomes
In addition, there is a full review system in place. The earlier you submit, the more feedback cycles you hit to evolve your proposal into the best possible candidate for selection to the programme.

Surprisingly, access within the submissions system is open so that anyone who logs in can view all proposals. Thankfully I had submitted my attempt prior to reading a review comment that included these scathing words "The abstract, on the other hand, made my ears bleed. I strongly recommend rewriting it from scratch with clearer, simpler phrases. I felt trapped in a Dilbert cartoon as I read it."

Let's Test Oz

Let's Test Oz include a comparatively sparse level of detail. Proposals should have a:
  • Title
  • Summary of your presentation’s key points, including a brief explanation of the context
  • Key learning outcomes that you hope your audience will walk away with
The submission is sent via email, its receipt is not acknowledged and there is no review loop. You are notified when you are accepted or otherwise.

CAST 2014

CAST 2014 requests abstracts to align to a theme of "The Art and Science of Software Testing". There are no guidelines on what an abstract should contain; the submissions form requests the abstract as a file upload but doesn't specify what file format. Submissions are acknowledged via an automated mail receipt. There is no formal review process, though Paul Holland seems very friendly.


It seems to me that each call for proposals became less specific. I understand that some of these differences will be driven by the size of the audience, the size of the conference organising committee and cost, but I'm wondering why the context driven community can't steal a few leaves from the agile book.

Without direction there must be huge variation in the content of proposals received. Without review there is no opportunity to refine the quirky idea into a solid submission. Where rejection is uniform across all who are unsuccessful, where is the opportunity to learn and refine?

As someone with little experience in submitting to conferences it would be nice to feel that the system supported my venture in to the unknown. Twitter has been awash with pleas to submit, yet on reading the details for the proposal calls I had no idea where to begin. Without guidance on what a proposal should contain, or feedback on what to change, those attempting to enter the arena are blindfolded.

Though there is help around if you ask for it, perhaps we could improve the process too? What do you think?