Conference Budgets

There has been conversation on Twitter recently about conferences who do not offer speaker compensation. If you haven't been part of this discussion I would encourage you to read Why I Don't Pay to Speak by Cassandra Leung, which provides a detailed summary.

I take an interest in these conversations from two perspectives: I regularly speak at international conferences and I co-organise the annual WeTest conferences for the New Zealand testing community.

As an organiser, the events that I help to run cover all speaker travel and accommodation. We make a special effort to care for our conference speakers and have built a reputation in the international testing community as being an event that is worth presenting at.

WeTest is a not-for-profit company that is entirely driven by volunteers. How do we afford to pay all of our speakers?

Humble Beginnings

Our 2014 WeTest conference was a half-day event in a single city.

We had 80 participants who paid $20 per person. They received a conference t-shirt along with a catered dinner of pizza and drinks.

All of our speakers were local to the city, so there were no travel or accommodation expenses. Our budget was balanced by the support of our primary sponsor, Assurity.

Our total budget for this event was approximately $3,000 where our income and expenses were:

WeTest Budget 2014

Stepping Up

By 2016 we felt that we had built an audience for a more ambitious event. We embarked on a full-day conference tour with the same program running in two cities within New Zealand.

We had 150 participants in each city who paid $150 per person. This was a significant jump in scale from our previous events, so we had to establish a formal scaffold for our organisation. WeTest was registered as a company, we created a dedicated bank account, and launched our own website.

This was also the first year that we invited international speakers. 25% of our speaker line-up, or three out of twelve speakers, traveled to New Zealand from overseas. Covering their travel and accommodation costs significantly altered the dynamics of our budget. Running the conference in two different cities meant that there were travel and accommodation costs for our New Zealand based speakers and organisers too.

Our total budget for this event was approximately $50,000 where our income and expenses were:

WeTest Budget 2016

The Big League

Our 2016 events sold out quickly and we had long waiting lists. To accommodate a larger audience, we grew again in 2017. This meant securing commercial venues, signing contracts, paying deposits, registering for a new level of tax liability and formalising our not-for-profit status.

In 2017 we had around 230 participants in each city. We introduced an earlybird ticket at $150 per person, so that our loyal supporters would not experience a price-hike and we could collect some early revenue to cover upfront costs. Our standard ticket was $250 per person.

40% of our speaker line-up, or four out of ten speakers, traveled to New Zealand from overseas. We incurred similar speaker travel and accommodation expenses to the previous year.

Our total budget for this event was approximately $100,000 where our income and expenses were:

WeTest Budget 2017

To re-iterate, WeTest is a not-for-profit organisation that is volunteer-led. The profit of our 2017 events will be reinvested into the testing community and help us to launch further events in the New Year.

In the discussion about speaker reimbursement we often discuss in the abstract. I hope that these examples provide specific evidence of how a conference might approach speaker reimbursement, whether they are a small community event or a larger endeavour.

At WeTest we have consistently balanced our budget without asking speakers to pay their own way. We are proud of the diverse speaker programs that have been supported by this approach. In 2018 we look forward to continuing to provide a free platform for our speakers to deliver great content.

Caring for conference speakers

I've been fortunate to have the opportunity to speak at a number of international conferences. I've traveled to the USA, Canada, India, Estonia, England, Australia, and Denmark, as well as speaking at many events around New Zealand.

My experiences have been generally good. Yet there are many things about speaking at conferences that I feel could be improved. As a co-organiser of the upcoming WeTest conferences, I've spent some time this year reflecting on where the opportunities are to do things better.

The most obvious is paying to speak. I've had to pay my own airfares and accommodation on a few occasions, particularly as a new speaker. Where reimbursement for expenses has been offered it is usually paid after the event, which means that I still need to be financially able to cover these expenses in the short term.

But there are a host of smaller parts that form the overall experience of speaking at an event.

I may not know whether I'm supposed to have my presentation material on my own laptop, on a USB drive, or submitted somewhere in advance. What is the type of connection to the projector? Will there be a microphone? A lecturn? A stage?

I may not know how big my audience is going to be: 10, 100, or more? What type of layout will they be in: tables of 10, rows of chairs, or a staggered amphitheater? What type of people will I be speaking to: testers, test managers, or others who work in software?

I may not know what sort of environment I will face. Is it a conference where presenters simply present, or will there be a Q&A or open season afterwards? Is there a culture of debate, argument or challenge? If so, will I be supported by a facilitator?

All of these unknowns about what I've signed up for can cause anxiety. They also make it difficult for me to picture the audience and tailor my material accordingly.

Then there are the series of small challenges that happen during the experience itself. Arriving from a long haul flight in an unfamiliar country and finding my accommodation. Locating the conference venue and the room in which I'll present. Determining whether I'll be introduced by someone or will introduce myself. Deciding how to manage time keeping. And so on.

So, what are we doing differently for WeTest?

One of the main priorities for our organising committee is to care for our speakers. As many of the WeTest organisers are also regular conference speakers, we've worked hard to remove the worries that may surround accepting a speaking engagement. We know our speakers are putting a lot of work into preparing their presentations. We think that this should be their only concern.

We've arranged and paid for our speaker flights and accommodation in advance. With one exception where a speaker had specific airline requirements, none of our speakers have been asked to foot any of these costs upfront.

We've communicated with our speakers regularly. Since their selection in June we've:

• agreed on benefits and expectations via a written speaker agreement,
• offered them the opportunity to check their session and biographical details on the event website prior to our go-live, 
• provided a mechanism for them to complete their complimentary registration, 
• shared details of the venue, audio visual setup and event timing, 
• prepared personal itineraries for travel, accommodation and any associated sponsorship commitments, and
• sent them a copy of our attendee communication.

Over the past four months I hope that this information has removed a lot of anxiety that can be associated with presenting at an event. As an organising team we've tried to space out these messages, to offer regular opportunities for our speakers to ask questions and eliminate any unknowns.

The speaker itineraries that we've prepared run from arrival in the conference city. We have arranged and paid transport to meet all of our speakers at the airport. For international guests this means they don't have to worry about how to find their hotel or immediately locate New Zealand currency when they land.

And on the conference day itself, we have a dedicated person assigned specifically to our speakers. One of our organising committee will be walking our speakers from their accommodation to the venue, leading the speaker briefing, and be available throughout the event to deal with any questions or problems that arise.

I'm confident that our efforts to look after our speakers will result in fantastic material this year and in years to come. I want to continue to create a safe space for new presenters to step forward from the New Zealand testing community. And I want our WeTest events to be a must for international presenters on the software testing conference circuit.

On a broader note, I hope that our efforts help to change the expectations of speakers for other events. If every organiser aimed to provide a similar level of care, or speakers came to expect this, the experience of speaking at a conference could be consistently better than it is today.

Lightning Talks for Knowledge Sharing

The end of March is the halfway point of the financial year in my organisation. It's also the time of mid-year reviews. I don't place much emphasis on the review process that is dictated, but I do see this milestone as a great opportunity to reflect on what I've learned in the past six months and reassess what I'd like to achieve in the remainder of the year.

I wanted to encourage the testers in my team to undertake this same reflection and assessment. I was also very curious about what they would identify as having learned themselves in the past six months. I wanted to see where people were focusing their self-development time and understand what they had achieved.

Then I thought that if I was curious about what everyone was doing, perhaps the testers would feel the same way about each other. So I started to think about how we could effectively share what had been learned by everyone across the team, without overloading people with information.

One of the main facets of my role as a Testing Coach is to facilitate knowledge sharing. I like to experiment with different ways of propagating information like our pairing experiment, coding dojos, and internal testing conference. None of these felt quite right for what I wanted to achieve this time around. I decided to try a testing team lightning talks session.

I was first exposed to the idea of lightning talks at Let's Test Oz. The organisers called for speakers who would stand up and talk for up to five minutes on a topic of their choice. A couple of my colleagues took up this challenge and I saw first-hand the satisfaction they had from doing so. I also observed that the lightning talk format created a one hour session that was diverse, dynamic and fun.

So in December last year I started to warn the testers that at the end of March they would be asked to deliver a five minute lightning talk on something that they had learned in the past six months. This felt like a good way to enforce some reflection and spread the results of this across the team.

I scheduled a half day in our calendars and booked a large meeting room. Three weeks out from the event I asked each tester to commit to a title for their talk along with a single sentence that described what they would speak about. I was really impressed by the diversity of topics that emerged, which reflected the diversity of activities in our testing roles.

One week ahead I asked those who wished to use PowerPoint slides to submit them so that I could create collated presentations. Only about half of the speakers chose to use slides, which I found a little surprising but this helped create some variety in presentation styles.

Then the day prior I finalised catering for afternoon tea and borrowed a set of 'traffic lights' from our internal ToastMasters club so that each presenter would know how long they had spoken for.

On the day itself, 27 testers delivered lightning talks. 

The programme was structured into three sessions, each with nine speakers, that were scheduled for one hour. This meant that there was approximately 50 minutes of talks, then a 10 minute break, repeated three times.

Having so many people present in such a short space of time meant that there was no time for boredom. I found the variety engaging and the succinct length kept me focused on each individual presentation. I also discovered a number of things that I am now curious to learn more about myself!

There were some very nervous presenters. To alleviate some of the stress, the audience was restricted to only the testing team and a handful of interested managers. I tried to keep the tone of the afternoon relaxed. I acted as MC and operated the lights to indicate how long people had been speaking for, keeping both tasks quite informal. 

There was also a good last minute decision to put an animated image of people applauding in the PowerPoint deck so that it would display between each speaker. This reminded people to recognise each presenter and got a few giggles from the audience.

After the talks finished, I asked the audience to vote on their favourite topic and favourite speaker of the day. I also asked for some input into our team plan for the next six months with particular focus on the topics that people were interested in learning more about. Though I could sense that people were tired, it felt like good timing to request this information and I had a lot of feedback that was relatively cohesive.

Since the session I've had a lot of positive comments from the testers who participated that it was a very interesting way to discover what their peers in other teams had been learning about. I was also pleased to hear from some of those who were most reluctant to participate that many of their fears were unfounded. 

From a coaching perspective, I was really proud to see how people responded to the challenge of reflecting on their own progress, identifying a piece of learning that they could articulate to others in a short amount of time, then standing up and delivering an effective presentation.

I'll definitely be using the lightning talks format for knowledge sharing again.

How to develop into a great speaker

Your first conference talk will give you exposure to writing an abstract, marketing your ideas, creating engaging slides, structuring a talk, speaking clearly, keeping to time, and so on. The more talks you do, the more experience you gain in those same activities.

But developing as a speaker is not just about opportunity for repetition. I see a change in speakers when they stop thinking so much about what they're going to say during their talk and start concentrating on how they're going to say it.

By that I mean, when a great speaker is on the stage their content is almost on autopilot. They're not worried about the points they need to cover on the next slide. Rather they're more aware of their delivery. They're operating at a meta-level.

So, what sort of things are these great speakers thinking about?

The audience

A new speaker is learning to feel comfortable making eye contact with the audience. A great speaker is learning to anticipate mood, read body language, understand the response of the audience, and adapt their presentation to react to that environment.

Look at the schedule for the event. How long ago did people have food? How many concurrent sessions have the participants sat through? Whereabouts in the day are you? Whereabouts in the conference? Think about these factors as you watch the audience arrive in the room. Is there energy or will you have to create it?

How do people sit down? Do you see many people with their arms folded or their legs crossed? How many people have an open posture and are tilted forwards in their seats? Are people receptive to your ideas or will you have to establish your credibility and use persuasive language?

When you change the slide in your presentation, how many eyes travel to the slide and then back to you as the speaker? Who isn't making eye contact with you at all, looking downwards, or out the window? Are people engaged or do you have to ask for their participation to draw them in?

Are there people who are still reading a slide when you switch to the next one? Are people zoning out between slide transitions? Look for signs of frustration, like people sighing or pulling out their phones. These may be indicators that you need to change your pace.

The audience won't change the key messages of a presentation, but a great presenter will allow them to massively influence its delivery.

The commentary

A new speaker will often include remarks in their own presentations that are about their delivery rather than about their content. It's analogous to playing the director's commentary across a movie, however in a conference presentation these remarks are often apologetic or self-deprecating.

Do you start your talk by undermining your own credibility or wondering aloud whether you're really an authority on your topic? Do you question the choices of the organisers who granted you a spot on the stage, or the choices of the audience for selecting your session to attend over other alternatives? Do you tell people how inexperienced you are at speaking or how nervous you feel about this presentation? Do you apologise for fumbling content or narrate your disappointment in failed technology?

I believe that eliminating this sort of commentary from your presentation creates a perception that you're a very confident speaker, regardless of how confident you actually feel.

I have a heuristic to mute this doubt track within my own presentations. When thinking about whether or not to vocalise something I like to first consider who I'm saying it for. Often the commentary is stuff I say to make myself more comfortable, or to settle myself in to the beginning of my presentation. It's not for the audience. And if it's not for the audience, I shouldn't say it.

The language

Related to audience, the same presentation may differ wildly in delivery through choice of language. Can you interact informally, use colloquialisms, and make jokes? Or should you take a more professional tone?

Choice of words is influenced by environment. Are you speaking at a MeetUp event or a formal conference? It's also influenced by culture. Your references and examples may change between a talk in New Zealand versus a talk in India.

You might also consider whether jargon appropriate for your audience. What terms will they be familiar with and which should you explain beyond an acronym?

Adjusting to these factors can make a big difference in how accessible and relatable your presentation is. A great speaker can deliver the same slides twice, in two different contexts, with what may feel like an entirely different speech. The key messages don't change, yet the words are altered for the greatest impact.


If you're feeling ready to tackle the next challenge in public speaking, start practicing these meta skills. Think about your audience, your commentary and your language in your next presentation.

What would you add?

Feedback for Conference Speakers

I spoke at a number of conferences over the past month or so. After each talk I received a variety of feedback, from a variety of channels. The genesis for this post is two pieces of feedback I received for the same talk at the Canterbury Software Summit.

From the conference survey responses:

"Good coverage of the topic; however: agile teams/tribes should be self-sustained. Katrina's presentation though was explaining management activities. What I missed was how the agile approach really works for BNZ, how they constantly improve, what issue and challenges they faced and face etc. Missing enthusiasm. Average slide quality."

From a direct message on Twitter:

"One of the guys at work I talked to today, appreciated your talk at Canterbury software summit. We're thinking of now trying some of the ideas you talked about. Katrina please keep using your gift of inspiring the testing community, as it makes our jobs more enjoyable and fruitful."

As you can see, one person was utterly underwhelmed while the other felt inspired and motivated to make changes in his organisation.

As a new speaker I had no frame of reference for feedback, or any notion of what to expect back from the audience when I delivered a talk. Had I received that first piece of feedback for my first presentation I would have been entirely disheartened.

Now that I've presented a few times, I'm starting to see patterns in when I receive feedback, what type of feedback it is, and how I can use it. To illustrate, here's the feedback I received after my 'Diversify' keynote talk at the recent WeTest Weekend Workshops.

Verbal

I find public speaking a taxing activity. At the end of the talk, my adrenaline is racing - I know it's all over and I am looking to get away from people for a few minutes to calm down. However, there is usually at least one person who comes to the front of the room to speak to me. 

I like that people do this. The things they wish to say are usually positive and it's good to get immediate validation that it all went okay. Unfortunately I usually don't remember the nice things that they've said, because my brain isn't working properly yet!

Occasionally I get immediate feedback of a different kind. At WeTest Weekend Workshops someone approached to suggest how I could improve my use of the handheld microphone. Strangely I always remember this sort of feedback, the things that aren't entirely positive, despite being in the same agitated mental state.

I consider the number of people who come to the front of the room after my talk to be a loose indicator of the emotional response of my audience. The more people, the more I feel like I spoke about something that really resonated for them.

Social Media

After my talk I like to find a quiet spot, take a few deep breaths, and then check the reaction from Twitter. I see three broad categories for the feedback that appear in my Twitter timeline.

Announcements

The first tweets are the people who simply say that they are attending my talk. Announcement tweets contain no judgement and no content. Often they contain a photo from near the start of the presentation.

As I've started to gain a wider following on Twitter, I think the number of people who announce that they're at my talk has increased. As a new speaker, very few people got excited about merely attending my sessions! I consider announcement tweets a loose indicator of my reputation in the community behind the conference.

Ideas

The next tweets will be the ideas from my talk. These might be pieces of content that resonated with people, summaries of my main points, or tweets that let people who are not in the audience know that they've been mentioned.

I consider idea tweets a loose indicator of how engaged people are in the content. In some respects I prefer that there are fewer of these type of tweets, as I believe that most people find it difficult to actively listen while also composing the perfect 140 characters on Twitter.

Reaction

Finally there are the tweets that come at the end of the talk. Reaction tweets are all about judgement, though on Twitter you're usually just going to get the happy vibes from people who loved it and felt inspired.

Reaction tweets are about the buzz. I consider these a replacement to coming to the front of the room after the presentation, and so treat them as the same loose indicator of the emotional response of my audience. The more reaction tweets I get, the better. Even if they're not all positive, at least I touched a nerve!

Event

If the event information has been published online, via Meet Up, Facebook, or some other alternative, there is usually an opportunity to post feedback.

I find that the feedback I receive via social media comes from people who feel a connection to me as an individual, or who are confident about expressing their opinions to a wide audience. By contrast the feedback I receive via the event page comes from people who I do not know well, those who need longer to process their reaction to the presentation, or those who are not on Twitter!

There is also a shift in language. People have had time to reflect, so their reaction is less emotive and more analytical. On Twitter people "love" the presentation while on Meet Up it's "great".

Providing feedback via Meet Up requires effort beyond the time frame of the event itself. I consider this feedback a loose indicator of how I've improved my standing in the community behind the conference.

Survey

Many conferences send out a post-event survey to all the attendees to help them improve their format, content and structure for the following year.

Survey feedback is anonymous and, of all the forms of feedback, gives the widest spectrum. It seems that once there is no association between your feedback and your name, people become remarkably honest.

Here's a selection of survey comments about the speakers at the WeTest Weekend Workshops event to illustrate this:

• Katrina's presentation was awesome. Very motivating 
• Keynote was useful and aligned with the theme. 
• Might have been even better if we had a more diverse speakers. 
• Would be lovely to see more "activity" type events over the vanilla "here's a talk" type events 
• The talks I attended were average from my perspective.
• Did not find it as useful as I thought it would be.

Suddenly there's a much richer picture that includes those who had a less enjoyable experience. At conferences without a survey form, the only negative feedback you receive may be the absence of positive feedback.

I consider survey feedback a loose indicator of what I can improve in my presentations. I don't listen to everything, and where there are clearly other factors at play I take the criticism with a grain of salt, but overall I find it a valuable source of information to help me refine my content and delivery.

Blogs

Finally, there are people who want to share the talk with others. I take blogs, and other post-event activities of this nature, as a form of feedback. I treat these as a loose indicator of lasting impact.

After WeTest, the following resources have appeared that referenced my keynote:

• Diversify - and so we all did at WeTest Weekend Workshops 2015 - Simone Wadham
• Diversify (YouTube) - Priyanka Ray

The Big Picture

The volume and type of feedback I get varies greatly between presentations. It's taken time to establish my own interpretations of an influx of information that might otherwise feel overwhelming. I use the types of feedback I've described to determine:

• The emotional response from my audience
• How engaged people are in my content
• My existing reputation in the community behind the conference
• Whether I've improved my reputation in the community behind the conference
• What I can improve in my presentations
• Whether I've had a lasting impact

Formality in open season at a peer conference

I attended the fifth annual Kiwi Workshop for Software Testing (KWST5) this week. Overall, I really enjoyed spending two days discussing the role of testing with a group of passionate people.

I took a lot from the content. But it's not what we discussed that I want to examine here, instead it's how we discussed it. As I was sharing the events of the final day with my husband he made a comment that troubled me. I took to Twitter to gauge how other people felt about his statement:

My husband thinks some exchanges in open season at #KWST5 sound unprofessional. In retrospect, maybe they were? I'm thinking about it.

— Katrina Clokie (@katrina_tester) August 4, 2015

Since this tweet created a fair amount of discussion, I thought I would take the time to gather my thoughts and those from others into a coherent narrative, and share some of the ways in which I would approach the same situation differently next time.

Who was there?

I found the dynamic at this year's conference different to previous years. It felt like I was going to spend two days with my friends. Among the attendees, there were only two people who I had never met before. Most of the people in the room were frequent attendees at KWST, or frequent attendees at WeTest, or people who help create or contribute to Testing Trapeze, or current colleagues, or former colleagues, or simply friends who I regularly chat to informally outside of work.

This meant that it was the first year that I wasn't nervous about the environment. It was also the first year that I didn't feel nervous about delivering a talk. Though I was a little anxious about the content of my experience report overall I would say that I felt relatively relaxed.

So, who exactly was in the room? James Bach, Oliver Erlewein, Richard Robinson, Aaron Hodder, Sarah Burgess, Andy Harwood, Adam Howard, Mark Boyt, Chris Priest, Mike Talks, Joshua Raine, Scott Griffiths, John Lockhart, Sean Cresswell, Rachel Carson, Till Neunast, James Hailstone, David Robinson and Katrina Clokie.

What happened?

I was the first speaker on the second day of the conference. My experience report was the first set in an agile context. The topic of the role of testing in agile had been touched on through the first day, but not explored.

I knew that there was a lot of enthusiasm for diving in to a real discussion, and was expecting a robust open season. In fact, the passion for the topic far exceeded my expectations. The particular exchanges that my husband questioned were in one particular period of the open season of my experience report.

Oliver proposed a model to represent roles in agile teams that kicked off a period of intense debate. During this time the only cards in use by participants were red, the colour that indicates the person has something urgent to say that cannot wait. I believe this spell of red cards exceeded 30 minutes, based on a comment from Mike who, when called as the subsequent yellow card, said "hooray, I've been waiting almost 40 minutes".

During this period of red cards, there were several occasions where multiple people who were waiting to speak were actively waving red cards. There were people interrupting one another. There were people speaking out of turn, without waiting to be called upon.

There were specific exchanges within this particular period that my husband questioned. I'm going to share four examples that relate specifically to my own behaviour.

The first happened relatively early in the red card period. Aaron made a statement that I started to respond to. When he attempted to interrupt my response, and he was not the first to interrupt me, I replied by raising my voice and telling him to shut up, so that I could finish what I was saying.

Perhaps halfway through the red card period, I had stopped responding to the people who were raising the red cards and the conversation was flowing among the participants themselves. Rich asked, in his role as facilitator, whether I agreed with what people were saying. I replied that no, I thought they were full of sh*t.

Near the end of the exchange I was asked whether I believed, on reflection, that I had behaved as a moron during the first experience I shared in my experience report. As a caveat my interpretation of this comment has been refuted in subsequent Twitter discussions.

Finally, there was a case where three people were speaking at once and none had used a card. I interjected with a comment that "we have cards for a reason" to shut down their conversation.

Was it a problem?

At the time, I didn't think there was a problem. I shared James' view that "it was an intense exchange done in a good and healthy spirit". I found that red card period of open season incredibly challenging, but I never felt unsafe.

On reflection though, I do think there was a problem.

Why?

My behaviour during open season contributed to an atmosphere where people were talking over one another and behaving informally. The lack of discipline in the heat of these exchanges meant that several people in the room withdrew from the discussion.

This goes directly against the spirit of a peer conference, which is designed for everyone to be included equally. I now feel that I was part of an exchange that excluded those who were unable or unwilling to voice an opinion in this atmosphere.

What would I do differently?

In future, I think that I need to remember to respect the formality of a peer conference. I felt that I was among friends and, because of this, I bought an informal attitude to my exchanges.

I believe this reflection is shared by some others who were present. On Twitter, Aaron said "I shouldn't interact with people I know during formal exchanges differently, and open season is a formal exchange". Sean said "Maybe we need to be more conscious of those relationship biases we bring to peer conferences? I'm guilty of it".

In future, if I felt overwhelmed by interruptions, I would stop and ask for support from the facilitator. On reflection, the very first time I felt compelled to raise my voice and start participating in the culture of talking across people would have been a good opportunity to pause and reset expectations for the discussion.

What do other people think?

@katrina_tester the talking over is the one thing I disliked of the last two days. I felt your frustration

— Sean (@seancresswell) August 4, 2015

@katrina_tester @HelenaJ_M @seancresswell The "shut up" I took as a comfortable exchange between two people who knew each other well

— Aaron Hodder (@AWGHodder) August 4, 2015

@katrina_tester 3/2 ;-) And I think you gave an exceptionally good ER. It did bring the discussion I was hoping for.

— Oliver Erlewein (@Oliver_NZ) August 4, 2015

@katrina_tester @awghodder @seancresswell What happened in that room was what has to happen to get deep into a subject.

— James Marcus Bach (@jamesmarcusbach) August 4, 2015

@katrina_tester @AWGHodder @seancresswell I guess we need a "whoa - ease up" clause. When people feel uncomfortable.

— Mike Talks (@TestSheepNZ) August 4, 2015

Maybe we should survey on whether ppl wanted tighter / looser moderation etc - some anonymous survey thing? #KWST5 https://t.co/4LeJxtKuxE

— John Lockhart (@johnlockhart) August 4, 2015

@katrina_tester @seancresswell Actually,perhaps CoC?I was surprised when Rich asked group not to swear at one point. 1/2

— Aaron Hodder (@AWGHodder) August 4, 2015

@TheTestDoctor @katrina_tester maybe there needs to be a "now you must listen to MEEEEE" stick, for the person doing the ER?

— Sean (@seancresswell) August 4, 2015

What do you think? How formal are your peer conferences? How formal should they be?

Women in Testing who Keynote

There has been a lot of discussion recently about women keynote speakers at testing conferences. If you haven't been following along, here are some of excellent and thought-provoking opinion pieces that have been published on this topic:

• Women and Conference Keynotes [Fiona Charles]
• Nice girls don't get the Keynote [Anne-Marie Charrett]
• Competent People and Conference Keynotes [James Bach]
• Becoming a Regular Keynote speaker [Paul Gerrard]
• Conferences and women of merit [Maaret Pyhäjärvi]

As a woman, I like to see women in keynote speaking roles. I feel they are more likely to speak about experiences that I can relate to. I like to hear from my role models in the community. I feel that having women keynote speakers sets the tone of the conference as clearly welcoming for women attendees, which improves my networking experiences.

As a conference organiser, I know it can be hard to find speakers outside of our own professional network. Because we tend to feel most comfortable with people like us, our professional networks tend to contain a high proportion of people like us. If we only utilise our networks to select keynote speakers we are likely to end up with a set of speakers that is not very diverse. And I'm not just speaking about gender diversity, or simply physical diversity, but also diversity of ideas.

That said, I feel some sympathy for organisers who feel they don't know who to ask. I wanted to do something practical to help. So I've compiled a list women who are experienced speakers, many who have delivered previous keynotes, to consider for keynote positions at your next testing conference.

I have tried to offer suggestions from around the world. If these women are unavailable, or you feel that they aren't suited to your conference, they are still likely to have a network from which they could suggest other awesome women in their region. Get in touch with them to expand your horizons and those of your conference attendees.

If you are not a conference organiser but you would like to hear from these women, you could suggest them as a speakers to your local conference convener or organising panel. Be proactive. Create the change that you want to see.

Don't tell me that there are no fantastic women in testing who can keynote at your event.

*****

Alex Schladebeck

Head of Test Consulting at BREDEX GmbH, Germany

LinkedIn
Twitter - @alex_schl
Presentations
Example Talk - EuroSTAR Conferences: What Agile Teams Can Learn From World of Warcraft

Amy Phillips

Head of Test at Songkick, United Kingdom
LinkedIn
Twitter - @itjustbroke
Speaking History
Example Talk - London Continuous Delivery MeetUp: Testing in a Continuous Delivery World

Anna Royzman

QA Manager at Liquidnet Holdings Inc., USA
LinkedIn
Twitter - @QA_nna
Example Talk - Software Test Professionals: Anna Royzman on Tester Awareness

Anne-Marie Charrett

Lead Test Engineer at Tyro Payments, Australia

LinkedIn

Twitter - @charrett

Example Talk - EuroSTAR Conferences: Coaching Software Testers

Christin Wiedemann

Regional Manager at PQA Testing, Canada

LinkedIn
Twitter - @c_wiedemann
Example Talk - A1QA Interview with Christin Wiedemann

Dawn Haynes

Principal Trainer & Consultant at PerfTestPlus, Inc., USA
LinkedIn
Twitter - @dawnmhaynes
Example Talk - CAST 2013 Keynote: Introspective Retrospectives: Lessons Learned and Re-Learned

Denali Lumma

Senior Manager of Engineering at Salesforce, USA
LinkedIn
Twitter - @denalilumma
Example Talk - San Francisco Selenium MeetUp: Keeping Selenium Tests 100% Blue

Dorothy Graham

Software testing consultant, speaker and author, United Kingdom
LinkedIn
Twitter - @dorothygraham
Presentations
Example Talk - STAREAST 2012 Keynote: What Managers Think They Know about Test Automation

Elisabeth Hendrickson

Engineering Director at Pivotal, USA
LinkedIn
Twitter - @testobsessed
Example Talk - AgileEE 2011 Keynote: Agile Testing, Uncertainty, Risk, and Why It All Works

Emily Bache

Software Developer, Consultant, Conference Speaker, Sweden

LinkedIn
Twitter - @emilybache
Example Talk - EuroPython 2014 Keynote: Will I still be able to get a job in 2024 if I don't do TDD?

Fiona Charles

Software test consultant, teacher, writer, speaker, Canada

LinkedIn
Twitter - @fionaccharles
Speaking History
Example Talk - EuroSTAR Conferences: Thinking Strategically About Testing

Gerie Owen

Business Solutions Analyst at Northeast Utilities, USA
LinkedIn
Twitter - @gerieowen
Example Talk - Belgium Testing Days: How Did I Miss That Bug? Overcoming Cognitive Bias In Testing

Goranka Bjedov

Capacity Engineer at Facebook, USA

LinkedIn

Example Talk - PNSQC 2011 Keynote: The Future of Quality

Isabel Evans

Independent Consultant, United Kingdom
LinkedIn
Example Talk - EuroSTAR Conferences: Working Ourselves Out of a Job A Passion for Improvement

Janet Gregory

Agile Coach and Process consultant at DragonFire Inc., Canada

LinkedIn

Twitter - @janetgregoryca
Speaking Engagements
Example Talk - AgileVancouver Keynote: I Don't Want to Talk about Bugs - Let's Change the Conversation

Johanna Rothman

Management Consulting, USA

LinkedIn

Twitter - @johannarothman

Speaking
Example Talk - StarWest 2012 Keynote: Becoming a Kick-Ass Test Manager

Karen N Johnson

Director of Mobile Quality at Orbitz Worldwide, USA
LinkedIn
Twitter - @karennjohnson
Presentations
Example Talk - StarWest 2012 Lightening Talk

Katrina Clokie

Automation Test Coach at Bank of New Zealand, New Zealand

Twitter - @katrina_tester

Speaking History

Example Talk - Interview on CASTLive 2014: Building a testing community

Lanette Creamer

QA Engineer at Sinclair Broadcast Group, USA

LinkedIn

Twitter - @lanettecream

Example Talk - Agile Vancouver: Small Agile Projects Delivering Quality in 3 weeks to 9 Months

Leah Stockley

Independent Context Driven Testing Consultant & Trainer, Singapore

LinkedIn

Twitter - @leahjstockley

Example Talk - Singapore Testing Forum May 2014: Software Engineering Culture

Lisa Crispin

Tester at Pivotal Labs, USA

LinkedIn

Twitter - @lisacrispin

Presentations
Example Talk - Agile Testing Days 2012: Debunking Agile Testing Myths

Liz Keogh

Lean / Agile Consultant, United Kingdom
Twitter - @lunivore
Example Talk - ACE! Conference Keynote: Superheated Test Tubes and Anti-Bumping Granules

Louise Perold

Test Lead at Rand Merchant Bank, South Africa

LinkedIn

Twitter - @lperold

Lynn McKee

Software Quality Professional, Canada
LinkedIn
Speaking History
Example Talk - EuroSTAR Cofnerences: Inspiring Passion in Test Teams

Maaret Pyhäjärvi 

Test Specialist at Granlund Oy, Finland

LinkedIn
Twitter
Speaking History
Example Talk - Scan Agile 2015 - Breaking Illusions: Testing is your most valuable asset!

Maria Kedemo

House of Test, Sweden

LinkedIn

Twitter - @mariakedemo

Speaking History

Example Talk - Erik Davis & Maria Kedemo explain their workshop for CAST2014

Nancy Kelln

Owner and Principal Consultant at Unimagined Testing, Canada

LinkedIn
Twitter - @nkelln
Speaking History
Example Talk - Interview on CASTLive 2012: Robots, Heuristics and Oracles

Parimala Hariprasad

Delivery Director at PASS Technologies, India

LinkedIn

Twitter - @curioustester

Example Talk - Agile Testing Alliance Bangalore 2014: Designing Agile Test Strategy for Mobile Apps

Selena Delesie

Management Consultant for Leadership & Agile Practices, Canada
LinkedIn
Twitter - @sdelesie
Example Talk - Square Pegs in Round Holes

Trish Khoo

Test Engineer at Google, USA
LinkedIn
Twitter - @hogfish
Example Talk - CAST 2014 Keynote: Scaling up with Embedded Testing

Ulrika Malmgren

Quality accelerator at Magine TV, Sweden
LinkedIn
Twitter - @Ulrikama
Example Talk - Web5Conference: Amplify your awesomeness with testing

NOTE: If you would like to be added to or removed from this list, or have any links or details updated, please let me know via email - katrinaclokie at gmail dot com - or leave a comment below

Answering the call for proposals

Lack of female speakers at technology conferences. A common topic of discussion, particularly among women who want to see more of their peers take the stage. I think that a first step to improving diversity would be to have more women responding to Call for Proposals (CFP) issued by conference organisers.

Writing a proposal can feel like a prohibitive hurdle to those who are new to speaking at conferences. A proposal does require some effort to compose, while offering no guarantee that the effort will be rewarded with a speaking engagement. Having written a few proposals, I’ve seen a common expectation in what they should contain, and come to realise that writing a proposal is not nearly as onerous as I originally imagined.

The purpose of a proposal is to pitch an idea to the organisers of the conference. You do not need to have an existing presentation prepared before proposing. Instead you imagine how you might communicate your experiences or knowledge to others, then describe this vision.

In my experience, a conference proposal usually has four key parts, which are:

• Format
• Title
• Abstract
• Learning Objectives 

How I tackle these when writing a proposal differs from how they are requested when submitting a proposal. When I write a proposal I begin with the abstract, which might also be referred to as the presentation description. 

A simple abstract has two paragraphs, where the first states some problem or opportunity then the second describes what the presenter will talk about. With this structure in mind, I roughly note down my ideas, often in bullet point format or sentence fragments dumped into a document.

From this skeleton, I work to create polished prose. An abstract is a marketing tool, so I aim to tell a compelling story that will make people want to attend my talk. I try to keep my language simple, easy to understand and persuasive.

Usually I only know the format that I want to adopt once I have completed the abstract. The available formats will differ between conferences, but may include a short or long presentation, half or full day tutorials, or hands on workshops. Consider how much you want to communicate to your audience, and which format the material is best suited to. Most new presenters will stick to a familiar method of delivery and choose to present from a set of slides.

Next I consider the learning objectives, which may also be described as the learning outcomes or key takeaways. This is generally the point where I determine whether a proposal has merit for submission. It's important to determine format before you tackle the learning objectives, as what you want attendees to take from the session will really depend on the time and resources available to you. I've written about Presentation Purpose previously.

When writing learning objectives, I like to aim for five succinct bullet points that explain what I feel people may learn from my session, where each begins with a different descriptive word. Bloom’s Taxonomy is a really helpful reference for giving me the correct language to express where I imagine people will find value.

Once my idea is defined with an abstract, format and learning objectives, then I attempt to label it with a title. This is my least favourite part about writing a proposal; I find it quite difficult to summarise my message into a catchy one-liner.

In my experience a one hour time boxed session is enough time to determine whether I have an idea with merit and draft a proposal that describes it to others. Remember that a minimum output is two paragraphs and five bullet points, which is not much at all!

Before submitting a proposal to organisers, I seek feedback from my peers. I am lucky to be part of a strong community of testers in Wellington who are happy to complete proposal reviews. If you can’t think of someone to ask to review your proposal, there are people with an interest in improving gender diversity in technology that will be willing to assist you:

• Speak Easy - a new initiative from Anne-Marie Charrett and Fiona Charles.
• A Line at the Ladies Room - a mentoring programme co-ordinated by Lorinda Brandon.

Proposal feedback will usually include phrasing, spelling and grammar; it’s amazing how many errors slip through the gap between what you meant and what you actually said. The reviewer should also highlight any areas of the proposal that are unclear. If your reviewer needs to ask a lot of questions to understand your proposal, then attempt to include your answers to their questions in the proposal itself. This will make it much clearer when you submit to the organisers.

Updating the proposal after feedback and submitting it can take as long as writing the proposal itself. Some conferences request a proposal via email while others enforce a standard submission format through an online form. Altogether, I try not to spend more than two hours pulling a proposal together.

You may be wondering, is this worth it if I don’t get accepted to a conference?

Even though my proposals are not always selected, I find the process of writing them to be valuable. A call for proposals is a worthy excuse to spend a relatively short amount of time reflecting on my work and considering which experiences and ideas I could share with others. Though I find it challenging to articulate what I want to present, writing a proposal prepares me for a number of other conversations. Having thought about how to frame my work to others, I can eloquently explain myself in meetings with senior stakeholders, client managers, and my own boss.

A Call for Proposals is an opportunity to have your voice heard by speaking at a conference. It is also a platform through which you can find your voice by practicing writing proposals. The review process may also help you create new connections with testers in the wider community, or strengthen relationships in your existing networks.

I believe the benefits of responding to a call for proposals far outweigh the investment. I hope that many of you will consider responding to the next call for proposals that interests you.

A version of this article was originally published in the January edition of Women Testers.

How to make a workshop hum

On Monday I co-facilitated a two hour workshop. It was good, but not great. I felt that the room was a bit flat, and wasn't entirely sure that my message had been conveyed effectively.

Today I had the opportunity to run the same content again in a different city. Thirty minutes before the first student arrived, I decided to tweak the material. I wanted this version of the workshop to hum.

And hum it did. Though I can't be sure whether it was the people in the room or the material, I thought I'd share the three things I changed. I certainly think they contributed to the shift in outcome.

Give and Take

The first change was near the beginning of the session. We had asked the participants to do a series of activities in fairly quick succession, which came about as a result of attempting to condense some old material into a smaller time frame. We had valued our interactive exercises above our static slides, which meant that there was no longer any significant presenter content between some of our activities.

On Monday, it felt like we were taking a lot from our students and giving very little in return. We wanted them to think about this, and then think about that, and then think about something else, with very little time in between for them to pause, digest and absorb. Though we ultimately wove all the pieces together, the balance felt wrong.

Today I re-instated one slide. Just one. Doing this created a few minutes of space between two activities that asked people to think pretty hard. By adding this piece of content, I gave something back as the presenter. I gave people enough time to feel that the output of the first activity was acknowledged, and gave their brains time to recover!

This change altered the attitude of participants. The second activity in this series was tackled with enthusiasm instead of reluctance.

Grow the Numbers

The second change was in the classroom dynamics for our exercises. In the Monday session we jumped between asking people to work as groups, as individuals, as groups, then as individuals again. Working individually makes people introspective, somber and comparatively withdrawn. Working in groups is collaborative, dynamic and engaging. By mixing the numbers in each of our activities, classroom participation was see-sawing.

Today I changed the activities specifically to create an increasing momentum through the module of material. I started with an informal group conversation, which worked as an icebreaker to have people comfortable with those around them. Then I ran an individual exercise, an exercise in pairs, then finally an exercise in larger groups.

Creating this progression significantly altered communication through the workshop. Student engagement evolved in a much more cohesive fashion; I had attention and participation to the very end.

Set up for Success

The third change was to our final exercise. It was migrated from another area of our training, but on Monday we found that it was a much harder problem in its new context than in its original one. In addition, we asked students to complete this last exercise alone. Having a final exercise that was both challenging and silent meant that the class finished on quite a flat, serious note.

Today I re-designed this exercise so that the students had a better chance of success. I left the answers from the previous problem on the whiteboard, as a prompt for their thinking. I provided an expanded mnemonic as a reference. I also switched to a group format so that they could use one another as resources, and actively encouraged them to collaborate.

These changes meant that the answers provided by each group at the end of the session reflected a real understanding of the concepts that I was trying to teach. Further, the students themselves recognised that they had grasped the material, and the room was buzzing with their shared success.


In the coming weeks I plan to revisit the rest of my training material and apply these same three principles across each module; give and take, grow the numbers, set up for success. Today makes me believe that this is how to make a workshop hum.

Presentation Purpose

As speakers, we get varying periods of time in which to make an impression. Over the past three months I have often been standing at the front of the room. When I speak I am usually talking about an approach to testing that most in the audience do not follow. I've been thinking about how to best present a new idea within a given time and what outcome I should try to achieve.

10 minutes - Doubt

This week I was invited to speak at a mini-conference that was run within a large organisation. Each speaker had 20 minutes; we were requested to present for 10 minutes then lead an open discussion on our chosen topic. I decided to frame my slides within the context of a problem that I was fairly confident at least some within the audience would be facing. My presentation included functional coverage models using mind maps, session based test management, automated checking, and how to combine the three effectively in a scrum development framework to pop the testing bubble.

On the day I was nervous. I wasn't sure that I had pitched the material at the right level to accommodate the varied audience. As I watched the first presenter I realised it wouldn't matter what I said. We didn't have enough time to explain a concept properly. My goal changed. It wouldn't be what I said, it would be how I said it.

People remember passion. It makes them wonder why they don't feel that way about their work. It makes them wonder what they're missing. This is one reason that trainers like James Bach and Brian Osman are often cited in the journey of software testers in New Zealand. A presenter who speaks with passion sows the seeds of doubt. As I stood up to present, I decided that was how I could win the room. Speak memorably and create uncertainty. It worked.

"Wow, I had no idea you were so passionate about testing".

"You gave me a lot to think about... and I'm glad my boss was here for that".

"We have that exact problem in our team right now. I'll be bringing this up in our next retrospective."

When you don't have long to make an impression, you want to give some key information and terminology that people can latch on to and research. But more importantly, be excited about what you are saying. Engage the audience and make them question what they are doing. If you only have 10 minutes, the best thing you can do is create doubt that leads people on their own journey of discovery.

60 minutes - Curiosity

During December, I have been teaching one hour testing sessions with a practical focus. These have included:

• Hendrickson variables and combinatorial testing using Hexawise
• Scenario touring and flow testing using state transition modelling
• Using oracles to identify bugs

When you have an hour, you don't focus on what you'll present or even how you say it. When you have an hour, you get your audience to do. You want people to leave the room eager to repeat the activity in their own role. 'I want to do that again'.

I had an attendee from my first class show up in the second week with a Hexawise screenshot. His testing nightmare had been simplified from several thousand possible variable combinations to less than 50. "Everyone should use this tool!" he said. I was a little concerned that he'd latched on to the tool alone. However he had done the reading too and had a good understanding of pairwise testing and what benefits it could offer in order to speak about the practice to his boss.

In an hour, you can run a hands on activity. You can demonstrate a practice. You can make people think and discuss. But, most importantly, you can make people curious enough to repeat what they've been shown, to continue to discover and learn on their own.

A day - Comfort

A day is a gift (or a crushing responsibility). I try to use a day to take people on a journey. I still want to sow a seed of doubt and create curiosity, but I want to go further. I try to anticipate what people will want to know next so that I can lead the expedition of discovery. I answer a lot of questions, we work until the class feel comfortable with a new idea.

Yesterday morning I ran a session about test cases. We started with test case execution, 6 pairs of testers simultaneously executing an identical set of 8 test cases against an online auction site (Adam wrote about this). The test results were incredibly varied, coverage was intentionally patchy and a number of bugs were missed. We talked about inattentional blindness, losing sight of our mission and the limitations of test cases.

"But those test cases weren't detailed enough!" claimed one student. "If the test cases were better, then we would have been fine". Excellent. I was hoping you'd claim that.

The next exercise asked each student to write a test case to verify one particular function of our national weather forecasting website, which they could interact with as they wrote. Handwriting a single test case took a remarkably long time. We then rotated the test cases through the group. Each tester had 2 minutes to complete the test case they held and record whether it was passed, failed, or unable to be executed. As the test cases circled the room, the mood went from frenetic energy to barely contained boredom. The results were as baffling as in the first exercise. We talked about procedural memory and change blindness, but I could see that some were still not convinced.

I had one more trick up my sleeve. An alien meets you and asks how to brush its teeth. Hilarity ensued as people attempted to brush their teeth following the instructions of their peers. I wanted to re-iterate the message that better test cases would not solve the problems we were seeing. I talked about having an external locus of control and how we absolve ourselves of personal responsibility when we see incredibly detailed instructions.

When you have time, you can take your audience on a journey and solidify your point of view. You can present an idea from multiple angles and address concerns of those in the audience. You can give enough information to make people comfortable with a different opinion, creating a base from which they can action change.

Doubt, curiosity and comfort. Are your goals the same?

Call for Proposals

It seems to be conference proposal season. I decided to submit for three conferences last week; Agile 2014, CAST 2014 and Let's Test Oz. Each had a very different process for submission.

Agile 2014

Agile 2014 is a large conference and the call for proposals is designed to support a large response. There are clear guidelines as to what a submission should include, with an explanatory paragraph against each of:

• Title
• Introduction
• Audience
• Presentation format
• History of the presentation
• Experience of the speaker
• Learning outcomes

In addition, there is a full review system in place. The earlier you submit, the more feedback cycles you hit to evolve your proposal into the best possible candidate for selection to the programme.

Surprisingly, access within the submissions system is open so that anyone who logs in can view all proposals. Thankfully I had submitted my attempt prior to reading a review comment that included these scathing words "The abstract, on the other hand, made my ears bleed. I strongly recommend rewriting it from scratch with clearer, simpler phrases. I felt trapped in a Dilbert cartoon as I read it."

Let's Test Oz

Let's Test Oz include a comparatively sparse level of detail. Proposals should have a:

• Title
• Summary of your presentation’s key points, including a brief explanation of the context
• Key learning outcomes that you hope your audience will walk away with

The submission is sent via email, its receipt is not acknowledged and there is no review loop. You are notified when you are accepted or otherwise.

CAST 2014

CAST 2014 requests abstracts to align to a theme of "The Art and Science of Software Testing". There are no guidelines on what an abstract should contain; the submissions form requests the abstract as a file upload but doesn't specify what file format. Submissions are acknowledged via an automated mail receipt. There is no formal review process, though Paul Holland seems very friendly.

@katrina_tester I can't promise anything but if you ask for feedback I can do my best to give some before the deadline.

— Paul Holland (@PaulHolland_TWN) December 4, 2013

Why?

It seems to me that each call for proposals became less specific. I understand that some of these differences will be driven by the size of the audience, the size of the conference organising committee and cost, but I'm wondering why the context driven community can't steal a few leaves from the agile book.

Without direction there must be huge variation in the content of proposals received. Without review there is no opportunity to refine the quirky idea into a solid submission. Where rejection is uniform across all who are unsuccessful, where is the opportunity to learn and refine?

As someone with little experience in submitting to conferences it would be nice to feel that the system supported my venture in to the unknown. Twitter has been awash with pleas to submit, yet on reading the details for the proposal calls I had no idea where to begin. Without guidance on what a proposal should contain, or feedback on what to change, those attempting to enter the arena are blindfolded.

Though there is help around if you ask for it, perhaps we could improve the process too? What do you think?