Showing posts with label Conferences. Show all posts
Showing posts with label Conferences. Show all posts

Wednesday, 6 December 2017

Conference Budgets

There has been conversation on Twitter recently about conferences who do not offer speaker compensation. If you haven't been part of this discussion I would encourage you to read Why I Don't Pay to Speak by Cassandra Leung, which provides a detailed summary.

I take an interest in these conversations from two perspectives: I regularly speak at international conferences and I co-organise the annual WeTest conferences for the New Zealand testing community.

As an organiser, the events that I help to run cover all speaker travel and accommodation. We make a special effort to care for our conference speakers and have built a reputation in the international testing community as being an event that is worth presenting at.

WeTest is a not-for-profit company that is entirely driven by volunteers. How do we afford to pay all of our speakers?

Humble Beginnings

Our 2014 WeTest conference was a half-day event in a single city.

We had 80 participants who paid $20 per person. They received a conference t-shirt along with a catered dinner of pizza and drinks.

All of our speakers were local to the city, so there were no travel or accommodation expenses. Our budget was balanced by the support of our primary sponsor, Assurity.

Our total budget for this event was approximately $3,000 where our income and expenses were:

WeTest Budget 2014

Stepping Up

By 2016 we felt that we had built an audience for a more ambitious event. We embarked on a full-day conference tour with the same program running in two cities within New Zealand.

We had 150 participants in each city who paid $150 per person. This was a significant jump in scale from our previous events, so we had to establish a formal scaffold for our organisation. WeTest was registered as a company, we created a dedicated bank account, and launched our own website.

This was also the first year that we invited international speakers. 25% of our speaker line-up, or three out of twelve speakers, traveled to New Zealand from overseas. Covering their travel and accommodation costs significantly altered the dynamics of our budget. Running the conference in two different cities meant that there were travel and accommodation costs for our New Zealand based speakers and organisers too.

Our total budget for this event was approximately $50,000 where our income and expenses were:

WeTest Budget 2016

The Big League

Our 2016 events sold out quickly and we had long waiting lists. To accommodate a larger audience, we grew again in 2017. This meant securing commercial venues, signing contracts, paying deposits, registering for a new level of tax liability and formalising our not-for-profit status.

In 2017 we had around 230 participants in each city. We introduced an earlybird ticket at $150 per person, so that our loyal supporters would not experience a price-hike and we could collect some early revenue to cover upfront costs. Our standard ticket was $250 per person.

40% of our speaker line-up, or four out of ten speakers, traveled to New Zealand from overseas. We incurred similar speaker travel and accommodation expenses to the previous year.

Our total budget for this event was approximately $100,000 where our income and expenses were:

WeTest Budget 2017

To re-iterate, WeTest is a not-for-profit organisation that is volunteer-led. The profit of our 2017 events will be reinvested into the testing community and help us to launch further events in the New Year.

In the discussion about speaker reimbursement we often discuss in the abstract. I hope that these examples provide specific evidence of how a conference might approach speaker reimbursement, whether they are a small community event or a larger endeavour.

At WeTest we have consistently balanced our budget without asking speakers to pay their own way. We are proud of the diverse speaker programs that have been supported by this approach. In 2018 we look forward to continuing to provide a free platform for our speakers to deliver great content.

Sunday, 9 October 2016

Caring for conference speakers

I've been fortunate to have the opportunity to speak at a number of international conferences. I've traveled to the USA, Canada, India, Estonia, England, Australia, and Denmark, as well as speaking at many events around New Zealand.

My experiences have been generally good. Yet there are many things about speaking at conferences that I feel could be improved. As a co-organiser of the upcoming WeTest conferences, I've spent some time this year reflecting on where the opportunities are to do things better.

The most obvious is paying to speak. I've had to pay my own airfares and accommodation on a few occasions, particularly as a new speaker. Where reimbursement for expenses has been offered it is usually paid after the event, which means that I still need to be financially able to cover these expenses in the short term.

But there are a host of smaller parts that form the overall experience of speaking at an event.

I may not know whether I'm supposed to have my presentation material on my own laptop, on a USB drive, or submitted somewhere in advance. What is the type of connection to the projector? Will there be a microphone? A lecturn? A stage?

I may not know how big my audience is going to be: 10, 100, or more? What type of layout will they be in: tables of 10, rows of chairs, or a staggered amphitheater? What type of people will I be speaking to: testers, test managers, or others who work in software?

I may not know what sort of environment I will face. Is it a conference where presenters simply present, or will there be a Q&A or open season afterwards? Is there a culture of debate, argument or challenge? If so, will I be supported by a facilitator?

All of these unknowns about what I've signed up for can cause anxiety. They also make it difficult for me to picture the audience and tailor my material accordingly.

Then there are the series of small challenges that happen during the experience itself. Arriving from a long haul flight in an unfamiliar country and finding my accommodation. Locating the conference venue and the room in which I'll present. Determining whether I'll be introduced by someone or will introduce myself. Deciding how to manage time keeping. And so on.

So, what are we doing differently for WeTest?

One of the main priorities for our organising committee is to care for our speakers. As many of the WeTest organisers are also regular conference speakers, we've worked hard to remove the worries that may surround accepting a speaking engagement. We know our speakers are putting a lot of work into preparing their presentations. We think that this should be their only concern.

We've arranged and paid for our speaker flights and accommodation in advance. With one exception where a speaker had specific airline requirements, none of our speakers have been asked to foot any of these costs upfront.

We've communicated with our speakers regularly. Since their selection in June we've:
  • agreed on benefits and expectations via a written speaker agreement,
  • offered them the opportunity to check their session and biographical details on the event website prior to our go-live, 
  • provided a mechanism for them to complete their complimentary registration, 
  • shared details of the venue, audio visual setup and event timing, 
  • prepared personal itineraries for travel, accommodation and any associated sponsorship commitments, and
  • sent them a copy of our attendee communication.

Over the past four months I hope that this information has removed a lot of anxiety that can be associated with presenting at an event. As an organising team we've tried to space out these messages, to offer regular opportunities for our speakers to ask questions and eliminate any unknowns.

The speaker itineraries that we've prepared run from arrival in the conference city. We have arranged and paid transport to meet all of our speakers at the airport. For international guests this means they don't have to worry about how to find their hotel or immediately locate New Zealand currency when they land.

And on the conference day itself, we have a dedicated person assigned specifically to our speakers. One of our organising committee will be walking our speakers from their accommodation to the venue, leading the speaker briefing, and be available throughout the event to deal with any questions or problems that arise.

I'm confident that our efforts to look after our speakers will result in fantastic material this year and in years to come. I want to continue to create a safe space for new presenters to step forward from the New Zealand testing community. And I want our WeTest events to be a must for international presenters on the software testing conference circuit.

On a broader note, I hope that our efforts help to change the expectations of speakers for other events. If every organiser aimed to provide a similar level of care, or speakers came to expect this, the experience of speaking at a conference could be consistently better than it is today.

Thursday, 18 August 2016

Post-merge test automation failures

Recently we implemented selenium grid for one of our automated suites. I've written about our reasons for this change, but in short we wanted to improve the speed and stability of our automation. Happily we've seen both those benefits.

We've also seen a noticeable jump in the number of pull requests that are successfully merged back to our master branch each day. This gives some weight to the idea that our rate of application code change was previously impeded by our test infrastructure.

The increase in volume occasionally causes a problem when two feature branches are merged back to master in quick succession. Our tests fail on the second build of the master branch post-merge.

To illustrate, imagine that there are two open pull requests for two feature branches: orange and purple. We can trigger multiple pull request (PR) builds in parallel, so the two delivery teams who are behind these feature branches can receive feedback about their code simultaneously.

When a PR build passes successfully and the code has been through peer review, it can be merged back to the master branch. Each time the master branch changes it triggers the same test suite that executes for a pull request.

We do not trigger multiple builds against master in parallel. If two pull requests are merged in quick succession the first will build immediately and the second will trigger a build that waits for the first to complete before executing. Sometimes the second build will fail.

1. Failing tests after multiple PR merges to master

As the person who had driven sweeping test infrastructure changes, when this happened the first time I assumed that the test automation was somehow faulty. The real issue was that the code changes in orange and purple, while not in conflict with each other at a source code level, caused unexpected problems when put together. The failing tests reflected this.

We hadn't seen this problem previously because our pull requests were rarely merged in such quick succession. They were widely spaced, which meant that when the developer pulled from master to their branch at the beginning of the merge process these type of failures were discovered and resolved.

I raised this as a topic of conversation during Lean Coffee at CAST2016 to find out how other teams move quickly with continuous integration. Those present offered up some possible options to resolve the problem as I described it.

Trunk based development

Google and Facebook move a lot faster than my organisation. Someone suggested that I research these companies to learn about their branching and merging strategy.

I duly found Google's vs Facebook's Trunk Based Development by Paul Hammant and was slightly surprised to see a relevant visualisation at the very top of the article:


2. Google's vs Facebook's Trunk Based Development by Paul Hammant

It seems that, to move very quickly with a large number of people contributing to a code base, trunk-based development is preferred. As the previous diagram illustrates, we currently use a mainline approach with feature branches. This creates larger opportunities for conflicts due to merging.

I had assumed that all possible solutions to these tests failing on master would be a testing-focused. However, a switch to trunk-based development would be a significant change to our practices for every person writing code. I think this solution is too big for the problem.

Sequential build

Someone else suggested that perhaps we were just going faster than we should be. If we weren't running any build requests in parallel and instead triggered everything sequentially, would there still be a problem?

I don't think that switching to sequential builds would fix our issue as the step to trigger the merge is a manual one. A pull request might have successfully passed tests but be waiting on peer review from other developers. In the event that no changes are required by reviewers, the pull request could be merged to master at a time that still creates conflict:

3. Sequential PR build with rapid merge timing

The pull request build being sequential would slow our feedback loop to the delivery teams with no certain benefit.

Staged Build

Another suggestion was to look at introducing an interim step to our branching strategy. Instead of feature branches to master, we'd have a staging zone that might work something like this:

4. Introducing a staging area

The staging branch would use sequential builds. If a test passes there, then it can go to master. If a test fails there, then it doesn't go to master. The theory is that master is always passing.

Where this solution gets a little vague is how the staging branch might automatically rollback a merge. I'm not sure whether it's possible to automatically back changes off a branch based on a test result from continuous integration. If this were possible, why wouldn't we just do this with master instead of introducing an interim step?

I'm relatively sure that the person who suggested this hadn't seen such an approach work in practice.

Do Nothing

After querying the cost of the problem that we're experiencing, the last suggestion that I received was to do nothing. This is the easiest suggestion to implement but one that I find challenging. It feels like I'm leaving a problem unresolved.

However, I know that the build can't always pass successfully. Test automation that is meaningful should fail sometimes and provide information about potential problems in the software. I'm coming to terms with the idea that perhaps the failures we see post-merge are valuable, even though they have become more prevalent since we picked up our pace.

While frustrating, the failures are revealing dependencies between teams that might have been hidden. They also encourage collaboration as people from across the product work together on rapid solutions once the master branch is broken.

While I still feel like there must be a better way, for now it's likely that we will do nothing.



Other posts from CAST2016:

Friday, 12 August 2016

Human centered test automation

The opening keynote at CAST2016 was Nicholas Carr. Though his talk was live streamed, unfortunately a recording is not going to be published. If you missed it, much of the content is available in the material of a talk he delivered last June titled "Media takes command".

Nicholas spoke about typology of automation, the substitution myth, automation complacency, automation bias and the automation paradox. His material focused on the application of technology in traditionally non-technical industries e.g. farming, architecture, personal training.

As he spoke, I started to wonder about the use of automation within software development itself. Specifically, as a tester, I thought about the execution of test automation to determine whether a product is ready to release.

Automation providing answers

Nicholas shared an academic study of a group of young students who were learning about words that are opposite in meaning e.g. hot and cold. The group of students were divided in two. Half of the students received flashcards to study that stated a word with the first letter of it's opposite e.g. hot and c. The other half of the students received flashcards that stated both words in their entirety e.g. hot and cold.

The students who were in the first group performed better in their exam than those in the second group. Academics concluded that this was because when we need to generate an answer rather than simply study an answer, then we are more likely to learn it. This phenomenon is labelled the generation effect.

On the flip side, the degeneration effect is where the answers are simply provided, as in many automated solutions. Nicholas stated that this approach is "a great way to stop humans from developing rich talents".

It's interesting to consider which of these effects are most prevalent in processing the results provided by our continuous integration builds. I believe that the intent of build results is to provide an answer: the build will pass or fail. However, I think the reality of the result is that it can rarely be taken at face value.

I like to confirm that a successful build has truly succeeded by checking the execution time and number of tests that were run. When a build fails, there is a lot of investigative work to determine the real root cause. I dig through test results, log files and screenshots.

I have previously thought that this work was annoying, but in the context of the degeneration effect perhaps the need to manually determine an answer is how we continue to learn about our system. If continuous integration were truly hands-off, what would we lose?

Developing human expertise

Nicholas also introduced the idea of human centered automation. This is a set of five principles by which we see the benefits of automation but continue to develop rich human expertise.

  1. Automate after mastery
  2. Transfer control between computer and operator
  3. Allow a professional to assess the situation before providing algorithmic assistance
  4. Don't hide feedback
  5. Allow friction for learning

This list is interesting to consider in the context of test automation. The purpose of having test automation is to get fast feedback, so I think it meets the fourth point listed above. But against every other criteria, I start to question our approach.

When I think about the test automation for our products, there is one suite in particular that has been developed over the past five years. This suite has existed longer than most of our testers have been working within my organisation. There is logic coded into the suite for which we no longer have a depth of human expertise. So we do not automate after mastery.

The suite executes without human interaction, there is no transfer of control between computer and operator. Having no tester involvement is a goal for us. Part of providing rapid feedback is reliably provide results within a set period of time regardless of whether there is a tester available.

The suite provides a result. There is human work to assess the result, as I describe above, but the suite has already provided algorithmic assistance that will bias our investigation. It has decided whether the build has passed or a failed.

Finally, the suite is relatively reliable. When the tests usually pass, there is no friction for learning. When the tests are failing and flaky, that is when testers have the opportunity to deeply understand a particular area of the application and associated test code. This happens, but ideally not very much.

So, what are the long term impacts of test automation on our testing skills? Are we forfeiting opportunities to develop rich human expertise in testing by prioritising fast, reliable, automated test execution?

I plan to think more about this topic and perhaps experiment with ways to make our automation more human centered. I'd be curious to hear if other organisations are already exploring in this area.



Other posts from CAST2016:

Thursday, 11 August 2016

Fostering professional development

One of the sessions that I attended at CAST2016 was titled "How do I reach the congregation when I'm preaching to the choir?" presented by Rob Bowyer and Erik Davis. One of the main themes of discussion focused on whether people should "sell" professional development to their colleagues or team.

In the introduction to the session, Rob and Erik spoke a little bit about their own contexts. They shared some of the challenges that they've encountered in trying to foster a culture of professional development in both their organisations and their local testing community.

Two particular challenges stuck out for me and I noted them down. Firstly, that "most of the people didn't care" about professional development. Secondly, that "I've been struggling to get people to see the value" in professional development. It struck me that these two challenges in creating a culture of learning could be related.

Do I see value?

I have an ISTQB Foundation certificate. I did this early in my career because I believed that getting this qualification was necessary to find employment in the software testing field. I could see the certificate being mentioned in a lot of job advertisements for testers. 

I saw a clear benefit to me in downloading the syllabus, doing some independent study and taking an exam. This activity was going to open up opportunities in a field of work that I might otherwise be unable to enter. I wanted to be a tester, so I wanted to get the certification.

At that time, I saw the value in this professional development for my career.

On the other hand, I have never completed the BBST Foundation course. I have heard a lot about this qualification and investigated the material that is available online. I have advocated for people in my team to attend this course and published the business benefits I used to argue for this opportunity. But I have not completed the course personally.

I did not learn about BBST Foundation until I had reached a point where I had learned many, but not all, of the concepts in the course via other means. I had heard a lot about the time investment required to complete the course successfully. When given the opportunity to take the class, I decided not to.

At that time, I did not see the value in this professional development for my career.

Do I care?

In the case of ISTQB, a manager might have assumed by my actions that I cared about my professional development. In the case of BBST, a manager might have assumed by my actions that I did not care about my professional development. Both conclusions are reached by assumptions, which are present in any communication.

The Satir Interaction Model describes what happens inside of us as we communicate - the process we go through as we take in information, interpret it, and decide how to respond [Ref].

Ref: "I think we have an issue" -- Delivering unwelcome messages
Fiona Charles

The steps in the Satir Interaction Model between intake and response are hidden. This means that the end result of the process that assigns meaning and significance can be quite surprising to the recipient, which can be a catalyst for conflict.

For example, imagine that I give a manager an input of "I do not want to take the BBST Foundation course". I would be surprised by a response from that manager expressing disappointment that I don't care about my professional development.

We can also climb a Ladder of Inference in our interactions, which refers the idea that there's "a common mental pathway of increasing abstraction, often leading to misguided beliefs". In essence, this is about leaping to conclusions.

For example, imagine the same manager who received my negative response to the BBST Foundation course receiving a promotional email for the RST course. They might extrapolate from my previous negative response that I will not want to attend RST, that I don't want to take any training courses, and that it would be a waste of time to forward me an email that describes this opportunity. I haven't had any input into this flow of reasoning. The manager has independently climbed a ladder of inference.

I think we need to be aware of both of these communication models when assessing an apparent lack of interest in professional development - particularly when we're labeling what we see as "most of the people didn't care".

Empathy & Understanding

Let's return to the question of whether there is a need to sell professional development. I don't think so. However, I agree with an alternative phrasing suggested in the session: that we should foster professional development.

When I sell, I am trying to be persuasive and articulate the merits of an activity. My communication is broadcast oriented. I want to share my reasoning and rationale. I try to explain why people should participate. My intent is to advertise.

That "I've been struggling to get people to see the value" is a failure to sell.

When I foster, I am seeking to encourage the development of an activity through understanding the obstacles that prevent it from happening. I want to be mindful of the ladder of inference and the judgments that I am applying to the responses of my colleagues and team. I want to be aware of where I've assigned significance and meaning might have distorted the message that I have been given, particularly when people are saying "no" to an opportunity.

That "most of the people didn't care" is a failure to foster.

I believe that there are relatively few people who truly don't care about their professional development. If there are people around you who you would label in this manner, I'd challenge you to think about how you have communicated and the responses that you've received.

What have they actually said? What meaning have you prescribed? Have you really understood?

I believe that in reflection and inquiry there is opportunity to successfully foster professional development.



Other posts from CAST2016:

Wednesday, 23 March 2016

How do you create a friendly conference?

One of the things that really impressed me about TestBash a few weeks ago was the warm and friendly tone of the event. I haven't been alone in my remarks on the environment created by Rosie Sherry, Vernon Richards and others.

As a conference organiser, I've been thinking a lot about what made each attendee feel this way. What were the specific actions that made such a noticeable difference when compared to other events. Here are five things that I've identified, which I'm hoping to try at the next event that I run.

Pre-Event Emails

Rosie sent an email per day to TestBash attendees in the week leading up to the event. These included the schedules for the workshops and conference, details of associated MeetUp events, invites to a slack channel, social media hashtags, information for a book swap, and a set of behavioural requests.

These emails created a sense of hype and expectation. They got everyone on the same page about the logistics of the event. They allowed people to start interacting online prior to the event itself, if they wished to do so.

These emails also served as the foundation for the community focus of the conference itself. In particular, here are the six things that Rosie asked from attendees in one of her pre-event messages:
  • I ask the old timers to reach out to those that look lost.
  • I ask for everyone to be brave and speak to anyone who looks like they could do with some company.
  • I ask you all to be human, kind, and helpful.
  • I ask you all to focus on making friends and having a good time.
  • I ask you all to create some incredible memories to remember, for yourselves and everyone else who attends.
  • I ask you all to speak to speakers. I can assure you they want to speak to you too.

This clearly set expectations for behaviour prior to the event.

Personalised Name Tags

The first task after registration on conference day was to create your own name tag using very large Ministry of Testing speech bubbles and a permanent marker. The name tags were handwritten. You could choose how to present yourself to others at the conference. First name only, full name or nickname. With or without social media details.

The name tags were large and colourful, which made them easy to locate on a person and easy to read. Though they clearly incorporated the Ministry of Testing brand, they didn't feel corporate. The variety of handwriting on display made something that is normally staid into something that felt informal and fun.

The name tags put a little piece of each personality on display.



No Tester Stands Alone

As the host of the day, Vernon Richards did a great job at specifically reminding people to interact with those who they didn't know. He reiterated the TestBash ethos from the pre-event emails that "No Tester Stands Alone".

This expectation was set at the start in his opening remarks and we were prompted of it prior to each break. Many of the people that I met were attending as the only tester from their company, yet I rarely encountered anyone by themselves.

Vernon demonstrated that you don't have to be afraid to remind people to be friendly.

Single Track

I had never been to a single track conference before. I was really amazed by how different it felt to be part of a large group of people who were all experiencing the same set of speakers. Having over 200 people in one place, focusing on one thing, for an entire day, creates a vibe in itself.

I also found that it changed the type of conversations I had in the breaks. Often at conferences the exchanges over tea and coffee are about what each person listened to during the last session. You'll hear snippets and impressions without really understanding what the other presentation was about. At TestBash we had all heard the same topics, so we had deeper conversations about the ideas, what could work in our own organisations, the doubts that we had, etc.

A set of shared experiences can offer opportunity to explore further together.

Open Social Events

There was a Pre-TestBash evening social and a Post-TestBash evening social. There was a Pre-TestBash run and a Post-TestBash brunch. All of these events were publicly advertised in the Software Testing Club MeetUp group.

Though they were located in a pub, the evening invitations weren't focused on drinking. They instead put emphasis on connecting with other testers. The invitations were open, inclusive, and there was room for everyone, even if it was sometimes slightly crowded!

By keeping those who wanted to socialise in one place, no one felt any fear of missing out. People were really present at these events instead of occupying social media in search of what else was happening.

There was a huge amount of support for people to connect outside of the event itself.


I'm hoping that these five ideas will help bring a little bit of the TestBash magic to my next testing event, and perhaps yours too?

Thursday, 22 October 2015

Feedback for Conference Speakers

I spoke at a number of conferences over the past month or so. After each talk I received a variety of feedback, from a variety of channels. The genesis for this post is two pieces of feedback I received for the same talk at the Canterbury Software Summit.

From the conference survey responses:
"Good coverage of the topic; however: agile teams/tribes should be self-sustained. Katrina's presentation though was explaining management activities. What I missed was how the agile approach really works for BNZ, how they constantly improve, what issue and challenges they faced and face etc. Missing enthusiasm. Average slide quality."

From a direct message on Twitter:
"One of the guys at work I talked to today, appreciated your talk at Canterbury software summit. We're thinking of now trying some of the ideas you talked about. Katrina please keep using your gift of inspiring the testing community, as it makes our jobs more enjoyable and fruitful."

As you can see, one person was utterly underwhelmed while the other felt inspired and motivated to make changes in his organisation.

As a new speaker I had no frame of reference for feedback, or any notion of what to expect back from the audience when I delivered a talk. Had I received that first piece of feedback for my first presentation I would have been entirely disheartened.

Now that I've presented a few times, I'm starting to see patterns in when I receive feedback, what type of feedback it is, and how I can use it. To illustrate, here's the feedback I received after my 'Diversify' keynote talk at the recent WeTest Weekend Workshops.

Verbal

I find public speaking a taxing activity. At the end of the talk, my adrenaline is racing - I know it's all over and I am looking to get away from people for a few minutes to calm down. However, there is usually at least one person who comes to the front of the room to speak to me. 

I like that people do this. The things they wish to say are usually positive and it's good to get immediate validation that it all went okay. Unfortunately I usually don't remember the nice things that they've said, because my brain isn't working properly yet!

Occasionally I get immediate feedback of a different kind. At WeTest Weekend Workshops someone approached to suggest how I could improve my use of the handheld microphone. Strangely I always remember this sort of feedback, the things that aren't entirely positive, despite being in the same agitated mental state.

I consider the number of people who come to the front of the room after my talk to be a loose indicator of the emotional response of my audience. The more people, the more I feel like I spoke about something that really resonated for them.

Social Media

After my talk I like to find a quiet spot, take a few deep breaths, and then check the reaction from Twitter. I see three broad categories for the feedback that appear in my Twitter timeline.

Announcements

The first tweets are the people who simply say that they are attending my talk. Announcement tweets contain no judgement and no content. Often they contain a photo from near the start of the presentation.



As I've started to gain a wider following on Twitter, I think the number of people who announce that they're at my talk has increased. As a new speaker, very few people got excited about merely attending my sessions! I consider announcement tweets a loose indicator of my reputation in the community behind the conference.

Ideas

The next tweets will be the ideas from my talk. These might be pieces of content that resonated with people, summaries of my main points, or tweets that let people who are not in the audience know that they've been mentioned.



I consider idea tweets a loose indicator of how engaged people are in the content. In some respects I prefer that there are fewer of these type of tweets, as I believe that most people find it difficult to actively listen while also composing the perfect 140 characters on Twitter.

Reaction

Finally there are the tweets that come at the end of the talk. Reaction tweets are all about judgement, though on Twitter you're usually just going to get the happy vibes from people who loved it and felt inspired.


Reaction tweets are about the buzz. I consider these a replacement to coming to the front of the room after the presentation, and so treat them as the same loose indicator of the emotional response of my audience. The more reaction tweets I get, the better. Even if they're not all positive, at least I touched a nerve!

Event

If the event information has been published online, via Meet Up, Facebook, or some other alternative, there is usually an opportunity to post feedback.

I find that the feedback I receive via social media comes from people who feel a connection to me as an individual, or who are confident about expressing their opinions to a wide audience. By contrast the feedback I receive via the event page comes from people who I do not know well, those who need longer to process their reaction to the presentation, or those who are not on Twitter!

There is also a shift in language. People have had time to reflect, so their reaction is less emotive and more analytical. On Twitter people "love" the presentation while on Meet Up it's "great".


Providing feedback via Meet Up requires effort beyond the time frame of the event itself. I consider this feedback a loose indicator of how I've improved my standing in the community behind the conference.

Survey

Many conferences send out a post-event survey to all the attendees to help them improve their format, content and structure for the following year.

Survey feedback is anonymous and, of all the forms of feedback, gives the widest spectrum. It seems that once there is no association between your feedback and your name, people become remarkably honest.

Here's a selection of survey comments about the speakers at the WeTest Weekend Workshops event to illustrate this:
  • Katrina's presentation was awesome. Very motivating 
  • Keynote was useful and aligned with the theme. 
  • Might have been even better if we had a more diverse speakers. 
  • Would be lovely to see more "activity" type events over the vanilla "here's a talk" type events 
  • The talks I attended were average from my perspective.
  • Did not find it as useful as I thought it would be.

Suddenly there's a much richer picture that includes those who had a less enjoyable experience. At conferences without a survey form, the only negative feedback you receive may be the absence of positive feedback.

I consider survey feedback a loose indicator of what I can improve in my presentations. I don't listen to everything, and where there are clearly other factors at play I take the criticism with a grain of salt, but overall I find it a valuable source of information to help me refine my content and delivery.

Blogs

Finally, there are people who want to share the talk with others. I take blogs, and other post-event activities of this nature, as a form of feedback. I treat these as a loose indicator of lasting impact.

After WeTest, the following resources have appeared that referenced my keynote:

The Big Picture

The volume and type of feedback I get varies greatly between presentations. It's taken time to establish my own interpretations of an influx of information that might otherwise feel overwhelming. I use the types of feedback I've described to determine:
  • The emotional response from my audience
  • How engaged people are in my content
  • My existing reputation in the community behind the conference
  • Whether I've improved my reputation in the community behind the conference
  • What I can improve in my presentations
  • Whether I've had a lasting impact

Wednesday, 5 August 2015

Formality in open season at a peer conference

I attended the fifth annual Kiwi Workshop for Software Testing (KWST5) this week. Overall, I really enjoyed spending two days discussing the role of testing with a group of passionate people.

I took a lot from the content. But it's not what we discussed that I want to examine here, instead it's how we discussed it. As I was sharing the events of the final day with my husband he made a comment that troubled me. I took to Twitter to gauge how other people felt about his statement:


Since this tweet created a fair amount of discussion, I thought I would take the time to gather my thoughts and those from others into a coherent narrative, and share some of the ways in which I would approach the same situation differently next time.

Who was there?

I found the dynamic at this year's conference different to previous years. It felt like I was going to spend two days with my friends. Among the attendees, there were only two people who I had never met before. Most of the people in the room were frequent attendees at KWST, or frequent attendees at WeTest, or people who help create or contribute to Testing Trapeze, or current colleagues, or former colleagues, or simply friends who I regularly chat to informally outside of work.

This meant that it was the first year that I wasn't nervous about the environment. It was also the first year that I didn't feel nervous about delivering a talk. Though I was a little anxious about the content of my experience report overall I would say that I felt relatively relaxed.

So, who exactly was in the room? James Bach, Oliver Erlewein, Richard Robinson, Aaron Hodder, Sarah Burgess, Andy Harwood, Adam Howard, Mark Boyt, Chris Priest, Mike Talks, Joshua Raine, Scott Griffiths, John Lockhart, Sean Cresswell, Rachel Carson, Till Neunast, James Hailstone, David Robinson and Katrina Clokie.

What happened?

I was the first speaker on the second day of the conference. My experience report was the first set in an agile context. The topic of the role of testing in agile had been touched on through the first day, but not explored.

I knew that there was a lot of enthusiasm for diving in to a real discussion, and was expecting a robust open season. In fact, the passion for the topic far exceeded my expectations. The particular exchanges that my husband questioned were in one particular period of the open season of my experience report.

Oliver proposed a model to represent roles in agile teams that kicked off a period of intense debate. During this time the only cards in use by participants were red, the colour that indicates the person has something urgent to say that cannot wait. I believe this spell of red cards exceeded 30 minutes, based on a comment from Mike who, when called as the subsequent yellow card, said "hooray, I've been waiting almost 40 minutes".

During this period of red cards, there were several occasions where multiple people who were waiting to speak were actively waving red cards. There were people interrupting one another. There were people speaking out of turn, without waiting to be called upon.

There were specific exchanges within this particular period that my husband questioned. I'm going to share four examples that relate specifically to my own behaviour.

The first happened relatively early in the red card period. Aaron made a statement that I started to respond to. When he attempted to interrupt my response, and he was not the first to interrupt me, I replied by raising my voice and telling him to shut up, so that I could finish what I was saying.

Perhaps halfway through the red card period, I had stopped responding to the people who were raising the red cards and the conversation was flowing among the participants themselves. Rich asked, in his role as facilitator, whether I agreed with what people were saying. I replied that no, I thought they were full of sh*t.

Near the end of the exchange I was asked whether I believed, on reflection, that I had behaved as a moron during the first experience I shared in my experience report. As a caveat my interpretation of this comment has been refuted in subsequent Twitter discussions.

Finally, there was a case where three people were speaking at once and none had used a card. I interjected with a comment that "we have cards for a reason" to shut down their conversation.

Was it a problem?

At the time, I didn't think there was a problem. I shared James' view that "it was an intense exchange done in a good and healthy spirit". I found that red card period of open season incredibly challenging, but I never felt unsafe.

On reflection though, I do think there was a problem.

Why?

My behaviour during open season contributed to an atmosphere where people were talking over one another and behaving informally. The lack of discipline in the heat of these exchanges meant that several people in the room withdrew from the discussion.

This goes directly against the spirit of a peer conference, which is designed for everyone to be included equally. I now feel that I was part of an exchange that excluded those who were unable or unwilling to voice an opinion in this atmosphere.

What would I do differently?

In future, I think that I need to remember to respect the formality of a peer conference. I felt that I was among friends and, because of this, I bought an informal attitude to my exchanges.

I believe this reflection is shared by some others who were present. On Twitter, Aaron said "I shouldn't interact with people I know during formal exchanges differently, and open season is a formal exchange". Sean said "Maybe we need to be more conscious of those relationship biases we bring to peer conferences? I'm guilty of it".

In future, if I felt overwhelmed by interruptions, I would stop and ask for support from the facilitator. On reflection, the very first time I felt compelled to raise my voice and start participating in the culture of talking across people would have been a good opportunity to pause and reset expectations for the discussion.

What do other people think?









What do you think? How formal are your peer conferences? How formal should they be?

Wednesday, 17 June 2015

Notes from Nordic Testing Days 2015

Nordic Testing Days 2015 was my first European testing conference, both as an attendee and a speaker. I really enjoyed my time in Estonia. It was fantastic to listen to, and learn from, a set of speakers that I have never heard before. I also enjoyed meeting a number of people that I have previously only known through Twitter.

As a speaker, I presented a two-hour workshop on the first day of the conference titled "Become someone who makes things happen". I was nervous about presenting this to an international audience. I needed people to interact with one another for the workshop to be successful, so it was a relief to have a group of participants who were prepared to discuss, debate and role-play scenarios.

On the second day of the conference I was a last minute replacement for a speaker who was ill, delivering a presentation titled "Sharing testing with non-testers in agile teams". This was a repeat of a talk I gave several times last year.

I'm particularly grateful to those who have included one of my sessions in their highlights of the conference:



Of the sessions that I attended, here are my key takeaways.

Security Testing

I started the conference by attending a full day tutorial titled Exploring Web Application (In)Security. Bill Matthews and Dan Billing presented some fundamentals of security testing by providing an intentionally vulnerable application. Each participant installed the application on a local virtual machine, which meant we could all independently exploit it - a fantastic learning environment.

This session was the first time I learned about the STRIDE security testing mnemonic, which I captured in a mind map format:



Effective Facilitation

Neil Studd presented a session, which I only later realised was his first full-length conference talk, on Weekend Testing Europe: A Behind-the-scenes Guide to Facilitating Effective Learning.

As a co-founder and intermittent organiser of WeTest Workshops MeetUp in Wellington, it was great to listen to some real-life experiences of selecting and facilitating practical workshops. I particularly liked the reminders about the limits of the facilitation role including "each attendee has their own light bulb moment, don't try and manufacture one" and "attendees are there to learn from each other and not just the facilitator".

I took the opportunity to talk to Neil after his presentation, as I had some specific questions for him. As a result I'm definitely planning to use the Weekend Testing Europe archive as a resource in future, both for internal and external workshops.

Gamification

Gamification is something I'd heard of, but never dug in to. Kristoffer Nordström shared his experience of engaging end users using gamification: game techniques in non-game situations to motivate people and drive behaviours.

I found his experience report really interesting and would encourage you to look through his slides in the Nordic Testing Days Archive. Though I'm not sure whether gamification will work in my organisation, or where it might be applied, this talk certainly gave me a better understanding of how others use it.

Bad Work & Quitters

Rob Lambert delivered a keynote on Why remaining relevant is so important. The point that particularly resonated with me was perhaps tangential to the main topic, his view of "bad work".

Rob talked about how it seems that quitting has become trendy with many people voicing their opinions about leaving a place that does "bad work". He questioned the definition of "bad work" by challenging how much of the concept was based on perception.

Rob also said that "sometimes 'bad work' is where the real change happens". This made me reflect on the opportunities I've had to make change. Perhaps it is from the worst situations that we can make the biggest difference.

Reflection

Erik Brickarp delivered a really interesting experience report on Going Exploratory. As he spoke, Erik repeatedly reiterated that he learned from attempting to implement change through regular reflection. Only when he stopped and thought about how he was working did he have the opportunity to realise how he could have approached things differently. 

Erik said "whenever I feel like I don't have time to reflect, that's a strong indication that I should stop and reflect." This was a good reminder to me. When I'm busy at work, that's when I need to take the time to pause and assess.

*****

I really enjoyed my experience at Nordic Testing Days. Thank you to Helena Jeret-Mäe for selecting my workshop as Content Owner, Kadri-Annagret Petersen for being my track co-ordinator, and Grete Napits for running a fantastic conference. I hope to be back again in the future.

Saturday, 18 April 2015

Women in Testing who Keynote

There has been a lot of discussion recently about women keynote speakers at testing conferences. If you haven't been following along, here are some of excellent and thought-provoking opinion pieces that have been published on this topic:


As a woman, I like to see women in keynote speaking roles. I feel they are more likely to speak about experiences that I can relate to. I like to hear from my role models in the community. I feel that having women keynote speakers sets the tone of the conference as clearly welcoming for women attendees, which improves my networking experiences.

As a conference organiser, I know it can be hard to find speakers outside of our own professional network. Because we tend to feel most comfortable with people like us, our professional networks tend to contain a high proportion of people like us. If we only utilise our networks to select keynote speakers we are likely to end up with a set of speakers that is not very diverse. And I'm not just speaking about gender diversity, or simply physical diversity, but also diversity of ideas.

That said, I feel some sympathy for organisers who feel they don't know who to ask. I wanted to do something practical to help. So I've compiled a list women who are experienced speakers, many who have delivered previous keynotes, to consider for keynote positions at your next testing conference.

I have tried to offer suggestions from around the world. If these women are unavailable, or you feel that they aren't suited to your conference, they are still likely to have a network from which they could suggest other awesome women in their region. Get in touch with them to expand your horizons and those of your conference attendees.

If you are not a conference organiser but you would like to hear from these women, you could suggest them as a speakers to your local conference convener or organising panel. Be proactive. Create the change that you want to see.

Don't tell me that there are no fantastic women in testing who can keynote at your event.

*****

Alex Schladebeck

Head of Test Consulting at BREDEX GmbH, Germany
LinkedIn
Twitter - @alex_schl
Presentations
Example Talk - EuroSTAR Conferences: What Agile Teams Can Learn From World of Warcraft

Amy Phillips

Head of Test at Songkick, United Kingdom
LinkedIn
Twitter - @itjustbroke
Speaking History
Example Talk - London Continuous Delivery MeetUp: Testing in a Continuous Delivery World

Anna Royzman

QA Manager at Liquidnet Holdings Inc., USA
LinkedIn
Twitter - @QA_nna
Example Talk - Software Test Professionals: Anna Royzman on Tester Awareness

Anne-Marie Charrett

Lead Test Engineer at Tyro Payments, Australia
Example Talk - EuroSTAR Conferences: Coaching Software Testers

Christin Wiedemann

Regional Manager at PQA Testing, Canada
LinkedIn
Twitter - @c_wiedemann
Example Talk - A1QA Interview with Christin Wiedemann

Dawn Haynes

Principal Trainer & Consultant at PerfTestPlus, Inc., USA
LinkedIn
Twitter - @dawnmhaynes
Example Talk - CAST 2013 Keynote: Introspective Retrospectives: Lessons Learned and Re-Learned

Denali Lumma

Senior Manager of Engineering at Salesforce, USA
LinkedIn
Twitter - @denalilumma
Example Talk - San Francisco Selenium MeetUp: Keeping Selenium Tests 100% Blue

Dorothy Graham

Software testing consultant, speaker and author, United Kingdom
LinkedIn
Twitter - @dorothygraham
Presentations
Example Talk - STAREAST 2012 Keynote: What Managers Think They Know about Test Automation

Elisabeth Hendrickson

Engineering Director at Pivotal, USA
LinkedIn
Twitter - @testobsessed
Example Talk - AgileEE 2011 Keynote: Agile Testing, Uncertainty, Risk, and Why It All Works

Emily Bache

Software Developer, Consultant, Conference Speaker, Sweden
LinkedIn
Twitter - @emilybache
Example Talk - EuroPython 2014 Keynote: Will I still be able to get a job in 2024 if I don't do TDD?

Fiona Charles

Software test consultant, teacher, writer, speaker, Canada
LinkedIn
Twitter - @fionaccharles
Speaking History
Example Talk - EuroSTAR Conferences: Thinking Strategically About Testing

Gerie Owen

Business Solutions Analyst at Northeast Utilities, USA
LinkedIn
Twitter - @gerieowen
Example Talk - Belgium Testing Days: How Did I Miss That Bug? Overcoming Cognitive Bias In Testing

Goranka Bjedov

Capacity Engineer at Facebook, USA

Isabel Evans

Independent Consultant, United Kingdom
LinkedIn
Example Talk - EuroSTAR Conferences: Working Ourselves Out of a Job A Passion for Improvement

Janet Gregory

Agile Coach and Process consultant at DragonFire Inc., Canada
Twitter - @janetgregoryca
Speaking Engagements
Example Talk - AgileVancouver Keynote: I Don't Want to Talk about Bugs - Let's Change the Conversation

Johanna Rothman

Management Consulting, USA
Speaking
Example Talk - StarWest 2012 Keynote: Becoming a Kick-Ass Test Manager

Karen N Johnson

Director of Mobile Quality at Orbitz Worldwide, USA
LinkedIn
Twitter - @karennjohnson
Presentations
Example Talk - StarWest 2012 Lightening Talk

Katrina Clokie

Automation Test Coach at Bank of New Zealand, New Zealand

Lanette Creamer

QA Engineer at Sinclair Broadcast Group, USA

Leah Stockley

Independent Context Driven Testing Consultant & Trainer, Singapore

Lisa Crispin

Tester at Pivotal Labs, USA
Presentations
Example Talk - Agile Testing Days 2012: Debunking Agile Testing Myths

Liz Keogh

Lean / Agile Consultant, United Kingdom
Twitter - @lunivore
Example Talk - ACE! Conference Keynote: Superheated Test Tubes and Anti-Bumping Granules

Louise Perold

Test Lead at Rand Merchant Bank, South Africa

Lynn McKee

Software Quality Professional, Canada
LinkedIn
Speaking History
Example Talk - EuroSTAR Cofnerences: Inspiring Passion in Test Teams

Maaret Pyhäjärvi 

Test Specialist at Granlund Oy, Finland
LinkedIn
Twitter
Speaking History
Example Talk - Scan Agile 2015 - Breaking Illusions: Testing is your most valuable asset!

Maria Kedemo

House of Test, Sweden

Nancy Kelln

Owner and Principal Consultant at Unimagined Testing, Canada
LinkedIn
Twitter - @nkelln
Speaking History
Example Talk - Interview on CASTLive 2012: Robots, Heuristics and Oracles

Parimala Hariprasad

Delivery Director at PASS Technologies, India

Selena Delesie

Management Consultant for Leadership & Agile Practices, Canada
LinkedIn
Twitter - @sdelesie
Example Talk - Square Pegs in Round Holes

Trish Khoo

Test Engineer at Google, USA
LinkedIn
Twitter - @hogfish
Example Talk - CAST 2014 Keynote: Scaling up with Embedded Testing

Ulrika Malmgren

Quality accelerator at Magine TV, Sweden
LinkedIn
Twitter - @Ulrikama
Example Talk - Web5Conference: Amplify your awesomeness with testing

NOTE: If you would like to be added to or removed from this list, or have any links or details updated, please let me know via email - katrinaclokie at gmail dot com - or leave a comment below

Friday, 6 February 2015

Answering the call for proposals

Lack of female speakers at technology conferences. A common topic of discussion, particularly among women who want to see more of their peers take the stage. I think that a first step to improving diversity would be to have more women responding to Call for Proposals (CFP) issued by conference organisers.

Writing a proposal can feel like a prohibitive hurdle to those who are new to speaking at conferences. A proposal does require some effort to compose, while offering no guarantee that the effort will be rewarded with a speaking engagement. Having written a few proposals, I’ve seen a common expectation in what they should contain, and come to realise that writing a proposal is not nearly as onerous as I originally imagined.

The purpose of a proposal is to pitch an idea to the organisers of the conference. You do not need to have an existing presentation prepared before proposing. Instead you imagine how you might communicate your experiences or knowledge to others, then describe this vision.

In my experience, a conference proposal usually has four key parts, which are:
  • Format
  • Title
  • Abstract
  • Learning Objectives 

How I tackle these when writing a proposal differs from how they are requested when submitting a proposal. When I write a proposal I begin with the abstract, which might also be referred to as the presentation description. 

A simple abstract has two paragraphs, where the first states some problem or opportunity then the second describes what the presenter will talk about. With this structure in mind, I roughly note down my ideas, often in bullet point format or sentence fragments dumped into a document.

From this skeleton, I work to create polished prose. An abstract is a marketing tool, so I aim to tell a compelling story that will make people want to attend my talk. I try to keep my language simple, easy to understand and persuasive.

Usually I only know the format that I want to adopt once I have completed the abstract. The available formats will differ between conferences, but may include a short or long presentation, half or full day tutorials, or hands on workshops. Consider how much you want to communicate to your audience, and which format the material is best suited to. Most new presenters will stick to a familiar method of delivery and choose to present from a set of slides.

Next I consider the learning objectives, which may also be described as the learning outcomes or key takeaways. This is generally the point where I determine whether a proposal has merit for submission. It's important to determine format before you tackle the learning objectives, as what you want attendees to take from the session will really depend on the time and resources available to you. I've written about Presentation Purpose previously.

When writing learning objectives, I like to aim for five succinct bullet points that explain what I feel people may learn from my session, where each begins with a different descriptive word. Bloom’s Taxonomy is a really helpful reference for giving me the correct language to express where I imagine people will find value.

Once my idea is defined with an abstract, format and learning objectives, then I attempt to label it with a title. This is my least favourite part about writing a proposal; I find it quite difficult to summarise my message into a catchy one-liner.

In my experience a one hour time boxed session is enough time to determine whether I have an idea with merit and draft a proposal that describes it to others. Remember that a minimum output is two paragraphs and five bullet points, which is not much at all!

Before submitting a proposal to organisers, I seek feedback from my peers. I am lucky to be part of a strong community of testers in Wellington who are happy to complete proposal reviews. If you can’t think of someone to ask to review your proposal, there are people with an interest in improving gender diversity in technology that will be willing to assist you:

Proposal feedback will usually include phrasing, spelling and grammar; it’s amazing how many errors slip through the gap between what you meant and what you actually said. The reviewer should also highlight any areas of the proposal that are unclear. If your reviewer needs to ask a lot of questions to understand your proposal, then attempt to include your answers to their questions in the proposal itself. This will make it much clearer when you submit to the organisers.

Updating the proposal after feedback and submitting it can take as long as writing the proposal itself. Some conferences request a proposal via email while others enforce a standard submission format through an online form. Altogether, I try not to spend more than two hours pulling a proposal together.

You may be wondering, is this worth it if I don’t get accepted to a conference?

Even though my proposals are not always selected, I find the process of writing them to be valuable. A call for proposals is a worthy excuse to spend a relatively short amount of time reflecting on my work and considering which experiences and ideas I could share with others. Though I find it challenging to articulate what I want to present, writing a proposal prepares me for a number of other conversations. Having thought about how to frame my work to others, I can eloquently explain myself in meetings with senior stakeholders, client managers, and my own boss.

A Call for Proposals is an opportunity to have your voice heard by speaking at a conference. It is also a platform through which you can find your voice by practicing writing proposals. The review process may also help you create new connections with testers in the wider community, or strengthen relationships in your existing networks.

I believe the benefits of responding to a call for proposals far outweigh the investment. I hope that many of you will consider responding to the next call for proposals that interests you.

A version of this article was originally published in the January edition of Women Testers.

Tuesday, 2 December 2014

Conferences build community

Community is important to me. The primary reason that I volunteer my time to organise testing events is to offer people an opportunity to meet their peers and share ideas. It is the opportunity for interaction that I value, and I think that conferences are an essential part of this. Conferences build community.

A successful conference is not just about delivery of great content. It also provides a platform for every attendee to genuinely connect with another; an interaction that is the start of a supportive, inspiring or challenging professional relationship. When reflecting on a conference I find that the presentations may have faded from memory, but the conversations that spawned ongoing associations are much easier to recall.

As a conference organiser, the responsibility for setting the tone of the conference weighs heavier on me than selecting the ideas. It seems that achieving success in the community aspect of a conference is much more difficult than the content.

And everything begins with the speaker selection.

I get frustrated when I feel that a list of speakers isn't a true mirror of the community behind a conference, but instead a distorted reflection suited to a fairground Hall of Mirrors. As a conference organiser,  I am looking for strong presenters with innovative ideas who truly reflect the diversity of our profession. I am constantly conscious of creating a speaker line up that engages the brain and accurately shows who we are as a group.

This is a challenge and, when I consider diversity, I consider many attributes. As a woman in testing, I certainly think about the gender ratio in our speaker selection. But I also think about years of experience in testing, where people are employed, ethnicity, age and reputation. If I want the conference to offer something for everyone in the community, then I have to consider everyone in the community by focusing on what distinguishes us from each other.

I don't feel that I have ever had to select a speaker who didn't deserve their place. I simply consider diversity alongside the experiences and ideas that people bring. I think about the audience for a topic rather than the topic in isolation. There are instances when a proposal holds little appeal to me personally, but I feel it would be a great session for others within the community, both for its content and the opportunity to establish the presenter as an active voice.

Ideas are rarely innovative in every context. So considering ideas alone is an injustice to the community that the conference is for. I believe that every organiser should actively think about the people that they serve when selecting speakers.

When asked "What did you enjoy about the conference?", attendees at the recent WeTest Weekend Workshops referenced the topics, discussions, session and learning. I think we had fantastic content. However the strongest theme in responses to this question was the people. I believe this feedback reflects our effort as organisers to put the people of the community at the heart of our decisions on their behalf.



What did you enjoy about the conference?
WeTest Weekend Workshops 2014