Thursday, 14 April 2016

An idea that didn't work

I've had a few encounters recently that reminded me how much people like to learn from the failures of others. So I thought I'd take this opportunity to share an idea that I thought was brilliant, then tell you why it instead turned out to be a real dud.

There is an expectation in my organisation that every tester will be capable of contributing to our automated checks. All of our automation frameworks are coded by hand, there are no recording tools or user-friendly interfaces to disguise that code has to be written.

However, we hire people from a variety of backgrounds and experiences, which means that not everyone has the ability to write code. They are all willing to learn and generally enthusiastic about the prospect, but some of the testers don't have this particular skill right now.

Since I started with the organisation in a coaching role I've had one persistent request in our test team retrospectives. Whether I ask "What can we improve?" or "How can I help you in your role?" the answer is "I want to learn more about automation".

In December last year I decided to tackle this problem.

I sent out a recurring invite to an Automation Lab. For two hours each fortnight on a Friday afternoon all of the testers were invited to study together. The invitation read:

This session is optional for permanent staff to make effective use of your self-development time and have a forum to ask for help in reaching your automation-related goal. This is a fortnightly opportunity to bring your laptop into a quiet lab environment and work with support from your Testing Coach and peers. Whether you're learning Java, scripting groovy, mastering mobile, or tackling SoapUI, it doesn't matter. You could use this lab to learn any language or tool that is relevant.

I ran the Automation Lab for five sessions, which spanned early January through to mid March. Despite there being 30 people in the test team, the largest Automation Lab was attended by just four. Though I was disappointed, I assumed that this low attendance was because people were learning via other means.

In late March we ran another test team retrospective activity. When I asked people what training they needed to do their roles, the overwhelming majority were still telling me "I want to learn more about automation".

As I read through the feedback I felt grumpy. I was providing this opportunity to study in the Automation Lab and it wasn't being used, yet people were still asking for me to help them learn! Then I started thinking about why this had happened.

I had assumed that the blockers to people in my team learning automation were time and support. The Automation Lab was a solution to these problems. I booked a dedicated piece of time and offered direct assistance to learn.

Unfortunately I had assumed incorrectly and solved the wrong thing.

As someone who learned to code at University, I haven't experienced online learning materials as a student. However the plethora of excellent resources that are available to people made me think that finding instruction wasn't a problem. Now I realise that without prior knowledge of coding the resources aren't just available, they're overwhelming. 

The real blocker to people in my team learning automation was direction. They didn't know where to begin, which resources were best for what they need to know, or the aspects of coding that they should focus on.

I had offered time and support without a clear direction. In fact, I had been intentionally broad in my invitation to accommodate the variety of interests in the team: "Whether you're learning Java, scripting groovy, mastering mobile, or tackling SoapUI, it doesn't matter."

I've changed tack.

We're about to embark on a ten week 'Java for Newbies' course. I've had ten testers register as students and another four volunteer as teaching assistants. I'm creating the course material a fortnight ahead of the participants consuming it by pulling theory, exercises and homework from the numerous providers of free online training.

I hope that this new approach to teaching will result in better attendance. I hope that the ten testers who have registered will be a lot more confident in Java at the end of ten weeks. I hope that giving a structured introduction to the basics will lay the foundation for future independent learning.

Most of all, I hope that I've learned from the idea that didn't work. 


  1. Thanks for sharing the experience Katrina. I am in a similar situation where I try to inspire people to learn but not getting many participants. I guess I need to change the strategy and be creative.

  2. This makes me think :) Which is a good thing, of course..

    Isn't this somehow related to the 'jack of all trades, master of none' problem? Without specific focus and direction, you won't reach your goals (in your case teaching your peers the basics of test automation). In my case it's something else, but the principle is the same.

    Thanks for sharing, Katrina.

  3. >> In late March we ran another test team retrospective activity. When I asked people what training they needed to do their roles, the overwhelming majority were still telling me "I want to learn more about automation"

    I've been going to tester meetups for more than 10 years and I've heard testers say "I want to learn more about automation" at every single meetup, even at meetups where there's been presentations on test automation.

    It inspired me to write two blog posts: one is about taking control of your own career (which very few testers actually do), another was about pedestals and how testers put test automation on a pedestal.

    So sorry to disappoint you in advance but I guarantee your testers will still be saying "I want to learn more about automation" even after your new and improved workshop.

    1. I don't necessarily want the feedback to go away, I just want what "more" means to change. This is why I said in the post "I hope that giving a structured introduction to the basics will lay the foundation for future independent learning." Fingers crossed.

  4. Great that you've hopefully identified a/the problem and not just given up :)

  5. Thanks SO much for sharing this.
    Totally hits a note for me... kind of freakishly serendipitously, as I was thinking of doing the SAME thing to solve the SAME problem today!


  6. I hear the same thing all the time at meetups and from team members. For a long time I wondered why people don't just learn this stuff online in their own time.. about a month ago I came to the same conclusion as you, and now I'm working on specific training courses in specific tools and languages. I'm not surprised to see that Sean is doing this as well :-) All over New Zealand, testers will be learning more about automation in the coming months!

    Thanks for articulating this so well.

  7. Nice share!

    What I had experienced in the past when training a QA team in automation is that sometimes one or two individuals feel threatened if they are not quite keeping up with other students, especially when it's colleagues and then they quickly become disheartened.

    What did help me is by creating a framework that implemented a Page Object Model pattern so that I could simplify the tests and get them rolling. Once they were comfortable I would explain the framework and demonstrate how messy the tests would be had I not done that. Even if they didn't quite understand inheritance etc, they still felt they were able to achieve something.

    Hope my experiences can help you with your upcoming course.

  8. Nice article! Actually I was in that situation of somebody saying "I want to learn automation". It's there almost in every job post and it's supposed to be one of the top software testing trends for 2016 and beyond. I've started listening to Joe Colantonio automation podcast and soon realized that to learn automation you should be understand coding. So I'm learning programming (Java) along with HTML and CSS. It's a process and won't happen that fast but I think it's a good way to go. I wish you good luck with training and coaching your team!

  9. Nice article, I remember when I was asking that same question however it didnt matter how many times I said it when asked what I wanted to do, the fundamental fact was that I didnt want to know more automation I wanted more responsibility and a better job. It just so happens to be that I saw automation as a way of getting that. It became a case of I want to be an automation tester not I want to be a better tester and worth the money I want to be paid because I can bring automation skills inside my testing toolbox. Now I am in a position where I can (and want to) learn automation in my free time because it is a another tool that I can bring to the table. It is a solution to a problem that I am tackling rather than a here is this language, here is this framework, here is the syntax overload which seems to be a barrier in learning it. Its also one thing to learn it but pointless unless you can use it everyday otherwise you just forget it. Maybe you can start off the question why do you want to learn automation first and go from there :)

  10. Katrina,
    I have been 'that' person saying 'I want to learn automation. I think, looking back, and looking forward, my personal challenge has been 'What does it mean to learn automation'. Now that I finally am in a role where I am learning on the job, I am finding that it is about understanding how to stand up the infrastructure to test (understanding a framework) and once stood up and starting to write test cases, what are the approaches for writing clear tests, methods, and validations.

    I would be interested in your thoughts on this.

  11. Half the time after writing automated scripts or framework improvements, my feedback becomes "I wish I could forget everything about automation"...

    There is a difference between "Learning to Code" and "Learning to write Test Automation". In general, learning language semantics (especially the basics) is usually an order of magnitude easier than solving the problem and creating maintainable code. Is the barrier or the fear factor learning coding (or Java), or is it the environments you need to get to get things running for experimentation, understanding the design patterns or the code already written? There still seems a danger of solving the wrong problem.

    That said, I find something like Python is going to be a good bit more approachable than Java. I guess Java is probably your implementation language, but there are implementations of interpreted languages that run on the JVM and may be able to call into Java classes. If you want to make automation more approachable for those without CS backgrounds, is it worth considering if you have got the right tool for the job?