tag:blogger.com,1999:blog-28445103440160168992024-03-18T19:38:02.718+13:00Katrina the TesterKatrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.comBlogger154125tag:blogger.com,1999:blog-2844510344016016899.post-35623203578093832312018-06-27T20:07:00.003+12:002018-06-27T20:07:47.724+12:003 ways to define your role without a RACI matrixI lead a team of Test Coaches, which is still a relatively unusual role in our industry. As a Test Coach I was part of a number of conversations about my role. As we scaled the team of coaches and I took a leadership position, these conversations increased in frequency and audience.<br />
<br />
Uncertainty about roles can happen when there are:<br />
<ul>
<li>new roles created, or </li>
<li>multiple people performing the same role in different areas of an organisation, or </li>
<li>a number of roles that interact closely with a shared purpose.</li>
</ul>
<br />
In all three situations there are alternatives to drawing up a traditional <a href="https://en.wikipedia.org/wiki/Responsibility_assignment_matrix">RACI matrix</a>, which can feel like marking territory when executed poorly. I prefer collaborative approaches to role definition. This article explains how to use an interactive activity with practical scenarios, a role expectation mapping canvas, and a rainbow roles overview.<br />
<br />
<h3>
Interactive activity with practical scenarios</h3>
When our test coaching function grew there were questions about how we would operate. Some people had preconceptions about our role being similar to test management. Others had concerns that we might overstep boundaries toward people leadership or setting technical direction.<br />
<br />
To kick-off our expanded services we scheduled a session for all the testers in the organisation, along with their line managers and delivery leadership, to explain test coaching. Attended by over 120 people, part of the agenda was an interactive activity to clarify and define our role.<br />
<br />
We asked the audience to pair up with someone that they wouldn't normally work with, preferably someone from another department. We gave each pair a questionnaire that listed 13 scenarios and asked them to consider whether or not they would involve a Test Coach in each situation.<br />
<br />
The scenarios in the activity were descriptive and practical. They were framed in the first person to encourage people to imagine themselves in each situation. Not just "bugs are being ignored", but "<b>my </b>bugs are being ignored".<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQZzCX8BZU_BGWTMfCVTCTe5mbZJ-NpuKqXyRxZlLFc-rCv5j1mFYQppuTNsKpTx26EynJRZ_4ciRd_QkmKqv-IWgpJZWVFnYsEHlzcBHUnBL5D0T09AYhwhaPvb53NLSQ46yRq5RA5ej4/s1600/TestCoachQuestionnaire.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="695" data-original-width="483" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQZzCX8BZU_BGWTMfCVTCTe5mbZJ-NpuKqXyRxZlLFc-rCv5j1mFYQppuTNsKpTx26EynJRZ_4ciRd_QkmKqv-IWgpJZWVFnYsEHlzcBHUnBL5D0T09AYhwhaPvb53NLSQ46yRq5RA5ej4/s640/TestCoachQuestionnaire.png" width="443" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Questionnaire for Test Coach role</td></tr>
</tbody></table>
<br />
Where a pair disagreed, they were allowed to record a two responses, however we encouraged discussion of differing opinions in the hope that a single answer could be found.<br />
<br />
Once this task was complete, we handed out sets of coloured paper squares to each pair. Each set included green, yellow, and pink sheets to represent yes, maybe, and no respectively.<br />
<br />
We used a set of slides to work through the questionnaire. Each question was displayed on a slide with a blue background, a neutral colour. I read out the question and each pair in the audience had to hold up a coloured card that corresponded to their answer. The card was green if they would engage a Test Coach, pink if they would not, and yellow if they were unsure.<br />
<br />
Asking each pair to vote kept the audience engaged, and helped us to understand where there was confusion about our role. Where the responses were mixed, or where they consistently differed to what we expected the response to be, we gained insight into situations where there might be overlap with other roles or where we might have to clarify the intent of our involvement.<br />
<div>
<br /></div>
After each vote, the next slide had the same question with a background colour that revealed the answer: green, yellow or pink. I provided a short explanation of our rationale for involvement in each situation and gave the audience the opportunity to ask any questions that they had.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtxYsxYb5Jt2dBYGjdy77FSgOCAYQdYoAEL4s-OGx9VZTH-Tk9G-K0_xAeXThIRKlwiO_FavVWmIDtOTU3iel7WZXpquB3CJwI9-VPkvgspOS1P6Ionrqlp465NwLXgoIUDYEC9rV925PJ/s1600/WhenToAskActivitySlides.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="600" data-original-width="1326" height="288" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtxYsxYb5Jt2dBYGjdy77FSgOCAYQdYoAEL4s-OGx9VZTH-Tk9G-K0_xAeXThIRKlwiO_FavVWmIDtOTU3iel7WZXpquB3CJwI9-VPkvgspOS1P6Ionrqlp465NwLXgoIUDYEC9rV925PJ/s640/WhenToAskActivitySlides.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Slides that explain when to ask a Test Coach</td></tr>
</tbody></table>
<br />
This approach to defining our role was well received and created a clear set of expectations about how to interact with the new Test Coaches.<br />
<br />
I have since applied the same type of exercise in a delivery tribe with five leadership roles. Both the team and the leaders were uncertain of who should be driving the outcome in some instances. I adapted the activity so that the columns of the questionnaire represented each role.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCopxeH_3BaTYIOEcC-tskP_RC0KpoNf2h9I4J5-5uoeJ3ekI5TS4zamxRlO1ftzpdk0ubuKs5WUlrWsSkZCuhvsOHlW5yamlCiSXtrqLQdG5FgSx8tX55_e1sYyVjlOBKcuh31l5CQRXZ/s1600/RACIAnotherTeam.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="562" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCopxeH_3BaTYIOEcC-tskP_RC0KpoNf2h9I4J5-5uoeJ3ekI5TS4zamxRlO1ftzpdk0ubuKs5WUlrWsSkZCuhvsOHlW5yamlCiSXtrqLQdG5FgSx8tX55_e1sYyVjlOBKcuh31l5CQRXZ/s640/RACIAnotherTeam.png" width="448" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Questionnaire for support roles in agile delivery</td></tr>
</tbody></table>
<br />
The audience split into pairs who discussed a set of scenarios, choosing which role they would expect an action from in each situation. We used five different coloured cards for the audience to share their thoughts, one colour per role. With more roles in scope and a larger audience, there was robust conversation about some of these scenarios and the activity was longer than when it was focused on a single role.<br />
<br />
<h3>
Role expectation mapping canvas</h3>
The test coaches in my organisation are part of a wider team that includes technical and practices coaching. As the team grew there was confusion, particularly from newer coaches, about our responsibilities. During a team offsite day, we split into our specialist coaching disciplines for an activity to discuss this.<br />
<br />
My manager facilitated the activity using the <a href="https://nomad8.com/role-expectation-mapping-canvas/">role expectation mapping canvas</a> developed by Tony O'Halloran based on ideas from Jimmy Janlen's description of <a href="http://blog.crisp.se/2014/03/11/jimmyjanlen/role-expectation-mapping">role expectation mapping</a>. Each subset of coaches in the team completed the activity together i.e. all of the Test Coaches worked on a single canvas.<br />
<br />
The canvas is separated into five parts:<br />
<ol>
<li>Roles that you work with</li>
<li>Externally visible signs of success</li>
<li>Things that you are responsible for</li>
<li>Expectations of the people that you interact with</li>
<li>Challenges that you face</li>
</ol>
<div>
<br /></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpqaGAYeJcJm0Sh5hfCOESB0qyPox6KQCA71FfSa312f5NoGgxy1FED8SQaiw7RNYZo5zJHXg_SnhDdqeN7K9RSaggxdkvksPI5NjJmzdrR7sI5tJCC-4eyrFp2mVy_eY1DgW4gBsYOI8L/s1600/Role-Expectation-Mapping-Canvas.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="424" data-original-width="600" height="451" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpqaGAYeJcJm0Sh5hfCOESB0qyPox6KQCA71FfSa312f5NoGgxy1FED8SQaiw7RNYZo5zJHXg_SnhDdqeN7K9RSaggxdkvksPI5NjJmzdrR7sI5tJCC-4eyrFp2mVy_eY1DgW4gBsYOI8L/s640/Role-Expectation-Mapping-Canvas.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://nomad8.com/role-expectation-mapping-canvas/">Role expectation mapping canvas</a></td></tr>
</tbody></table>
<br />
As the manager of the test coaching function and the longest serving test coach, I decided to act as the scribe for this activity. I refrained from contributing to the conversation and instead focused on recording it. This was difficult, but my silence created space for the other test coaches to express their ideas, opinions and doubts.<br />
<br />
As each test coach worked in a different neighbourhood the purpose of the activity was to understand what was common. Across each department in our organisation there are consistent stakeholder roles, indicators of success, responsibilities, expectations and challenges. The canvas was useful to focus conversation and capture these, to create clarity and alignment in our function.<br />
<br />
This canvas was designed to be applied in a delivery team to clarify relationships between different roles. Though I am yet to use it in that context, I can see how it might be successful.<br />
<br />
<h3>
Rainbow roles</h3>
A tribe that I interact with as a Test Coach has other test leadership roles at a tribe and team level, along with delivery managers who have an interest in quality. This environment had created confusion in the test chapter of the tribe and conflict in direction for testing.<br />
<br />
To resolve this, a meeting was scheduled for the people directly involved in test leadership. The facilitator drove the discussion towards a RACI matrix, which was distributed to the participants of the meeting after the session.<br />
<br />
In this context the RACI matrix was usefully divisive for the leadership audience. It clearly showed which role was responsible for which task. However I felt that we needed to present something else to the rest of the
tribe. Something that illustrated how we had agreed to work collaboratively towards a shared purpose, rather than defining our separate territories.<br />
<div class="MsoNormal">
<o:p></o:p></div>
<div class="MsoNormal">
<o:p><br /></o:p></div>
<div class="MsoNormal">
<o:p>I converted the information from the RACI matrix into a one-page overview with five streams that showed how the test leadership team would work together on common testing goals: strategy, standards, day-to-day support, recruitment, and personal development.</o:p></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1F3haka_gbzMtOBQDhydlW9rTPTtUor7Zd2gVuprSWfhgrEecwq3IiGj9YHCXdXIc7qW9vpBSJ-UEZl9I2qm4QTzUyHI6o-jDntyu9WaHx63YpXk-tFY0NnnC_mMGwGFgVhcnQiLDJsfN/s1600/RainbowRoles.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="516" data-original-width="831" height="395" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1F3haka_gbzMtOBQDhydlW9rTPTtUor7Zd2gVuprSWfhgrEecwq3IiGj9YHCXdXIc7qW9vpBSJ-UEZl9I2qm4QTzUyHI6o-jDntyu9WaHx63YpXk-tFY0NnnC_mMGwGFgVhcnQiLDJsfN/s640/RainbowRoles.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Rainbow Roles</td></tr>
</tbody></table>
<br />
It isn't miles away from the original, but the information is presented in simple language and intentionally illustrates our different contributions towards a single result. The rainbow colours started as a joke to further emphasize our aspirations of harmony, but became core to the identity of this diagram: rainbow roles.<br />
<br />
It is not unusual for roles to become murky as the organisation evolves. In each example we found clarity through collaboration using tools that emphasized how different people work together. I encourage you to treat confusion or conflict as an opportunity to positively engage with others about your role and theirs.<br />
<br />
For another take on this topic, you might also want to read <a href="https://www.riotgames.com/en/work-with-us/disciplines/dev-management/riots-agile-team-leadership-model-a-story-of-challenging-convention">Riot's agile team leadership model: A story of challenging convention</a> by Ahmed Sidky and Tricia Broderick.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com5tag:blogger.com,1999:blog-2844510344016016899.post-87075317783537033472018-06-07T16:31:00.002+12:002018-06-07T16:31:55.480+12:00The world of test automation capabilityImagine a traditional capability matrix. A spreadsheet where each column is skill, each row is the name of an individual, and then the map is populated by people who rate their competence. These spreadsheets are useful for seeing trends within a team, both strengths and opportunities for improvement, but there are limitations.<br />
<br />
A matrix captures state, but not journeys. It shows skills, but not always how the skills are applied. It lists what is important in a specific domain, but not how other people are using different skills to solve the same problem. The limitations of the matrix can stifle thinking about capability where evolution, application, and diversity of skill are important.<br />
<br />
We could try to capture this information in our spreadsheet. Telling these stories by colour-coding or cross-referencing multiple tabs. Or we could think about capability from another angle.<br />
<br />
The world of test automation capability is a model that illustrates the skills and experience of a test team using layers of the earth: core, mantle and crust. At the core are the testers with the least knowledge about coding and test automation frameworks. Paths to the surface travel through the mantle of programming languages towards the crust of applying coding skills.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieb5ekbD37TBWJLLm9_or4nuwzSVGmC7PcGIyeJtuYoz58wsmuFBhyNAfV1xs_D0v2BbOPetEV2hKVeo7gUMR7UzUF8kNdrX7zvNSVO2ARWv0NK26kE4ASZF6CL6KA5CI3SX2g5EWGHlaa/s1600/WorldOfTestAutomationCapability.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="561" data-original-width="614" height="365" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieb5ekbD37TBWJLLm9_or4nuwzSVGmC7PcGIyeJtuYoz58wsmuFBhyNAfV1xs_D0v2BbOPetEV2hKVeo7gUMR7UzUF8kNdrX7zvNSVO2ARWv0NK26kE4ASZF6CL6KA5CI3SX2g5EWGHlaa/s400/WorldOfTestAutomationCapability.png" width="400" /></a></div>
<br />
<br />
Divisions exist within the layers. The mantle of programming languages might include Java, JavaScript, Visual Basic, Ruby, and Swift. The crust of test automation frameworks might include suites for web-based user interfaces, iOS and Android mobile applications, desktop applications, and APIs. Splitting each layer of the model proportionally allows us to see relative strengths and weaknesses in capability and tooling.<br />
<br />
Divisions within the mantle and crust can be shared by different teams. In my organisation testers are split across four different departments. There is some overlap in the programming languages and automation frameworks in use across these different areas. In forming a strategy for training and framework evolution, we want to consider the distribution of testers within this model.<br />
<br />
Part of the test coaching function in my organisation is to formalise paths between layers of the world, to support testers to learn a programming language and apply this knowledge to implementation of a test automation framework. These paths are created through training courses, coding dojos and knowledge sharing opportunities. Informal paths will exist where testers are self-taught or prototyping their own frameworks.<br />
<br />
To illustrate, our current world of test automation capability can be visualised as:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4ofmyoGP2ju25W5gQE-yInwlDIxsGF1Yyu1DmC2rPiGbeihdMvd5aJVt3H37qfiaoBnkXbhlRa4awo9Effm9o5Bafi5NwPKuwn9BGpwnWxjk9OnkA6-uW8PNElosfThVo9B-uckjAkvX4/s1600/WorldOfTestAutomationCapability_Example.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="557" data-original-width="704" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4ofmyoGP2ju25W5gQE-yInwlDIxsGF1Yyu1DmC2rPiGbeihdMvd5aJVt3H37qfiaoBnkXbhlRa4awo9Effm9o5Bafi5NwPKuwn9BGpwnWxjk9OnkA6-uW8PNElosfThVo9B-uckjAkvX4/s400/WorldOfTestAutomationCapability_Example.png" width="400" /></a></div>
<br />
<br />
The model shows testers at the core who have no coding or framework experience. In the mantle, the strongest programming language is Java, though capability also exists in four other languages. Our division in programming language is usually by department, with different areas adopting different solutions. At the crust, the largest population are working on WebDriver Frameworks, which includes testers who work on web-based applications across all areas of the organisation. The crust also includes API frameworks as an emerging area, UFT as a legacy framework choice, and the mobile teams who have specialised tooling in xCode and Espresso.<br />
<br />
The world of test automation capability is a snapshot at a given point in time. Divisions within the model will shift as programming languages and automation frameworks evolve. People move through the model as their knowledge and experience develops. Paths will change as learning needs transform.<br />
<br />
It is quite a different snapshot to a matrix though. It is one that includes relationships between training and skill level, between skills and frameworks, and between people across different departments. The evolution, application, and diversity of skill is easier to see.<br />
<br />
Our Test Automation Strategy is formed within the context of this world, describing the forces that the Test Coaches will apply to the model to facilitate and direct movement. Our strategy differs by department, applying a different lens on the overarching visual.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhevhe-6Nsqc988NYwhMmzCrV5jEjCTcO6VTshE6R9ubej7Gmff-uzF_mJBo4CnceFII5fC0vEz6hftURDYBqsWVlSuTDYu18HDkvpHxSrMeeEyRsHQa4svJI3W25AhkA17k7iCNGBj95QG/s1600/WorldOfTestAutomationCapability_WithForces.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="200" data-original-width="769" height="164" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhevhe-6Nsqc988NYwhMmzCrV5jEjCTcO6VTshE6R9ubej7Gmff-uzF_mJBo4CnceFII5fC0vEz6hftURDYBqsWVlSuTDYu18HDkvpHxSrMeeEyRsHQa4svJI3W25AhkA17k7iCNGBj95QG/s640/WorldOfTestAutomationCapability_WithForces.png" width="640" /></a></div>
<br />
<br />
Though how we rate our skill relative to others is useful, there is rich narrative in mapping skills within an environment then talking about how that environment is changing. The world of test automation capability is a new perspective that creates an opportunity for a new story.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com5tag:blogger.com,1999:blog-2844510344016016899.post-66529687116082299822018-05-20T16:59:00.001+12:002018-05-20T17:02:37.618+12:009 quick ideas for flexible testingWhen you're a tester in an agile team, it can be easy to fall into a comfortable testing pattern. The transparency of daily stand-up and reflection of retrospectives can create an illusion of continuous improvement. These routines make us feel that we work in a flexible way but, if we dig a little deeper, we may not be as adaptable as we think.<br />
<br />
If you think back to the last time that you felt uncomfortable at work, there's a strong probability that this feeling was associated with a change that you were experiencing. A flexible approach means that you are willing to accept and adopt change regularly, which means that you routinely experience discomfort.<br />
<br />
When was the last time that you were surprised by the outcome of your retrospective or quizzed by a colleague in your stand-up? When was the last time that a stakeholder asked questions about your test artifacts? If you can't remember being challenged recently then you, and your team, might be stuck.<br />
<br />
Being flexible is not just about activities or outcomes. Imagine that you used to plan your testing in an Excel spreadsheet and now you capture test ideas in a mind map. Does this make you a flexible tester? Not necessarily.<br />
<br />
To be a versatile thinker you need to regularly inspect your own habits and create opportunities to collaborate with different people. If you cultivate flexibility as an attitude and make it part of the way that you work, you'll become more aware of how you think. You can change <i>what </i>you deliver, but a flexible tester will also challenge <i>how </i>they deliver.<br />
<div>
<br /></div>
<div>
How can you do that?</div>
<div>
<br /></div>
<div>
Here are nine quick, practical ideas that may help you develop your flexibility as a tester:</div>
<div>
<br />
<ol>
<li>Change the order of your test approach to break a routine.</li>
<li>Ask for advice from a non-tester on how to diagnose a bug.</li>
<li>Actively seek test ideas from non-testers outside your agile team e.g. UX, Ops.</li>
<li>Copy the format of a colleague's test report to see your own results in a new light.</li>
<li>Pair with a tester in another team to see a different test approach first-hand.</li>
<li>Invite someone else to test your product, then debrief about what they found.</li>
<li>Experiment with a tool that you haven't tried before.</li>
<li>Take a second look at something that you thought was a bad suggestion.</li>
<li>Ask for constructive feedback about your testing.</li>
</ol>
<br />
Being in an agile team does not guarantee that you are behaving in an agile way. Try to develop the habits that cultivate flexibility, so that you continue to learn and your testing continues to evolve.<br />
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com0tag:blogger.com,1999:blog-2844510344016016899.post-52953623257710798232018-05-12T16:41:00.000+12:002018-05-12T16:43:31.486+12:00No unit tests? No problem!A couple of weeks ago I created a Twitter poll about unit tests that asked:<br />
<br />
<div style="text-align: center;">
<i>"Is code without unit tests inherently bad code?" </i>
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
Is code without unit tests inherently bad code?</div>
— Katrina Clokie (@katrina_tester) <a href="https://twitter.com/katrina_tester/status/991240053249589248?ref_src=twsrc%5Etfw">May 1, 2018</a></blockquote>
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
</div>
The conversations that emerged covered a number of interesting points, which challenged some of my assumptions about unit tests and how we evaluate code.<br />
<br />
<h3>
What is bad code?</h3>
When I framed my original question, I deliberately chose the phrase "<i>inherently </i>bad code". I was trying to emphasize that the code would be objectively bad. That the absence of unit tests would be a definitive sign, one of a set of impartial measures for assessing code.<br />
<br />
In my organisation, most of our agile development teams include unit tests in their Definition of Done. Agile practitioners define a Definition of Done to understand what is required for a piece of work to be completed to an acceptable level of quality. In this context, the absence of unit tests is something that the agile development team have agreed would be bad.<br />
<br />
A Definition of Done may seem like an unbiased measure, but it is still a list that is collectively agreed by a team of people. The code that they create isn't good or bad in isolation. It is labeled as good or bad based on the criteria that this group have agreed will define good or bad for them. The bad code of one team may be the good code of another, where the Definition of Done criteria differs between each.<br />
<br />
Is the code inherently bad when it doesn't do what the end user wanted? Not necessarily. What if it the unexpected is still useful? There are a number of famous products that were originally intended for a completely different purpose e.g. bubble wrap was originally marketed as wallpaper [<a href="http://www.businessinsider.com/successful-products-that-were-originally-intended-for-a-completely-different-purpose-2016-3/?r=AU&IR=T">Ref</a>].<br />
<br />
I believe there is no such thing as <i>inherently </i>bad code. It is important to understand how the people who are interacting with your code will judge its value.<br />
<br />
<h3>
Why choose to unit test?</h3>
Many people include unit testing in their test strategy as a default, without thinking much about what type of information the tests provide, the practices used to create them, or risks that they mitigate.<br />
<br />
Unit tests are usually written by the same developer who is writing the code. They may be written prior to the code, in a test driven development approach, or after the code. Unit tests define how the developer expects the code to behave by coding the "known knowns" or "things we are aware of and understand" [<a href="https://dojo.ministryoftesting.com/dojo/lessons/not-sure-about-uncertainty">Ref</a>].<br />
<br />
By writing unit tests the developer has to think carefully about what their code should do. Unit tests catch obvious problems with an immediate feedback loop to the developer, by running the tests locally and through build pipelines. If the developer discovers issues and resolves them as the code is being created, this offers opportunities for other people to discover unexpected or interesting problems via other forms of testing.<br />
<br />
Where there are different types of automated testing, across integration points or through the user interface, unit tests offer an opportunity to exercise a piece of functionality at the source. This is especially useful when testing a function that behaves differently as data varies. Rather than running all of these variations through the larger tests, you may be able to implement these checks at a unit level.<br />
<br />
Unit tests require the developer to structure their code so that it is testable. These implementation patterns create code that is more robust and easier to maintain. Where a production problem requires refactoring of existing code, the presence of unit tests can make this a much quicker process by providing feedback that the code is still behaving as expected.<br />
<br />
The existence of unit tests does not guarantee these benefits. It is entirely possible to have a lot of unit tests that add little value. The developer may have misunderstood how to implement the tests, worked in isolation, or designed their test coverage poorly. The merit of unit tests is often dependent on team culture and other collaborative development practices.<br />
<div>
<br />
<h3>
No unit tests? No problem!</h3>
Though there are some solid arguments for writing unit tests, their absence isn't always a red flag. In some situations we can realise the benefits of unit testing through other tools.<br />
<br />
Clean implementation patterns that make code easier to maintain may be enforced by static analysis tools. These require code to follow a particular format and set of conventions, rejecting anything that deviates from the agreed norm before it is committed to the code base. These tools can even detect some of the same functional issues as unit tests.<br />
<br />
Rather than writing unit tests to capture known behaviour, you may choose to push this testing up into an integration layer. Where the data between dependent systems includes a lot of variation, shifting the tests can help to examine that the relationship is correct rather than focusing on the individual components. There is a trade-off in complexity and time to execution, but simple integrated tests can still provide fast feedback to the developers in a similar fashion to unit testing.<br />
<br />
When dealing with legacy code that doesn't include unit tests, trying to retrofit this type of testing may not be worth the effort. Similarly if the code is unlikely to change in the future, the effort to implement unit tests might not provide a return through easy maintainability, as maintenance will not be required.<br />
<br />
There may be a correlation between unit tests and code quality, but one doesn't cause the other. "Just because two trends seem to fluctuate in tandem ... that doesn’t prove that they are meaningfully related to one another" [<a href="https://www.fastcodesign.com/3030529/hilarious-graphs-prove-that-correlation-isnt-causation">Ref</a>].<br />
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com9tag:blogger.com,1999:blog-2844510344016016899.post-35601216999949245862018-04-25T18:56:00.000+12:002018-04-25T19:16:52.954+12:00How do you choose a test automation tool?When you’ve agreed <a href="http://katrinatester.blogspot.co.nz/2018/02/how-do-you-decide-what-to-automate.html">what you want to automate</a>, the next thing you’ll need to do is choose a tool. As a tester most of the conversations I observed from a distance, between managers and people selecting a tool, focused on only one aspect.<br />
<div>
<br />
Cost. <br />
<br />
Managers do care about money, don’t get me wrong. But choosing a tool based on cost alone is foolish, and I believe that most managers will recognise this. <br />
<br />
Instead of making cost the primary decision making attribute, where possible defer the question of cost until you’re asking for investment. Your role as the testing expert in a conversation about choosing a tool is to advocate for the other aspects of the tool, beyond cost, that are important to consider.<br />
<br />
<h3>
Support</h3>
You want to choose a tool that is well supported so that you know help is available if you encounter problems.<br />
<br />
If you’re purchasing a tool from a vendor, is it formally supported? Will your organisation have a contractual arrangement with the tool vendor in the event that something goes wrong? What type of response time can you expect when you encounter issues, or have questions? Is the support offered within your time zone? In New Zealand, there is rarely support offered within our working hours, which makes every issue an overnight feedback loop. This has a big impact in a fast-paced delivery environment. <br />
<br />
If it’s an open source tool, or a popular vendor tool, how is it supported by documentation and the user community? When you search for information about the tool online, do you see a lot of posts? Are they from people across the world? Are their posts in your language? Have questions posted in forums or discussion groups been responded to? Does the provided documentation look useful? Can you gauge whether there are people operating at an expert level within the toolset, or is everything you see online about people experimenting or encountering problems at an entry level? <br />
<br />
The other aspect of support is to consider how often you’ll need it. If the tool is well designed, hopefully it’s relatively intuitive to use. A lack of online community may mean that the tool itself is of a quality where people don’t need to seek assistance beyond it. They know what to do. They can reach an expert level using the resources provided with the tool. <br />
<br />
<h3>
Longevity</h3>
How long has the tool been around? There’s no right or wrong with longevity. If you’re extending a legacy application you may require a similarly aged test tool. If you’re working in a new JavaScript framework you may require a test tool that’s still in beta. But it’s important to go into either situation with eyes open about the history of the tool and it’s possible future. <br />
<br />
Longevity is not just about the first release, but how the tool has been maintained and updated. How often is the tool itself changing? When did the last update occur? Is it still under active development? As a general rule, you probably don’t want to pick a tool that isn’t evolving to test a technology that is. <br />
<br />
<h3>
Integration</h3>
Integration is a broad term and there are different things to consider here. <br />
<br />
How does the tool integrate with the other parts of your technology? Depending on what you’re going to test, this may or may not be important. In my organisation, our web services automation usually sits in a separate code base to our web applications, but our selenium-based browser tests sit in the same code base as the product. This means that it doesn’t really matter what technology our web services automation uses, but the implementation of our selenium-based suite must remain compatible with the decisions and direction of developers and architects.</div>
<div>
<br /></div>
<div>
What skills are required to use the tool and do they align with the skills in your organisation? If you have existing frameworks for the same type of testing that use a different tool, consider whether a divergent solution really makes sense. Pause to evaluate how the tool might integrate with the people in your organisation, including training and the impact of shifting expectations, before you look further afield.</div>
<div>
<br />
Integration is also about whether the test tool will integrate with the development environment or software development process that the team use. If you want the developers in your team to get involved in your test automation, then this is a really important factor to consider when choosing a tool. The smaller the context switch for them to contribute to test code, the better. If they can use the same code repository, the same IDE development environment on their machine, and their existing skills in a particular coding language, then you’re much more likely to succeed in getting them active in your solution. Similarly, if the tool can support multiple people working on tests at the same time, do clean merging from multiple authors, offer a mechanism for review or feedback, these are all useful points to consider related to integration of the tool to your team. <br />
<br />
<h3>
Analyse</h3>
In a conversation with management about choosing a tool, your success comes from your ability to articulate why a particular tool is better than another, not just in it’s technical solution. Demonstrate that you’ve thought broadly about the implications of your choice, and how it will impact on your organisation now and in the future. <br />
<br />
It’s worth spending time to prepare for a conversation about tools. Look at all the options available in the bucket of possible tools and evaluate them broadly. Make cost the lesser concern, by speaking passionately about the benefits of your chosen solution in terms of support, longevity and integration, along with any other aspects that may be a consideration in your environment. <br />
<br />
You may not always get the tool that you’d like, but a decision that has been made by a manager based on information you’ve shared, that has come from a well-thought through analysis of the options available, is a lot easier to accept than a decision made blindly or solely based on cost. </div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com1tag:blogger.com,1999:blog-2844510344016016899.post-87415259500407544202018-04-11T21:01:00.001+12:002018-04-11T21:01:43.376+12:00Setting strategy in a Test PracticePart of my role is figuring out where to lead testing within my organisation. When thinking strategically about testing I consider:<br />
<br />
<ul>
<li>how testing is influenced by other activities and strategies in the organisation,</li>
<li>where our competitors and the wider industry are heading, and</li>
<li>what the testers believe to be important.</li>
</ul>
<div>
<br /></div>
<div>
I prefer to seek answers to these questions collaboratively rather than independently and, having recently completed a round of consultation and reflection, I thought it was a good opportunity to share my approach.</div>
<div>
<br /></div>
<h3>
Consultation</h3>
<div>
In late March I facilitated a series of sessions for the testers in my organisation. These were opt-in, small group discussions, each with a collection of testers from different parts of the organisation.</div>
<div>
<br /></div>
<div>
The sessions were promoted in early March with a clear agenda. I wanted people to understand exactly what I would ask so that they could prepare their thoughts prior to the session if they preferred. While primarily an information gathering exercise, I also listed the outcomes for the testers in attending the sessions and explained how their feedback would be used.</div>
<div>
<br /></div>
<div>
***</div>
<div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<b><span style="font-size: 12pt;">AGENDA<u></u><u></u></span></b></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<b>Introductions<u></u><u></u></b></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
Round table introduction to share your name, your team, whereabouts in the organisation you work<u></u><u></u></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<br /></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<b>Transformation Talking Points<u></u><u></u></b></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
8 minute time-boxed discussion on each of the following questions:<u></u><u></u></div>
<ul style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px; margin-bottom: 0cm; margin-top: 0cm;" type="disc">
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">What is your biggest challenge as a tester right now?<u></u><u></u></li>
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">What has changed in your test approach over the past 12 months?<u></u><u></u></li>
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">How are the expectations of your team and your stakeholders changing?<u></u><u></u></li>
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">What worries you about how you are being asked to work differently in the future?<u></u><u></u></li>
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">What have you learned in the past 12 months to prepare for the future?<u></u><u></u></li>
<li class="MsoNormal" style="margin: 0px 0px 0px 15px;">How do you see our competitors and industry evolving?<u></u><u></u></li>
</ul>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<u></u> <span style="font-size: 12.8px;"> </span></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<u></u></div>
<div class="MsoNormal" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
If you attend, you have the opportunity to connect with testing colleagues outside your usual sphere, learn a little about the different delivery environments within <organisation>, discover how our testing challenges align or differ, understand what the future might look like in your team, and share your concerns about preparation for that future. The Test Coaches will be taking notes through the sessions to help shape the support that we provide over the next quarter and beyond.</div>
***</div>
<div>
<br /><div>
The format worked well to keep the discussion flowing. The first three questions targeted current state and recent evolution, to focus testers on present and past. The second three questions targeted future thinking, to focus testers on their contribution to the continuous changes in our workplace and through the wider industry. Every session completed on time, though some groups had more of a challenge with the 8 minute limit than others!</div>
<div>
<br /></div>
Just over 50 testers opted to participate, which is roughly 50% of our Test Practice. There were volunteers from every delivery area that included testing, which meant that I could create the cross-team grouping within each session as I intended. There were ten sessions in total. Each ran with the same agenda, a maximum of six testers, and two Test Coaches.</div>
<div>
<br /></div>
<h3>
Reflection</h3>
From ten hours of sessions with over 50 people, there were a lot of notes. The second phase of this exercise was turning this raw input into a set of themes. I used a series of questions to guide my actions and prompt targeted thinking.<br /><br /><b>What did I hear?</b> I browsed all of the session notes and pulled out general themes. I needed to read through the notes several times to feel comfortable that I had included every piece of feedback in the summary.<div>
<br /></div>
<div>
<div>
<b>What did I hear </b><i style="font-weight: bold;">that I can contribute to</i><b>?</b> With open questions, you create an opportunity for open answers. I filtered the themes to those relevant to action from the Test Practice, and removed anything that I felt was beyond the boundaries of our responsibilities or that we were unable to influence.</div>
<div>
<br /></div>
<div>
<b>What didn't I hear?</b> The Test Coaches regularly seek feedback from the testers through one-on-one sessions or surveys. I was curious about what had come up in previous rounds of feedback that wasn't heard in this round. This reflected success in our past activities or indicated changes elsewhere in the organisation that should influence our strategy too.</div>
<div>
<br /></div>
<div>
<b>How did the audience skew this?</b> Because the sessions were opt-in, I used my map of the entire Test Practice to consider whose views were missing from the aggregated summary. There were particular teams who were represented by individuals and, in some instances, we may seek a broader set of opinions from that area. I'd also like to seek non-tester opinion, as in parts of the organisation there is shared ownership of quality and testing by non-testers that makes a wider view important.</div>
<div>
<br /></div>
<div>
<b>How did the questions skew this?</b> You only get answers to the questions that you ask. I attempted to consider what I didn't hear by asking only these six questions, but I found that the other Test Coaches, who didn't write the questions themselves, were much better at identifying the gaps.</div>
<div>
<br /></div>
<div>
From this reflection I ended up with a set of about ten themes that will influence our strategy. The themes will be present in the short-term outcomes that we seek to achieve, and the long-term vision that we are aiming towards. The volume of feedback against each theme, along with work currently in progress, will influence how our work is prioritised.</div>
<div>
<br /></div>
<div>
I found this whole process energising. It refreshed my perspective and reset my focus as a leader. I'm looking forward to developing clear actions with the Test Coach team and seeing more changes across the Test Practice as a result.</div>
<div>
<br /></div>
<div>
</div>
</div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com0tag:blogger.com,1999:blog-2844510344016016899.post-13609625336497028782018-02-16T14:07:00.000+13:002018-02-16T14:19:13.196+13:00How do you decide what to automate?When you start to think about a new automated test suite, the first conversations you have will focus on what you're going to automate. Whether it's your manager that is requesting automation or you're advocating for it yourself, you need to set a strategy for test coverage before you choose a tool.<br />
<br />
There are a lot of factors that contribute to the decision of what to automate. If you try to decide your scope in isolation, you may get it wrong. Here are some prompts to help you think broadly and engage a wide audience.<br />
<br />
<h3>
Product Strategy</h3>
The strategy for the product under test can heavily influence the scope of the associated test automation solution.<br />
<br />
Imagine that you're part of a team who are releasing a prototype for fast feedback from the market. The next iteration of the product is likely to be significantly different to the prototype that you're working on now. There is probably a strong emphasis on speed to release.<br />
<br />
Imagine that you're part of a team who are working on the first release version of a product that is expected to become a flagship for your organisation over the next 5 - 10 years. The product will evolve, but in contrast to the first team it may be at a steadier pace and from a more solid foundation. There is probably a strong emphasis on technical design for future growth.<br />
<br />
Imagine that you're part of a team who will deliver the first product in a line of new products that will all use the same technology and software development process. An example may be a program of work that switches legacy applications to a new infrastructure. The goal may be to shift each piece in a like-for-like manner. There is probably a strong emphasis on standard practices and common implementation.<br />
<br />
What you automate and the way that you approach automation will vary in each case.<br />
<br />
You need to think about, and ask questions about, what's happening at a high-level for your product so that you can align your test automation strategy appropriately. You probably won't invest heavily in framework design for a prototype. Your automation may be shallow, quick and dirty. The same approach would be inappropriate in other contexts.<br />
<br />
At a slightly lower level of product strategy, which features in your backlog are most important to the business? Are they also the most important to your customers? There may be a list of features that drive sales and a different set that drive customer loyalty. Your automation may target both areas or focus only on the areas of the application that are most important to one set of stakeholders.<br />
<br />
Can you map the upcoming features to your product to determine where change might happen in your code base? This can help you prepare appropriate regression coverage to focus on areas where there is a high amount of change.<br />
<br />
<h3>
Risk Assessment</h3>
When testers talk about the benefits of automation, we often focus on the number of defects it finds or the time that we’re saving in development. We describe the benefits in a way that matters to us. This may not be the right language to talk about the benefits of automation with a manager. <br />
<br />
In the 2016 World Quality Report the number one reason that managers invest in testing is to protect the corporate image. Testing is a form of reputation insurance. It’s a way to mitigate risk. <br />
<br />
When you’re deciding what to automate, you need to think about the management perspective of your underlying purpose and spend your effort in automation in a way that directly considers risk to the organisation. To do this, you need to engage people in a conversation about what those risks are.<br />
<br />
There are a lot of resources that can help support you in a conversation about risk. James Bach has a paper titled <a href="http://www.satisfice.com/articles/hrbt.pdf">Heuristic Risk-Based Testing</a> that describes an approach to risk assessment and offers common questions to ask. I've used this as a resource to help target my own conversations about risk.<br />
<br />
Both the strategy and the risk conversation are removed from the implementation detail of automation, but they set the scene for what you will do technically and your ongoing relationships with management. These conversations help you to establish credibility and build trust with your stakeholders before you consider the specifics of your test automation solution. Building a strong foundation in these working relationships will make future conversations easier.<br />
<br />
How do you consider what to automate at a more technical level?<br />
<br />
<h3>
Objective Criteria</h3>
As a general rule, start by automating a set of objective statements about your application that address the highest risk areas.<br />
<br />
Objective measures are those that can be assessed without being influenced by personal feelings or opinion. They’re factual. The kind of measure that you can imagine in a checklist, where each item is passed or failed. For example, on a construction site a hard hat must be worn. When you look at a construction worker it is easy to determine whether they are wearing a hat or not.<br />
<br />
On the flip side, subjective measures are where we find it difficult to articulate the exact criteria that we’re using to make a decision. Imagine standing in front of a completed construction project and trying to determine whether it should win the “Home of the Year”. Some of the decision making in this process is subjective.<br />
<br />
If you don’t know exactly how you’re assessing your product, then a tool is unlikely to know how to assess the product. Automation in a subjective area may be useful to assemble a set of information for a person to evaluate, but it can’t give you a definitive result like an objective check will do.<br />
<br />
A common practice to support conversations about objective criteria is the gherkin syntax for creating examples using Given, When, Then. A team that develop these scenarios collaboratively through <a href="https://lizkeogh.com/behaviour-driven-development/">BDD</a>, or <a href="http://jonkruger.com/blog/2012/01/04/the-three-amigos/">Three Amigos</a>, etc. are likely to end up with a more robust set of automation than a person who identifies scenarios independently.<br />
<br />
<h3>
Repetitive Tasks</h3>
Repetition can indicate an opportunity for automation, both repetition in our product code and repetition in our test activities. <br />
<br />
In the product, you might have widgets that are reused throughout an application e.g. a paginated table. If you automate a check that assess that widget in one place, you can probably reuse parts of the test code to assess the same widget in other places. This creates confidence in the existing use of the widgets, and offers a quick way to check behaviour when that same widget is used in future. <br />
<br />
In our testing, you might perform repetitive data setup, execute the same tests against many active versions of a product, or repeat tests to determine whether your web application can be used on multiple browsers. These situations offer an opportunity to develop automation that can be reused for many scenarios.<br />
<br />
By contrast, if we have a product that is unique, or an activity that’s a one-off, then we may not want to automate an evaluation that will only happen once. There are always exceptions, but as a general rule we want to seek out repetition. <br />
<br />
There is benefit to collaboration here too, particularly in identifying repetitive process. Your colleagues may perform a slightly different set of tasks to what you do, which could mean that you're unaware of opportunities to automate within their area. Start conversations to discover these differences.<br />
<br />
<h3>
Opportunity Cost</h3>
When you automate it’s at the expense of something else that you could be doing. In 1998 Brian Marick wrote a paper titled <a href="https://www.oio.de/public/softwaretest/When-Should-A-Test-Be-Automated.pdf">When should a test be automated?</a>. In his summary at the conclusion of the paper he says:<br />
<br />
<i>“The cost of automating a test is best measured by the number of manual tests it prevents you from running and the bugs it will therefore cause you to miss.” </i><br />
<br />
I agree this is an important consideration in deciding what to automate and believe it is often overlooked.<br />
<br />
When you decide what to automate, it’s important to discuss what proportion of time you should spend in automating and the opportunity cost of what you're giving up. Agreeing on how long you can spend in developing, maintaining, and monitoring your automation will influence the scope of your solution.<br />
<br />
<h3>
Organisational Change</h3>
So far we've considered five factors when deciding what to automate: product strategy, risk, objectivity, repetition, and opportunity cost. All five of these will change.<br />
<br />
Product strategy will evolve as the market changes. Risks will appear and disappear. As the features of the product change, what is objective and repetitive in its evaluation will evolve. The time you have available, and your opportunity costs, will shift.<br />
<br />
When deciding what to automate, you need to consider the pace and nature of your change. <br />
<br />
Is change in your organisation predictable, like the lifecycle of a fish? You know what steps the change will take, and the cadence of it. In 2 weeks, this will happen. In 6 months, that will happen. <br />
<br />
Or is change in your organisation unpredictable, like the weather? You might think that rain is coming, but you don’t know exactly how heavy it will be or when it will happen, and actually it might not rain at all. <br />
<br />
The pace and nature of change may influence the flexibility of your scope and the adaptability of your implementation. You can decide what to automate now with a view to how regularly you will need to revisit these conversations to keep your solution relevant. In environment with a lot of change, it may be less important to get it right the first time.<br />
<br />
<h3>
Collaboration</h3>
Deciding what to automate shouldn’t be a purely technical decision. Reach out for information from across your organisation. Establish credible relationships with management by starting conversations in their language. Talk to your team about objective measures and repetitive tasks. Discuss time and opportunity costs of automation with a wide audience. Consider the pace of change in your environment. <br />
<br />
It it important to agree the scope of automation collaboratively.<br />
<br />
Collaboration doesn’t mean that you call a meeting to tell people what you think. It isn't a one-way information flow. Allow yourself to be influenced alongside offering your opinion. Make it clear what you believe is suited to automation and what isn’t. Be confident in explaining your rationale, but be open to hearing different perspectives and discovering new information too.<br />
<br />
You don't decide what to automate. The decision does not belong to you individually. Instead you lead conversations that create shared agreement. Your team decide what to automate together, and regularly review those choices.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com6tag:blogger.com,1999:blog-2844510344016016899.post-61172081834290084122018-01-30T20:26:00.002+13:002018-01-30T20:27:09.733+13:00A stability strategy for test automationAs part of the continuous integration strategy for one of our products, we run stability builds each night. The purpose is to detect changes in the product or the tests that cause intermittent issues, which can be obscured during the day. Stability builds give us test results against a consistent code base during a period of time that our test environments are not under heavy load.<br />
<br />
The stability builds execute a suite of web-based user interface automation against mocked back-end test data. They run to a schedule and, on a good night, we see six successful builds:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTT-Z3KhZ1xz7VEWhvIptL_ZZwPVE8krer3vgQGJmFe_0SgGgJ7GB5eoB0ahavta56rlXLv67qzOqs3sre-SnuOccGeekK-D8QL82qJiZg78EqzCJqfHazD10pn2U9U33Bs6JbX2vOU0Vn/s1600/successbuilds.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="178" data-original-width="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTT-Z3KhZ1xz7VEWhvIptL_ZZwPVE8krer3vgQGJmFe_0SgGgJ7GB5eoB0ahavta56rlXLv67qzOqs3sre-SnuOccGeekK-D8QL82qJiZg78EqzCJqfHazD10pn2U9U33Bs6JbX2vOU0Vn/s1600/successbuilds.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
The builds do not run sequentially. At 1am and 4am we trigger two builds in close succession. These execute in parallel so that we use more of our Selenium Grid, which can give early warning of problems caused by load and thread contention.<br />
<br />
When things are not going well, we rarely see six failed builds. As problems emerge, the stability test result trend starts to look like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPwxyacgwN7x0tVvzNreK55Nq0Jr9Hi8mJiREaMrckof5wDLaAtCeKY07GjK6qE4bXDwb_dMvy73ecYZKfj4QLwdf04c7n4VLz1Gavrn4bi0WEuF8aiEZCaWyrpE50A48W_7aziHYRDjpZ/s1600/failbuildtrend.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="248" data-original-width="504" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPwxyacgwN7x0tVvzNreK55Nq0Jr9Hi8mJiREaMrckof5wDLaAtCeKY07GjK6qE4bXDwb_dMvy73ecYZKfj4QLwdf04c7n4VLz1Gavrn4bi0WEuF8aiEZCaWyrpE50A48W_7aziHYRDjpZ/s1600/failbuildtrend.png" /></a></div>
<br />
In a suite of over 250 tests, there might be a handful of failures. The number of failing tests, and the specific tests that fail, will often vary between builds. Sometimes there is an obvious pattern e.g. tests with an image picker dialog. Sometimes there appears to be no common link.<br />
<br />
Why don't we catch these problems during the day?<br />
<br />
These tests are part of a build pipeline that includes a large unit test suite. In the build that is run during the day, the test result trend is skewed by unit test failures. The developers are actively working on the code and using our continuous integration for fast feedback.<br />
<br />
Once the unit tests are successful, intermittent issues in the user interface tests are often resolved in a subsequent build without code changes. This means that the development team are not blocked, once the build executes successfully they can merge their code.<br />
<br />
The overnight stability build is a collective conscience for everyone who works on the product. When the build status changes state, a notification is pushed into the shared chat channel:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYQbs7-g22laelERCoTH3yRRVxuhDsjAdbuPuTmxEIGj3vEoFw4xil_EYX2usIppCcVRYy3Nxl20fb8-FyvuOs8lYbQ_7M-X3xTj6U7IBgXn1Z2cxU0vlFwYnwGnKoqBPGDFy6z21jbZ6Y/s1600/failbuildnotification.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="362" data-original-width="407" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYQbs7-g22laelERCoTH3yRRVxuhDsjAdbuPuTmxEIGj3vEoFw4xil_EYX2usIppCcVRYy3Nxl20fb8-FyvuOs8lYbQ_7M-X3xTj6U7IBgXn1Z2cxU0vlFwYnwGnKoqBPGDFy6z21jbZ6Y/s1600/failbuildnotification.png" /></a></div>
<br />
Each morning someone will look at the failed builds, then share a short comment about their investigation in a thread of conversation spawned from the original notification message. The team decide whether additional investigation is warranted and how the problem might be addressed.<br />
<br />
It can be difficult to prioritise technical debt tasks in test automation. The stability build makes problems visible quickly, to a wide audience. It is rare that these failures are systematically neglected. We know from experience that ignoring the problems has a negative impact on cycle time of our development teams. When it becomes part of the culture to repeatedly trigger a build in order to get a clean set of test results, everything slows down and people become frustrated.<br />
<br />
If your user interface tests are integrated into your pipeline, you may find value in adopting a similar approach to stability. We see benefits in early detection, raising awareness of automation across a broad audience, and creating shared ownership of issue resolution.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com4tag:blogger.com,1999:blog-2844510344016016899.post-69589790321785231872018-01-18T21:29:00.000+13:002018-01-18T21:34:00.550+13:00Three types of coding dojo for test automationThe Test Coaches in my organisation provide support for our test automation frameworks. We create tailored training material, investigate new tools in the market, agree our automation strategy, monitor the stability of our suites, and establish practices that keep our code clean.<br />
<br />
A lot of this work is achieved in partnership with the testers. We create opportunities for shared learning experiences. We facilitate agreement of standards and strategy that emerge through conversation. I like using coding dojos to establish these collaborative environments for test automation.<br />
<br />
When I run a coding dojo, all the testers of a product gather with a single laptop that is connected to a projector. The group set a clear objective for a test automation task that they would like to complete together. Then everyone participates in the code being written, by contributing verbally and taking a turn at the keyboard. Though we do not adopt the strict <a href="http://codingdojo.org/CodingDojoPrinciples/">principles of a coding dojo</a> as they were originally defined, we operate almost exactly as described in this short video titled '<a href="https://www.youtube.com/watch?v=gav9fLVkZQc">How to run a coding dojo</a>'.<br />
<br />
When using this format for three different types of test automation task - training, refactoring, and discovery - I've observed some interesting patterns in the mechanics of each dojo.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM7byYw7SszG3hLhU83ziKre_IfSDLySYFhJtCelu4kqyqJdBrSTzMFBig-ZbJ5HmZruLrqOzdPXL4BeLAZ6h4OL9nNElzdOmRZdNqoO0ol-LJ9zF-GxNjWh62ia6DLyM4vq3_2NnFjpWm/s1600/DifferentTypesOfDojo.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="950" data-original-width="800" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiM7byYw7SszG3hLhU83ziKre_IfSDLySYFhJtCelu4kqyqJdBrSTzMFBig-ZbJ5HmZruLrqOzdPXL4BeLAZ6h4OL9nNElzdOmRZdNqoO0ol-LJ9zF-GxNjWh62ia6DLyM4vq3_2NnFjpWm/s400/DifferentTypesOfDojo.jpg" width="336" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Three types of coding dojo for test automation</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<h3>
Training</h3>
Where there are many testers working on a product with multiple types of test automation, individuals may specialize e.g. user interface testing or API testing. It isn't feasible for every person to be an expert in every tool.<br />
<br />
Often those who have specialized in one area are curious about others. A coding dojo in this context is about transfer of knowledge to satisfy this curiosity. It also allows those who don't regularly see the code to ask questions about it.<br />
<br />
The participants vary in skill from beginner to expert. There may be a cluster of people at each end of the spectrum based on whether they are active in the suite day-to-day.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaZ8HnZG5Sp9yigp0E0Vy8hRq6azH_3PO4__H7TaDW3eBxgdlJtdQlY9zTw2xmljzAqBRagiVNrPiA2e0oDVgzsJ12FY6dHZNhHZ2CwM3QMFy-tJX593bDnKMUnUqyhwWKlFSTGPn6SaZq/s1600/DojoTraining.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="300" data-original-width="798" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaZ8HnZG5Sp9yigp0E0Vy8hRq6azH_3PO4__H7TaDW3eBxgdlJtdQlY9zTw2xmljzAqBRagiVNrPiA2e0oDVgzsJ12FY6dHZNhHZ2CwM3QMFy-tJX593bDnKMUnUqyhwWKlFSTGPn6SaZq/s400/DojoTraining.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Training dojo</td></tr>
</tbody></table>
<br />
Communication in this dojo can feel like it has a single direction, from expert to learner. Though everyone participates, it is comfortable for the expert and challenging for the beginner. In supporting the people who are unfamiliar, the experts need to provide a lot of explanation and direction.<br />
<div>
<br /></div>
<div>
This can create quite different individual experiences within the same shared environment. A beginner may feel flooded by new information while an expert may become bored by the slow pace. Even with active participation and rotation of duties, it can be difficult to facilitate this session so that everyone stays engaged.</div>
<div>
</div>
<h3>
Refactoring</h3>
When many people contribute to the same automation suite, variation can emerge in coding practices through time. Occasionally refactoring is required, particularly where older code has become unstable or unreadable.<br />
<br />
A coding dojo in this context is useful to agree the patterns for change. Rather than the scope and nature of refactoring being set by the first individual to tackle a particular type of problem, or dictated by a Test Coach, a group of testers collectively agree on how they would like to shape the code.<br />
<br />
Though the skill of the participants will vary, they skew towards expert level. The audience for refactoring work is usually those who are regularly active in the code - perhaps people across different teams who all work with the same product.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx41ScszW8ASP4xug-YylJkbVyV9B8CoPFyyuq-PfbMsnwFiJ_HhkdHFZ_wIHiCtn2lt_hN7dW8n70zk0WmrJo3TkWagYn4xLjLLgUXpBHr0lV_RQ4Gb-613udE88IqYmAyvnMQXZubuc_/s1600/DojoRefactoring.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="298" data-original-width="800" height="148" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhx41ScszW8ASP4xug-YylJkbVyV9B8CoPFyyuq-PfbMsnwFiJ_HhkdHFZ_wIHiCtn2lt_hN7dW8n70zk0WmrJo3TkWagYn4xLjLLgUXpBHr0lV_RQ4Gb-613udE88IqYmAyvnMQXZubuc_/s400/DojoRefactoring.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Refactoring dojo</td></tr>
</tbody></table>
<br />
Communication in this dojo is convergent. There are usually competing ideas and the purpose of the session is to reach agreement on a single solution. As everyone participates in the conversation, the outcome will often include ideas from many different people.<br />
<br />
In this example I've included one beginner tester, who might be someone new to the team or unfamiliar with the code. Where the context is refactoring, these people can become observers. Though they take their turn at the keyboard, the other people in the room become their <a href="http://llewellynfalco.blogspot.co.nz/2014/06/llewellyns-strong-style-pairing.html">strong-style pair</a>, which means that "for ideas to reach the computer they must go through someone else's hands".<br />
<br />
<h3>
Discovery</h3>
As our existing suites become out-dated, or as people hear of new tools that they would like to experiment with, there is opportunity for discovery. An individual might explore a little on their own, then a coding dojo is an invitation to others to join the journey.<br />
<br />
The participants of this dojo will skew to beginner. The nature of prototyping and experimentation is that nobody knows the answer!<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGpjvKCg9EgO246ip4V7Vyk1uHUB1vVRpOSFtd9_koKigPxdqg5SVTzPOVhCM5zqjrI_oqpAWQP8ZNOYP0jXUHqBmKWHfwc65IPohDcn16dgE9yxuxRqw-IYpYh-j1kMT6DbIzx7W1sxt9/s1600/DojoDiscovery.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="292" data-original-width="798" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGpjvKCg9EgO246ip4V7Vyk1uHUB1vVRpOSFtd9_koKigPxdqg5SVTzPOVhCM5zqjrI_oqpAWQP8ZNOYP0jXUHqBmKWHfwc65IPohDcn16dgE9yxuxRqw-IYpYh-j1kMT6DbIzx7W1sxt9/s400/DojoDiscovery.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Discovery dojo</td></tr>
</tbody></table>
<br />
Communication in this dojo is divergent, but directed towards a goal. People have different ideas and want to try different things, but all suggestions are in the spirit of learning more about the tool.<br />
<br />
The outcome of this dojo is likely to be <i>greater </i>understanding rather than <i>shared </i>understanding. Though we probably won't agree, but we'll all know a little bit more.<br />
<br />
For training, refactoring, and discovery, I enjoy the dynamics of a coding dojo format. I would be curious to know how these experiences match your own, or where you've participated in a dojo for test automation work in a different context.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com0tag:blogger.com,1999:blog-2844510344016016899.post-88932565382663756042018-01-04T18:33:00.000+13:002018-11-06T19:26:07.805+13:0030 articles for tech leaders written by womenWhen I was first promoted to a leadership role in tech, I looked for leadership resources that were written by women with advice targeted to a tech environment.<br />
<br />
It took some time to discover these articles, which resonated with me and have each contributed to my leadership style in some way. They are written by a variety of women in the US, UK, Europe and New Zealand, many have ties to the software testing community.<br />
<br />
This list includes several themes: leadership, communication, learning, inclusion, and recruitment. I would love your recommendations for other articles that could be added.<br />
<br />
<a href="https://www.linkedin.com/pulse/why-bother-doing-better-lynne-cazaly/">Why we should care about doing better</a> - Lynne Cazaly<br />
<a href="https://medium.com/@marlenac/follow-the-leader-de9f79e90e6">Follow the leader</a> - Marlena Compton<br />
<a href="http://larahogan.me/blog/desk-moves/">Dealing with surprising human emotions: desk moves</a> - Lara Hogan<br />
<a href="https://kingatest.wordpress.com/2017/03/06/you-follow-the-leader-because-you-want-to/">You follow the leader because you want to</a> - Kinga Witko<br />
<div>
</div>
<a href="http://www.estherderby.com/2011/03/entering-groups.html">Entering Groups</a> - Esther Derby<br />
<a href="https://www.jrothman.com/articles/2010/03/agile-managers-the-essence-of-leadership-2/">Agile Managers: The Essence of Leadership</a> - Johanna Rothman<br />
<a href="https://blog.figure.nz/recovering-from-a-toxic-job-c92ef696120e">Recovering from a toxic job</a> - Nat Dudley<br />
<a href="https://lizkeogh.com/2017/03/05/yes-and/">Yes, and...</a> - Liz Keogh<br />
<a href="https://speakerdeck.com/kwugirl/code">Ask vs. Guess cultures</a> - Katherine Wu<br />
<a href="http://finding-marbles.com/2017/02/16/how-to-engage-someone-who-doesnt-participate-gnarly-retrospective-problems/">"I just can't get her to engage!" - Gnarly Retrospective Problems</a> - Corinna Baldauf<br />
<a href="https://testingthemind.wordpress.com/2017/09/24/eight-reasons-why-no-ones-listening-to-you/">Eight reasons why no one's listening to you</a> - Amy Phillips<br />
<a href="http://quality-intelligence.blogspot.com.au/2010/01/dont-argue-with-sleepwalkers.html">Don't argue with sleepwalkers</a> - Fiona Charles<br />
<div>
<a href="http://www.estherderby.com/?s=change+artist+super+powers">Change Artist Super Powers</a> - Esther Derby<br />
<a href="https://blog.mapbox.com/how-i-work-with-someone-who-is-learning-d6c53e460625">How I work with someone who is learning</a> - Emily McAfee</div>
<a href="http://emilywebber.co.uk/learning-knit-reminded-learning/">What learning to knit has reminded me about learning</a> - Emily Webber<br />
<a href="http://akaptur.com/blog/2015/10/10/effective-learning-strategies-for-programmers/">Effective learning strategies for programmers</a> - Allison Kaptur<br />
<a href="https://medium.com/@cwodtke/five-models-for-making-sense-of-complex-systems-134be897b6b3">Five models for making sense of complex systems</a> - Christina Wodtke<br />
<a href="https://www.ctohanian.com/blog/2017/11/24/stepping-out-of-your-comfort-zone">The comfort zone</a> - Christina Ohanian<br />
<a href="http://www.cassandrahl.com/blog/wtf-are-you-doing-tell-your-teams/">WTF are you doing? Tell your teams!</a> - Cassandra Leung<br />
<a href="http://thagomizer.com/blog/2017/09/29/we-don-t-do-that-here.html">We don't do that here</a> - Aja Hammerly<br />
<a href="http://firstround.com/review/heres-how-to-wield-empathy-and-data-to-build-an-inclusive-team/">Here's how to wield empathy and data to build an inclusive team</a> - Ciara Trinidad<br />
<a href="http://larahogan.me/blog/inclusion-math/">Tracking compensation and promotion inequity</a> - Lara Hogan<br />
<a href="https://medium.com/this-is-hard/the-other-side-of-diversity-1bb3de2f053e">The other side of diversity</a> - Erica Joy<br />
<a href="https://modelviewculture.com/pieces/hiring-isn-t-enough">Hiring isn't enough</a> - Catt Small<br />
<a href="http://blog.alicegoldfuss.com/ladies-is-gender-neutral/">'Ladies' is gender neutral</a> - Alice Goldfuss<br />
<a href="https://www.linkedin.com/pulse/where-does-white-privilege-show-up-kristin-hull-phd/">Where does white privilege show up?</a> - Kirstin Hull<br />
<div>
<a href="https://techbeacon.com/recruiting-diverse-engineering-candidates-what-tech-companies-still-get-wrong">Recruiting diverse engineering candidates: What tech companies still get wrong</a> - Angie Jones</div>
<a href="http://trishkhoo.com/2017/01/better-hiring-with-less-bias/">Better hiring with less bias</a> - Trish Khoo<br />
<a href="https://textio.ai/1000-different-people-the-same-words-6149b5a1f351">1000 different people, the same words</a> - Kieran Snyder<br />
<a href="https://medium.com/thoughts-on-society/why-do-women-try-to-get-ahead-by-pulling-men-down-a1345b36b91b">Why do women try to get ahead by pulling men down?</a> - Missy Titus<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com2tag:blogger.com,1999:blog-2844510344016016899.post-60750584115494536492017-12-13T18:07:00.000+13:002017-12-13T18:07:54.929+13:00Pairing for skill vs. Pairing for confidenceI went to a <a href="https://www.meetup.com/WeTest-Workshops/">WeTest</a> leadership breakfast this morning. We run in a Lean Coffee format and today we had a conversation about how to build confidence in people who have learned basic automation skills but seem fearful of applying those skills in their work.<br /><br />I was fortunate to be sitting in a group with <a href="https://www.linkedin.com/in/vicki-hann-81a28174/">Vicki Hann</a>, a Test Automation Coach, who had a lot of practical suggestions. To build confidence she suggested asking people to:<br /><ul>
<li>Explain a coding concept to a non-technical team mate</li>
<li>Be involved in regular code reviews</li>
<li>Practice the same type of coding challenge repeatedly</li>
</ul>
<br />Then she talked about how she buddies these people within her testing team.<br /><br />Traditionally when you have someone who is learning you would buddy them with someone who is experienced. You create an environment where the experienced person can transfer their knowledge or skill to the other.<br /><br />In a situation where the person who is learning has established some basic knowledge and skills, their requirements for a buddy diversify. The types of activities that build confidence can be different to those that teach the material.<br /><br />Confidence comes from repetition and experimentation in a safe environment. The experienced buddy might not be able to create that space, or the person who is learning may have their own inhibitions about making mistakes in front of their teacher.<br /><br />Vicki talked about two people in her organisation who are both learning to code. Rather than pairing each person with someone experienced, she paired them with each other. Not day-to-day in the same delivery team, but they regularly work together to build confidence in their newly acquired automation skills.<br /><br />In their buddy session, each person explains a piece of code that they’ve written to the other. Without an experienced person in the pair, both operate on a level footing. Each person has strengths and weaknesses in their knowledge and skills. They feel safe to make mistakes, correct each other, and explore together when neither know the answer.<br /><br />I hadn’t considered that there would be a difference in pairing for skill vs. pairing for confidence. In the past, I have attempted to address both learning opportunities in a single pairing by putting the cautious learner with an exuberant mentor. I thought that confidence might be contagious. Sometimes this approach has worked well and others not.<br /><br />Vicki gave me a new approach to this problem, switching my thinking about confidence from something that is contagious to something that is constructed. I can imagine situations where I’ll want to pair two people who are learning, so that they can build their confidence together. Each person developing a belief in their ability alongside a peer who is going through the same process.<br /><br /><div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com0tag:blogger.com,1999:blog-2844510344016016899.post-46917974903276835432017-12-06T21:15:00.001+13:002017-12-07T06:12:27.355+13:00Conference BudgetsThere has been conversation on Twitter recently about conferences who do not offer speaker compensation. If you haven't been part of this discussion I would encourage you to read <a href="http://www.cassandrahl.com/blog/why-i-dont-pay-to-speak/">Why I Don't Pay to Speak</a> by Cassandra Leung, which provides a detailed summary.<br />
<br />
I take an interest in these conversations from two perspectives: I regularly speak at international conferences and I co-organise the annual <a href="http://wetest.co.nz/">WeTest</a> conferences for the New Zealand testing community.<br />
<br />
As an organiser, the events that I help to run cover all speaker travel and accommodation. We make a special effort to <a href="http://katrinatester.blogspot.co.nz/2016/10/caring-for-conference-speakers.html">care for our conference speakers</a> and have built a reputation in the international testing community as being an event that is worth presenting at.<br />
<br />
WeTest is a not-for-profit company that is entirely driven by volunteers. How do we afford to pay all of our speakers?<br />
<br />
<h3>
Humble Beginnings</h3>
Our 2014 WeTest conference was a half-day event in a single city.<br />
<br />
We had 80 participants who paid $20 per person. They received a conference t-shirt along with a catered dinner of pizza and drinks.<br />
<br />
All of our speakers were local to the city, so there were no travel or accommodation expenses. Our budget was balanced by the support of our primary sponsor, Assurity.<br />
<br />
Our total budget for this event was approximately $3,000 where our income and expenses were:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB6_nt3KNFhQLMMDIvuqsZ1sue1yy9iXbEGWR-BcVDu_OSK41WiCwDJ_aV3wABL-PAI832JJWVrCnH3DHPiTQrB6rH0Lx9hmmH57kP886XooFYOHTZuf2VMRmHlJlu2HJfeHhCKEzWp_mE/s1600/WeTestBudget2014.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB6_nt3KNFhQLMMDIvuqsZ1sue1yy9iXbEGWR-BcVDu_OSK41WiCwDJ_aV3wABL-PAI832JJWVrCnH3DHPiTQrB6rH0Lx9hmmH57kP886XooFYOHTZuf2VMRmHlJlu2HJfeHhCKEzWp_mE/s640/WeTestBudget2014.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">WeTest Budget 2014</td></tr>
</tbody></table>
<br />
<h3>
Stepping Up</h3>
By 2016 we felt that we had built an audience for a more ambitious event. We embarked on a full-day conference tour with the same program running in two cities within New Zealand.<br />
<br />
We had 150 participants in each city who paid $150 per person. This was a significant jump in scale from our previous events, so we had to establish a formal scaffold for our organisation. WeTest was registered as a company, we created a dedicated bank account, and launched our own website.<br />
<br />
This was also the first year that we invited international speakers. 25% of our speaker line-up, or three out of twelve speakers, traveled to New Zealand from overseas. Covering their travel and accommodation costs significantly altered the dynamics of our budget. Running the conference in two different cities meant that there were travel and accommodation costs for our New Zealand based speakers and organisers too.<br />
<br />
Our total budget for this event was approximately $50,000 where our income and expenses were:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWBGo0WSuUMC-hFmaYxj44F5lG8XqUopRvSW_reiWbg8T8Y-CxSmYb6c8f64aXVGCKjInV2U_N2GZBChOFyrT3WpjTA-0bt-kGOasnOLPnC0OPi48TC6ZtVYgCDBmKmMx2DVJSWsD0TwHj/s1600/WeTest2016Budget.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWBGo0WSuUMC-hFmaYxj44F5lG8XqUopRvSW_reiWbg8T8Y-CxSmYb6c8f64aXVGCKjInV2U_N2GZBChOFyrT3WpjTA-0bt-kGOasnOLPnC0OPi48TC6ZtVYgCDBmKmMx2DVJSWsD0TwHj/s640/WeTest2016Budget.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">WeTest Budget 2016</td></tr>
</tbody></table>
<br />
<h3>
The Big League</h3>
Our 2016 events sold out quickly and we had long waiting lists. To accommodate a larger audience, we grew again in 2017. This meant securing commercial venues, signing contracts, paying deposits, registering for a new level of tax liability and formalising our not-for-profit status.<br />
<div>
<br /></div>
<div>
<div>
In 2017 we had around 230 participants in each city. We introduced an earlybird ticket at $150 per person, so that our loyal supporters would not experience a price-hike and we could collect some early revenue to cover upfront costs. Our standard ticket was $250 per person.</div>
<div>
<br /></div>
<div>
40% of our speaker line-up, or four out of ten speakers, traveled to New Zealand from overseas. We incurred similar speaker travel and accommodation expenses to the previous year.<br />
<br />
Our total budget for this event was approximately $100,000 where our income and expenses were:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkh5jwr4DSKlCYJcV7dSqUyV6BP99FdcvX55922H71SBlh_bX8FQwXYb8K_DVJrkDlTnEYhLW4TZCbhFxrnzBGAa0jmvZsOkahLuae5FgzG46sgZbIQPkyPNlJYNGnW_qTmGHviYtHdRwl/s1600/WeTest2017Budget.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhkh5jwr4DSKlCYJcV7dSqUyV6BP99FdcvX55922H71SBlh_bX8FQwXYb8K_DVJrkDlTnEYhLW4TZCbhFxrnzBGAa0jmvZsOkahLuae5FgzG46sgZbIQPkyPNlJYNGnW_qTmGHviYtHdRwl/s640/WeTest2017Budget.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">WeTest Budget 2017</td></tr>
</tbody></table>
<br />
To re-iterate, WeTest is a not-for-profit organisation that is volunteer-led. The profit of our 2017 events will be reinvested into the testing community and help us to launch further events in the New Year.<br />
<br />
In the discussion about speaker reimbursement we often discuss in the abstract. I hope that these examples provide specific evidence of how a conference might approach speaker reimbursement, whether they are a small community event or a larger endeavour.<br />
<br />
At WeTest we have consistently balanced our budget without asking speakers to pay their own way. We are proud of the diverse speaker programs that have been supported by this approach. In 2018 we look forward to continuing to provide a free platform for our speakers to deliver great content.<br />
<br /></div>
</div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com1tag:blogger.com,1999:blog-2844510344016016899.post-3944506260307562922017-10-29T14:00:00.004+13:002017-10-29T14:00:49.449+13:00Strategies for automated visual regressionIn my organisation we have adopted automated visual regression in the test strategy for three of our products. We have made different choices in implementing our frameworks, as we use automated visual regression testing for a slightly different purpose in each team. In this post I introduce the concept of automated visual regression then give some specific examples of how we use it.<br />
<br />
<h3>
What is visual regression?</h3>
The appearance of a web application is usually defined by a cascading style sheet (CSS) file. Your product might use a different flavour of CSS like SCSS, SASS, or LESS. They all describe the format and layout of your web-based user interface.<br />
<br />
When you make a change to your product, you are likely change how it looks. You might intentionally be working on a design task e.g. fixing the display of a modal dialog. Or you might be working on a piece of functionality that is driven through the user interface, which means that you need to edit the content of a screen e.g. adding a nickname field to a bank account. In both cases you probably need to edit the underlying style sheet.<br />
<br />
A problem can arise when the piece of the style sheet that you are editing is used in more than one place within the product, which is often the case. The change that you make will look great in the particular screen that you're working in, but cause a problem in another screen in another area of the application. We call these types of problems visual regression.<br />
<br />
It is not always easy to determine where these regression issues might appear because style sheets are applied in a cascade. An element on your page may inherit a number of display properties from parent elements. Imagine a blue button with a label in Arial font where the colour of the button is defined for that element but the font of the button label is defined for the page as a whole. Changing the font of that button by editing the parent definition could have far-reaching consequences.<br />
<br />
We use automated visual regression to quickly identify differences in the appearance of our product. We compare a snapshot taken prior to our change with a snapshot taken after our change, then highlight the differences between the two. A person can look through the results of these image comparisons to determine what is expected and what is a problem.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfds5ECdcR_YzGSE7dHaFEuOwJEdp1hPE6B3RUqzJJNaqW5Pm9p5SvTxM1gpbDALzWIwacgMpolPu3eq7IWsZTmj7Srl63oDV204TDxw1eHNgdbIwvdaTLwtfdneWO-wgmVeQ9LnBANcYE/s1600/ExampleVR.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="175" data-original-width="1000" height="112" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfds5ECdcR_YzGSE7dHaFEuOwJEdp1hPE6B3RUqzJJNaqW5Pm9p5SvTxM1gpbDALzWIwacgMpolPu3eq7IWsZTmj7Srl63oDV204TDxw1eHNgdbIwvdaTLwtfdneWO-wgmVeQ9LnBANcYE/s640/ExampleVR.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Manufactured example to illustrate image comparison</td></tr>
</tbody></table>
<br />
<h3>
Team One Strategy</h3>
The first team to adopt automated visual regression in my organisation was our public website, a product with a constantly evolving user interface.<br />
<br />
The test automation strategy for this product includes a number of targeted suites. There are functional tests written in Selenium that examine the application forms, calculators, and other tools that require user interaction. There are API tests that check the integration of our website to other systems. We have a good level of coverage for the behaviour of the product.<br />
<br />
Historically, none of our suites specifically tested the appearance of the product. The testers in the team found it frustrating to repetitively tour the site, in different browsers, to try to detect unexpected changes in how the website looked. <a href="https://www.youtube.com/watch?v=IGQmdoK_ZfY">Inattentional blindness</a> meant that problems were missed.<br />
<br />
The team created a list of the most popular pages in the site based on our analytics. This list was extended, so that it included at least one page within each major section of the website, to define an application tour for the automated suite to capture screenshots for comparison.<br />
<br />
The automated visual regression framework was implemented to complete this tour of the application against a configurable list of browsers. It launches BrowserStack, which means that it is able to capture images against desktop, tablet, and mobile browsers. The automated checks replace a large proportion of the cross-browser regression testing that the testers were performing themselves.<br />
<br />
The team primarily use the suite at release, though occasionally make use of it during the development process. The tool captures a set of baseline images from the existing production version of the product and compares these to images from the release candidate. The image comparison is made at a page level: a pixel-by-pixel comparison with a fuzz tolerance for small changes.<br />
<br />
<h3>
Team Two Strategy</h3>
The second team to adopt automated visual regression was our UI toolkit team. This team develop a set of reusable user interface components so that all of our products have a consistent appearance. The nature of their product means that display problems are important. Even a difference of a single pixel can be significant.<br />
<br />
The tester in the this team made automated visual regression the primary focus of their test strategy. They explored the solution that the first team had created, but decided to implement their own framework in a different way.<br />
<br />
In our toolkit product, we have pages that display a component in different states e.g. the button page has examples of a normal button, a disabled button, a button that is being hovered on, etc. Rather than comparing the page as a whole with a fuzz tolerance, this tester implemented an exact comparison at a component level. This meant that the tests were targeted and would fail with a specific message e.g. the appearance of the disabled button has changed.<br />
<br />
The initial focus for this framework was getting component level coverage across the entire toolkit with execution in a single browser. This suite was intended to run after every change, not just at release. The tester also spent some time refining the reporting for the suite, to usefully abstract the volume of image comparisons being undertaken.<br />
<br />
Once the tests were reliable and the reporting succinct, the tester extended the framework to run against different browsers. Cross-browser capability was a lower priority than in the Team One.<br />
<br />
<h3>
Team Three Strategy</h3>
A third team are starting to integrate automated visual regression into their test strategy. They work on one of our authenticated banking channels, a relatively large application with a lot of different features.<br />
<br />
This product has mature functional test automation. There are two suites that execute through the user interface: a large suite with mocked back-end functionality and a small suite that exercises the entire application stack.<br />
<br />
For this product, implementing automated visual regression for a simple application tour is not enough. We want to examine the appearance of the application through different workflows, not just check the display of static content. Rather than repeating the coverage provided by the existing large test suite, the team extended the framework to add an automated visual regression test.<br />
<br />
This suite is still under development and, of the three solutions, it is the largest, the slowest, and requires the most intervention by people to execute. The team created a configuration option to switch on screenshot collection as part of the existing functional tests. This generates a set of images that will either represent the 'before' or the 'after' state, depending on which version of the application is under test.<br />
<br />
Separate to the collection of images is a comparison program that takes the two sets of screenshots and determines where there are differences. The large suite of functional tests means that there are many images to compare, so the developers came up with an innovative approach to perform these comparisons quickly. They first compare a hash string of the image then, in the event that these differ, they perform the pixel-by-pixel comparison to determine what has changed.<br />
<br />
In this team the automated visual regression has a fractured implementation. The collection and comparison happen separately. The focus remains on a single browser and the team continue to iterate their solution, particularly by improving the accuracy and aesthetics of their reporting.<br />
<br />
<h3>
Conclusion</h3>
We use automated visual regression to quickly detect changes in the appearance of our product. Different products will require different strategies, because we are looking to address different types of risk with this tool.<br />
<br />
The three examples that I've provided, from real teams in my organisation, illustrate this variety in approach. We use visual regression to target:<br />
<ul>
<li>cross-browser testing, </li>
<li>specific user interface components, and</li>
<li>consistent display of functional workflows. </li>
</ul>
As with any test automation, if you're looking to implement automated visual regression consider the problem that you're trying to solve and target your framework to that risk.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com3tag:blogger.com,1999:blog-2844510344016016899.post-47199490514736278682017-10-12T07:05:00.000+13:002017-10-17T19:09:01.568+13:00Identifying and influencing how people in your team contribute to test automation<div style="text-align: center;">
<span style="font-family: "trebuchet ms" , sans-serif; font-size: x-small;">This is a written version of my keynote at The Official 2017 European Selenium Conference in Berlin, Germany. </span></div>
<div style="text-align: center;">
<span style="font-family: "trebuchet ms" , sans-serif; font-size: x-small;">If you'd prefer to watch the talk, it is available on the <a href="https://www.youtube.com/watch?v=YhQHgYZyt1U">Selenium YouTube channel</a>.</span></div>
<i><br /></i>
<br />
<div style="text-align: center;">
<i><i>How do your colleagues contribute to test automation?</i></i></div>
<div style="text-align: center;">
<span style="font-style: italic;"><br /></span></div>
<div style="text-align: center;">
<i>Who is involved in the design, development and maintenance of your test suites?</i></div>
<div style="text-align: center;">
<span style="font-style: italic;"><br /></span></div>
<div style="text-align: center;">
<i>What would happen if people in your team changed how they participate in test automation?</i></div>
<div style="text-align: center;">
<span style="font-style: italic;"><br /></span></div>
<div style="text-align: center;">
<i>How could you influence this change?</i></div>
<div style="text-align: center;">
<br /></div>
This article will encourage you to consider these four questions.<br />
<br />
<h3>
Introduction</h3>
When I was 13 years old I played field hockey. I have a lot of fond memories of my high school hockey team. It was a really fun team to be part of and I felt that I really belonged.<br />
<br />
When I was 10 years old I really wanted to play hockey. I have clear memories of asking my parents about it, which is probably because the conversation happened more than once. I can remember how much I wanted it.<br />
<br />
What happened in those three years, between being a 10 year old who wanted to play hockey and a 13 year old creating fond memories in a high school hockey team? Three things.<br />
<br />
The first barrier to playing hockey was that I had none of the gear. I lived in a small town in New Zealand, both my parents were teachers, and hockey gear was a relatively large investment for my family. To participate in the sport I needed a stick, a mouth guard, shin guards, socks. Buying all this equipment gave me access to the sport.<br />
<br />
Once I had the gear, I needed to learn how to use it. It turned out that my enthusiasm for getting onto the field did not translate into a natural ability. In fact, initially I was quite scared of participating. I had to learn to hit the ball and trap it, the different positions on the field, and what to do in a short corner. Learning these skills gave me the confidence to play the game, which meant that I started to enjoy it.<br />
<br />
The third reason that I ended up in a hockey team when I was 13 years old was because that is where my friends were. As a teenager, spending time with my friends after school was excellent motivation to be part of a hockey team.<br />
<br />
Access, skills, and motivation. These separated me at 10 years of age from me at 13 years of age. These separated a kid who really wanted to participate in a sport from someone who felt like they were part of a team.<br />
<br />
This type of division is relatable. Team sports are an experience that many of us share, both the feelings of belonging and those of exclusion. Access, skills, and motivation also underpin other types of division in our lives.<br />
<br />
<h3>
Division in test automation</h3>
If you look at a software development team at a stand-up meeting, they are all standing together. People are physically close to each other, not on opposite sides of a chasm. But within that group are divisions, and different divisions depending on the lens that you use.<br />
<br />
If we think about division in test automation for a software development team then, given what I’ve written about so far, you imagine something like this:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkNdzs2Mlhk8tFf1nkIcd5ZACGg8ZRggsmIex9ZmYsjpLyidol6E9ttZfDwTvTveVKOhtxTHpqabIX4y-r2MnLq8StGQO6l69hM7rAyO0iGNBJAw8TDheZbBv9P8CIdMO5IFlAHANjDzGT/s1600/FarmyardLinear.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkNdzs2Mlhk8tFf1nkIcd5ZACGg8ZRggsmIex9ZmYsjpLyidol6E9ttZfDwTvTveVKOhtxTHpqabIX4y-r2MnLq8StGQO6l69hM7rAyO0iGNBJAw8TDheZbBv9P8CIdMO5IFlAHANjDzGT/s400/FarmyardLinear.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A linear diagram of division</td></tr>
</tbody></table>
<br />
People are divided into categories and progress through these pens from left to right. To be successful in test automation I need access to the source code, I need the skills to be able to write code, and I need to be enthusiastic about it. Boom!<br />
<br />
Except, perhaps it’s not that simple or linear.<br />
<br />
What if I’m a new tester to a team, and I have the coding skills but not permissions to contribute to a repository? What if I’m enthusiastic, but have no idea what I’m doing? It’s not always one, then the other, then the other. I am not necessarily going to acquire each attribute in turn.<br />
<br />
Instead, I think the division looks something like this.<br />
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5VTIZLkP6KOJRxL1gkfKWCS_qjKBDH69qg64cOu0dAESIz129S81FtwHbN_4MgOkZIICOH909m0sPUxfykPFSk8WkALw7QvQgITc314IHB7Nnm5BvUy4ONu1ScOQoqYD-prWKxSh-BA8S/s1600/FarmyardVenn.png" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5VTIZLkP6KOJRxL1gkfKWCS_qjKBDH69qg64cOu0dAESIz129S81FtwHbN_4MgOkZIICOH909m0sPUxfykPFSk8WkALw7QvQgITc314IHB7Nnm5BvUy4ONu1ScOQoqYD-prWKxSh-BA8S/s400/FarmyardVenn.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A Venn diagram of division</td></tr>
</tbody></table>
<br />
A Venn diagram of division by access, skills and motivation. An individual could have any three of these, or any combination of the three, or none at all.<br />
<br />
To make sense of this, I’d like to talk in real-life examples from teams that I have been part of, which feature five main characters:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiErE_g9RkfxXruwBl69bYFP7Bi-epmVr4mzyxkLO6ykoOejFfFPwdu32X0wN2msyqG7gmxR3YDbWk9xgd5s2UtAYrcsHkK2ufdpqbw5zeihiOaxKEdxgaAuxUbnSB6ELKLhhhDfhl6M7X9/s1600/FarmyardCharacters.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="185" data-original-width="480" height="153" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiErE_g9RkfxXruwBl69bYFP7Bi-epmVr4mzyxkLO6ykoOejFfFPwdu32X0wN2msyqG7gmxR3YDbWk9xgd5s2UtAYrcsHkK2ufdpqbw5zeihiOaxKEdxgaAuxUbnSB6ELKLhhhDfhl6M7X9/s400/FarmyardCharacters.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Five characters of an example team</td></tr>
</tbody></table>
<br />
The gray goose represents the manager. The burgundy red characters represent the business: the dog is the product owner, the horse is the business analyst. Orange chickens are the developers, and the yellow deer are the testers.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg40xi3BJyziT02bsukdK_FZNMPudhBqOvAPn4t58uBpa9_UcTd4AcxWbWuGksAeiBdZ3aMeesX2R_DmwCVj2ORwRE8Dcs1ZuDw2HmbfnS0FHyZ0V12xBWlz6yEJWutvXXaIZEJTbchoDmr/s1600/TeamOne.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg40xi3BJyziT02bsukdK_FZNMPudhBqOvAPn4t58uBpa9_UcTd4AcxWbWuGksAeiBdZ3aMeesX2R_DmwCVj2ORwRE8Dcs1ZuDw2HmbfnS0FHyZ0V12xBWlz6yEJWutvXXaIZEJTbchoDmr/s400/TeamOne.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team One</td></tr>
</tbody></table>
<h4>
Team One</h4>
This was an agile development team in a large financial institution. I was one of two testers. We were the only two members of the team who were committing code to the test automation suite. We are the two yellow deer right in the middle of the Venn diagram with access, skills and motivation.<br />
<br />
The developers in this team could have helped us, they had all the skills. They didn’t show any interest in helping us, but also we didn’t give them access to our code. The three orange chickens at the top of the Venn diagram show that the developers had skills, but no motivation or access.<br />
<br />
The business analyst didn’t even know that we had test automation, and there was no product owner in this team. However there was a software development manager and they were a vocal advocate for the test automation to exist, though they didn’t understand it. The burgundy red horse at the top right is outside of the diagram, the grey goose is in motivation alone.<br />
<br />
The test automation that this farmyard created was low-level, it executed queries against a database. As we were testing well below the user interface where the business felt comfortable, they were happy to have little involvement. The code in the suite was okay. It wasn’t as good as it could have been if the developers were more involved, but it worked and the tests were stable.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqaqHIVwUsgh066y7_YzzfQADg0KF9DnWKPbtk8vqNy2_90NMuHzP26MIDnAZw55OOwZKuDxx0Sc4tAmlxX6EevXAt6yH0lBGtz-DDpn001Ku-PVJgbAlcUyCRgqfcMPa5N0doDerVfy_L/s1600/TeamTwo.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqaqHIVwUsgh066y7_YzzfQADg0KF9DnWKPbtk8vqNy2_90NMuHzP26MIDnAZw55OOwZKuDxx0Sc4tAmlxX6EevXAt6yH0lBGtz-DDpn001Ku-PVJgbAlcUyCRgqfcMPa5N0doDerVfy_L/s400/TeamTwo.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Two</td></tr>
</tbody></table>
<h4>
Team Two</h4>
This was a weird team for me. You can see that as a tester, the yellow deer, I had the skills and motivation to contribute to test automation, but no access. I was bought in as a consultant to help the existing team of developers and business analysts to create test automation.<br />
<br />
The developers and business analysts had varying skills. There were a couple of developers who were the main contributors to the suites. The business analysts had access and were enthusiastic, but they didn’t know how to write or read code. Then there were a couple of developers who had the access and skills, but firmly believed that test automation was not their job, they’re the chickens on the left without motivation.<br />
<br />
This team built a massive backlog of technical debt in their test automation because the developers who were the main contributors preferred to spend their time doing development. The test code was elegant, but the test coverage was sparse.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxhf3QnojI7lj7x3SSSr9weaYVYJtEgutRKUDJ3ezGYa7btuxi7p_BCeUmRYGq7Iz8cbKDEQOpwoDzDmlQ4c2A88H3OlOXegLpwrISwLu-Y4U1qqYAvHGgFqU8guyPTUuRDojisQF6T02/s1600/TeamThree.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDxhf3QnojI7lj7x3SSSr9weaYVYJtEgutRKUDJ3ezGYa7btuxi7p_BCeUmRYGq7Iz8cbKDEQOpwoDzDmlQ4c2A88H3OlOXegLpwrISwLu-Y4U1qqYAvHGgFqU8guyPTUuRDojisQF6T02/s400/TeamThree.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Three</td></tr>
</tbody></table>
<h4>
Team Three</h4>
In this team everyone had access to the code except the project manager, but skills and motivation created division.<br />
<br />
I ended up working on this test suite with one of the business analysts. He bought all the domain and business knowledge, helped to locate test data, and made sure that the suites had strong test coverage across all of the peculiar scenarios. I bought the coding skills to implement the test automation.<br />
<br />
In this team I couldn’t get any of the developers interested in automation. Half had the skills, half didn’t, but none of them really wanted to dive in. The product owner and the other BA who had access to the code were not interested either. They would say that they trusted what the two of us in the middle were producing, so they felt that they didn’t need to be involved.<br />
<br />
I believe that the automation we created was pretty good. We might have improved with the opportunity to do peer review within a discipline. The business analyst reviewed my work, and I reviewed his, but we didn’t have deep cross-domain understanding.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSTSrJUegV-yeLffXaF7fiEqBMnfo0yMCxevBkTlWmZF5nPmxPU3MvIALyQJU0hhY8PiofuXaNi1zEugIQca3IPGq3UggJ1jFQo_s495t3jV-SjNAjYZUbISPhtESvaMYRTsKgMw1ZTxi7/s1600/TeamFour.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSTSrJUegV-yeLffXaF7fiEqBMnfo0yMCxevBkTlWmZF5nPmxPU3MvIALyQJU0hhY8PiofuXaNi1zEugIQca3IPGq3UggJ1jFQo_s495t3jV-SjNAjYZUbISPhtESvaMYRTsKgMw1ZTxi7/s400/TeamFour.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Four</td></tr>
</tbody></table>
<h4>
Team Four</h4>
This was a small team where we had no test automation. We had some unit tests, but there wasn’t anything beyond that. This meant that we did a lot of repetitive testing that, in retrospect, seems a little silly.<br />
<br />
I was working with two developers. We had a business analyst and a product owner, but no other management alongside us. The technical side of the delivery team all had access to the code base and the skills to write test automation, but we didn’t have time or motivation to do so. The business weren’t pushing for it as an option.<br />
<br />
You may have heard similarities to your situation in these experiences. Take a moment to consider your current team. Where would you put your colleagues in a test automation farmyard?<br />
<br />
<h3>
Contributors to test automation</h3>
Next, think about how people participate in test automation dependent on where they fall into this model. Originally I labelled the parts of the diagram as access, skills, and motivation:<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigV6SLI0IiDVCEbZlpAyMY7UW2emCgZFWVLl99S9Dgm_BrKGJ0UAmiouf2aFQ4zYYeFNFmJUUYUlcIArKTqsxK8T43VjXmODx9FdGAvtOTkNBbTGnwqSdv7c907hKbvN44mu_0LKHxE9hm/s1600/VennRolesOriginalASM.png" imageanchor="1"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigV6SLI0IiDVCEbZlpAyMY7UW2emCgZFWVLl99S9Dgm_BrKGJ0UAmiouf2aFQ4zYYeFNFmJUUYUlcIArKTqsxK8T43VjXmODx9FdGAvtOTkNBbTGnwqSdv7c907hKbvN44mu_0LKHxE9hm/s400/VennRolesOriginalASM.png" width="400" /></a></div>
<br />
If I switch these labels to roles they might become:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzBaRHV2OA8Py6VK9jdU656EACUMjYbbgivo-ktRTWl9nzQMkI3D4dDtLgS2D9vDpVWJhyGn4edvDHGMEUk2OoSAJbVWX8HbFgrIlEcNa4lASeHwuw6K95oCTLKX4LDdgLC-G_5SlNPqQ9/s1600/VennRolesV1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzBaRHV2OA8Py6VK9jdU656EACUMjYbbgivo-ktRTWl9nzQMkI3D4dDtLgS2D9vDpVWJhyGn4edvDHGMEUk2OoSAJbVWX8HbFgrIlEcNa4lASeHwuw6K95oCTLKX4LDdgLC-G_5SlNPqQ9/s400/VennRolesV1.png" width="400" /></a></div>
<br />
A person who only has access to test automation is an observer. They're probably a passive observer, as they don't have the skills or the motivation to be more involved.<br />
<br />
A person who only has skills is a teacher. They don't have access or motivation, but they can contribute their knowledge.<br />
<br />
A person who only has motivation is an advocate. They're a source of energy for everyone within the team.<br />
<br />
Where these boundaries overlap:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3tv2q6Q84p5Sqet0sxKSQ7yvtoN4CkZowUVWfczdeshRl4_nObIDgLB-2_bSoREogSXCEGust0dzQD_iN7Irj19XnXk_F9UapoazbIL0Xku0ZnXCzriXV97-kABSTdzRcEY38FVo_EmSE/s1600/VennRolesV2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3tv2q6Q84p5Sqet0sxKSQ7yvtoN4CkZowUVWfczdeshRl4_nObIDgLB-2_bSoREogSXCEGust0dzQD_iN7Irj19XnXk_F9UapoazbIL0Xku0ZnXCzriXV97-kABSTdzRcEY38FVo_EmSE/s400/VennRolesV2.png" width="400" /></a></div>
<br />
A problem solver is someone with access and skills who is not motivated to get involved with test automation day-to-day. These people are great for helping to debug specific issues, reviewing pull requests, or asking specific questions about test coverage. Developers often sit in this role.<br />
<br />
Coaches have skills and motivation, but no access. They’re an outside influence to offer positive and hopefully useful guidance. If you consider a wider set of colleagues, you might treat a tester from another development team as a coach.<br />
<br />
Inventors are those who have access and motivation, but no skills. These are the people who can see what is happening and get super excited about it, but don’t have skills to directly participate. In my experience they’ll throw out ideas, some are crazy and some are genius. These people can be a source of innovation.<br />
<br />
And in the middle are the committers. These are generally the people who keep the test suite going. They have access, skills, and motivation.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipSkQEq8Tx7fWdM_-5D1pkaJkrDDTPPwHk5_dz7K_VnwS_fM9-6Wre7xZIHPy-9A0u9RxA_4O9bMUeEuTeSl-tquuayRqHLJcbbYX2yck_f307nxn6fjXLvUzFuYPnkAn_YZY8rfWDvtC3/s1600/RolesDivisionTestAutomation.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipSkQEq8Tx7fWdM_-5D1pkaJkrDDTPPwHk5_dz7K_VnwS_fM9-6Wre7xZIHPy-9A0u9RxA_4O9bMUeEuTeSl-tquuayRqHLJcbbYX2yck_f307nxn6fjXLvUzFuYPnkAn_YZY8rfWDvtC3/s400/RolesDivisionTestAutomation.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">How people contribute to test automation</td></tr>
</tbody></table>
<br />
<h3>
Changing roles</h3>
Now that you have labels for the way that your colleagues contribute to your test automation, consider whether people are in the "right" place.<br />
<br />
I’m not advocating for everyone in your team to be in the middle of the model. Being a committer is not necessarily a goal state, there is value in having people in different roles. However there might be specific people who you can shift within the model that would create a big impact for your team.<br />
<br />
Consider how people were contributing to test automation in the four teams that I shared earlier.
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBtoyAMuraGgCqFrEMxe5_ud5_Uu0ZZFlNl5XzDHBkCzu2Y-8pX6T6-4NvEVT1zagTKADRoc5vIJM6mZL8xXJsNlik-w9X2FtrwMB2jd1avAJs2prvz1EtQZuC7eAFUGPhaga0zwTkByus/s1600/TeamOneMove.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBtoyAMuraGgCqFrEMxe5_ud5_Uu0ZZFlNl5XzDHBkCzu2Y-8pX6T6-4NvEVT1zagTKADRoc5vIJM6mZL8xXJsNlik-w9X2FtrwMB2jd1avAJs2prvz1EtQZuC7eAFUGPhaga0zwTkByus/s400/TeamOneMove.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team One</td></tr>
</tbody></table>
<br />
In team one, all the developers were teachers. They had skills, but nothing else. In retrospect if I was choosing one thing to change here, we should have given at least one of the developers access to the code so that they could step into a problem solving role and provide more hands-on help.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCdQWvDacFTytE5IwefHQEE10CMfJaUmKFB0mqS4ccZzyjKMb_xq7ORodXIHQ4KrNAMdJeYXuwOzliHMXn7bP41IMyNw1QSY423ld-77LicuszoLaF-lVVQmjYVeZkR2bQNlXvfY-y24MD/s1600/TeamTwoMove.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCdQWvDacFTytE5IwefHQEE10CMfJaUmKFB0mqS4ccZzyjKMb_xq7ORodXIHQ4KrNAMdJeYXuwOzliHMXn7bP41IMyNw1QSY423ld-77LicuszoLaF-lVVQmjYVeZkR2bQNlXvfY-y24MD/s400/TeamTwoMove.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Two</td></tr>
</tbody></table>
<br />
In team two, I found it frustrating to coach the team without being able to directly influence the code. In retrospect, I could have fought harder for access to the code base. I think that as a committer I would have had greater impact on the prioritisation of testing work and the depth of test coverage provided by automation.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj65AaQvdNvw1IB3DUAx-i6zSagv3YbpFc8wUh6XAzG3cBRjxCt73BFXMP-0qtgu66HnVQMqtvFb0RaWSkgshXSns0OBWX2dMhzimFhrbyj5VNsK46HO86f_vbgO9ii1ChCHeBqeBs-2Nis/s1600/TeamThreeMove.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj65AaQvdNvw1IB3DUAx-i6zSagv3YbpFc8wUh6XAzG3cBRjxCt73BFXMP-0qtgu66HnVQMqtvFb0RaWSkgshXSns0OBWX2dMhzimFhrbyj5VNsK46HO86f_vbgO9ii1ChCHeBqeBs-2Nis/s400/TeamThreeMove.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Three</td></tr>
</tbody></table>
<br />
In team three, it would have been good to have a peer reviewer from the same discipline. Bringing in a developer to look at the implementation of tests, and/or another business analyst to look at business coverage of the tests, could have made our test automation more robust.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhthgV8MBHjJEdiS733r-HnifRHX71RHVEeZbMmgRwhgx5nHBpyB9YmtmU7lkWBHdOd63otPJsWbORRXCLOjQjMD1Vs4DO-080XoH4iJxbE7L_y8DN2rBmn__OCL3XkvlUrEDJi39DmyZUc/s1600/TeamFourMove.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="270" data-original-width="480" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhthgV8MBHjJEdiS733r-HnifRHX71RHVEeZbMmgRwhgx5nHBpyB9YmtmU7lkWBHdOd63otPJsWbORRXCLOjQjMD1Vs4DO-080XoH4iJxbE7L_y8DN2rBmn__OCL3XkvlUrEDJi39DmyZUc/s400/TeamFourMove.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Team Four</td></tr>
</tbody></table>
<br />
In team four, we needed someone to advocate for automation. Without a manager, I think the product owner was the logical choice for this. If they’d been excited about test automation and created an environment where it had priority, I think it would have influenced the technical team members towards automation.<br />
<br />
Think about your own team. Who would you like to move within this model? Why? What impact do you think that would have?<br />
<br />
<h3>
Scope of change</h3>
To influence, you first need to think about what specifically you are trying to change. Let's step back out to the underlying model of skills, access, motivation. These three attributes are not binary themselves. If you are trying to influence change for a person in one of these dimensions, then you need to understand what exactly you are targeting, and why.<br />
<br />
<h4>
Access</h4>
What does it mean to have access to code? Am I granted read-only permissions to a repository, or can I edit existing files, or even create new ones? Does access include having licenses to tools, along with permission to install and setup a development environment locally.
<br />
<br />
In some cases, perhaps access just means being able to see a report of test results in a continuous integration server like Jenkins. That level of access may be enough to involves a business analyst or a product owner in the scope of automated test coverage.
<br />
<br />
When considering access, ask:
<br />
<ul>
<li>What are your observers able to see?</li>
<li>What types of problems can your problem solvers respond to?</li>
<li>How does access limit the ideas of your inventors?</li>
</ul>
<br />
<h4>
Skill</h4>
Skill is not just coding skill.
<a href="https://twitter.com/northern_tester">Ash Winter</a> has developed a wheel of testing which I think is a useful prompt for thinking more broadly about skill:
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFFlwoq1FBpNzqMMFbZWKg7ogEXr6MAqJxdJ_EYyc0K__irdU4HHF9_pIukTNx9MVV4iy54JY09hI4PRWpBMKhdeRyveFthqqZUQ0YZTPwJCPDumdt8gSu7YZiNw5ttxUuofCSSDm3dVZ6/s1600/WheelOfTestingAshWinter.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="843" data-original-width="1018" height="330" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFFlwoq1FBpNzqMMFbZWKg7ogEXr6MAqJxdJ_EYyc0K__irdU4HHF9_pIukTNx9MVV4iy54JY09hI4PRWpBMKhdeRyveFthqqZUQ0YZTPwJCPDumdt8gSu7YZiNw5ttxUuofCSSDm3dVZ6/s400/WheelOfTestingAshWinter.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Wheel of Testing by Ash Winter</td></tr>
</tbody></table>
<br />
Coding is one skill that helps someone contribute to test automation. Skill also includes test design, the ability to retrieve different types of test data, creating a strategy for test automation, or even generating readable test reports.
<br />
<br />
How do the skills of your teachers, coaches, and problem solvers differ? Where do you have expertise, and where is it lacking? What training should your team seek?<br />
<br />
<h4>
Motivation</h4>
Motivation is not simply "I want test automation" or "I don’t want test automation". There's a spectrum - how much does a person want it? You might have a manager who advocates for 100% automated test coverage, or a developer who considers anything more than a single UI test to be a waste of time.
<br />
<br />
How invested are your advocates? Should they be pushing for more or backing off a little?<br />
<br />
How engaged are your coaches and inventors?
<br />
<br />
<h3>
Wider Perspective</h3>
The other thing to consider is who isn’t inside the fences at all. The examples that I shared above featured geese, dogs, horses, chickens, deer. Who is not in this list? Are there other animals around your organisation who should be part of your test automation farmyard?<br />
<br />
Test automation may be helpful for your operations team to understand the behaviour of a product prior to its release. If you develop executable specifications using BDD, or something similar, could they be shared as a user manual for call centre and support staff?
<br />
<br />
A wider perspective can also provide opportunities for new information to influence the design of your test automation. Operations and support staff may think of test scenarios that the development team did not consider.
<br />
<br />
<h3>
Conclusion</h3>
Considering division helps us to feel empathy for others and to more consciously split ourselves in a way that is "right" for our team. Ask whether there are any problems with the test automation in your team. If there are no problems, do you see any opportunities?<br />
<br />
Next, think about what you can do to change the situation. Raise awareness of the people around you who don't have access that would be useful to them. Support someone who is asking for training or time to contribute to automation. Ask or persuade a colleague to move themselves within the model.<br />
<br />
Testers have a key skill required to be an agent of change, we ask questions daily.<br />
<br />
<div style="text-align: center;">
<i>How do your colleagues contribute to test automation? </i></div>
<div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<i><i>Who is involved in the design, development and maintenance of your test suites?</i></i></div>
<i>
</i></div>
<div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<i><i>What would happen if people in your team changed how they participate in test automation?</i></i></div>
<i>
</i></div>
<div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<i><i>How could you influence this change?</i></i></div>
<i>
</i></div>
<div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com3tag:blogger.com,1999:blog-2844510344016016899.post-13985281149231329772017-09-17T09:02:00.003+12:002017-10-21T17:00:48.430+13:00How to start a Test Coach roleI received an email this morning that said, in part:<br />
<br />
<div dir="auto" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<i>I've been given the opportunity to trial a test coaching approach in my current employer (6-7 teams of 4-5 devs). </i></div>
<div dir="auto" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<i><br /></i></div>
<div dir="auto" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<i>I had a meeting with the head of engineering and she wants me to act almost like a test consultant in that I'm hands off. She expects me to be able to create a system whereby I ask the teams a set of questions that uncover their core testing problems. <span style="font-size: 12.8px;">She's also looking for keys metrics that we can use to measure success.</span></i></div>
<div dir="auto" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<span style="font-size: 12.8px;"><i><br /></i></span></div>
<div dir="auto" style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8px;">
<div dir="auto" style="font-size: 12.8px;">
<i>My question is whether you have a set of questions or approach that allows teams to uncover their biggest testing problems? <span style="font-size: 12.8px;">Can you suggest reading material or an approach?</span></i></div>
<div>
<br /></div>
</div>
On this prompt, here is my approach to starting out as a Test Coach.<br />
<div>
<br /></div>
<h3>
Avoid Assessment</h3>
<div>
A test coach role is usually created by an organisation who are seeking to address a perceived problem. It may be that the testers are slower to respond to change, or that testers are less willing to engage in professional development, or that the delivery team does not include a tester and the test coach is there to introduce testing in other disciplines. </div>
<div>
<br /></div>
<div>
Generally, the role is created by a manager who sits beyond the delivery teams themselves. They have judged that there is something missing. I think it is a bad idea to start a test coach role with a survey of testing practices intended to quantify that judgement. You might represent a management solution to a particular problem that does not exist in the eyes of the team. </div>
<div>
<br /></div>
<div>
Your first interaction will set the tone of future interactions. If you begin by asking people to complete a survey or checklist, you pitch your role as an outsider. Assessments are generally a way to claim power and hierarchy, neither of which will benefit a test coach. You want to work alongside the team as a support person, not over them.</div>
<div>
<br />
Assessment can also be dangerous when you enter the role with assumptions of what your first actions will be. If you think you know where you need to start, it can be easy to interpret the results of an assessment so that it supports your own bias.<br />
<br />
But if not by assessment, how do you begin?<br />
<br />
<h3>
Initiation Interviews</h3>
Talk to people. One-on-one. Give them an hour of your time and really listen to what they have to say. I try to run a standard set of questions for these conversations, to give them a bit of structure, but they are not an assessment. Example questions might include:<br />
<br />
<ul>
<li>Whereabouts do you fit in the organisation and how long have you been working here?</li>
<li>Can you tell me a bit about what you do day-to-day?</li>
<li>What opportunities for improvement do you see?</li>
<li>What expectations do you have for the coaching role? How do you think I might help you?</li>
<li>What would you like to learn in the next 12 months?</li>
</ul>
<br />
I don't ask many questions in the hour that I spend with a person. Mostly I listen and take notes. I focus on staying present in the conversation, as my brain can tend to wander. I try to keep an open mind to what I am hearing, and avoid judgement in what I ask.<br />
<br />
In this conversation I consciously try to adopt a position of ignorance. Even if I think that I might know what a person does, or what improvements they should be targeting, or even where they should be focused on their own development. I start this conversation with a blank slate. Some people have said "This is silly, you know what I do!", to which I say "Let's pretend that I don't". This approach almost always leads me to new insights.<br />
<br />
This is obviously a lot slower than sending out a bulk survey and asking people to respond. However, it gives you the opportunity as a coach to do several important things. You demonstrate to the people that you'll be working with that you genuinely want their input and will take the time to properly understand it. You start individual working relationships by establishing an open and supportive dialog. And you give yourself an opportunity to challenge your own assumptions about why you've been bought into the test coach role.<br />
<br />
Then how do you figure out where to start?<br />
<br />
<h3>
Finding Focus</h3>
When my role changed earlier in the year, I completed 40 one-on-one sessions. This generated a lot of notes from a lot of conversations, and initially the information felt a little overwhelming. However, when I focused on the opportunities for improvement that people spoke about, I started to identify themes.<br />
<br />
I grouped the one-on-one discussions by department, then created themed improvement backlogs for each area. Each theme included a list of anonymous quotes from the conversations that I had, which fortuitously gave a rounded picture of the opportunities for improvement that the team could see.<br />
<br />
I shared these documents back with the teams so that they had visibility of how I was presenting their ideas, then went to have conversations with the management of each area to prioritise the work that had been raised.<br />
<br />
What I like about this approach is that I didn't have to uncover the problem myself. There was no detective work. I simply asked the team what the problems were, but instead of framing it negatively I framed it positively. What opportunities for improvement do you see? Generally people are aware of what could be changed, even when they lack the time or skills to drive that change themselves.<br />
<br />
Then asking for management to prioritise the work allows them to influence direction, but without an open remit. Instead of asking "What should I do?", I like to ask "What should I do <i>first</i>?".<br />
<br />
<h3>
Measuring Success</h3>
The final part of the question I received this morning was about determining success of the test coach role. As with most measures of complex systems, this can be difficult.<br />
<br />
I approach this by flipping the premise. I don't want to measure success, I want to celebrate it.<br />
<br />
If you see something improve, I think part of the test coach role is to make sure that the people who worked towards that improvement are being recognised. If an individual steps beyond their comfort zone, call it out. If a team have collectively adopted and embedded a new practice, acknowledge it.<br />
<br />
Make success visible.<br />
<br />
I believe that people want to measure success when they are unable to see the impact of an initiative. As a test coach, your success is in the success of others. Take time to reflect on where these successes are happening and celebrate them appropriately.<br />
<br />
<br />
That's my approach to starting in a test coach role. Avoiding assessment activities. Interviewing individuals to understand their ideas. Finding focus in their responses, with prioritisation from management. Celebrating success as we work on improvements together.<br />
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com2tag:blogger.com,1999:blog-2844510344016016899.post-66651248088372274192017-08-24T21:15:00.001+12:002017-08-25T11:14:57.988+12:00The pricing model for my bookI have encountered mixed opinion about whether I should continue to offer <a href="https://leanpub.com/testingindevops">my book</a> for free, along with curiosity as to whether I am earning any money by doing so. In this post I share some information about purchases of my book, rationale for my current pricing model, and my argument for why you should invest in a copy.<br />
<br />
<h3>
What do people pay?</h3>
At the time of writing this blog, my book has 1,719 readers.<br />
<br />
On average, 1 in 5 people have chosen to pay for my book. Of those who pay, 53% pay the recommended price, 10% pay more, and 37% pay less.<br />
<br />
Based on a rough estimate of the number of hours that I spent writing and revising the book, I calculate that my current revenue gives an income of $8 an hour.<br />
<br />
I incurred minimal cost in the writing process: the LeanPub subscription, gifts for my reviewers and graphic designer, and a celebratory dinner after I published. If I subtract these expenses, my income drops to $5 an hour.<br />
<br />
The minimum wage in New Zealand is $15.75 per hour. This means that based on current sales, I would have been better paid by working in a fast food outlet than in writing my book. I don't think that's a particularly surprising outcome for a creative pursuit.<br />
<br />
<h3>
Why offer it for free?</h3>
I wrote the book to collate ideas and experiences about testing in DevOps into a single place. I wanted to encourage people to develop their understanding of an emerging area of our industry. My primary motivation was to share information and to do that I think it needs to be accessible.<br />
<br />
I have been told that offering something for free creates a perception that it is not valuable. That people are likely to download the book, but not to read it, as they haven't invested in the purchase. That people won't trust the quality of a book that a writer is willing to offer for nothing.<br />
<br />
I weigh these arguments against someone being unable to access the information because they cannot afford it, or someone who is unwilling to enter their payment details online, or someone using cost as an excuse to avoid learning. These reasons make me continue to offer free download as an option.<br />
<br />
I recognise that being able to set a minimum price of zero is a privilege. If I was writing for a living, then this would not be an option available to me. I do worry that my actions impact those writers who would be unable to do the same.<br />
<br />
<h3>
Why pay?</h3>
If my book is available for free, then why should you pay for a copy?<br />
<br />
I believe that I have written a useful resource, but every writer is likely to feel that way about their book! Rather than just taking my word for it, I have collected reader testimonials from around the world including the USA, Netherlands, UK, India, Pakistan, Ecuador, and New Zealand.<br />
<br />
The testimonials endorse the practical content, industry examples, and the breadth of my research in my book. They compliment my writing style as being clear and easy-to-read. They state that there is a wide audience for the book - anyone who holds a role in a software development team.<br />
<br />
Alongside their opinions, you can preview a sample chapter and the table of contents prior to purchase. I hope that all of this information creates a persuasive case for the value contained within the book itself.<br />
<br />
If you believe that the book will be useful to you, then I believe that the recommended price is fair.<br />
<br />
<a href="https://leanpub.com/testingindevops">A Practical Guide to Testing in DevOps</a> is now available on LeanPub.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com2tag:blogger.com,1999:blog-2844510344016016899.post-25532599502887848092017-08-15T16:29:00.000+12:002017-08-15T16:29:19.762+12:00Encouraging testers to share testingIn theory, agile development teams will work together with a cross-functional, collaborative approach to allow work to flow smoothly. In practice, I see many teams with a delivery output that is limited by the amount of work that their testers can get through.<br />
<br />
If testers are the only people who test, they can throttle their team. This can happen because the developers and business people who are part of the team are unwilling to perform testing activities. It can also be the tester who is unwilling to allow other people to be involved in their work. I have experienced both.<br />
<br />
There is a lot of <a href="http://katrinatester.blogspot.co.nz/2015/11/testing-for-non-testers-pathway.html">material to support non-testers in test activities</a>. I feel that there is less to support the tester so that they feel happy to allow others to help them. I'd like to explore three things that could prevent a tester from engaging other people to help with test activities.<br />
<br />
<h3>
Trust</h3>
Do you trust your team? I hope that most of you will work in a team where your instinct is to answer that question with 'yes'. What if I were to ask, do you trust your team <i>to test</i>?<br />
<br />
In the past, I have been reluctant to hand testing activities to my colleagues for fear that they will miss something that I would have caught. I worried that they would let bugs escape. I trusted them in general, but not specifically with testing.<br />
<br />
On reflection, I think my doubt centered on their mindset.<br />
<br />
In exploratory testing, often a non-tester will take a confirmatory approach. This means that if the product behaves correctly against the requirements, it passes testing. It is easy for anyone, regardless of their testing experience, to fall into a habit of checking off criteria rather than interrogating the product for the ways in which it might fail.<br />
<br />
In test automation, it is usually the developer who has the skill to assist. My observation is that developers will generally write elegant test automation. They can also fall into the trap of approaching the task from a confirmatory perspective. Automation often offers an opportunity to quickly try a variety of inputs, but developers don't always look at it from this perspective.<br />
<br />
If you share these doubts, or have others that prevent you from trusting your team to test, how could you change your approach to allow others to help you?<br />
<br />
One thing that I have done, and seen others do, is to have short handovers at either end of a testing task. If a non-tester is going to pick up a test activity, they first spend ten minutes with the tester to understand the test plan and scope. When the non-tester feels that they have finished testing, they spend a final ten minutes with the tester to share their coverage and results.<br />
<br />
These short handovers can start to build trust as they pass the testing mindset to other people in the team. As time passes, the tester should find that their input in these exchanges decreases to a point where the handovers become almost unnecessary.<br />
<br />
<h3>
Identity</h3>
If your role in the team is a tester, this identity can be tightly coupled to test activities. What will you do if you don't test? If other people can test, then why are you there at all?<br />
<br />
I particularly struggled with this as a consultant. I would be placed into an agile development team as a tester, but often the most value that I could deliver would be in encouraging other people to test. This felt a bit like cheating the system by getting other people to do my role. I believe that a lot of people write about the evolving tester role because of this dissonance.<br />
<br />
The clearest way that I have to challenge the tester identity in a constructive way is the concept of <a href="http://katrinatester.blogspot.co.nz/2015/08/elastic-role-boundaries.html">elastic role boundaries</a> that I co-developed with Chris Priest. This concept highlights the difference between tasks and enduring commitments. We can be flexible in taking ownership of small activities while still retaining a specialist identity in a particular area of the team.<br />
<br />
In simpler terms, a colleague helping with a single testing activity does not make them a tester. I do not see a threat to our role in sharing test activities. I believe that a tester retains their identity and standing in the team as a specialist even when they share testing work.<br />
<br />
<h3>
Vulnerability</h3>
The third barrier that I see is that testers are unwilling to ask for help. This isn't an attribute that is unique to testers, many people are unwilling to ask for help. In an agile team, failing to pull your colleagues to help with testing may limit your ability to deliver.<br />
<br />
Even when you think that it is clear that you need help, don't assume that your colleagues understand when you are under pressure. In my experience, people are often blissfully unaware of the workload of others in the team. Even when the day begins with a stand-up meeting, it can be difficult to determine how much work sits behind each task that a person has in progress. Make it clear that you would welcome assistance.<br />
<br />
You may be reluctant to ask for help because you see it as a sign of weakness. It might feel like everyone else in the team can maintain a certain pace while you are always the person who needs a hand. In my experience, asking for help is rarely perceived as weakness by others. More often, I have seen teams respond with praise for bravery, eagerness to alleviate stress, and relief that they have been given permission to help.<br />
<br />
It can also be difficult to share testing when you work in a wider structure alongside other agile teams that have the same composition as your team. You might believe that your team is the only one where the testers cannot handle testing themselves. In this situation, remember <a href="http://katrinatester.blogspot.co.nz/2014/06/the-ratio-myth.html">the ratio myth</a>. There are a lot of variables in determining the ratio of testers to the rest of the team. Sometimes a little bit of development can spin off a lot of testing, or vice versa. Test your assumptions about how other teams are operating.<br />
<br />
If you are vulnerable, you encourage others to behave in the same way. A tester sharing testing can encourage others to seek the help that they need too. If you're unwilling to ask someone for help directly, start by making it clear to your team what your workload is and extend a general invitation for people to volunteer to assist.<br />
<br />
***<br />
<br />
If you work in an agile development team and feel reluctant to share test activities with your colleagues, you might be creating unnecessary pressure on both yourself and your team by limiting the pace of delivery. I'd encourage you to reflect on what is preventing you from inviting assistance.<br />
<br />
If you doubt that your colleagues will perform testing to your standard, try a handover. If you worry about the necessity of your role if you share testing, perhaps elastic role boundaries will explain how specialists can share day-to-day tasks and retain their own discipline. If you are reluctant to ask for help directly, start by making your workload clear so that your team have the opportunity to offer.<br />
<br />
I encourage you to reflect on these themes and how they influence the way that you work, to get more testers to share testing.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com1tag:blogger.com,1999:blog-2844510344016016899.post-66394351841161881912017-07-23T20:24:00.000+12:002017-07-23T20:24:27.032+12:00Exploring the top of the testing pyramid: End-to-end and user interface testingA few weeks ago I found myself in a circular conversation about end-to-end testing. I felt like I was at odds with my colleague, but couldn't figure out why. Eventually we realised that we each had a different understanding of what end-to-end meant in the testing that we were discussing.<br />
<br />
That conversation lead to this poll on Twitter:<br />
<br />
<blockquote class="twitter-tweet" data-lang="en">
<div dir="ltr" lang="en">
Do you use 'end-to-end' test to mean:<br />
1) test of full infrastructure stack, or <br />
2) test of entire customer workflow?</div>
— Katrina Clokie (@katrina_tester) <a href="https://twitter.com/katrina_tester/status/884670200431296512">July 11, 2017</a></blockquote>
<script async="" charset="utf-8" src="//platform.twitter.com/widgets.js"></script>
<br />
The poll results show that roughly a quarter of respondents considered end-to-end testing to be primarily about the infrastructure stack, while the remaining majority considered it from the perspective of their customer workflow. Odds are that this ratio means I'm not the only person who has experienced a confusing conversation about end-to-end tests.<br />
<br />
I started to think about the complexity that is hidden at the top of the testing pyramid. The model states that the smallest number of tests to automate are those at the peak, labelled as end-to-end (E2E) or user interface (UI) tests.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><img src="https://www.mountaingoatsoftware.com/uploads/blog/Testpyramid.jpg" style="margin-left: auto; margin-right: auto; text-align: center;" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://www.mountaingoatsoftware.com/uploads/blog/Testpyramid.jpg">Testing Pyramid by Mike Cohn</a></td></tr>
</tbody></table>
<br />
These labels are used in the test pyramid interchangeably, but end-to-end and user interface testing are not synonymous. I can think of seven different types of automation that might be labelled by one or either of those two terms:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirbTig7i6Zf0E1F2gcPt2BbU3D1gS6adEz4mA8SuC8L3ENSjus5gCQ-PQfr0SqhwUIKyWX34yRhHUmxHXchwGy7qxEMe0kiVi6GmUFnfcaii0RHc1kGfpG7e516p4-25kuFNUDMWQ4W9HD/s1600/UIE2E.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="482" data-original-width="607" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirbTig7i6Zf0E1F2gcPt2BbU3D1gS6adEz4mA8SuC8L3ENSjus5gCQ-PQfr0SqhwUIKyWX34yRhHUmxHXchwGy7qxEMe0kiVi6GmUFnfcaii0RHc1kGfpG7e516p4-25kuFNUDMWQ4W9HD/s1600/UIE2E.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Seven example of user interface and/or end-to-end testing</td></tr>
</tbody></table>
<br />
The table above might be confusing without examples, so here are a few from my own experience.<br />
<div>
<br /></div>
<div>
<b>1. User interface; Not full infrastructure stack; Not entire workflow</b></div>
<div>
<br /></div>
<div>
In my current organisation we have a single page JavaScript application that allows the user to perform complex interactions through modal overlays. We run targeted tests against this application, through the browser, using static responses in place of our real middleware and mainframe systems. </div>
<div>
<br /></div>
<div>
This suite is not end-to-end in any sense. We focus on the front-end of our infrastructure and test pieces of our customer workflows. We could call this suite user interface testing.</div>
<div>
<br /></div>
<div>
<b>2. User interface; Full infrastructure stack; Not entire workflow</b></div>
<div>
<br /></div>
<div>
I previously worked with an application that was consuming data feeds from a third party provider. We were writing a user interface that displayed the results of calculations that relied on these feeds as input. We had a suite of tests that ran through the user interface to test each result in isolation.</div>
<div>
<br /></div>
<div>
Multiple calculations made up a workflow, so the tests did not cover an entire customer journey. However they did rely on the third-party feed being available to return test data to our application, so they were end-to-end from an infrastructure perspective. In this team we used the terms user interface tests and end-to-end tests interchangeably when talking about this suite.</div>
<div>
<br /></div>
<div>
<b>3. User interface; Not full infrastructure stack; Entire workflow</b></div>
<div>
<br /></div>
<div>
In my current organisation we have an online banking application written largely in Java. Different steps of a workflow, such as making a payment, each display separately on a dedicated page. We have a suite of tests that run through the common workflows to test that everything is connected correctly.</div>
<div>
<br /></div>
<div>
This suite executes tests through the browser against the deployed web application, but uses mocked responses instead of calling the back-end systems. It is a suite that targets workflows. We could call this a user interface or end-to-end suite.</div>
<div>
<br /></div>
<div>
<b>4. User interface; Full infrastructure stack; Entire workflow</b></div>
<div>
<br /></div>
<div>
In the same product as the first example, there is another suite of automation that runs through the user interface against the full infrastructure stack to test the entire customer workflow. We interact with the application using test data that is provisioned across all the dependent systems and the tests perform all the steps in a customer journey. We could call this suite user interface or end-to-end testing.</div>
<div>
<br /></div>
<div>
<b>5. No user interface; Full infrastructure stack; Not entire workflow</b></div>
<div>
<br /></div>
<div>
In my current organisation we have a team working on development of open APIs. There is no user interface for the API as a product. The team have a test suite that interacts with their APIs and relies on the supporting systems to be active: databases, middleware, mainframe, and dependencies to other APIs.</div>
<div>
<br /></div>
<div>
These tests are end-to-end from an infrastructure perspective, but their test scope is narrow. They interrogate successful and failing responses for individual requests, rather than looking at the sequence of activities that would be performed by a customer.</div>
<div>
<br /></div>
<div>
<b>6. No user interface; Not full infrastructure stack; Entire workflow</b></div>
<div>
<br /></div>
<div>
Earlier in my career I worked in telecommunications. I used to install, configure, and test the call control and charging software within cellphone networks. We had an in-house test tool that we could use to trigger scripted traffic through our software. This meant that we could bypass the radio network and use scripts to test that a call would be processed correctly, from when a person dialed the number to when they ended the call, without needing to use a mobile device.</div>
<div>
<br /></div>
<div>
These automated tests were end-to-end tests from a workflow perspective, even though there was no user interface and we weren't using the entire network infrastructure.</div>
<div>
<br /></div>
<div>
<b>7. No user interface, Full infrastructure stack; Entire workflow</b></div>
<div>
<br /></div>
<div>
The API team in the fifth example have a second automated suite where a single test performs multiple API requests in sequence, passing data between calls to emulate a customer workflow. These tests are end-to-end from both the infrastructure and the customer experience perspective.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
As these examples illustrate, end-to-end and user interface testing can mean different things depending on the product under test and the test strategy adopted by a team. If you work in an organisation where you label your test automation with one of these terms, it may be worth checking that there is truly a shared understanding of what your tests are doing. Different perspectives of test coverage can create opportunities for bugs to be missed.</div>
<div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com2tag:blogger.com,1999:blog-2844510344016016899.post-38199442493190595622017-07-12T18:01:00.004+12:002017-07-12T18:01:47.987+12:00Test Automation CanvasTest automation frameworks grow incrementally, which means that their design and structure can change over time. As testers learn more about the product that they are testing and improve their automation skills, this can reflect in their code.<br />
<br />
Recently I've been working with a group of eight testers who belong to four different agile teams that are all working on the same set of products. Though the testers regularly meet to share ideas, their test automation code had started to diverge. The individual testers had mostly been learning independently.<br />
<br />
A manager from the team saw these differences emerging and felt concerned that the automated test coverage was becoming inconsistent between the four teams. The differences they saw in testing made them question whether there were differences in the quality of delivery. They asked me to determine a common approach to automated test coverage by running a one hour workshop.<br />
<br />
I am external to the team and have limited understanding of their context. I did not want to change or challenge the existing approach or ideas from this position, particularly given the technical skills that I could see demonstrated by the testers themselves. I suspected that there were good reasons for what they were doing, but perhaps not enough communication.<br />
<br />
I decided that a first step would be to create an activity that would get the testers talking to each other, gather information from these conversations, then summarise the results to share with the wider team.<br />
<br />
To do this, I thought a bit about the attributes of a test automation framework. The primary reason that I had been engaged was to discuss test coverage. But coverage is a response to risk and constraints, so I wanted to know what those were too. I was curious about the mechanics of the suites: dependencies, test data, source control, and continuous integration. I had also heard varying reports about who was writing and reviewing automation in each team, so I wanted to talk about engagement and maintenance of code.<br />
<br />
I settled on a list of nine key areas:<br />
<br />
<ol>
<li><b>RISKS</b> - What potential problems does this suite mitigate? Why does it exist?</li>
<li><b>COVERAGE</b> - What does this suite do?</li>
<li><b>CONSTRAINTS</b> - What has prevented us from implementing this suite in an ideal way? What are our known workarounds?</li>
<li><b>DEPENDENCIES</b> - What systems or tools have to be functional for this suite to run successfully?</li>
<li><b>DATA</b> - Do we mock, query, or inject? How is test data managed?</li>
<li><b>VERSIONING</b> - Is there source control? What is the branching model for this suite?</li>
<li><b>EXECUTION</b> - Is the suite part of a pipeline? How often does it run? How long does it take? Is it stable?</li>
<li><b>ENGAGEMENT</b> - Who created the suite? Who contributes to it now? Who is not involved, but should be?</li>
<li><b>MAINTAINABILITY</b> - What is the code review process? What documentation exists?</li>
</ol>
<br />
I decided to put these prompts into an A3 canvas format, similar to a <a href="https://blog.leanstack.com/why-lean-canvas-vs-business-model-canvas-af62c0f250f0">lean canvas</a> or an <a href="http://jpattonassociates.com/opportunity-canvas/">opportunity canvas</a>. I thought that this format would create a balance between conversation and written record, as I wanted both to happen simultaneously.<br />
<br />
Here is the blank Test Automation Canvas that I created:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTeqPebU9EWbSZFd8CPRSuylUWLTVA8s2kZEge7V92o6pm3PxytlB_0nTaXkW2GwfxPqtN2J8FUKroIi5m3pBxG4pJjb62rPQEgd58h8dwIhudpRSejpSPS0IyvEJ4fqQCBmrWD6_2MA1x/s1600/TestAutomationCanvas.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="914" data-original-width="1361" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTeqPebU9EWbSZFd8CPRSuylUWLTVA8s2kZEge7V92o6pm3PxytlB_0nTaXkW2GwfxPqtN2J8FUKroIi5m3pBxG4pJjb62rPQEgd58h8dwIhudpRSejpSPS0IyvEJ4fqQCBmrWD6_2MA1x/s640/TestAutomationCanvas.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A blank Test Automation Canvas</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
On the day of the workshop, the eight testers identified four separate automation suites under active development. They then self-selected into pairs, with each pair taking a blank canvas to complete.<br />
<br />
It took approximately 20 minutes to discuss and record the information in the canvas. I asked them to complete the nine sections in the order that they are numbered in the earlier list: risks, coverage, constraints, dependencies, data, versioning, execution, engagement, and maintainability.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGdnuUp-6rPsESMKXfSOVUUEYwh5rOTuu1FWaNaLdZQ_fYED2eRCY9s3PAiBTi4jgp1M6MMDwp7h4tw4UDy75KjHJsIJ1w1wGTTJY0dImvA9OMTBzyobj9zIq9WJzhHlrZxDuzlNd-bkCL/s1600/CompletedTestAutomationCanvas.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="578" data-original-width="800" height="460" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGdnuUp-6rPsESMKXfSOVUUEYwh5rOTuu1FWaNaLdZQ_fYED2eRCY9s3PAiBTi4jgp1M6MMDwp7h4tw4UDy75KjHJsIJ1w1wGTTJY0dImvA9OMTBzyobj9zIq9WJzhHlrZxDuzlNd-bkCL/s640/CompletedTestAutomationCanvas.jpg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Examples of completed Test Automation Canvas</td></tr>
</tbody></table>
<br />
Then I asked the pairs to stick their completed canvas on the wall. We spent five minutes circling the room, silently reading the information that each pair had provided. As everyone had been thinking deeply about one specific area, this time was to switch to thinking broadly.<br />
<br />
In the last 15 minutes, we finished by visiting each canvas in turn as a group. I asked two questions at each canvas to prompt group discussion: is anything unclear and is anything missing. This raised a few new ideas, and some misunderstanding between different teams, so notes were added into the canvas'.<br />
<br />
After the workshop, I took the information from the canvas' to create a single A3 summary of all four automation frameworks, plus the exploratory testing that is performed using a separate tool:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMRfPggFocbggZbaDbbJmBkHYtQnL4PLv0ex3C0-9m0Q7w5VOB71B_cJiNbL8KyZTNrDH7wtEgaddu5iqAjWGV_QI-aqEH1XjgVfZXOKWd4TgeyydYuGNBcvp64dHP-RKlozL9PR5rXuCB/s1600/TestAutomationSummary.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="511" data-original-width="800" height="408" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMRfPggFocbggZbaDbbJmBkHYtQnL4PLv0ex3C0-9m0Q7w5VOB71B_cJiNbL8KyZTNrDH7wtEgaddu5iqAjWGV_QI-aqEH1XjgVfZXOKWd4TgeyydYuGNBcvp64dHP-RKlozL9PR5rXuCB/s640/TestAutomationSummary.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Example of Test Automation Summary</td></tr>
</tbody></table>
<br />
In the image above, each row is a different framework. The columns are rationale, coverage, dependencies, mechanics, and improvement opportunities. Within mechanics are versioning, review, pipeline, contributors and data.<br />
<br />
I shared this summary image in a group chat channel for the testers to give their feedback. This led to a number of small revisions and uncovered one final misunderstanding. Now I think that we have a reference point that clearly states the collective understanding of test automation among the testers. The next step is to share this information with the wider team.<br />
<br />
I hope that having this information recorded in a simple way will create a consistent basis for future iterations of the frameworks. If the testers respect the underlying rationale of the suite and satisfy the high-level coverage categories, then slight differences in technical implementation are less likely to create the perception that there is a problem.<br />
<br />
The summary should also support testers to give feedback in their code reviews. I hope that it provides a reference to aid constructive criticism of code that does not adhere to the statements that have been agreed. This should help keep the different teams on a similar path.<br />
<br />
Finally, I hope that the summary improves visibility of the test automation frameworks for the developers, business people, and managers who work in these teams. I believe that the testers are doing some amazing work and hope that this reference will promote their efforts.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com4tag:blogger.com,1999:blog-2844510344016016899.post-14146353853878727492017-06-23T16:01:00.003+12:002017-06-23T16:12:43.244+12:00The Interview RoadshowRecently I have been part of a recruitment effort for multiple roles. In May we posted two advertisements to the market: automation tester and infrastructure tester. Behind the scenes we had nine vacancies to fill.<br />
<br />
This was the first time that I had been involved in recruiting for such a large number of positions simultaneously. Fortunately I was working alongside a very talented person in our recruitment team, <a href="https://www.linkedin.com/in/tburgessedinburgh/?ppe=1">Trish Burgess</a>, who had ideas about how to scale our approach.<br />
<br />
Our recruitment process for testers usually includes five steps from a candidate perspective:<br />
<ol>
<li>Application with CV and cover letter</li>
<li>Screening questions</li>
<li>Behavioural interview</li>
<li>Practical interview</li>
<li>Offer</li>
</ol>
We left the start of the process untouched. There were just over 150 applications for the two advertisements that we posted and, after reading through the information provided, we sent three screening questions to a group of 50 candidates. We asked for responses to these questions by a deadline, at which point we selected who to interview.<br />
<br />
Usually we would run the two interviews separately. Each candidate would be requested to attend a behavioural interview first then, depending on the feedback from that, a practical interview as a second step. Scheduling for the interviews would be agreed between the recruiter, the interviewers, and the candidates on a case-by-case basis.<br />
<br />
As we were looking to fill nine vacancies, we knew that this approach wouldn't scale to the number of people that we wanted to meet. We decided to trial a different approach.<br />
<br />
<h3>
The Interview Roadshow</h3>
Trish proposed that we run six parallel interview streams. To achieve this we would need twelve interviewers available at the same time - six behavioural and six practical - to conduct the interviews in pairs.<br />
<br />
The first hurdle was that we didn't have six people who were trained to run our practical interview, as we usually ran them one-by-one. I asked for volunteers to join our interview panel and was fortunate to have a number of testers come forward. I selected a panel of eight where four experienced interviewers were paired with four new interviewers. The extra pair gave us cover in case of unexpected absence or last minute conflicts.<br />
<br />
We assembled a larger behavioural interview panel too, which gave us a group of 16 interviewers in total. Several weeks in advance of the interview dates, while the advertisements were still live, Trish booked three half-day placeholder appointments into all their diaries:<br />
<ul>
<li>Friday morning 9.30am - 12pm</li>
<li>Monday afternoon 1pm - 3.30pm</li>
<li>Wednesday morning 9.30am - 12pm</li>
</ul>
<br />
In the weeks leading up to the interviews themselves, the practical interviewer pairs conducted practice interviews with existing staff as a training exercise for the new interviewers. We also ran a session with all the behavioural interviewers to make sure that there was a consistent understanding of the purpose of the interview and that our questions were aligned.<br />
<br />
From the screening responses I selected 18 people to interview. We decided to allocate the candidates by their experience into junior, intermediate, and senior streams, then look to run a consistent interview panel for each group. This meant that the same people met all of the junior candidates, and similarly at other levels.<br />
<br />
The easiest way to illustrate the scheduling is through an example.<br />
<br />
For the first session on Friday morning we asked the candidates to arrive slightly before 9.30am. Trish and I met them in the lobby, then took them to a shared space in our office to give them a short explanation of how we were running the interviews. I also took a photo of each candidate, which I used later in the process when collating feedback.<br />
<br />
Then we delivered the candidates to their interviewers. We gave the interviewers a few minutes together prior to the candidate arriving, for any last minute preparation, so the interviews formally began ten minutes after the start of their appointment (at 9.40am).<br />
<br />
Here is a fictitious example schedule for the first set of interviews:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirFFrkCCNPXz75UwwOWMcxq863NlEuk6hYIQ3csciIVKzjblNo-_7iVSwymh11MSMv1-YxRyA8UmJRAJa7-axsgM2c4Vd-28BGmvONvc2uCU0lU7wVN3fSrJTznad-AwoG2ALlxn4PdmUl/s1600/ExampleInterviewScheduleOne.png" imageanchor="1"><img border="0" data-original-height="499" data-original-width="602" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirFFrkCCNPXz75UwwOWMcxq863NlEuk6hYIQ3csciIVKzjblNo-_7iVSwymh11MSMv1-YxRyA8UmJRAJa7-axsgM2c4Vd-28BGmvONvc2uCU0lU7wVN3fSrJTznad-AwoG2ALlxn4PdmUl/s1600/ExampleInterviewScheduleOne.png" /></a></div>
<br />
<br />
The first interviews finished by 10.40am, at which point the interviewers delivered the candidate back to the shared space. We provided morning tea and they had 20 minutes to relax prior to their next interview at 11am. Trish and I were present through the break and delivered the candidates back to the interviewers.<br />
<br />
Here is a fictitious example schedule for the second interviews:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7t1Jv6fpcQHZqampxtTwYyFE69O0Bv3kigJ3FL5j6ghq4tD527kRQzIEtAuHjlHGpu3moGC7UiHA5GhrdCJI9gf8WBg6D_VQ2JmsguE2b9jUJyqXrPRDi4kzLSMipMrUaU5VnTmlAGPh_/s1600/ExampleInterviewScheduleTwo.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="499" data-original-width="602" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7t1Jv6fpcQHZqampxtTwYyFE69O0Bv3kigJ3FL5j6ghq4tD527kRQzIEtAuHjlHGpu3moGC7UiHA5GhrdCJI9gf8WBg6D_VQ2JmsguE2b9jUJyqXrPRDi4kzLSMipMrUaU5VnTmlAGPh_/s1600/ExampleInterviewScheduleTwo.jpg" /></a></div>
<br />
<br />
The second interview session finished by 12pm, at which point the interviewers would farewell the candidate and collate their feedback from both sessions.<br />
<br />
<h3>
Retrospective Outcomes</h3>
The main benefit to people involved in the interview roadshow was that it happened within a relatively short time frame. Within four working days we conducted 36 interviews. As a candidate, this meant fast feedback on the outcome. As an interviewer, it meant less disruption of my day-to-day work.<br />
<br />
We were happily surprised that we had 18 candidates accept the interview offer immediately. We had assumed that some people would be unavailable, as when we schedule individual interviews there is a lot of back-and-forth. Trish had given an indication of the interview schedule when asking the screening questions. The set times seemed to motivate candidates make arrangements so that they could attend.<br />
<br />
By running two interviews in succession, the candidate only had to visit our organisation once. In our usual process recruitment process a candidate might visit twice: the first time for a behavioural interview and the second for a practical interview. One trip means fewer logistical concerns around transport, childcare, and leaving their current workplace.<br />
<br />
On the flip side, running two interviews in succession meant that people had to take more time away from their current role in order to participate. We had feedback from one candidate that it was a long time for them to spend away from the office.<br />
<br />
There were three areas that we may look to improve.<br />
<br />
Having six candidates together in the pre-interview briefing and refreshment break was awkward. These were people who didn't know each other, were competing for similar roles, and were in the midst of an intense interview process. The conversation among the group was often stilted or non-existent - though perhaps this is a positive thing for candidates who need silence to recharge?<br />
<br />
In our usual process the hiring manager would always meet the person that was applying for the vacancy in their team. In this situation, we had individual hiring managers who were looking for multiple roles at multiple levels - junior, intermediate, senior. With the interview roadshow approach, we had some successful candidates who were proposed to a role where the hiring manager hadn't met them. Though this worked well for us, as there was a high degree of trust among the interviewers, it may not in other situations.<br />
<br />
The other thing that became difficult in comparison to our usual approach was dealing with internal applicants. We had multiple applications from within the organisation and it was harder to handle these in a discrete way with such a large panel of interviewers. The roadshow approach to interviewing also made these people more visible in their aspirations, though we tried to place them in rooms that were away from busy areas.<br />
<br />
Overall, I don't think that we could have maintained the integrity of our interview process for such a large group of candidates by any other means. The benefits of scaling to an interview roadshow outweigh the drawbacks and it is something that I think we will adopt again in future, as required.<br />
<br />
I personally had a lot of fun in collating the candidate feedback, seeing which candidates succeeded, and suggesting how we could allocate people to teams. Though it is always hard to decline the candidates that are unsuccessful, I think we have a great set of testers coming in to join us as a result of this process and I'm looking forward to working with them.<br />
<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com3tag:blogger.com,1999:blog-2844510344016016899.post-48536660498984751552017-06-08T20:46:00.001+12:002017-06-08T20:48:10.202+12:00Using SPIN for persuasive communicationI can recall several occasions early in my career where I became frustrated by my inability to persuade someone to my way of thinking. Reflecting on these conversations now, I can still bring to mind the feelings of agitation as I failed. I thought I had good ideas. I would make my case, sometimes multiple times, to no avail. I was simply not very good at getting my way.<div>
<br /></div>
<div>
The frustration came from my own failure, but I was also frustrated by seeing others around me succeed. They could persuade people. I couldn't figure out why people were listening to them, but not me. I was unable to spot the differences in our approach, which meant that I didn't know what I should change.<br /><div>
<br /></div>
<div>
Some years later, in my role as a test consultant, I had the opportunity to attend a workshop on the fundamentals of sales. The trainer shared an acronym, SPIN, which is a well-known sales technique developed in the late 1980s.</div>
</div>
<div>
<br /></div>
<div>
SPIN was a revelation to me and I believe that it has significantly improved my ability to persuade. In this post I'll explain what the acronym stands for and give examples of how I apply SPIN in a testing context.</div>
<div>
<br /></div>
<h3>
What is SPIN?</h3>
<div>
SPIN stands for situation, problem, implication, and need.</div>
<div>
<br /></div>
<div>
A SPIN conversation starts with explaining what you see. Describe the situation and ask questions to clarify where you're unsure. Avoid expressing any judgement or feelings - this should be a neutral account of the starting point.</div>
<div>
<br /></div>
<div>
Then discuss the problems that exist in the current state. Where are the pain points? Share the issues that you see and draw out any that you have missed. Try to avoid making the problems personal, as this part of the conversation can be derailed into unproductive ranting.</div>
<div>
<br /></div>
<div>
Next, think about what the problems mean for the business or the team. Consider the wider organisational context and question how these problems impact key measures of your success. Where is the real cost? What is the implication of keeping the status quo.</div>
<div>
<br /></div>
<div>
Finally, describe what you think should happen next. This is the point of the conversation where you present your idea, or ideas, for the way forward. What do you think is needed?</div>
<div>
<br /></div>
<div>
To summarise in simple terms, the parts of SPIN are:</div>
<div>
<ul>
<li>Situation - What I see</li>
<li>Problem - Why I care</li>
<li>Implication - Why you should care</li>
<li>Need - What I think we should do</li>
</ul>
<div>
<br /></div>
</div>
<h3>
A SPIN example</h3>
<div>
My first workplace application of SPIN was at a stand-up meeting. I was part of a team that were theoretically running a fortnightly scrum process. In reality it was a water-scrum-fall where testing kept being flooded at the end of each sprint.</div>
<div>
<br /></div>
<div>
I had been trying, unsuccessfully, to change our approach to work. Prior to this particular stand-up I sat down and noted some thoughts against SPIN. With my preparation in mind, at the stand-up I said something like:</div>
<div>
<i><br /></i></div>
<div>
<i>"It seems like the work isn't being delivered to testing until really late in the sprint, and then everything arrives at once. This means that we keep running out of time for testing, or we make compromises to finish on time. </i></div>
<div>
<i><br /></i></div>
<div>
<i>If we run out of time, then we miss our sprint goal. If we compromise on test coverage, then we all doubt what we are delivering. Both of these outcomes have a negative impact on our team morale. At the end of each fortnight I feel like we are all pretty flat. </i></div>
<div>
<i><br /></i></div>
<div>
<i>I'd like us to try having developers work together on tasks so that we push work through the process, rather than individual developers tackling many tasks in the backlog at once. That way we should see work arrive in testing more regularly through the sprint. What do you think?"</i></div>
<div>
<br /></div>
<div>
To my amazement, this was the beginning of a conversation where I finally convinced the developers to change how they were allocating work.</div>
<div>
<br /></div>
<div>
Did you spot the SPIN in that example?</div>
<div>
<br /></div>
<div>
<ul>
<li><b>Situation </b>- <i>What I see </i>- It seems like the work isn't being delivered to testing until really late in the sprint, and then everything arrives at once.</li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Problem </b>- <i>Why I care</i> - This means that we keep running out of time for testing, or we make compromises to finish on time. </li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Implication </b>- <i>Why you should care</i> - If we run out of time, then we miss our sprint goal. If we compromise on test coverage, then we all doubt what we are delivering. Both of these outcomes have a negative impact on our team morale. At the end of each fortnight I feel like we are all pretty flat. </li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Need </b>- <i>What I think we should do </i>- I'd like us to try having developers work together on tasks so that we push work through the process, rather than individual developers tackling many tasks in the backlog at once. That way we should see work arrive in testing more regularly through the sprint.</li>
</ul>
</div>
<div>
<br /></div>
<div>
In the first few conversations where I applied SPIN, I had to spend a few minutes preparing. I would write SPIN down the side of a piece of paper and figure out what I wanted to say in each point. This meant that I could confidently deliver my message without feeling like I was citing the different steps of a sales technique.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZfZ1VIIc9c3dSIpSsvz9GozLG3QDubAUVVNkVtR-nqp3Z3dbKv3r_TGBaDkGgp2djwjOlN5CCH_7aqN3FmHvP5YtXIhBwHH7-TXFnzamlM7JNytzigJEycwCqfFpGfraDOMuf5Zi-_Mox/s1600/SPINexample.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="300" data-original-width="300" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZfZ1VIIc9c3dSIpSsvz9GozLG3QDubAUVVNkVtR-nqp3Z3dbKv3r_TGBaDkGgp2djwjOlN5CCH_7aqN3FmHvP5YtXIhBwHH7-TXFnzamlM7JNytzigJEycwCqfFpGfraDOMuf5Zi-_Mox/s200/SPINexample.jpg" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Preparing for a conversation using SPIN</td></tr>
</tbody></table>
<div>
<br /></div>
<h3>
SPIN in a retrospective</h3>
<div>
As I became confident with structuring my own conversations using SPIN, I started to observe the patterns of success for others. Retrospectives provided a lot of data points for both successful and unsuccessful attempts at persuasion.</div>
<div>
<br /></div>
<div>
Many retrospective formats encourage participants to write their thoughts on sticky notes. When prompted with a question like "What could we do differently" I noticed that different people would usually note down their ideas using a single piece of SPIN. Where an individual consistently chose the same piece of SPIN in their note taking, they created a perception of their contributions among the audience. </div>
<div>
<br /></div>
<div>
Let me explain this with an example. Imagine a person who takes the prompt "What could we do differently" and writes three sticky notes:</div>
<div>
<ol>
<li>We all work from home on Wednesday</li>
<li>The air conditioning is too cold</li>
<li>Our product owner was sick this week</li>
</ol>
</div>
<div>
All three are observations, the 'situation' of SPIN that describe what they see. Though they might be thinking more deeply about each, without any additional information the wider team are probably thinking "so what?"</div>
<div>
<br /></div>
<div>
Similarly, if your sticky notes are mostly problems, then your team might think that you're whiny. If your sticky notes are mostly solutions, then your team might think that you're demanding. In the absence of a rounded explanation your contribution can be misinterpreted.</div>
<div>
<br /></div>
<div>
I'm not suggesting that you write every retrospective sticky note using the SPIN format!</div>
<div>
<br /></div>
<div>
I use SPIN in a retrospective in two ways. Firstly to remind myself to vary the type of written prompt that I give myself when brainstorming on sticky notes, to prevent the perception that can accompany a consistent approach. Secondly to construct a rounded verbal explanation of the ideas that I have, so I have the best chance of persuading my team.</div>
<div>
<br /></div>
<h3>
SPIN with gaps</h3>
<div>
There may be cases where you cannot construct a whole SPIN.</div>
<div>
<br /></div>
<div>
Generally I consider the points of SPIN with an audience in mind. When I think about implication, I capture reasons that the person, or people, that I am speaking to should care about what I'm saying. If I'm unable to come up with an implication, this is usually an indicator that I've picked the wrong audience. When I can't think of a reason that they should care, then I need to pick someone else to talk to.</div>
<div>
<br /></div>
<div>
Sometimes I can see a problem but I'm not sure what to do about it. When this happens, I use the beginning of SPIN as a way to start a conversation. I can describe the situation, problems, and implications, then ask others what ideas they have for improvement. It can be a useful way to launch a brainstorming activity.</div>
<div>
<br /></div>
<h3>
Conclusion</h3>
<div>
SPIN is one facet of persuasive communication. It offers guidance on what to say, but not how to say it. In addition to using SPIN, I spent a lot of time considering the delivery of my arguments in order to improve the odds of people accepting my ideas.</div>
<div>
<br /></div>
<div>
Though I rarely have to write notes in the SPIN format as I did originally, I still use SPIN as a guide to structure my thinking. SPIN stops me from jumping straight to solutions and helps me to consider whether I have the right audience for my ideas. I've found it a valuable technique to apply in a variety of testing contexts.</div>
<div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com3tag:blogger.com,1999:blog-2844510344016016899.post-48869025996596364122017-05-11T21:25:00.001+12:002017-05-11T21:27:59.439+12:00Three styles of automationAt <a href="http://lets-test.com/?page_id=5191">Let's Test</a> next week I have the privilege of presenting a hands-on workshop titled "Three styles of automation". The abstract for the session reads:<br />
<div style="padding-left: 30px;">
<i><br />A lot of people use Selenium WebDriver to write their UI automation. But the specific implementation language and coding patterns will differ between organisations. Even within the same organisation, a set of front-end tests can look different between different products.<br /><br />Katrina will share three different approaches to Java-based UI automation using Selenium WebDriver from her organisation. She will explain the implementation patterns, the reasons that differences exist between repositories, and the benefits and drawbacks of each approach.<br /><br />Participants will download three different suites that each implement a simple test against the same web application. Once they have a high-level understanding of each code base, they will have the opportunity to execute and extend the test suite with targeted hands-on exercises.</i>
</div>
<br />
In this post I share the code and resources that I'll be using in this workshop. Though you won't get the same level of support or depth of understanding as a workshop participant, I hope you will find something of interest.<br />
<br />
<h3>
Background</h3>
<div>
These three automated suites are written against a tool provided by the New Zealand tax department -the <a href="http://brc.ird.govt.nz/web-determinations/startsession/PAYE_Calculator/en-GB/Attribute~b1%40Rules_ProceduralRules_VisibilityRules_doc~global~global">IRD PAYE and Kiwisaver deductions calculator</a>. Each suite contains a single test that enters the details for a test employee and confirms that PAYE is calculated correctly.</div>
<div>
<br />
Each suite is reflective of a framework that we use for test automation in my organisation: Pirate, AAS, or WWW. These public versions do not contain any proprietary code; they've been developed as a training tool to provide a safe place for testers to experiment.<br />
<div>
<br /></div>
<div>
Each training suite was created almost a year ago, which means they're showing their age. They still run with Selenium 2 against Firefox 45. We're in the process of upgrading our real automation to Selenium 3, and switching to Chrome, but these training suites haven't been touched yet.</div>
</div>
<br />
The three suites illustrate the fundamental differences in how we automate for three of our products. Some of these differences are based on conscious design decisions. Some are historic differences that would take a lot of work to change. The high-level implementation information about each suite is:<br />
<br />
Pirate <br />
<ul>
<li>Uses Selenium Page Factory to initialise web elements in page objects </li>
<li>Methods that exit a page will return the next page object </li>
<li>Has an Assertion Generator to automatically write assertions </li>
<li>Uses rules to trigger @Before and @After methods for tests </li>
</ul>
<br />
AAS <br />
<ul>
<li>Uses a fetch pattern to retrieve page objects </li>
<li>Provides WebDriverUtils to safely retrieve elements from the browser </li>
<li>Tests are driven by an HTML concordion specification </li>
<li>Uses inheritance to trigger @Before and @After methods for tests </li>
</ul>
<br />
WWW <br />
<ul>
<li>Uses Selenide as a wrapper for Selenium to simplify code in page objects </li>
<li>Uses a Selenide Rule to configure Web Driver </li>
<li>Uses @Before and @After methods in the tests directly</li>
</ul>
<div>
<br /></div>
<h3>
Installation</h3>
<div>
There are <a href="https://docs.google.com/document/d/1ACBw7tP3tkxqNlMXIAU4b38JN_AWisZ0Vc87PyW9G9Q/edit#heading=h.si1azlx1tyde">pre-requisite installation instructions</a> to help you get the code running on your own machine. To get the tests executing within each framework, you may have to download and install:<br />
<br />
<ul>
<li>git</li>
<li>Java</li>
<li>Firefox 45</li>
<li>An IDE e.g. IntelliJ</li>
</ul>
You can download the three suites from GitHub. If you haven't used GitHub before, you may need to create an account in order to clone <a href="https://github.com/katrinaclokie">the three repositories</a>.<br />
<br />
<h3>
Comparison</h3>
<div>
The beauty of these training frameworks is that it is easy to compare the three implementations. If you are familiar with the way that one works, you can easily map your understanding to learn the others. </div>
<div>
<br /></div>
<div>
In each suite you will see different page object implementation. The <span style="font-family: "courier new" , "courier" , monospace;">enterUserAndTaxDetails </span>method in the <span style="font-family: "courier new" , "courier" , monospace;">UserAndTaxYearPage</span> is a good example of the different approaches to finding and using web elements:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC5MgKQ1lpHFkmEcN8Un78fHCDaTUA203XwFPy_xeDTvjOQuZ0NT8MLI_F8tMjFpwu8eM2aohN84buw8e8o6XXchMqxdZu9Sqw3q-yrvgHdPlYcbwhu9LTm0PjyqPhLfBwWH2w4-Ck24TM/s1600/SameCodeThreeWays_PageObjects.png" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC5MgKQ1lpHFkmEcN8Un78fHCDaTUA203XwFPy_xeDTvjOQuZ0NT8MLI_F8tMjFpwu8eM2aohN84buw8e8o6XXchMqxdZu9Sqw3q-yrvgHdPlYcbwhu9LTm0PjyqPhLfBwWH2w4-Ck24TM/s640/SameCodeThreeWays_PageObjects.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The same functionality implemented in three different ways</td></tr>
</tbody></table>
<div>
<br /></div>
<div>
There are different types of assertions in the tests. Pirate assertions are created by an automated assertion generator, in AAS the English language <a href="http://concordion.org/">concordion</a> specification holds the assertions, and WWW make use of simple JUnit asserts.</div>
<div>
<br /></div>
<div>
The navigation through the application varies too. Pirate passes page objects, AAS implements fetch methods, and WWW simply use the <a href="http://selenide.org/">Selenide</a> open method. </div>
<br />
These differences are apparent when reading through the code, but to really understand them you are best to get hands-on. As a starting point, try adding to the existing test so that a 3% employee KiwiSaver
deduction is selected, then make sure that deduction is
reported correctly in the summary.<br />
<div class="MsoNormal">
<o:p></o:p></div>
<br />
<h3>
Conclusion</h3>
I don't claim that any of these frameworks are a best practice for UI automation. However, they represent a real approach to tests that are regularly executed in continuous integration pipelines in a large organisation. I wish that more people were able to share this level of detail.</div>
<div>
<br /></div>
<div>
I find the similarities and differences, and the rationale for each, to be fascinating. Given the variety within our team, it makes me wonder about the variety worldwide. There are so many different ways to tackle the same problem.</div>
<div>
<br /></div>
<div>
<div>
This is a whistle-stop tour of a three hour workshop. I hope to see some of you at <a href="http://lets-test.com/">Let's Test</a> to have the opportunity to explain in person! If you cannot attend and have questions, or suggestions for improvements in our approach, please leave a comment below or ask <a href="https://twitter.com/katrina_tester">via Twitter</a>.<br />
<br /></div>
</div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com11tag:blogger.com,1999:blog-2844510344016016899.post-31645110947862019782017-04-22T22:15:00.000+12:002017-05-11T20:06:59.982+12:00Introducing testers to developersWhile completing my Computer Science degree, I created a lot of software without testers. At the end of my qualification, I searched for my first role as a developer with a limited understanding of the other roles in IT that I would work alongside. I didn't know what a tester was, what they did, or how they might help me. I don't think this is an unusual position for a graduate developer to be in. <div>
<div>
<br />At my first job we didn't have testers. Had I spent a longer period in that company, I may have become an experienced developer who still didn't understand what testers were. There are a lot of organisations that create software without employing dedicated testers. I don't think that a poor understanding of testers is an unusual position for an experienced developer to be in.<br /><br />Developers who have never worked with testers are likely to have an understanding of testing, but as an activity rather than a role. Testing happens as part of their work rather than being lead by a separate person. Why would testing be delegated when a developer can successfully write and release quality software on their own?<br /><br /><div>
If you are a developer with this mindset or history, it can be really challenging to encounter a tester for the first time. Allow me to introduce you through analogy.</div>
</div>
<div>
<br /></div>
<h3>
No Tester</h3>
<div>
A restaurant chef is a creator. They make a meal that is delivered to a customer. Quality is determined by the skill of the chef, the quality of their ingredients, and the practices that they follow.</div>
<div>
<br /></div>
<div>
In many restaurants the food is made, delivered to the table, and consumed. The earliest feedback that the chef receives is from the customer. In the event that they have a negative opinion of the meal, it can be difficult to fix the problem. The damage is often done.</div>
<div>
<br /></div>
<div>
There are parallels to software development without testers. Quality is determined by the skill of the developers, the quality of their platform, and the practices that they follow. Feedback comes directly from the customer, who can be unforgiving.</div>
<div>
<br /></div>
<div>
Restaurants can succeed in this model, as can software development.</div>
<div>
<br /></div>
<h3>
Waterfall Tester</h3>
<div>
As a student, I worked part-time as a waitress. In one of the restaurants where I worked the restaurant manager used a small workstation that was located directly outside the double doors to the kitchen.</div>
<div>
<br /></div>
<div>
The chef in this restaurant was temperamental and territorial, so the restaurant manager would rarely venture into the kitchen. However, during meal service, she would intercept the wait staff as we moved plates from the kitchen to our customers.</div>
<div>
<br /></div>
<div>
If the meals for a table were ready at different times, she would hold service until they could be delivered together. If the presentation of a dish was sloppy, she would tidy it up. In the rare event that she was unhappy with portion size or the quality of cooking, she would return the plate to the chef. </div>
<div>
<br /></div>
<div>
The chef didn't like having his food returned. He would often over-correct in spite. You think the sweet and sour pork portion was too small?! Then you'll get double-sized dishes for the next hour! However he would ultimately settle into delivering a more appropriate size of meal.</div>
<div>
<br /></div>
<div>
On reflection, this restaurant manager was my first experience with a separate tester. Once development was complete she examined the final product. With a fresh perspective, she could identify problems that the kitchen had missed. </div>
<div>
<br /></div>
<div>
The restaurant manager contributed to the quality of the product through small actions of her own or by reporting problems back to the chef so that he could make improvements. He may not have always responded well to these reports, but he did alter the meals, which improved what the customer received.</div>
<div>
<br /></div>
<div>
Testers bring the same contributions to software development - a fresh perspective to identify problems, fast feedback from someone with a customer focus, and the ability to make their own small contributions to the overall quality of the customer experience.</div>
</div>
<div>
<br /></div>
<h3>
Agile Tester</h3>
<div>
The restaurant manager was improving quality of a finished dish. What if there was a way to improve the meal as it was being prepared? Then a chef could adapt as they worked, rather than trying to alter their end result.</div>
<div>
<br /></div>
<div>
Sometimes there are multiple elements of a meal cooking at the same time. As a chef, it can be difficult to keep track and sometimes things get burned. What if someone else had set a timer, noticed when it had lapsed, and could intervene?</div>
<div>
<br /></div>
<div>
Sometimes there ingredients in a dish that look similar and might be easily confused. This week I prepared a meal with two spices: tagine spice mix and couscous spice mix. What if someone else had noticed when I opened the wrong packet?</div>
<div>
<br /></div>
<div>
Sometimes there is a requirement for consistency between meals. How can a chef be sure that the crème brûlée they create today is the same as yesterday? What if someone else could check that the recipe, portion size, and presentation met a minimum standard?</div>
<div>
<br /></div>
<div>
Hypothetically, this all sounds reasonable. Realistically, if you're a chef who likes to cook alone, this might sound like a nightmare. Perhaps you would rather have to throw away the occasional plate of food that was burned or tasted strange than accommodate another person in your creative space.</div>
<div>
<br /></div>
<div>
There is a parallel to the role of a tester in agile development. Success relies on collaborative working relationships that can be difficult to negotiate. Dealing with feedback from a tester can be frustrating, particularly when a developer is not used to receiving it. The tester will need to explain and demonstrate how their perspective can contribute to an improved product. </div>
<div>
<br /></div>
<div>
It can be difficult for the developer to adapt their approach and accommodate feedback. If they find that the information from a tester is not relevant, timely, or delivered in a constructive manner, then they should let them know. It takes time to form a useful working relationship and investment is required from both sides.</div>
<div>
<br /></div>
<h3>
Every Tester</h3>
<div>
Whether a tester is contributing in agile or waterfall, they want to help deliver the best possible product to customers. If you are a developer who is encountering a tester for the first time, the first thing to assume is positive intent.</div>
<div>
<br /></div>
<div>
Testers bring a fresh perspective to identify problems, fast feedback from someone with a customer focus, and the ability to make their own small contributions to the overall quality of the customer experience. Allow them to influence the product. Ultimately they help to create a better outcome than what you could achieve alone.</div>
<div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com5tag:blogger.com,1999:blog-2844510344016016899.post-34734936429400353862017-04-05T19:52:00.004+12:002017-04-05T19:53:32.504+12:00Test Coaching Competency FrameworkA few weeks ago I was wondering how to explain the skills required for a Test Coach role. <a href="https://twitter.com/TobySinclair_">Toby Sinclair</a> introduced me to <a href="https://twitter.com/lyssaadkins">Lyssa Adkin</a>'s Agile Coaching Competency Framework. It's a simple diagram of the core experience and skills that an agile coach might have:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzbqh7zrH8GzV0R4Mf5u-D0gC9pEwiCMRoRS9qrMHeTj6Nm60gFqXI3dfj_yuawnSn9-c-hKOnpBffPBCYnLNGNffRQrEaautognhqNGiFjXOFFP9RHu5cSQfHTZgYsgVhxjkRiBTwbbcM/s1600/AgileCoachingCompetencyModel.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzbqh7zrH8GzV0R4Mf5u-D0gC9pEwiCMRoRS9qrMHeTj6Nm60gFqXI3dfj_yuawnSn9-c-hKOnpBffPBCYnLNGNffRQrEaautognhqNGiFjXOFFP9RHu5cSQfHTZgYsgVhxjkRiBTwbbcM/s400/AgileCoachingCompetencyModel.png" width="400" /></a></td></tr>
<tr><td class="tr-caption">Ref: <a href="https://www.slideshare.net/agileindia/hiring-or-growing-right-agile-coach-by-lyssa-adkins-and-michael-spayd">Hiring or Growing Right Agile Coach by Lyssa Adkins and Michael Spayd</a></td></tr>
</tbody></table>
<br />An individual can take this diagram and complete a relative assessment of themselves or others by shading in each slice, for example:<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK5f-mUCS9T1OgTD76kQGH_w15950Z74oTJthMFRAvNngv_iFpkVe3qP2A2tBSVwGhhn2CIEL2hfGppNdi36Z1Aol-996s0lVT07Ah15R63gn4ul1ozfontJ-6UmzVZpMaBgkf1wjYC_xY/s1600/AgileCoachingCompetencyModel_Example.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK5f-mUCS9T1OgTD76kQGH_w15950Z74oTJthMFRAvNngv_iFpkVe3qP2A2tBSVwGhhn2CIEL2hfGppNdi36Z1Aol-996s0lVT07Ah15R63gn4ul1ozfontJ-6UmzVZpMaBgkf1wjYC_xY/s400/AgileCoachingCompetencyModel_Example.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Ref: <a href="https://www.slideshare.net/agileindia/hiring-or-growing-right-agile-coach-by-lyssa-adkins-and-michael-spayd" style="font-size: 12.8px;">Hiring or Growing Right Agile Coach by Lyssa Adkins and Michael Spayd</a></td></tr>
</tbody></table>
<div>
<br /><div>
In <a href="https://www.slideshare.net/agileindia/hiring-or-growing-right-agile-coach-by-lyssa-adkins-and-michael-spayd">a presentation about the framework</a>, Lyssa Adkins and <a href="https://twitter.com/mspayd">Michael Spayd</a> explain how it can be used as a self-assessment tool, for hiring and developing agile coaches, or to help communities of agile coaches to grow.</div>
</div>
<div>
<br /></div>
<div>
I like the simplicity of the framework and could see an opportunity to apply it to the testing domain to explain test coaching. As a small disclaimer, I have never heard Lyssa and Michael explain their framework in person, so I'm not entirely sure how true my interpretation is to their original intent.</div>
<div>
<br /></div>
<h3>
Test Coaching Competency Framework</h3>
<div>
The test version of the framework is almost identical to the original:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRkzQR8SdV1w7DQNVQTisesmP-p3cnLj9l0c7U3iJnrxQxWgpJSvHExXOA5xEaRqaneWNoQF7v2W2-jlB4r5aPz0E1Eoy95WnT1IgFTlQJb1taecXJDlR_wmn-I6-9YBRZfcpm7pXaslYQ/s1600/TestCoachCompetencyModel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRkzQR8SdV1w7DQNVQTisesmP-p3cnLj9l0c7U3iJnrxQxWgpJSvHExXOA5xEaRqaneWNoQF7v2W2-jlB4r5aPz0E1Eoy95WnT1IgFTlQJb1taecXJDlR_wmn-I6-9YBRZfcpm7pXaslYQ/s400/TestCoachCompetencyModel.png" width="400" /></a></div>
<div>
<br /></div>
<div>
Starting at the top, testing practitioner reflects the relevant experience that a person has. Different people will have different interpretations of what this means. Shading of this slice might reflect number of years in industry, number of organisations, variety of roles, etc.</div>
<div>
<br /></div>
<div>
If occupying a role is reflected at the top of the framework, the knowledge that you build through that experience is reflected at the bottom as technical, business, and transformation mastery. In simple terms, shading of the practitioner slice shows what you've done, shading in the mastery slices shows what you've learned as a result.</div>
<div>
<br /></div>
<div>
Technical mastery in testing might include:</div>
<div>
<ul>
<li>test automation frameworks</li>
<li>coding skills</li>
<li>continuous integration and delivery tools</li>
<li>development practices e.g. source control, code review</li>
<li>basic understanding of solution architecture</li>
</ul>
<div>
Business mastery in testing might include:</div>
</div>
<div>
<ul>
<li>shift-left testing skills e.g. Lean UX, BDD</li>
<li>user experience, usability, and accessibility testing</li>
<li>targeted exploratory testing and development of test strategy</li>
<li>business intelligence and analytics</li>
<li>relevant domain knowledge</li>
</ul>
<div>
Transformation mastery in testing might include being a leader or participant in:</div>
</div>
<div>
<ul>
<li>switching test approach e.g. waterfall to agile</li>
<li>continuous improvement in development practices</li>
<li>engaging beyond the development team e.g. DevOps</li>
<li>change management</li>
</ul>
<div>
<div>
<br /></div>
<div>
On the left of the framework are the skills that are important to communicate mastery to others: training and mentoring. The distinction is in the size of the audience. Training is prepared material that is delivered to a group of people. Mentoring is transfer of knowledge from one individual to another. Both training and mentoring require the coach to have knowledge of the topic that they're helping someone with.</div>
<div>
<br /></div>
<div>
On the right of the framework are the skills that are important to enable others to find their own solutions: facilitating and coaching. The same distinction in size is present: facilitating is to a group of people and coaching is between individuals. A person conducting these activities is guiding a group or an individual towards the discovery of their own solutions. Skill in facilitation and coaching does not come from personal mastery of a topic, but instead in developing the ability to draw the best from people through questioning.</div>
<div>
<br /></div>
<h3>
Using the framework</h3>
<div>
I needed to be able to explain the skills of a Test Coach in order to recruit. I have used my version of this framework as an activity in eight Test Coach interviews so far. I draw the framework on a piece of paper, explain what is included in each slice as above, then ask the candidate to assess themselves and talk about their decisions.</div>
<div>
<br /></div>
<div>
Here's an example of how one candidate responded:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWF1LW7HY59DvMvZLNdz24AYUwuz3rE4iR08-RY6mRxw63jip0WJFu-P7wja8hKBbAqeaB7309VRnJvytFHXaJ5BveQpE7wN8LdrtFQGhJ1IbVq18uUCrSeaymrZ-RexDixkLBkHmlrLFq/s1600/TestCoachCompetencyModel_Example.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="365" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWF1LW7HY59DvMvZLNdz24AYUwuz3rE4iR08-RY6mRxw63jip0WJFu-P7wja8hKBbAqeaB7309VRnJvytFHXaJ5BveQpE7wN8LdrtFQGhJ1IbVq18uUCrSeaymrZ-RexDixkLBkHmlrLFq/s400/TestCoachCompetencyModel_Example.jpg" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
I've found the test coaching competency framework is an excellent alternative to the general prompt of "tell us a bit about yourself". I use it at the start of the interview to give more structure to the opening conversation and clearly illustrate what I'm interested in hearing from the candidate. </div>
<div>
<br /></div>
<div>
I've found that it empowers people to talk more freely about both their achievements and opportunities for growth. I like that it gives control of the conversation to the candidate for a period of time rather than asking them to respond to specific questions for the entire interview.</div>
</div>
</div>
<div>
<br /></div>
<div>
What do I do with the results? I haven't been seeking an individual who is strong in every area, but I am looking to build a team of coaches with complementary skills and experiences. The shaded frameworks are helpful in determining how a group of individuals might become a team. Once a team is formed, I imagine that this information could help guide initial allocation of work and allow me to identify some opportunities for cross-training.</div>
<div>
<br /></div>
<div>
I have found the test coaching competency framework useful and would recommend it to others who are looking to build test coaching teams.</div>
<div>
<br /></div>
Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com2tag:blogger.com,1999:blog-2844510344016016899.post-12258748047095169982017-03-12T16:01:00.000+13:002017-03-12T16:01:17.174+13:00How do you hire a junior tester?I recently participated in a workshop that was co-hosted by the New Zealand Qualifications Authority (NZQA) and New Zealand IT Professionals (ITP). They're exploring the idea of creating a new qualification for software testing within our tertiary education system.<br />
<br />
One of the topics of conversation that I found interesting was the question of how employers currently recruit junior testers. By junior, I mean a person with no experience in testing, regardless of age or other work history. As there are no dedicated testing qualifications* at present, where do new testers come from?<br />
<br />
I sat in a discussion group with Aaron Hodder, Adam Howard, and Anna Marshall. We came up with a list of ways that we found the junior testers in our organisations that included, in no particular order:<br />
<ul>
<li><b>the business </b>- subject matter experts who show aptitude for testing</li>
<li><b>overseas </b>- new arrivals to New Zealand may start in a junior role</li>
<li><b>internships </b>- through programs like <a href="https://summeroftech.co.nz/">Summer of Tech</a></li>
<li><b>consulting firms</b> - local consultancies who run their own graduate training and placement</li>
<li><b>graduates </b>- those who are fresh from study</li>
</ul>
<br />
In my experience, finding candidates for a junior testing role is not a problem. When I've advertised a role that is suited to the wide audience defined above, I've had a lot of applicants. Testing is seen as a pathway into the IT industry, so there is a lot of interest.<br />
<br />
I think that the workshop question is more interesting when it is considered in a slightly different way. As there are no dedicated testing qualifications at present, by what criteria do you recruit junior testers? In other words, how do you decide which person to hire?<br />
<br />
I assess the testers who work in our agile delivery teams by six criteria that can be broadly summarised as:<br />
<ol>
<li>Testing knowledge and experience</li>
<li>Automation knowledge and experience</li>
<li>Agile knowledge and experience</li>
<li>Domain knowledge</li>
<li>People skills</li>
<li>Growth mindset</li>
</ol>
<br />
I'm not looking to hire people who hit all of those criteria. I am looking to create testing teams with complementary individual strengths that mean we are collectively strong in all of those criteria. This applies across all testers, not just juniors.<br />
<br />
When I hire a junior, I'm generally looking at the latter criteria in the list. While a junior may have knowledge of testing, automation and agile, it is likely to be entirely theoretical. I probably have strong testing, automation, and agile skills in the existing testing team anyway. The strengths that a junior might bring are their domain knowledge, people skills, and growth mindset.<br />
<br />
I've talked previously about <a href="http://katrinatester.blogspot.co.nz/2015/12/why-you-should-hire-junior-testers.html">why you should hire junior testers</a>. The benefits that I see in making junior testers part of the team, which largely focus on their attitude and behaviour, can be difficult to quantify. Similarly, the attributes that I am looking for in order to realise those benefits can be difficult to quantify, which means that they generally aren't assessed in an IT qualification.<br />
<br />
I look for juniors who can communicate and work well in a team. People who are eager to learn, pick up new ideas quickly, and constantly search for a better way of doing things. Perhaps people with these traits are more likely to pursue higher education, but not necessarily. A person can pass a qualification without possessing these particular skills.<br />
<br />
So, would it be useful to create a software testing qualification? Perhaps. If the qualification had a strong syllabus that was delivered in a practical manner, then hiring someone with this qualification might save some time when explaining basic concepts.<br />
<br />
Would the presence of a software testing qualification change how I hire junior testers? Perhaps. The presence of such a qualification might increase the chance that I interview a candidate as I would spot it when screening CVs, but I'm not sure that it would have a significant bearing on the final criteria by which I hire.<br />
<br />
Does New Zealand need a new software testing qualification? Perhaps. When I received the invitation to the workshop I thought it sounded like a great idea. As a trainer I was excited about the challenge of creating a syllabus. But the more that I've thought about it from the perspective of an employer, the less sure I become.<br />
<br />
Now I'm curious. How do you hire a junior tester? By what criteria do you choose a candidate? Is there a software testing qualification in your country? Do you think there should be? I welcome your comments below.<br />
<br />
<br />
*****<br />
<br />
* ISTQB is a certification, not a qualification. A certification is an official document attesting to a status or level of achievement. A qualification is a pass of an examination or an official completion of a course, especially one conferring status as a recognized practitioner of a profession or activity.<br />Katrina Clokiehttp://www.blogger.com/profile/13817473142273516519noreply@blogger.com4