Wednesday 28 January 2015

Visual Test Models & State Transition Diagrams

This article was originally printed in the November edition of Testing Circus

Visual test models and state transition diagrams are two means of visualising information. Though they may appear similar at a glance, the structure and purpose of each is unique. When used in software testing, they act as tools to guide entirely different test techniques.

This article compares these two types of visualisation in the context of a real-world example based on purchasing a book from e-commerce retailer Amazon.

Visual Test Models

Testing often begins by focusing on function first. Sometimes this is because testing is being driven by a requirements document or the acceptance criteria of a user story. Sometimes it’s because not all of the functionality is present in an iterative delivery model, so the testers have to concentrate on one piece at a time. Sometimes it is just due to personal preference.

When focusing on function, we are testing to see what the software can do.  James Bach lists four key points of function testing in the Heuristic Test Strategy Model:

  1. Identify things that the product can do (functions and sub-functions). 
  2. Determine how you’d know if a function was capable of working. 
  3. Test each function, one at a time. 
  4. See that each function does what it’s supposed to do and not what it isn’t supposed to do.

A visual test model begins as a useful way to quickly capture the things that a product can do, acting as a note taking tool during investigation of a function.

Imagine that we have received the Amazon Shopping Cart function to test. A screenshot of a cart with one item awaiting purchase is shown below:




A visual test model that identifies things that the product can do might look like this:


Once the visual test model shows what the product can do, it can then be used to capture how the tester could know whether each function was capable of working. Test ideas are added to each branch and might include negative tests – those ideas that determine that the function doesn’t do things that it shouldn’t.

When switching between documenting function and capturing test ideas it may be useful to switch colour, to clearly delineate between different types of information. A continued example for the Amazon Shopping Cart is shown below:



From this point the continued use of the model will vary dependent on the test approach. It may be extended to show exploratory test sessions and progress of testing. It may drive discussion around which ideas are suited to automated checking. It may be used as a reference to create pre-scripted test cases in a test case management tool.

A visual test model is a good way to capture quick thinking about what a product can do. It is a simple mechanism to capture test ideas that drive function testing.

State Transition Diagrams

Once the pieces of the product have been functionally verified, the tester will want to examine how they interact together. The shopping cart, the checkout and the order review page may all work in isolation, but unless they also integrate correctly with each other a customer will not be able to purchase a book.

Flow testing is where a tester does one thing after another. The Heuristic Test Strategy Model lists three key points about the flow testing technique:

  1. Perform multiple activities connected end-to-end; for instance, conduct tours through a state model.
  2. Don’t reset the system between actions.
  3. Vary timing and sequencing, and try parallel threads.

State transition diagrams offer a way to visualise paths through a product. Determining how activities are connected end-to-end will help the tester to identify varied flow through the application that may otherwise have been missed.

To create a state transition diagram start by identifiying the scenario for a successful interaction. When purchasing a book using Amazon, the simplest flow through the product may look like this:



Then investigate the other scenarios that a user could encounter at each point in this simple flow. An expanded diagram may look like this:



Because the state transition diagram is focused on interaction, it is a good visualisation for flow testing. Traversing the diagram through varied paths, while adopting different timing and number of users, is a good way to explore a piece of software through a different lens.

Monday 19 January 2015

Generic Testing Personas

Personas are a tool to consciously adopt the habits and feelings of different people. When used during testing, they can help us to discover different types of problems.

Traditionally persona are developed for a specific application, each describing a rich background and context that shape the actions of user. I think there is value in using the same concept to model stereotypical software users for testing purposes alone.

Here are six generic testing personas that you could adopt when completing a testing task.

Manager Maria

Maria is a busy executive who interacts with the application between meetings. She is impatient and often not focused on her task, completing activities in haste. Maria will:

  • Stick to the quickest workflow through the application
  • Use shortcut keys
  • Fill in the minimum number of fields to get a result
  • Make mistakes in her efforts to get things completed quickly
  • Require fast responses and may repeat an action if the application takes too long to respond
  • Often be called to a meeting midway through a task

Hipster Hilary

Hilary likes to investigate new functionality and areas of the application that are outside of the mainstream. She is an early adopter and an avid explorer. Hilary will:

  • Investigate new features as soon as they become available
  • Explore all possible paths through a workflow to determine which she prefers
  • Frequently use areas of the application that are less popular
  • Have unusual data input compared to other clients e.g. different units of measure
  • Be accessing the application from an unusual browser, operating system or device

Careful Claire

Claire enjoys routine. She uses the same workflows each time that she interacts with the application, taking care to ensure that she is consistent and the information she provides is complete. Claire will:

  • Stick to popular features of the application
  • Notice and investigate any visible changes to these features, e.g. a new button is added
  • Complete every field possible when entering information
  • Be verbose when asked to enter notes of her own, e.g. a reason for editing a record.
  • Be patient with long response times

Sneaky Shirley

Shirley likes to break things. She knows about common security holes in software and likes to explores the applications that she uses to feel confident about their ability to protect her information. Shirley will:

  • Enter SQL and JavaScript injection in to application input fields 
  • Manipulate URLs to attempt to access private information
  • Violate constraints on input fields by entering invalid information
  • Try to generate as many error messages as possible

International Ioana

Ioana is on an overseas vacation. She periodically uses the application for specific reasons, e.g. to retrieve a piece of information or complete a single task. Ioana will:

  • Use the application outside of local business hours
  • Be accessing the application from multiple locations and time zones
  • Use a variety of browsers, operating systems and devices
  • Occasionally have poor network connectivity that is slow and unreliable
  • Be using a variety of keyboard layouts
  • Enter personal information that includes foreign language characters

Elder Elisabeth

Elisabeth is of an older generation with relatively little knowledge of computing. She has trouble understanding many software applications. Elisabeth will:

  • Use the application slowly, taking time to read each screen
  • Frequently use the 'Back' button to remind her previous information
  • Have the interface font of the application enlarged via browser settings or zoom
  • Require simple and clear interfaces in order to successfully complete a task
  • Seek out online help to assist her
  • Be using outdated technology including an older browser and operating system


Who else would you add to this list?

Wednesday 7 January 2015

Behind the Scenes: Editor of Testing Trapeze

Our June edition of Testing Trapeze clashes with my European holiday plans, so Shirley Tricker has kindly agreed to take the reins as Guest Editor. This has prompted me to think about and record what I actually do as Editor of Testing Trapeze.

Testing Trapeze started in a rapid fashion. I had the idea on the 7th of January last year and our first edition published on the 15th of February. I didn’t really know what I was getting myself in to when we launched, but I had a clear vision of what I wanted to create. My role as Editor has evolved as I learned what was involved in realising my vision. Though I expect it will continue to change, this is a snapshot of how things currently run.

Planning

The first step in creating a magazine is finding people to write. I like to approach potential contributors at least three months in advance of our submission deadline.

Testing Trapeze follows a consistent structure. There are always five articles written by two New Zealanders, two Australians and one international contributor. There are other informal criteria that I try to meet in each edition - people from different cities, at least two women, no more than one person from my organisation, some people with a strong Twitter following, someone who is writing for a magazine for the first time, etc.

Delivering a quality line up within these parameters does require some forward planning. I use a spreadsheet to track past, present and potential contributors, as shown below:



Confirming Contributors

We share who will be contributing to our next edition in a ‘Next time in Testing Trapeze’ teaser. This serves a dual purpose by promoting our writers and making a public commitment on their behalf.

Prior to publishing this teaser, I send an email to all the people who will appear to confirm that they are actually still happy to write for our next edition. They usually are, though on occasion there is a late withdrawal and I have to find an alternative writer.

Submission Review

In the week prior to our submission deadline the writers send in their articles. I think it is important to gratefully acknowledge the receipt of each article immediately. I also remind the writer of our review process, and tell them to expect a reviewed version of their article within a week.

I then pair the writer and their article with a reviewer. I find this the trickiest part of my role as Editor, to match personalities and material so that the review process is a positive experience for both sides.

When I send the article to a reviewer, I ask them to comment on, or modify, the article directly. There is an expectation that any comments or changes will have an appropriate tone, so that their review output can be shared directly with the writer.

Sometimes the reviewer will know who wrote the article. If there are existing relationships between writer and reviewer that may cloud honesty then I generally keep it anonymous. When the writer is a first time contributor, I might ask the reviewer to focus on selecting a few pieces of feedback that are framed in an encouraging way.

In every instance I try to remain a middleman in the review process. There are many reasons for this. The reviewer may make comments they believe are fair and constructive that the writer interprets as harsh criticism. The sheer volume of review comments may be overwhelming and disheartening for the writer. The writer may become confused where the reviewer hasn't provided enough detail to guide them in making changes. I read through each review to create a consistent experience for contributors - reframing, adding or removing feedback as I feel is appropriate.

When sending a reviewed article back to the writer I always position the feedback as a set of suggestions. The writer has the final say in how many changes they'd like to make after a review. Ultimately it's their article and they need to be happy with what is published. I request a final version of the article within a week and often have to plead for author biographies and photographs too, as people are notoriously reluctant to provide these.

Design and Layout

Adam Howard does the bulk of the work to create the layout and design of Testing Trapeze. I kick off the process for a new edition by choosing a colour palette, then providing a set of images for Adam to select from. This is one of my favourite things to do.

I find our images using Google Image search, which includes tools to filter by colour palette and licensing. For example, if I want images about flying that are primarily red and labelled for non-commercial reuse:




I also share with Adam an initial opinion on the order that I would like our writers to appear on the cover and in the magazine. Often these change when I complete a final review, but Adam requires a starting point to lay out a draft.

Halfway through last year I changed the logic I applied to ordering contributors on our cover. Our earlier editions featured the international contributor in the headline position. I started to feel that this undermined our focus as a magazine for Australia and New Zealand. Now the international contributor is always listed last. For other names, the order is loosely based on how well known they are in the local testing community.

I choose the order of articles in the magazine based on their topic and tone. I apply some general rules to this. The first article often has a broad appeal. Sometimes there are a pair of articles that cover a similar area, despite there being no theme to our editions. These are split across the second and fourth position in the magazine so that they are separated in the reader’s mind. I often place the longest article in the edition in the middle.

Editorial

Though it is a relatively easy task, I dread writing the editorial. I think this stems from my own view when reading a magazine - the editorial is in the way of what I really want to read. As Editor, I don’t feel like the star of the magazine. I’m simply creating a platform for others to shine. I try to keep my editorial short and focused on acknowledging all our contributors. I don’t want to distract from our consistently amazing content that speaks for itself.

Final Review

As I receive the final submissions from writers these are saved into a shared Google Drive folder. Adam creates a draft version of the magazine, then I do a final review. When we published our first edition this was an intense process. Now that we’re established, this review usually includes a handful of cosmetic changes.

This is also the point at which I change my mind about ordering of our articles, though I can rarely articulate the reasons why I want to shuffle things around. I feel that a rhythm appears when the articles are strung together. Our final note is static, it’s always our international contributor. Sometimes the other notes have to move for the magazine to really sing.

Publish

Ajoy Singha, the Editor of Testing Circus, agreed to host Testing Trapeze on the Testing Circus website as an associated publication. He did this without seeing a single edition, and his instant support of my idea to create a new magazine was incredibly encouraging.

When we have a final edition I login to the Testing Circus site to create a draft post that includes a photo of our cover and a list of the articles that are included. Initially we adopted the same format as Testing Circus for these posts and, now that we’ve published a few editions, I simply copy and paste from previous issues of Testing Trapeze to create new ones.

I email the final PDF to Ajoy for him to upload to the Testing Circus site. He updates the draft post with a link, then leaves it for me to review and publish. This works well, particularly as we are working across different timezones. We want to publish when our readers in Australia and New Zealand are actually awake!

Marketing

Once we publish, the final task is to let people know that our latest edition is available. I email all the contributors to the edition, both writers and reviewers, to let them know we have published. I tweet and update our Facebook page

As people start to read the edition, I amplify any positive feedback that we receive. This echo usually lasts about a week, by which point I assume that we have reached everyone who is interested. I try to avoid generating noise in our social media accounts.

This approach to marketing relies heavily on Twitter, so I have recently set up a mailing list for Testing Trapeze subscriptions to capture readers who don't use this platform. This list will send one email per edition, or just six emails per year. I hope that we will see many people choose to use this option.

Testing Trapeze is a magazine that I am really proud to be a part of.