Heuristics and Oracles

Heuristics and oracles may seem like inaccessible concepts for new testers. The words sound academic, removed from the reality of what a tester does every day. In fact they are immensely useful tools for critical thinking.

What are heuristics and oracles, and why should you learn more about them?

Heuristics

Imagine that I want to eat a pickle. My pickles are stored in a large glass jar. In my household the last person to eat a pickle was my husband. He has closed the jar tight. On my first attempt I fail to open it.

What do I do next?

I check that I'm turning towards the left to loosen the lid and try again. Then I retrieve a tea towel to establish a better grip when twisting the lid of the jar. Finally, in some frustration, I go and locate my husband. He successfully opens the jar.

When faced with a jar that will not open there are a number of things that I know are worth trying. These are my jar opening heuristics. When I am instructed to test a software application there are a number of things that I know are worth trying. These are my test heuristics.

Heuristics are simply experience-based techniques for problem solving, learning, and discovery. [Ref.] 

Every tester will have their own set of heuristics that guide their testing every day. These are innate and developed through experience. The value of learning more about heuristics is in discovering how other people think, and becoming capable of describing our own thinking.

When I run out of inspiration during my testing, there are numerous heuristics that might prompt my next move. Rather than relying on my own brain, two of my favourite resources list a variety of techniques to apply based on the experiences of other testers. These are:

• Test Heuristics Cheat Sheet Elisabeth Hendrickson
• Heuristic Test Strategy Model James Bach

This insight into how others think allows me to introduce variety in my own approach. Instead of consistently finding the same sort of bugs, I broaden my horizons. It's like a single person adopting the mantra of "two heads are better than one". James Lyndsay illustrates this difference with a nifty visualisation in his blog post Diversity matters, and here's why.

Heuristics also give me the words to describe my testing. When questioned about how I discovered a bug my response had always been a nonchalant shrug; "I just played with it". Heuristics changed the way I communicated my testing to others. Once I could clearly articulate what I was doing I gained credibility.

Oracles

Imagine that I go to lunch with a friend. I enter a restaurant at 12pm on Thursday. After an hour enjoying a meal, I leave the restaurant at 1pm on Friday. Although I have experienced only one hour, the world around me has shifted by a day.

How do I know there's a problem here? 

I may have several notifications on my mobile phone from friends and family wondering where I am. I may have a parking ticket. I may spot somebody reading a Friday newspaper.

There are a number of ways in which I might determine that I have skipped a day of my life. These are my time travelling oracles. There are a number of ways in which I might determine that I have discovered a bug in a software application. These are my test oracles.

Oracles are simply the principle or mechanism by which we recognise a problem. [Ref.]

The test oracles that I consciously use most often are those described by Michael Bolton in Testing without a Map. This article describes the original mnemonic of HICCUPPS; history, image, comparable products, claims, user expectations, product, purpose, and statutes. The list has since been extended, which Michael describes in his blog post FEW HICCUPPS.

When I find a bug during my testing I always consider why I think that I have found a bug. I don't like to cite my "gut feeling" or claim "it's a bug because I said so!". Oracles help me to discover the real reason that I think there is a problem.

Knowing my oracle means that I can clearly explain to developers and business stakeholders why the users of the application may agree that I have found a bug. This means that I am much more effective in advocating for the bugs I raise, so they usually result in a change being made.

If I cannot associate the problem with an oracle, then it makes me question whether I have really found a problem at all. I believe this self-imposed litmus test removes a lot of noise from my bug reporting.