This article focuses on the approach to bug reporting with a closer look at the rejected and deferred bugs.
It was originally published in www.teatimewithtesters.com
What is heuristic?
The word heuristic is derived from the Greek word 'heuriskein' meaning 'to discover'.
A commonsense rule/set of rules intended to increase the probability of solving some problem (definition via the WordWeb 5.52).
I was introduced to the heuristic approach of solving software (testing) problems when I first met Pradeep Soundararajan.
Based on my personal experience with bug reporting, I have collated the points and the examples below to ponder upon while reporting bugs.
D - DISCOMFORT
I - INQUISITORY
S - SOME CASES
S - SUGGESTION
C - CASE CLOSED
O - ORIGINALLY
H - HISTORY
D – DISCOMFORT
Consider the word 'Error' being misspelled as 'ERORR'. This misspelled word is causing discomfort to you and requires to be logged as a bug. It is possible that this may not be seen as discomfort by some and the bug if logged could get fixed or rejected or deferred.
To understand what a mis-spelt word could cost, take a look at this picture in the link below.
And that's the price Chile paid for the mis-spelt word.
When in distress whether to log a bug or not, remember to:
- Question with an intention to gather information.
- When we know better we test better.
- The information gathered could help us in understanding the impact the bug might cause if left unattended.
Question who the users, target audience and the customers are.
Know that your definition of discomfort can be different from the others.
Question the stakeholders, business owner, product owner, yourself, users and the involved in order to gather information to answer arising questions.
If it seems clear to you that a bug needs to be fixed then log it and move ahead with logging other bugs, which you think could add value to the software being developed.
Points to ponder:
Have you noticed this perspective on how fast/slow a mis-spelt word on a very popular website/stand alone application versus a not so popular website/stand alone application is handled?
Have you observed if you added value by logging other bugs along with the misspelling, this caused both the bugs being handled versus only the misspelling issue being ignored?
What factors are affecting the decision makers?
Have you questioned the decision makers?
If not, it is worth finding answers from the decision makers.
Consider reading the below excerpt from an article by Michael Larsen on his blog www.mkltesthead.com
“Spelling errors are often items that really get an executive irritated, especially in things like End User License Agreements. I learned this a number of years ago when, because of misspellings and typos in the legal agreement, the net result was the fact that we had created a loophole where we were making a tacit agreement that was never intended (and that we would be liable if something were to happen if the users did not do the steps necessary).
These areas are not glamorous, and sometimes they can be quite tedious. While that may be true, these are also areas that are considered most likely to impact revenue, and therefore will be given close scrutiny by executives. If we are taking the time necessary to look in these areas and report on what we see, we will probably get more traction and movement on these issues. Why? Because these are the issues that really matter. It may not seem fair or intuitive, but you can almost never go wrong reporting and championing bugs that the CEO has mentioned will be problematic for them if they were to be released.”
I – INQUISITORY
Users must verify if the information found on the Internet is authentic. If/when producing the information found on the Internet as a proof for a bug raised.
Consider the below link here as an example to probing.
The information related to the prefix list 985-989 is missing in the tabular column and was missing at the time the issue (missing information) was reported to be fixed.
Post the fix, the readers now have access to this information: Prefixes not listed above are reserved by GS1 Global Office for allocations in non-member countries and for future use.
This information is VITAL and is essential to any retail organisation.
Optional further reading:
Question with an intent to gather the RIGHT information.
Who are the contributors to Wikipedia?
Views expressed on Wikipedia or on the Internet could be an individuals perspective and/or an organisation adding their own view.
Diligently work on providing evidences and be thorough with the investigation. Build your first and subsequent investigation reports with references leading to why/why not the information on the Internet is correct or incorrect.
Also know, when to stop probing.
S - SOME CASES
Consider a bug description like the below:
On 'Nokia Lumia 520' device, in some cases the user is unable to slide down to the whole list of paired devices which are connected via the Bluetooth on that device.
Wait! In most cases the users and/or the programmers might ignore (as there is a work around it) this bug so why fix at all?
Avoid the usage of words like some cases, most cases from the testing vocabulary.
What are some cases?
What are most cases?
Who is the judge of such cases?
Is there a way to concretely confirm what some and most cases are?
If the answer is No – then avoid the usage of such misleading words from verbal and non-verbal communication and while reporting bugs.
Instead, it helps if some/most cases are expanded to include the details of what it really means.
S – SUGGESTION
Consider this bug rejection reason - "Suggestion rather than a bug"
The bug validator conveys that the raised bug is not a bug but a suggestion.
Provide reason/s for logging the bug with appropriate reasoning and ask for to receive a concrete understanding behind deferring a bug as not fixed.
A tester could have logged the bug cause he/she did not have all the required/sufficient information to judge the failed test as a suggestion rather than a bug. This in itself could be due to insufficient documentation.
It could be a missing requirement in the requirements document.
The scope of testing is not specified or is limited.
It could be any of these or other reasons.
Hence it is essential that the validator too provides a reason for marking the bug as a suggestion. Ask for and receive very specific rejection reasons.
C - CASE CLOSED
Failures can teach a lifetime's lessons.
Care to defend the rejected bug if required.
Read the rejected reason when a bug is rejected.
If/when there is no reason for rejection, ask for it in written communication.
Re-open a closed bug if found detrimental to the current requirement or you foresee it as a future potential candidate of becoming a deferred bug.
And remember this when trying to recreate that assumed ir-reproducible bug.
Image courtesy: http://cartoontester.blogspot.in The above cartoon is designed by Andy Glover.
O – ORIGINALLY
Have you heard this rejection reason before?
Originally, on my machine, the requirement is/was satisfied and hence this is not a bug.
Know your test environments.
a) Did the programmer/tester say this only to dodge further questions?
b) 'Originally' is in consideration to what?
c) Avoid usage of words like: Unorganised, primarily, to begin with, earlier and works on my machine when logging and rejecting a bug.
In agile, requirements don't always stay true to their original description.
Ask for what Originally means?
H – HISTORY
Learn the history of the product.
While testing, try not to be biased by the launch date, versions and previous state of the product. Bias and un-bias if required. Test your assumptions regularly.
Find the answer to:
Should I be performing sympathetic testing as this product was launched 80 minutes/hours/days ago?
- Irrespective of any such preconceptions, log the bug.
- Remember your right to information. Question whenever required.
- If there is no scope to question or there is a block recognise it. Try to unblock.
- There could be a case when a bug remains in unaddressed / in a new state or has not been fixed. Follow up on such bugs, be persistent and log bugs *dispassionately.
How to use DISSCOH heuristic when logging a bug or approaching a rejected bug?
I would like to suggest that you try this out and check the usage in your context when reporting a bug.
*RIMGEA: A Bug reporting Heuristic
My gratitude goes to Carsten Feilberg and James Marcus Bach who mentored me on bug logging and bug titling respectively.
Andy Glover (cartoontester.blogspot.com) and Michael Larsen (www.mkltesthead.com) for granting permission to use the cartoon and the excerpt from their blogs respectively.
Other mentions and further optional readings about Heuristics include:
Happy and efficient bug logging.