Saturday, 4 April 2015

About automation and mismeasurements

Testing Newsletter - 1

  • 100/100 test cases automated - testing team awarded.
  • Management says 12.73 failure rate is a must for every testing cycle.
  • Testers denied from attending training's on analytical and critical thinking skills.
If such is the criteria for accolade and criticism we as professionals, leaders have failed ourselves.

Are these people who make such suggestions / decisions educated in software testing?
The testing team must seek answers and challenge such decisions made.

To people, who think automation is the solution to any testing problem. Read, read often and enough to know the origin, history and usage of automation. Then apply this knowledge to learn: if automation is the solution to the problem at hand. 

Read enough to know the pros and cons of test automation. Then use it wisely.

About automation and mis-measurements

Consider this example below to understand what we can do to educate ourselves and those around us.

Preparation of Doughnut (or Vada)
We can automate the process of doughnut preparation, but everyone who consumes the doughnut has a different need / taste / allergic to the ingredients used. The same doughnut prepared out of the same ingredients and using the same preparation process is not the only way to prepare a doughnut. Nor is test automation the only solution to all testing problems.

Extending this analogy to testing, we can learn that with the introduction of variations and complications we can begin to gather information about the product. With this information we can learn the context, in which a doughnut is prepared, served and to whom and under what conditions. These parameters define the CONTEXT. 

Are we as inheritors of this knowledge, understand and apply the knowledge of context before defining, suggesting automation? If yes, then good.

If no is the answer to the above question, then let's begin to read regularly and apply the knowledge. Every user is not in the same environment nor is in the same context when preparing, serving and consuming the doughnut. Then, why suggest and sell test automation as a solution without exhaustive research of the organization, product, team, technology, environment and the users.

Try not to suggest test automation by reading the profit sheet of organization X. Do your own study and put your power to influence to good use.

Measure under pressure
Measurement under pressure is a recipe for disaster. 
Though not immediately but eventually metrics / measurements made under pressure lead to corrupting the system and the process followed and used. 

Metrics must serve a purpose, have well defined parameters, state the environment of use and conditions in which it can be put to use aptly.
Record the data as clearly as possible. 
Metrics should not be merely used to achieve, over or under achieve an / any objective. 
Or used to meet the standards of an audit ready document.

"Whenever possiblebe clear" -  Confucius
When defining the requirements, test strategy, automation framework, test reports - be clear.


Suvarna Durga said...

Not only in Automation , for a tester, its very important to have better understanding, ask questions and have clarity, a tester should
never and ever assume anything.

Get Software Services said...

This is very useful article.Thank you so much for sharing this article

Jyothi said...

True Suvarna. Thanks for you comments Suvarna Durga and Get Software Services.