Monday, 20 August 2018

D, E, F’s of Software Testing

All we think and talk about is a, b, c's of x, y, z's, and for a change here are the D, E, F’s of Software Testing.


D for Data
Data Testing - Do extensive testing of the input and output data and provide coverage in terms of the below. 

For its legal usage and infringement of the right to data privacy. What data can be used and when and who can use it and why?
Test the data - how is data collected, stored/saved, transmitted, used and disposed of.
Test the data - for correctness and reliability. Not everything on the Internet is from a trusted source.
Test the input data - for idiosyncratic patterns.
Test the output data - if it's spewing out relevant output for the input fed.
Test the data - input fields for the invisible forces attacking them.
Test the data - for who prepared it, who is the end user, are they the authorized end user, where did we get the data from, where are we storing it, how are we storing it, how better can we store  it, how securely can we store it, how else can we use it, how legitimate the data in transit is, how do we distribute it across systems, third party and otherwise, are the users aware of how their personal/valuable information is stored and used, who/which tool generated the input data, is this tool licensed, how is the data generated, who are the makers of this tool, who else can access the data and how is the data disposed of.
Test the data - for its quality.

The question for us to think about here is:
Why are we collecting this data from a user?
Answering this question can help in solution-ing the software for apt usage and can help reduce the cost incurred in designing, implementing, testing and when the solution is released to the users.

E for Evidence
Evidence collection of the testing performed is of utmost importance to testing. It is also significant to understand the value-addition of testing evidence produced and provided to the client.

Testers need to be trained to use tools to capture and produce evidence. From the project initiation phase to completion and beyond, evidence of the testing done, coverage provided, scope altered, tools used, checklists filled are filed by testers for the consumption of an end-user. The test lead needs to ensure that the test evidence is easily accessible to the consumer (for the client) and in time.

Be punctilious of the testing approaches you learn and use, as it creates value. Including the context-driven test approach which sets up the tester to form and base the testing by cautiously understanding the context. Failing to state the context prior to performing testing and during execution, can set the product up to have lousy quality.

With evidence collection, the testers learn the fine distinction between testing and providing proof. Without which, testing is deemed valueless.

The question for us to think about here is: 
What steps you take or have taken to gather evidence?

F for Fact
Lest we forget the value addition of the reviews, root cause analysis of the bugs which reveal that lack of fact-checking can be fateful to any project.

Testers as fact checkers can ultimately lead them to be and become better quality contributors.

Fact checking as part of testing can trigger a tester to be hawk-eyed during the whole process and make one quality conscious. 

The statement of work
Collated product and testing requirements
Use cases referred and used
User stories created and suspended
Agreed user acceptance criteria
Test ideas, charters designed for execution with valid and invalid inputs
Product understanding and bug review
Test reports and graphs generated
Risks encountered and mitigation planned
Experience and closure reports

The side effect of doing this helps a tester to better time track the testing activities one is involved with. And raise relevant questions related to estimation which is a faux-pas when planning for testing is considered.

All of the testing related activities needs to be fact checked for. Testers, practice checking for facts before filing a bug or sending out a test report. Tester vocabulary used when testing and not testing plays a prime role in doing so.

The question for you here is:
Dear testers, have you fact checked today?

Wednesday, 1 August 2018

Understanding The Context

Establishing the context is essential when conversing, writing and especially when testing.

Reasons why establishing the context prior to testing can be missed out:
  • Ignorant or Unaware –Testers/Leaders are not yet aware of the science behind establishing a context.
  • Code and test enthusiasts hastening to verify and validate the product against set requirements.
  • By not encouraging the idea of questioning at every stage of software development, one could dampen the outcome of testing. Relevant and well-timed questioning leads to challenging the status quo and learn new dimensions and the changing contexts that are akin to testing. 
  • Change in requirements requires a change in context, which is seldom accommodated in ardent and uncompromising work cultures.

The above scenarios can occur in instances when:
  • The team has not set clear team and individual objectives prior to testing.
  • The code delivered to be tested is delayed and has passed the test execution start date. 
  • When the manager is pushing for testing to be completed and delivered at the earliest. 
  • Every member of the team wants to be in the good books of everyone else and is not asking lost time to test.
  • There is a mentor missing whose guidance can help in fire-fighting.

What is the context?

*Context - The set of facts or circumstances that surround a situation or an event. 
*Definition via the WordWeb.

De-constructing Context
  • Making sense of the system under test is itself the beginning of testing and is the first step in knowing the system.
  • Understanding the circumstances that surround a situation or an event - In this context, the system under test is needed to be understood in the environment it operates in, in order to test the system in an interactive mode.

Why is it important to establish the context when testing?

It is required in order to construct solutions for testing problems and to understand the system’s isolated and interactive behavior. 

Testing is to learn and to know the system under test.
I would like to suggest further reading of Chapter 1 from the book ‘An Introduction to General Systems Thinking’ authored by Gerald Marvin Weinberg to understand, to know, to extend, to limit the learning/testing and how far we can/need to go when subjecting the system under test to contextual mode.

An analogy
Recently, I was asked to prepare interview questions on logical reasoning and this request came as a 1 liner. I went ahead without questioning (or establishing the context) and spent a considerable amount of time defining, redefining, learning about questions, various types, and modes of questioning and learned the definition of logic, reasoning, facts, ways of thinking, asking and framing questions. Found various online resources to learn about questioning and interviews and I finally submitted the 20 questions that I prepared with some original questions in the list.
It was only later and when I asked for a review did I learn that since the context was missing in the written communication(the e-mail request sent) the effort was rendered waste, however original. The email clearly lacked the information that was required about the context and I had assumed the context of my current understanding and this whole effort was rendered a waste of valuable time for all involved.

Key Takeaways
  • Do not hesitate to ask relevant questions.
  • Establish context whenever needed.
  • This tweet from JeanAnn Harrison sums up the other lesson learned.

Optional reading