Software Testers – Do we (As)sure Quality?

I am a Software Tester. But do/can I assure quality of the s/w that I test? For that matter, is it practically possible for a tester to assure quality of a product? Before going into much detail, I would like to quote some widely accepted definitions of Testing, Quality and Quality Assurance for the better understanding of these terms:

“Testing is questioning a Product in order to evaluate it” –
James Bach

“Software Testing is a process of empirical, technical investigation of the product under test conducted to provide stakeholders with quality-related information.” –
Dr. Cem Kaner

"Quality is value to some person". –
Gerald M. Weinberg

“Quality assurance (QA) is the activity of providing evidence needed to establish confidence among all concerned, that quality-related activities are being performed effectively. All those planned or systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. Quality Assurance is a part and consistent pair of quality management proving fact-based external confidence to customers and other stakeholders that a product meets needs, expectations, and other requirements. QA assures the existence and effectiveness of procedures that attempt to make sure - in advance - that the expected levels of quality will be reached.” – Wikipedia

I have seen testers/testing groups claiming to be the quality guardian of the project. They try to show/prove that testers are the only bunch of people who actually care for quality. Even they take pride in claiming themselves as part of the Quality Assurance team and ask for the power to block a release if the product quality is low! Actually we should not blame the testers here for this misconception. There are organizations that categorize testers into teams like Quality Assurance groups, Quality Controller groups blah blah blah…

But honestly speaking, I tend to get confused by this categorization. I don’t think that a tester has the ability, power or control to assure the quality of a product. Here are some questions for those testers who believe they are assuring quality of the product that they are testing:

a) Can you (the tester) measure quality of the product? If yes, how? [Please don’t show me those so-called quality metrics as proof; I don’t believe much in them as they can be easily faked!]
b) How can you justify that the product quality is too low and hence its release should be blocked? How do you measure the quality as “too low”, “medium” or even “high”? When someone says the quality of the product as “too low”, can you think of a context when the quality could have been “still lower”? When you say the quality is “good”, can you think of a scenario when it could have been “better”?
c) As a tester, do you have the authority to provide necessary training to the project team so as to assure a quality product?
d) Have you seen a tester with “hire and fire” authority?
e) Do you have the authority to take a decision on whether to fix a bug or not?
f) Do you have the authority to get more testers and more time for testing, for the test project, as and when required?
g) Are you ready to assume responsibility in case the product fails? [In cases, where the testers claim to assure the quality of the product and had blocked the release of the product due to it’s “low quality”! What if the product fails in its subsequent release?]

Hence, I don’t think that a tester can assure quality of a product. I believe in Dr. Cem Kaner’s saying:

“Whatever QA is, it's not testing.”

As testers we perform technical investigation of a product, to gather quality-related information. Our duty is to report a problem. We serve as technical advisors for the management. Testers gather useful information, which might assist the management in taking an important project-related decision. But we DON’T ensure quality. Quality of the product is something, which is entirely out of our control. We might prove that a product doesn’t work! But can we prove that a product does work perfectly? A product might be working as per its specifications. But that does not necessarily make it a high quality product! Quality is a relative term and is multi dimensional. Think of a video game that is too robust and rock solid in its architecture! But if it fails to entertain the user playing it, will you still call it a high quality product? Think over it.

Happy Testing…
Share on Google Plus

About Unknown

Debasis has over a decade worth of exclusive experience in the field of Software Quality Assurance, Software Development and Testing. He writes here to share some of his interesting experiences with fellow testers.

9 Comments:

  1. Just updated your blog in the Chennai bloggers list.
    http://chennaibloggers.blogspot.com/

    It would be great if you can provide a link to the chennai bloggers list.

    Thanks,
    Karthikeyan.

    ReplyDelete
  2. Well, I do not agree with you on lots of ground for this particular article.

    "Testing can show the presence of errors but not their ABSENCE."

    As long as the project/system is within your company's frame the developer is responsible for any loss/delay or any other point and once it has reached the to the client the QA is responsible for the issues reported from the other side.

    So, we in our company follow one specific rule

    "Delay any release but provide a quality release coz people won't remember the time frame which was lost for not delivering on time but they shall always remember the thing they are going to use for a long period is not of the QUALITY standards which they were expecting."

    So, why should the testers/QA should not take harsh decision of withholding a release? It's afterall their responsibility if they are not provided ample time to complete their work.

    And having attitude/egoistic it's human tendency which changes from person to person.

    Not all fingers are alike. So you can look at it anyways. So does we human.

    Sorry for going offtrack from technical to non technical.

    No hard feelings dude.

    I spoke what I felt.

    Cheers!!

    Thanks and Regards,
    Prabhath Murthy
    (prab_murthy@yahoo.co.in)

    ReplyDelete
  3. @ Prabhath,

    Thanks for venting your anger upon me! :) Looks like I have touched a sensitive issue here!

    "Testing can show the presence of errors but not their ABSENCE."

    Yes! I still stand on this point. I can show you dozens of bugs in a product that I am testing. But I have not met a tester who can claim the ABSENCE of bugs in a product! If that was true, then all the products that are shipped must have been "100% Bug Free", a term, that makes me tremble with horror! Any testing team that claims or even expects a "100% Bug Free" Product is unknowingly shooting at it's own feet!

    "Delay any release but provide a quality release coz people won't remember the time frame which was lost for not delivering on time but they shall always remember the thing they are going to use for a long period is not of the QUALITY standards which they were expecting."

    So, why should the testers/QA should not take harsh decision of withholding a release? It's afterall their responsibility if they are not provided ample time to complete their work.


    This strategy might have had worked in your case. But I can think of contexts, where this strategy has good chance to fail!

    CORRECTIONS: Form your whole comment, I am getting a feeling that somehow you have missed to capture the intention of the post! I am wondering whether it was because of some mistake in my part to convey my message clearly! I could see you using Testers and QA interchangeably! But throughout the post I have tried to say that Testers are NOT QA! In my understanding QA is something, which is more of a management job. Management is responsible for managing the whole project. Let them take care of the quality part too! As testers we can report problems with the product.

    Take this scenario. Just a day before your proposed release date, the testing team finds 3 critical bugs! Will you try to block the release? Well, if I were you, I would rather try to report the bugs with as much accurate and comprehensive information as I can along with the worst possible impact of the bug on the end user if the Product is released without a fix! Now I would like to let the onus on the management, client and other stakeholders of the project. In most cases, they would not like to ship the product with these known showstoppers! However, if they still want the product to be shipped, why should I stop them from doing so? Let them ship the Product and let them be responsible for any possible future mishaps! Either way, I would not try to block the release. My job is to provide accurate, comprehensive, and timely information about the product under development. Not to act like a quality guardian!

    As a closing note, I feel that we (the testers) don't have the authority to behave as the quality guardian of the project (or even block the release) UNLESS we have positive and valid answers to my questions [a-g asked in the above post].

    My above words are against your beliefs, and NOT against YOU as a person. Thanks for opposing my beliefs (look, I am not saying "thanks for opposing me") and making me think more on this topic. Happy Testing! :)

    Regards,
    Debasis.

    ReplyDelete
  4. @ Debasis,

    Let me clear up some cache between us

    You(as a tester) is the base of the QA. And you(as a tester) are responsible for not finding any critical bug till the last moment but again the golden words come into picture "Testing is never 100%" and it takes MMT(mind money and time).

    The management is involved in just 2 things 1) Delivery and 2) Count your account balance.

    You set up the building block for and provide enough confidence for the management to release or NOT to release a bill.

    I, as a tester, have set myself on cool mind. Patience and calm are key to success as a tester. So never your words are taken on serious ground.

    Chill relax and no harsh feelings.

    Lets just exchange words and thoughts and not our weapons ;)

    *Peace*

    Thanks and Regards,
    Prabhath Murthy

    ReplyDelete
  5. @ Prabhath,

    We are human and there will be bugs. To the extent that quality assurance fails at its primary purpose -- bug prevention -- it must achieve a secondary goal of bug detection.
    - Boris Beizer, in his Book "Software Testing Techniques"

    I think we are trying to tell the same thing but from a different perspective. e.g. take this paraphrase:

    You(as a tester) is the base of the QA. And you(as a tester) are responsible for not finding any critical bug till the last moment but again the golden words come into picture "Testing is never 100%" and it takes MMT(mind money and time).

    I am slightly changing the second one by replacing a *singe word*! See, if this makes sense to you:

    You set up the building block for and provide enough confidence for the management to release or NOT to release a Product.

    I, as a tester, have set myself on cool mind. Patience and calm are key to success as a tester. So never your words are taken on serious ground.

    I respect your virtues.

    Chill relax and no harsh feelings. Lets just exchange words and thoughts and not our weapons ;)


    If any of my words did appear as weapons, I doubt that was due to a possible bug in my brain's thought-to-word synthesizer! Have to investigate it soon. Thanks for reporting! :) Happy Testing...

    Regards,
    Debasis.

    ReplyDelete
  6. HI Guys ,

    I am a QA anlyst by profession and I have often come accross managers who feels that Quality is the Responcibility of the QA team . I have been searching in the intenet for any articles which describes the distribution of Responcibility among different teams in a project in delivering a high quality product . Sadly I havent come accoross such a one. From quality gurus i have heard time and time that Testers can report defects , however the primary responcibility in developping a quality product lies with the developpers . To a great extent i agree on this . If you come accorss some kind of a document that demarcates the responcibilities among Dev , QA , Project Management teams in ensuring quality could you please publish it please.

    (KG)
    xtreme_00001@yahoo.com

    ReplyDelete
  7. KG - Look into Six Sigma, it is all about Quality throughout the organization.

    http://en.wikipedia.org/wiki/Six_Sigma

    ReplyDelete
  8. @ Butterfly,

    The major problem with adopting a model like Six Sigma in a Software Development Process is that - Six Sigma relies way too much on statistics. It considers and expects all persons involved in the Product to be with equal skill sets and simply fails to understand that NOT every person can be with equal level of intelligence! And to me it appears to be the biggest drawback with a model like Six Sigma. In real life we just cannot afford to undermine the difference in intelligence level of two different persons (can be a tester, a programmer etc) which is bound to happen! We are testers not machines. We should not forget that Six Sigma was successful in manufacturing based companies may be because; most of the operations are done via machines (which can be of equal skill sets!) Imagine a machine that manufactures a mobile phone keypad. If we compare two such machines they might be more or less same in their intelligence level. Take two testers. Can we expect the same? If still we consider them as same and go on to figure out our own statistical method to reduce defects, you might guess the consequences!

    -Debasis

    ReplyDelete
  9. Quite an insight, Debasis. I offer my differences/views to some generalized comments you make here:

    a. Agreed - Testers and QA are different in their roles but for sure the onus lies on both for ensuring a quality product, and in reality both of them does this.

    b. The path/travellers to quality travels this road: developer -> tester -> qa i.e. a developer ensures quality by trying to build/develop a product meeting to the maximum possible extent with the requirements. Tester does his role towards ensuring quality by testing all (humanly) possible scenarios to confirm with the requirements and finally QA analyzes test findings to approve/reject acceptability. Does this any way seem like tester not ensuring quality? I think, NO.

    c. Replies to your question that you pose for testers are as below:
    1. Yes - 'Quality' is the extent to which the final product meets its 'specified' requirements. If you as a tester provide an honest and expert coverage of testing all the specified requirements, you can definitely claim that you provide a 'good' quality product. Yes, good is subjective, but here our usage is the universal 'good'.

    2. Not necessarily all businessman possess good business acumen. If the product requirements are not defined properly, you need to have that done first. Definitely testers won't do that. Once that is taken care of and if you are doing as responded in point 1 with not many critical or blocker defect/bug findings, you can claim a product to be of high quality.

    3 & 4. No and No.

    Many other aspects I wanted to offer. I agree to your article in essence that you are drawing a fine line between QA and tester but the claim that tester does not contribute to quality or exercises no control over defining or approving quality is something which I strongly disagree with.

    I surely have to agree...you provide an awesome insight into all these articles. Great work, please continue this favor :-)

    Vishal Rajput
    vishalraj21@gmail.com

    ReplyDelete

NOTE: Comments posted on Software Testing Tricks are moderated and will be approved only if they are on-topic. Please avoid comments with spammy URLs. Having trouble leaving comments? Contact Me!