ISO 9001: 2015 and “Effectiveness”

Control chart, data, analysis

Question

ISO 9001:2015 references measurement of “effectiveness” throughout the standard. My question is what methods/techniques are typically used to measure “effectiveness”? Is it purely a quantitative analysis of metrics, or does it also involve subjective evaluation that may not be driven by statistics or metrics? Also, is it expected that effectiveness be measured for each process/procedure?

Answer

Thank you for your question.

Let’s begin with the definition of effectiveness from ISO 9000:2015.

Effectiveness is “the extent to which planned activities are realized and planned results are achieved.”  This definition requires that you know what the expected outcome (or objective/target) you want for a given task, project or process.  The first step is to know what you want to achieve, and then ask yourself what would be the best means to determine the extent to which that objective was met?

Yes, it is expected that effectiveness be (suitably) measured for each process. This is at the heart of the Plan-Do-Check-Act cycle.  Also, please read ISO 9001 Clause 4.1 and Clause 0.3.

As far as quantitative vs. qualitative metrics are concerned, you will have to decide what best measures the outcome of the process.  Also consider the cost of data collection.  Often, easy-to-collect qualitative data can suffice.   And don’t be deterred if quantitative data cannot be obtained.  As Dr. Deming so wisely stated: “Sometimes we have to settle for inexact measures of exactly the right things”.

Denis Devos

A Fellow of the American Society for Quality
Devos Associates Inc.
(519) 476-8951
www.DevosAssociates.com

For more information on this topic, please visit ASQ’s website.

Defining Qualification, Verification, and Validation

Q: I understand the hierarchy, but I would be hard pressed, if asked, to give a clear definition of the terms: qualification, verification, and validation. Can one of the experts help explain these terms? Thank you.

A: This is a great question and I hope I’ll be able to help you.

To begin, I refer you to ISO 9000:2005 Quality management systems – Fundamentals and vocabulary.  As you may already know, this document is used to define/describe many terms used in the ISO 9000 series, including the three words you question.

In 9000:2005, under clause 3.8 Terms relating to examination, we find:

3.8.4 verification
Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled
NOTE 1  The term “verified” is used to designate the corresponding status.
NOTE 2  Confirmation can comprise activities such as
–          performing alternative calculations,
–          comparing a new design specification with a similar proven design specification,
–          undertaking tests and demonstrations, and
–          reviewing documents prior to issue.

3.8.5 validation
Confirmation, through the provision of objective evidence, that the specified requirements for a specific intended use or application have been fulfilled
NOTE 1 The term “validated” is used to designate the corresponding status.
NOTE 2 The use conditions for validation can be real or simulated.

Validation definition, as provided by ASQ's Quality Glossary.

3.8.6 qualification process
Process to demonstrate the ability fulfill specified requirements
NOTE 1 The term “qualified” is used to designate the corresponding status.
NOTE 2 Qualification can concern person, products, processes or systems.
EXAMPLE  Auditor qualification process, material qualification process.

I’ll try to expand on these definitions in hopes of making things a bit more clear.  Keep in mind that qualification, verification, and validation are individual processes, but the explanations below (from Boston Scientific) should help you recognize their individuality as well as their interdependence.

Validation is an act, process, or instance to support or collaborate something on a sound authoritative basis.

Verification is the act or process of establishing the truth or reality of something.

Qualification is an act or process to assure something complies with some condition, standard, or specific requirements.

For example:

A design verification verifies that a frozen (static) design meets top level product specifications.

A process validation validates that the on-going (dynamic) manufacturing process produces product that meets product/print specifications and consist of installation qualifications, operational qualifications, process performance qualifications, a product performance qualification and perhaps process verifications.

An installation qualification qualifies that equipment was installed correctly and are a subset of a process validation (or possibly a test method validation).

Validation Examples:
•         Design validation, sterilization validation, test method validation, software validation, and process validation.

Verification Examples:
•         Design verification and process verification.

Qualification Examples:
•         Installation qualification, operational qualification, process performance qualification, product performance qualification, and supplied material qualification.

After reading all of this, I am confident you would be able to explain qualification.  An old and trusty phrase to help summarize the other two is: Validation – Are we producing the right product?; Verification – Are we producing the product right?

Bud Salsbury
ASQ Senior Member, CQT, CQI