Ask the Standards Experts

Quality in Radical Innovation

Employees, Training, Working, Learning, Duties, Tasks, DFSS, Innovation

Question

Design for Six Sigma (DFSS) involves the discovery, development, and understanding of critical to quality areas and fosters innovation. However, studies have shown that using focus groups, interviews, and etc., based on current users only bring forth ideas relative to incremental innovation, as the only knowledge that most customers have is of current products. But we know that the greatest potential for return is in radical innovation.

My question is: what useful tools are there for determining critical to quality areas of radical innovation products, or products that are new to market where customers have little to no knowledge of?

Answer

These are great questions that are not easy to answer as posed.

One of the dilemmas I’ve seen with companies building radical innovation without enough knowledge to identify the important quality aspects is that the company is often under intense pressure to get to market. In some cases, the innovation presents clear aspects that have to be controlled to create an acceptable product. In some cases, the issues are unknown.

I do not agree the work within a group only reflects the knowledge already present. One of the best tools in these situations is carefully crafted questions posed to those most familiar with the new technology. Given my personal bias, I would ask: “What will fail? Why?” and then ask about material, process, and feature performance variation. Focusing on the failure mechanisms and variation will often lead the team to uncover those aspects of the product that require well crafted specifications and monitoring.

Not a fancy tool, just a question or two. Yet, the focus is on what will cause the innovation to not meet the customer’s expectations. What could go wrong? Make it visible, talked about, and examined. Creating a safe atmosphere (no blame or personal attacks) to explore failure permits those most vested in making the product work examine the boundaries and paths that lead to failure.

Once the process of safely examining failures starts, a range of tools assist with the refinement and prioritization. Failure Modes and Effects Analysis (FMEA) and Highly Accelerated Life Testing (HALT), provide means to further discover areas to explore the paths to failure. I mention creating a safe environment first, because using FMEA and HALT when someone’s reputation or status is threatened generally leads to these tools being very ineffective.

One more thought on a safe environment for the exploration of failures. Focus on the process, materials and interaction with customers and their environment. “How can we make this better, more resilient, more robust, etc.?” Not, “Why did you design it this way?” or, “This appears to be a design mistake.” All involved have the same goal to create a quality product or service, yet there may be a lot unknown related to those conditions that lead to product failure. An open and honest exploration to discover the margins and product weaknesses is most effective in a safe environment for those concerned. And, by the way, this includes vendors, contractors, suppliers, and all those involved with the supply chain, development and manufacturing processes.

Fred Schenkelberg
Voting member of U.S. TAG to ISO/TC 56
Voting member of U.S. TAG to ISO/TC 69
Reliability Engineering and Management Consultant
FMS Reliability
fmsreliability.com

For more on this topic, please visit ASQ’s website.

Defining Qualification, Verification, and Validation

Q: I understand the hierarchy, but I would be hard pressed, if asked, to give a clear definition of the terms: qualification, verification, and validation. Can one of the experts help explain these terms? Thank you.

A: This is a great question and I hope I’ll be able to help you.

To begin, I refer you to ISO 9000:2005 Quality management systems – Fundamentals and vocabulary. As you may already know, this document is used to define/describe many terms used in the ISO 9000 series, including the three words you question.

In 9000:2005, under clause 3.8 Terms relating to examination, we find:

3.8.4 verification
Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled
NOTE 1 The term “verified” is used to designate the corresponding status.
NOTE 2 Confirmation can comprise activities such as
–          performing alternative calculations,
–          comparing a new design specification with a similar proven design specification,
–          undertaking tests and demonstrations, and
–          reviewing documents prior to issue.

3.8.5 validation
Confirmation, through the provision of objective evidence, that the specified requirements for a specific intended use or application have been fulfilled
NOTE 1 The term “validated” is used to designate the corresponding status.
NOTE 2 The use conditions for validation can be real or simulated.

Validation definition, as provided by ASQ's Quality Glossary.

3.8.6 qualification process
Process to demonstrate the ability fulfill specified requirements
NOTE 1 The term “qualified” is used to designate the corresponding status.
NOTE 2 Qualification can concern person, products, processes or systems.
EXAMPLE Auditor qualification process, material qualification process.

I’ll try to expand on these definitions in hopes of making things a bit more clear. Keep in mind that qualification, verification, and validation are individual processes, but the explanations below (from Boston Scientific) should help you recognize their individuality as well as their interdependence.

Validation is an act, process, or instance to support or collaborate something on a sound authoritative basis.

Verification is the act or process of establishing the truth or reality of something.

Qualification is an act or process to assure something complies with some condition, standard, or specific requirements.

For example:

A design verification verifies that a frozen (static) design meets top level product specifications.

A process validation validates that the on-going (dynamic) manufacturing process produces product that meets product/print specifications and consist of installation qualifications, operational qualifications, process performance qualifications, a product performance qualification and perhaps process verifications.

An installation qualification qualifies that equipment was installed correctly and are a subset of a process validation (or possibly a test method validation).

Validation Examples:
• Design validation, sterilization validation, test method validation, software validation, and process validation.

Verification Examples:
• Design verification and process verification.

Qualification Examples:
• Installation qualification, operational qualification, process performance qualification, product performance qualification, and supplied material qualification.

After reading all of this, I am confident you would be able to explain qualification. An old and trusty phrase to help summarize the other two is: Validation – Are we producing the right product?; Verification – Are we producing the product right?

Bud Salsbury
ASQ Senior Member, CQT, CQI

ISO 9001 & Time to Retrieve Records

Q: I am looking for an interpretation for ISO 9001:2008 Quality management systems–Requirements, clause 4.2.4 Control of records: “Records shall remain legible, readily identifiable and retrievable.”

What is considered readily retrievable (i.e., 24 hrs, 48 hrs, 8 hrs, 1 hr)? I have a customer who thinks traceability records should be available within an hour of a request. I interpret readily as 24 hrs. The current ISO and TS specifications do not indicate a time, so a reasonable time to me is 24 hrs to pull the information together.

In addition, the customer’s supplier requirements also do not have any specified time for document retrieval. I did contact our third party registrar auditor and he indicated that 24 hrs would be considered readily retrievable as long as there were no customer specific requirements.

A: There appears to be some confusion between records being “readily retrievable” vs. a customer’s request for the delivery of copies of records. These are two separate issues.

The first issue: What is meant by “readily retrievable?” ISO 9001 does not prescribe any specific timeline or define the term “readily retrievable.” However, the intent of this requirement is to ensure that objective evidence is available to provide proof of conformance or evidence that requirements have been met. If the organization is unable to provide records upon request during an audit, the auditor will very likely document this as a nonconforming condition. Records must be available upon demand.

The second issue is response time to customer requests for records. Although records or evidence of conformance may be “readily retrievable” within the organization, the response time needed for an organization to provide copies of records to a customer may vary based upon the organization’s work load and availability of resources. So, it may take an organization an hour, a day or a week to deliver copies of records to a customer. In the event that the timely delivery of records is critical, requirements for the delivery of records should be stated in a contract or in a PO to provide a timeline or a delivery schedule. The delivery of copies of records or documents to customers is not addressed in ISO 9001, clause 4.2.4.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

For more on this topic, please visit ASQ’s website.

AS9100C: Scoring the Aerospace QMS

Airplane, aerospace, AS9100

Q: I’m reviewing the scoring method used for auditing AS9100C – Requirements for Aviation, Space and Defense Organizations, and I don’t see any verbiage to show what would be considered an acceptable overall score. I’m curious to know if the score is more subjective to the discretion of the auditor or if the threshold for “acceptable” or “not acceptable” exists somewhere as a guideline. Thank you to anyone able to offer insight.

A: The AS9101D auditing standard (currently not sold by ASQ) has scoring to provide an indicator of how robust your quality management system is operating (QMS), which is based upon the findings identified during your audit. There is not a required score to “pass” the audit and receive certification. The AS9101D score is recorded in the OASIS database, which your current and potential customers may review.

AS9100C requires the use of the AS9101D auditing standard, which has eliminated scoring.

Buddy Cressionnie
International Aerospace Quality Group Americas AS9100 Lead
Voting member of the U.S. TAG to ISO/TC 176
Southlake, TX

For more on this topic, please visit ASQ’s website.

Calibration of AutoCAD Software

About ASQ's Ask the Standards Expert program and blog

Q: To what extent must an engineering firm, specializing in railway infrastructure and transportation, have its AutoCAD software “calibrated” or verified?

Also, what about software designed to calculate earthwork quantities for railway alignments laid out on topographic mapping for all levels of studies – pre-feasibility through preliminary engineering (not for final design, operation simulation and design dynamic system models)? This type of software is utilized by competent draft persons and engineers, but it is not verified prior to use or periodically calibrated.

We don’t confirm “the ability of computer software to satisfy the intended application…”

Your assistance or reference is appreciated

A: AutoCAD is considered “Commercial -Off-The-Shelf” (COTS) software. It is purchased without modification and cannot be modified by the end-user. A similar example would be Excel spreadsheet software. The COTS software by itself should be considered validated and used as is provided it is configured per the software manufacturer’s instructions.

The functionality of the software (distance, volume, formulae and other functions) is fit to be used as intended. If an application is created using COTS software (Excel Templates, AutoCAD applications), then it must be validated and records of validation must be kept.

It should also be noted that definitions of verification and validation are not clearly understood. So, I am repeating them here:

ISO/IEC Guide 99:2007—International vocabulary of metrology—Basic and general concepts and associated terms, defines these terms as:

Verification: provision of objective evidence that a given item fulfills specified requirements

Validation: verification, where the specified requirements are adequate for an intended use

Further explanation:

Validation is a quality assurance process of establishing evidence that provides a high degree of assurance that a product, service, or system accomplishes its intended requirements. This often involves acceptance of fitness for purpose with end users and other product stakeholders.

It is sometimes said that validation can be expressed by the query “Are you building the right product?” and verification by “Are you building it right?”

“Building the right thing” refers back to the user’s needs, while “Are we building the product right?” checks that the specifications are correctly implemented by the system. In some contexts, it is required to have written requirements for both as well as formal procedures or protocols for determining compliance.

Dilip A Shah
ASQ CQE, CQA, CCT
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio
http://www.emc3solutions.com

For more on this topic, please visit ASQ’s website.

ISO 9001 and CMMI Certifications

Manufacturing, inspection, exclusions

Question

Our company is working toward certification to ISO 9001:2008 Quality management systems–Requirements and Capability Maturity Model Integration (CMMI) certifications.

I have studied ISO 9001 and mapped it to CMMI goals and practices. It appears to me that some sections of ISO point to CMMI level 3 process areas and practices, e.g.:

Clause 5.6.1 Management review – General relates to organizational process areas
Clause 7.2.1 Determination of requirements related to the product is relative to requirements development, which is a level 3 process area
A large part of clause 7.3 Design and development maps to CMMI level 3 process areas

My question is:

Does an organization need to be at CMMI level 3 in order to be ISO 9001:2008 certified? I am not saying certified CMMI level 3, but capable of performing at CMMI level 3?

Thank you so much.

Answer

Although the guidelines contained in CMMI may help to prepare an organization toward ISO 9001 certification, there are several major differences between CMMI and ISO 9001.

ISO 9001 is an internationally recognized standard for quality management systems. While CMMI is a Carnegie Mellon University registered trade mark.

ISO 9001 has specific requirements for documented procedures for the control of documents, control of records, control of nonconforming products, internal audits, corrective actions and preventive actions. In addition, a quality policy, measurable objectives, and management reviews are required.

CMMI is focused on process improvement, while ISO 9001 focuses on customer satisfaction, process improvement, product conformity and the continual improvement of the quality management system. An organization could be CMMI certified or “capable” as mentioned in the inquiry, but still be some distance way from readiness for ISO 9001 certification.

I hope this helps.

Bill Aston
ASQ Senior Member
Managing Director of Aston Technical Consulting Services
Kingwood, TX
www.astontechconsult.com

Here’s more information about ISO 9001.

ISO 17025; Rounding Measurements

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories

Q: At the lab I work for, certified to ISO 17025:2005 General requirements for the competence of testing and calibration laboratories, the documented quality assurance system does not allow the rounding of numbers. For example, the requirement for the weight of an adhesive material is 25 to 35 grams, and the actual weight is 24.6 grams.

The engineering member of the team feels this is acceptable because 25 grams is specified with two significant figures; 24.6 grams, expressed as two significant figures is 25 grams. If the intent was not to round off in the tenths place, the document would read “25.0” and rounding would be in the hundredths.

A: If the requirement (specification) is 25 to 35 grams, the need to specify accurately (24.6 grams) is not as critical and the number can be rounded to 25 grams. We would assume that the nominal desired value would be 30 grams. (Personal opinion: the 25 to 35 gram requirement is a fairly loose tolerance, but I do not know the application).

But, this raises more questions:

How was the weight measured? Was the reported value an average of repeated measurements? Was the measuring instrument capable of reading two or three significant digits? What was the measurement uncertainty of the measurement? Was the measurement uncertainty higher than the 25 to 35 grams requirement?

If the reported measurement was an average of n number of measurements made with a two significant digit measuring scale, the reported averaged is always carried to an extra significant digit. If it was three significant digits, then round to four significant digits.

If the measurement uncertainty was +/- 7 grams, the reported value could fall between 17.6 to 31.6 grams. This scenario would require a better measurement process with smaller measurement uncertainty.

For general number rounding conventions, NIST offers Publication SP811 (appendix B.7 on page 43) which provides a good reference. It can be downloaded as a free PDF.

Exploring ACM and BPM

Q: Can someone explain how to use Adaptive Case Management (ACM) and how to incorporate it in a process diagram?

A: First, let’s take a look at Business Process Management (BPM), which approaches the problem of improving an organization’s work from a strongly process centric point of view. The first thing you think about is the process. In a certain way, it is the process which defines whether two instances are similar or not.

Data flows into and out of a process. The process represents the goal of a particular sequence of actions, but that goal is not itself an information resource. The process instance contains process relevant data, as well as application data, but it generally assumes that the data duplicates data that has its source elsewhere. This is the main point about “integration” of the process into other information resources.

BPM might be visualized as in this diagram:

Adaptive Case Management (ACM) also tries to improve the performance of an organization, but instead of considering the process to be primary, it is the case information that is primary. This case information is an information resource which will be accessed over the duration of use, and in many situations will become the official record (system of record) for that work.

There can be processes, but the processes are brought to the case, and run in the context of the case, rather than the other way around.

An ACM system might be envisioned as in this diagram:

Both approaches deal with:

Process relevant data
Allowing for process requirements
Producing historical information that can be analyzed to determine the efficiency of the group involved
Availability to multiple people
People who are notified of tasks
Capturing the results of tasks

At a technical level, these are similar or perhaps even identical. But at a methodological level, i.e., how you approach a given problem, they are at the opposite ends of a spectrum.

In BPM, the process is primary. Thus, it is predetermined and static, while the data flows through it. However, with ACM it is the data that is primary. This tends to persist for a long period of time, but the processes are brought to the data. In many cases with ACM, the processes are not even fully predefined, but must be defined on the fly.

The net result is that BPM and ACM are useful for different kinds of business situations. For example:

Highly predictable and highly repeatable business situations are best supported with BPM, e.g., signing up for cell phone service: it happens thousands of times a day, and the process is essentially fixed.
Unpredictable and unrepeatable business situations are best handled with ACM, e.g., investigation of a crime requires following up on various clues, going down various paths which are not predictable in advance. There are various tests and procedures to use, but they will be called upon only when required.

Jack B. ReVelle, Ph.D.
A Consulting Statistician
ReVelle Solutions, LLC
Santa Ana, CA
www.ReVelleSolutions.com

For more on this topic, please visit ASQ’s website.

Guidance on Z1.4 Levels

Chart, graph, sampling, plan, calculation, z1.4

Q: My company is using ANSI/ASQ Z1.4-2008 Sampling Procedures and Tables for Inspection by Attributes, and we need some clarification on the levels and the sampling plans.

We are specifically looking at Acceptable Quality Limits (AQLs) 1.5, 2.5, 4.0, and 6.5 for post manufacturing of apparel, footwear, home products, and jewelry.

Do you have any guidelines to determine when and where to use levels I, II, and III? I understand that level II is the norm and used most of the time. However, we are not clear on levels I and III versus normal, tightened, and reduced.

Are there any recommended guidelines that correlate between levels I, II, III and single sampling plans, normal, tightened, and reduced?

The tables referenced in the standard show single sampling plans for normal, tightened, and reduced, can you confirm that these are for level II (pages 11, 12, 13)?

Do you have any tables showing the levels I and III for normal, tightened, and reduced?

A: Level I is used when you need less discrimination or when you are not as critical on the acceptance criteria. This is usually used for cosmetic defects where you may have color differences, but it is not noticeable in a single unit. Level III is used when you want to be very picky. This is a more difficult level to get acceptance with, so it needs to be used sparingly or it can cost you a lot of money.

Each level has a normal, tightened and reduced scheme. I am not sure about what you are asking for with respect to correlation to levels I, II and III and normal, tightened and reduced. The goal is to simply inspect the minimum amount to get an accept or reject decision. Since inspection costs money, we do not want to do too much. Likewise, we do not want to reject much since that also costs money both in product availability and extra shipping.

Yes, the tables on pages 11, 12 and 13 are for normal, tightened, and reduced, but if you look at the letters for sample size, you will note that in most cases there are different letters for the levels I, II, and III. Accept and reject numbers are based on the defect level and the sample size. The switching rules tell you when you can switch to either a reduced or tightened level. The tables can handle not just the levels I, II , and III, but also the special levels.

Jim Bossert
SVP Process Design Manger, Process Optimization
Bank of America
ASQ Fellow, CQE, CQA, CMQ/OE, CSSBB, CMBB
Fort Worth, TX

OHSAS 18001 and ISO 9001 Work Environment Requirements

Workplace safety, OHSAS 18001, work environments

Q: We had the opportunity to get the certification for OHSAS 18001:2007 Occupational health and safety management systems — Requirements. While looking at the clause interaction between ISO 9001 Quality management systems–Requirements and OHSAS 18001 given at the end of the standard, I did not find any interaction between the standards for clause 6.4 work environment in ISO 9001.

Am I missing anything or is there any reason for it?

A: I am a U.S. Technical Expert for ISO 9001 and associated quality management system (QMS) standards and have been involved with QMS standards since 1975.

In my opinion, the answer to your question is that the developers of OHSAS 18001:2007 did not feel that ISO 9001 clause 6.4 related to 18001. This, incidentally, I find puzzling.

The requirement in ISO 9001:2008 Quality management systems–Requirements clause 6.4 reads: The organization shall determine and manage the work environment needed to achieve conformity to product requirements.

In other words, you should make sure that your employees have an adequate work environment for producing your products. They should have adequate room temperature, lighting, and etc.

The 2005 report: Integrated Management Systems (IMS) – Potential Safety Benefits Achievable from Integrated Management of Safety, Health, Environment and Quality (SHE&Q) from Environment Directorate, Organisation For Economic Cooperation And Development, Paris, includes the following which might be of interest to you:

“OHSAS 18001 and National Standards

During drafting of the original BS 8800 a major division of opinion arose as to whether or not independent assessment and certification of an organisation’s OSHMS should be encouraged, as for QMS and EMS. Some viewed such certificates as valuable, particularly in the context of effective supply chain management, others believed that existing certification processes: added minimal value, required excessive resources and resulted in unused manuals – so new certification processes should be resisted. It proved impossible to reconcile these views within BS8800, which was structured and published as a non-certifiable standard.

As a result, an international consortium of certification bodies, including the commercial arm of BSI, produced the OHSAS 18001 specification in 1999, followed by implementation guidelines OHSAS 18002 in 2000. Neither document is an official British Standard, but OHSAS 18001 either is, or is likely to become, a national standard in other countries, notably in Pacific Rim. A recent survey by BSI identified that over 8000 OSHMS certificates have been issued in 70 countries, to many different standards and guidance, and that some 46% are to OHSAS 18001.

With the revision of BS 8800, from which it is derived, it might be presumed that OHSAS would be updated automatically. A review is indeed planned, but the decision on when to publish a revision will take into account other factors, including the needs of current new users to have time to ‘bed down’ their internal processes before revising them to meet an improved standard. When a revision is agreed, it is likely to include some alignment with other high-quality national standards such as AUS/NZ 4801, to aid recognition as a truly global standard.

A new US standard was published in 2005: ANSI/AIHA Z10 – Occupational Health and Safety Systems. The format includes both a standard and associated guidance, but is not intended as a basis for certification. It is fully compatible with ISO 9001/14001 and takes account of the other national/global OSHMS documents outlined in this section.”

OHSAS 18001:2007 is not an ISO standard. It appears to be simply an update of OHSAS 18001:2000. Its development was driven by the British Standards Institute which publishes the standard and profits directly from its distribution and sales.

Part of the answer to your question is to evaluate for yourself:

1) Why did you go to the expense to be certified to 18001:2007 and who were the customers that you were satisfying by doing this?

2) What is the expectation of these customers?

From a practical standpoint, consider embracing the concept in ISO 9001 clause 6.4. I would expect that providing your employees adequate conditions for producing products can only improve your product offerings and help to enhance customer satisfaction.

Joe Tsiakals
Voting member of the U.S. TAG to ISO/TC 176 (ASQ)
Voting member of the U.S. TAG to ISO/TC 210 (AAMI)