calibration Archives -

AS9100 7.1.5.2 Measurement Traceability

Airplane, aerospace, AS9100

Question

Can a manufacturer use “reference only” M&M equipment to accomplish in-process checks as long as these items are verified later on by “Inspection” using calibrated and/or verified equipment?

Would all of the measurements have to be verified by properly calibrated equipment?

Answer

The practice of using “reference-only” devices for in-process measures is not noncompliant with the standard. The formal inspections to accept product would require use of calibrated equipment.

In clause 7.1.5.1, the standard clearly states “the organization shall determine and provide the resources needed to ensure valid and reliable results when monitoring or measuring is used to verify the conformity of products and services to requirements.” And in clause 7.1.5.2, “hen measurement traceability is a requirement, or is considered by the organization to be an essential part of providing confidence in the validity of measurement results…”

Users and auditors should look at formal inspections steps and final buy-off to determine if any nonconformities were escapes from in-process measurements being conducted without calibrated devices. Customer complaints can also be analyzed to see if any escapes were a result of failed in-process inspections. If so, then the organization did not properly determine and provide the appropriate resources.

Buddy Cressionnie

For more information on this topic, please visit ASQ’s website.

Calibration Questions

Automotive inspection, TS 16949, IATF 16949

Question

I work at a hydraulic cylinder manufacturer. The company has homemade thread and ring gages in house that we are using for production that are not sent out for calibration but have a homemade master that is used to check them with once a year which does not get sent out either. I have been here six months and am thinking these gages and masters are a violation of ISO. Am I correct?

Answer

The short answer is yes. The intent of ISO 9001:2015’s subclause 7.1.5 is to ensure that your company determines and provides suitable resources to ensure valid and reliable monitoring and measuring results, when evaluating the conformity of your products; and 7.1.5.2’s that is to ensure that your company provides measurement traceability when it is a requirement or when your company determines it to be necessary to have confidence in the validity of the measurement results. It seems that your practice for controlling the homemade thread and ring gages cannot fully fulfill those purposes. This is how I would address the situation:

Assign a unique identifier to each homemade thread and ring gage. Maybe you can do that through your Document Control process.
Ensure that those gages are protected from deterioration or damage when they are not in use.
Have the homemade master measured by a service able to provide you with reliable certified measurements. That will make that gauge traceable to national or international standards. It will also allow you to demonstrate that the piece is fit for its intended purpose. That means, you will be able to use this piece as the standard during the in-house calibration of the rest of the gages.
Conduct an “in-house calibration” of each gage you use in production. You will need to issue an in-house calibration certificate for each one of those pieces, indicating on those documents how you achieve traceability to NIST or equivalent. If possible, identify the error for each one of those individual measurements you perform during calibration. Do not forget to include a statement indicating that the gauge was found suitable/unsuitable for use. That will demonstrate that each gage is fit for its intended purpose.
Include ALL the gages in your calibration program. Make them subject to all the applicable provisions of your Quality System.

This approach will allow you to demonstrate that your thread and ring gages are properly controlled and maintained. If controlling those gages has not been an issue in the past, there is no guarantee that the situation will remain the same in the future. That is managing risk 😉

Aura Stewart

For more on this topic, please visit ASQ’s website.

“As Found” Calibration Data – Available for a Fee?

Automotive inspection, TS 16949, IATF 16949

Q: I have been an auditor of ISO/ANSI/ASQ 9001:2008 Quality management systems–Requirements since 1992 and recently began consulting hospitals who seek ISO 9001 certification.

My experience with auditing to ISO 9001 is mostly in the manufacturing sector. When I audited against ISO 9001 clause 7.6 control of monitoring and measuring equipment, I routinely included questions regarding the process for assessing the validity of previous measurement results when equipment did not conform to established limits. I found no real issues with this until lately.

Now, clients say that calibration service providers do not routinely provide “as found” data in the report that’s sent to clients/customers. I have been told that “as found” data only becomes available to the client/customer for an additional charge (and it’s not cheap).

Obviously, organizations cannot comply with the ISO 9001 requirement to perform the aforementioned assessment without this data. Since this has only come to my attention recently, I am wondering about the ethics and legality of withholding specific information in the calibration report – unless an additional fee is paid.

Could you please provide some insight or justification for this business practice?

A: It is always a good idea to evaluate one’s suppliers. This requirement is in ISO 9001 clause 7.4 purchasing. The May 2010 Quality Progress Measure for Measure column, “Supplier Demand,” provides guidance on evaluating and selecting calibration providers accredited to ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories. In addition, the ILAC-P14:12/2010 policy document requires ISO/IEC 17025 accredited laboratories to provide measurement uncertainty data with the measurement results as of December 1, 2011.

The customer should specify their requirements in their purchasing documents for calibration. ISO/IEC 17025 has contract review requirements that accredited laboratories must meet in order to to comply with clause 4.4 of ISO/IEC 17025.

In order for the laboratory to make an out of tolerance decision, it has to measure “as found” data. Even if the laboratory does not report it, it is required to retain it per ISO/IEC 17025 clause 5.10.4.2, second paragraph:

“When a statement of compliance with a specification is made omitting the measurement results and associated uncertainties, the laboratory shall record those results and maintain them for possible future reference.”

So, for a start, it is a good idea to use ISO/IEC 17025 accredited calibration providers and specify the customer’s requirements. Some provide “as found – as left” data routinely. Others may charge because they may claim that it takes extra time. But, if a competing laboratory provides it as part of the service, the other laboratories will follow suit or lose market share.

If the ISO/IEC 17025 accredited providers have to make a compliance decision on an item being calibrated, why would they not record the data? Even if it’s not provided, they are required to retain it for future reference in case of an inquiry. Calibration providers (whether accredited or not) that do not provide “as found – as left” data should probably be avoided. One does not know if they provided a legitimate calibration or they “stickered” the calibrated item and produced a generic certificate.

Other laboratories complying with ANSI Z540-1 or ANSI Z540.3 requirements are also required to provide “as found – as left” data. Otherwise, they are not fully complying with Z540 requirements.

The September 2010 Quality Progress Measure for Measure column, “Calibration Evaluation,” discusses evaluating non-accredited calibration providers and what to look for when assessing them.

Dilip A Shah
ASQ CQE, CQA, CCT
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio
http://www.emc3solutions.com

For more on this topic, please visit ASQ’s website.

Difference Between ISO/IEC 17025 and ISO 10012

Q: I am updating the instrumentation section of a product fabrication specification to replace a cancelled military specification (MIL-STD 45662) that specified calibration systems requirements. I am looking for an industry standard that provides requirements/guidance for documentation of our established schedules and procedures for all of our measuring and test equipment and measurement standards.

I am looking into ANSI/ISO/ASQ Q10012-2003: Measurement management systems — Requirements for measurement processes and measuring equipment and ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories, and I would like guidance on usage and application of these standards.

A: The two standards in question, ISO 10012 and ISO 17025 have different scopes.

While the scope of both documents includes language that can perhaps cause confusion, what follows is the salient text from both that illuminates the difference between the two.

From the scope of ISO 10012:

“It specifies the quality management requirements of a measurement management system that can be used by an organization performing measurements as part of the overall management system, and to ensure metrological requirements are met.”

From scope of ISO 17025:

“This International Standard is for use by laboratories in developing their management system for quality, administrative and technical operations.”

ISO 10012 focuses on the requirements of the measurement management system. You can consider it a system within the quality management system. It defines requirements relevant to the measurement management system in language that may illustrate interrelations to other parts of an overall quality management system.

ISO 10012 is a guidance document and not intended for certification. An organization, for example, could have a quality management systems that is certified to ISO 9001:2008. Even if the organization chooses to adhere to the requirements of ISO 10012, the certification to ISO 9001 does not imply certification to the requirements of ISO 10012.

ISO 17025 describes the requirements for a quality management system that can be accredited (a process comparable but different from certification). It encompasses all aspects of the laboratory.

The competence referred to in the title of the standard relates to the competence of the entire system – not just training of personnel. It addresses such factors as contracts with customers, purchasing, internal auditing, and management review of the entire quality management system – ISO 10012 does not.

In summary, ISO 10012 is a guidance document that addresses one element (namely management of a measurement system) of a quality management system. ISO 17025 defines requirements for entire quality management system that can be accredited.

Denise Robitaille
Vice Chair, U.S. TAG to ISO/TC 176 on Quality Management and Assurance
SC3 Expert – Supporting Technologies

Related Content:

Expert Answers: Metrology Program 101, Quality Progress

Measure for Measure: First Step Toward Disaster, Quality Progress

10 Quality Basics, Quality Progress

Standards Column: Using the Whole ISO 9000 Family of Quality Management System Standards, Quality Engineering

ISO 9001 7.6a Calibration and Traceability

Gage R&R, Torque Wrence

Q: ANSI/ISO/ASQ Q9001-2008 Quality management systems — Requirements, clause 7.6a states, in part:

“Where necessary to ensure valid results, measuring equipment shall

a) be calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards traceable to international standards or national measurement standards…”

Does this sub clause require that the calibration process be performed in accordance with international or national calibration procedures? Or does it require that the measurement standards (hardware) used for calibration be traceable to international or national measurement standards (hardware)?

A: The standard is clear that it is the traceability of the calibration standards they are looking for.

Note: By definition, the traceability needs to eventually lead to an accredited lab who will be following procedures such as those set forth in ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories.

Your internal calibration processes can best be guided by acquiring a copy of ANSI/NCSL Z540.3.

I hope this helped answer your questions.

Bud Salsbury
ASQ Senior Member, CQT, CQI

Related Content:

For more on this topic, visit ASQ’s website.

ISO 17025 Clause 5.4.2 – Selection of Methods

Q: We are working with the Mexican Accreditation Entity (EMA) for certification to ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories. Clause 5.4.2 states: The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.

We are a testing laboratory and work with Method 21 – Determination of Volatile Organic Compound, EPA 40 CFR Ch.1 ( 01/07/04 Edition ) Test: Monitoring of Fugitive Emissions.

The question is: What would be the best way or a way to confirm the method? Or, to put it another way, how can we satisfy the requirements in clause 5.4.2 ?

A: The questioner is referring to clause 5.4.2 from ISO/IEC 17025:2005. An excerpt of this clause is below. Please refer to ISO/IEC 17025:2005 for the full clause.

5.4.2 Selection of methods

“…Methods published in international, regional or national standards shall preferably be used….. Laboratory-developed methods or methods adopted by the laboratory may also be used if they are appropriate for the intended use and if they are validated…. The customer shall be informed as to the method chosen. The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.…”

Since the questioner is using the published methods, there is no need for validation of the method unless the method is modified.

However, the proficiency of being able to apply the published method needs to be demonstrated. This can be demonstrated by a documented Gage R & R study, Analysis of Variance (ANOVA) or Design of Experiments (DOE) study as appropriate to show proficiency in being able to utilize the test method properly.

The results from these studies may also be used to estimate the uncertainty of measurement for the tests. Reporting uncertainty of measurement with both test and calibration results is a requirement in ISO/IEC 17025:2005. The ILAC P14 document is a good guidance document on reporting uncertainty.

For more on this topic, please visit ASQ’s website.

Calibration of AutoCAD Software

About ASQ's Ask the Standards Expert program and blog

Q: To what extent must an engineering firm, specializing in railway infrastructure and transportation, have its AutoCAD software “calibrated” or verified?

Also, what about software designed to calculate earthwork quantities for railway alignments laid out on topographic mapping for all levels of studies – pre-feasibility through preliminary engineering (not for final design, operation simulation and design dynamic system models)? This type of software is utilized by competent draft persons and engineers, but it is not verified prior to use or periodically calibrated.

We don’t confirm “the ability of computer software to satisfy the intended application…”

Your assistance or reference is appreciated

A: AutoCAD is considered “Commercial -Off-The-Shelf” (COTS) software. It is purchased without modification and cannot be modified by the end-user. A similar example would be Excel spreadsheet software. The COTS software by itself should be considered validated and used as is provided it is configured per the software manufacturer’s instructions.

The functionality of the software (distance, volume, formulae and other functions) is fit to be used as intended. If an application is created using COTS software (Excel Templates, AutoCAD applications), then it must be validated and records of validation must be kept.

It should also be noted that definitions of verification and validation are not clearly understood. So, I am repeating them here:

ISO/IEC Guide 99:2007—International vocabulary of metrology—Basic and general concepts and associated terms, defines these terms as:

Verification: provision of objective evidence that a given item fulfills specified requirements

Validation: verification, where the specified requirements are adequate for an intended use

Further explanation:

Validation is a quality assurance process of establishing evidence that provides a high degree of assurance that a product, service, or system accomplishes its intended requirements. This often involves acceptance of fitness for purpose with end users and other product stakeholders.

It is sometimes said that validation can be expressed by the query “Are you building the right product?” and verification by “Are you building it right?”

“Building the right thing” refers back to the user’s needs, while “Are we building the product right?” checks that the specifications are correctly implemented by the system. In some contexts, it is required to have written requirements for both as well as formal procedures or protocols for determining compliance.

For more on this topic, please visit ASQ’s website.

ISO 17025; Rounding Measurements

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories

Q: At the lab I work for, certified to ISO 17025:2005 General requirements for the competence of testing and calibration laboratories, the documented quality assurance system does not allow the rounding of numbers. For example, the requirement for the weight of an adhesive material is 25 to 35 grams, and the actual weight is 24.6 grams.

The engineering member of the team feels this is acceptable because 25 grams is specified with two significant figures; 24.6 grams, expressed as two significant figures is 25 grams. If the intent was not to round off in the tenths place, the document would read “25.0” and rounding would be in the hundredths.

A: If the requirement (specification) is 25 to 35 grams, the need to specify accurately (24.6 grams) is not as critical and the number can be rounded to 25 grams. We would assume that the nominal desired value would be 30 grams. (Personal opinion: the 25 to 35 gram requirement is a fairly loose tolerance, but I do not know the application).

But, this raises more questions:

How was the weight measured? Was the reported value an average of repeated measurements? Was the measuring instrument capable of reading two or three significant digits? What was the measurement uncertainty of the measurement? Was the measurement uncertainty higher than the 25 to 35 grams requirement?

If the reported measurement was an average of n number of measurements made with a two significant digit measuring scale, the reported averaged is always carried to an extra significant digit. If it was three significant digits, then round to four significant digits.

If the measurement uncertainty was +/- 7 grams, the reported value could fall between 17.6 to 31.6 grams. This scenario would require a better measurement process with smaller measurement uncertainty.

For general number rounding conventions, NIST offers Publication SP811 (appendix B.7 on page 43) which provides a good reference. It can be downloaded as a free PDF.

Using the 10:1 Ratio Rule and the 4:1 Ratio Rule

Q: Can you explain when I should be using the 10:1 ratio rule and the 4:1 ratio rule within my calibration lab? We calibrate standards as well as manufacturing gages.

A: First, I will use the right nomenclature. What the user means is 10:1 and 4:1 Test Accuracy Ratio (TAR). That is, one uses standards 4 or 10 times as accurate as the Unit Under Test (UUT) to calibrate it with.

Unfortunately, the answer to the user’s question is NEVER if we were to use newer metrologically accepted practices.

The TAR is replaced by Test Uncertainty Ratio (TUR). The ANSI/NCSLI Z540.3:2006 definition of TUR is:

“The ratio of the span of the tolerance of a measurement quantity subject to calibration, to twice the 95% expanded uncertainty of the measurement process used for calibration.”

*NOTE: This applies to two-sided tolerances.

The TUR is represented as a mathematical equation below:

Test Uncertainty Ratio (TUR) represented as an equation

Because of advances in technology, one can purchase highly precise and accurate instrumentation at the end user level, it gets challenging to find standards 4 or 10 times as precise with which to calibrate it and maintain metrological traceability at the same time (definition per ISO Guide 99:2007, Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty).

Proper measurement uncertainty analysis of the UUT (including standards used with its uncertainty) identifies all the errors associated with the measurement process and ensures confidence that calibration is within the specification desired by the end user.

ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories, clause 5.10.4.2, third paragraph, also states that “When statements of compliance are made, the uncertainty of measurement shall be taken into account.”

This would also ensure confidence in the calibration employing the metrological and statistical practices recommended.

The other rule of thumb not to be confused in this discussion is to measure/calibrate with the right resolution. In the ASQ Quality Progress March 2011 Measure for Measure column, I wrote more on resolution with respect to specification and measurement uncertainty. The general rule of the thumb is if you want to measure/calibrate a 2-decimal place resolution device, you need at least 3-decimal place or higher resolution device.

This is a very good question posed and it is also unfortunately the most misunderstood practice among a lot of folks performing calibration.

Measurement Tolerances and Techniques

Q: I am looking for some information regarding blueprint tolerances and measurement tools used to measure certain features.

For example, can the same type of tolerance be applied over the length of 450 mm as it could be for a distance of 3 mm? Is there additional measurement error or gage error that needs to be applied for longer distances? If one uses a 1” micrometer for measuring a feature, does it make a difference in the measurement error by using the top end of the instrument versus using it to measure just very small features?

A: Thank you for your questions about measurement tolerances. First of all, since your questions were multi-layered, my answers will be as well. Nonetheless, I think I should be able to help you.

As for using the same type of tolerance for a dimension of 450 mm and a dimension of 3 mm, there is more than one answer. We’re talking about 17.7165 inches vs. .118 inches. The 3 F’s must first be considered. That is Form, Fit, and Function. In other words, where will this product be used? If this will be for a medical product or for anything whatsoever where safety is a factor, the design engineer will most likely use a tighter tolerance. So both dimensions could be ± .001 or a more liberal ± .010. The difference between the two sizes would just change the way they are manufactured. For example: a 17.717 inch product with a tolerance of ± .030 could probably be rough machined or even made on a burn table. If the size or location of the smaller dimension is critical, you would machine it with appropriate equipment and hold a tighter tolerance. OK, enough Manufacturing 101 lingo.

With regard to measurement error, larger/longer dimensions can introduce the possibility of increased measurement error. However, if a “qualified” and experienced individual is doing the measurement, that should not be a major factor. The same basic skills and standards would apply. The type of measurement equipment can make a difference. In other words; if you use a Dial Caliper, you can probably rely on it to be accurate within .001-.002 inches. If you use a 0-1 inch micrometer, you should be able to trust it’s accuracy within .0001 inch.

A qualified metrologist and/or a quality technician would know to check a micrometer at numerous points over its measuring range. Measurement error should not increase significantly from one end to the other. If it does, there is something wrong with the calibration or with the tool itself.

I know the above can be perceived as general answers, but I am confident you will see the specifics there as well.

Bud Salsbury
ASQ Senior Member, CQT, CQI

For more on this topic, please visit ASQ’s website.