dilip shah Archives -

Charging Lab Fees

Calipers, Measuring, Measurement, Accreditation, ISO 17025

Question

We are in the process of getting our new laboratory ISO 17025 certified. At this moment, our lab is a part of a parent organization. However, it has been requested that the lab should have its own name and with the ability to charge fees, etc. My question is would that be possible without establishing the lab as its own company (LLC, division, etc.)? If we charge fees, would we be considered a third party lab in relation to our parent company with our own, HR, admins, IT, etc separate from the parent company?

Answer

The ISO/IEC 17025 accreditation (not certification) process allows for captive laboratory under a parent organization to become accredited. ISO/IEC 17025:2005 Clause 4.1.4 states:

“If the laboratory is part of an organization performing activities other than testing and/or calibration, the responsibilities of key personnel in the organization that have an involvement or influence on the testing and/or calibration activities of the laboratory shall be defined in order to identify potential conflicts of interest.

NOTE 1 Where a laboratory is part of a larger organization, the organizational arrangements should be such that departments having conflicting interests, such as production, commercial marketing or financing do not adversely influence the laboratory’s compliance with the requirements of this International Standard.

NOTE 2 If the laboratory wishes to be recognized as a third-party laboratory, it should be able to demonstrate that it is impartial and that it and its personnel are free from any undue commercial, financial and other pressures which might influence their technical judgement. The third-party testing or calibration laboratory should not engage in any activities that may endanger the trust in its independence of judgement and integrity in relation to its testing or calibration activities.”

There is no requirement that the “laboratory should have its own name and have ability to charge fees”. It is up to the laboratory to organize in a manner that it sees fit as long potential conflicts with the parent organization are identified. Notes 1 and 2 provide more guidance for this Clause. Nor is there a requirement that it have its own IT, HR and other departments as long as those arrangements are identified and any potential conflict of interest resolved to the Accrediting Body’s interest.

Dilip A. Shah
ASQ Fellow, ASQ-CQE, CQA, CCT,
President, E = mc3 Solutions,
Technical Director, Sapphire Proficiency Testing Services
Chair, ASQ Measurement Quality Division (2016, 2012-2013, 2007, 2004-2005)
Past Member of the A2LA Board of Directors (2006-2014)
Tel: 330-328-4400
Fax: 1-888-226-9533
E-mail: emc3solu@aol.com

For more on this topic, please visit ASQ’s website.

Auditor’s Responsibilities

Root cause analysis figure

Question

Is it an auditor’s responsibility to seek the “root cause” while conducting an audit?

Answer

An auditor should not seek the root cause for an audit finding. An auditor’s responsibility is to verify compliance with a requirement (e.g. ISO 9001 standard) and determine if there is compliance with the requirement or not. In doing so, there is objectivity in making that assessment.

If an auditor determines the root cause, it introduces subjectivity and potential conflict of interest to the audit process and in correcting an issue. In addition, the auditor may not have the full information about the issue thus the “root cause determined by the auditor” may not correct the non-compliance to the requirement.

Best Regards,

Dilip

Dilip A. Shah ASQ Fellow, ASQ-CQE, CQA, CCT,
President, E = mc3 Solutions,
Technical Director, Sapphire Proficiency Testing Services
Past Chair, ASQ Measurement Quality Division (2012-2013)
Past Member of the A2LA Board of Directors (2006-2014)
Tel: 330-328-4400
Fax: 1-888-226-9533
E-mail: emc3solu@aol.com

For more on this topic, please visit ASQ’s website.

The Role of an Observer During an Audit

Audit, audit by exception

Question

A customer of ours wants to participate as an observer in an upcoming audit. I’ve not been able to find much information about the role of observer – what they can and cannot do.

For instance, I assume that they cannot ask questions during the audit interview process. Does anyone have an appropriate checklist for an observation – list of dos and don’ts?

Answer

The auditors should be notified of a presence of the observer in advance. There are times where this may not be allowed depending on the type of the audit.

The customer should sign a confidentiality agreement on not disclosing any information outside the audit process. The rules should be established as part of this confidentiality agreement.

An observer (customer) may not engage in any part of the audit.

The observer may not interfere in any aspect of the audit (may not inject, provide opinions, argue a finding, speak for or against a finding, use the audit information for a future punitive measure).

If questioned during the audit, the observer should explain the role as observer. Ideally this should be brought to the attention of the auditor in advance.

These basic rules ensure that the audit is not compromised in any way and the customer’s request to witness the audit is conducted in a professional manner.

Dilip A Shah
ASQ Fellow, ASQ CQE, CQA, CCT
President, E = mc3 Solutions,
Technical Director, Sapphire Proficiency Testing Services.
Past Chair, ASQ Measurement Quality Division (2012-2013)
Former Member of the A2LA Board of Directors (2006-2014)

For more on this topic, visit ASQ’s website.

“As Found” Calibration Data – Available for a Fee?

Automotive inspection, TS 16949, IATF 16949

Q: I have been an auditor of ISO/ANSI/ASQ 9001:2008 Quality management systems–Requirements since 1992 and recently began consulting hospitals who seek ISO 9001 certification.

My experience with auditing to ISO 9001 is mostly in the manufacturing sector. When I audited against ISO 9001 clause 7.6 control of monitoring and measuring equipment, I routinely included questions regarding the process for assessing the validity of previous measurement results when equipment did not conform to established limits. I found no real issues with this until lately.

Now, clients say that calibration service providers do not routinely provide “as found” data in the report that’s sent to clients/customers. I have been told that “as found” data only becomes available to the client/customer for an additional charge (and it’s not cheap).

Obviously, organizations cannot comply with the ISO 9001 requirement to perform the aforementioned assessment without this data. Since this has only come to my attention recently, I am wondering about the ethics and legality of withholding specific information in the calibration report – unless an additional fee is paid.

Could you please provide some insight or justification for this business practice?

A: It is always a good idea to evaluate one’s suppliers. This requirement is in ISO 9001 clause 7.4 purchasing. The May 2010 Quality Progress Measure for Measure column, “Supplier Demand,” provides guidance on evaluating and selecting calibration providers accredited to ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories. In addition, the ILAC-P14:12/2010 policy document requires ISO/IEC 17025 accredited laboratories to provide measurement uncertainty data with the measurement results as of December 1, 2011.

The customer should specify their requirements in their purchasing documents for calibration. ISO/IEC 17025 has contract review requirements that accredited laboratories must meet in order to to comply with clause 4.4 of ISO/IEC 17025.

In order for the laboratory to make an out of tolerance decision, it has to measure “as found” data. Even if the laboratory does not report it, it is required to retain it per ISO/IEC 17025 clause 5.10.4.2, second paragraph:

“When a statement of compliance with a specification is made omitting the measurement results and associated uncertainties, the laboratory shall record those results and maintain them for possible future reference.”

So, for a start, it is a good idea to use ISO/IEC 17025 accredited calibration providers and specify the customer’s requirements. Some provide “as found – as left” data routinely. Others may charge because they may claim that it takes extra time. But, if a competing laboratory provides it as part of the service, the other laboratories will follow suit or lose market share.

If the ISO/IEC 17025 accredited providers have to make a compliance decision on an item being calibrated, why would they not record the data? Even if it’s not provided, they are required to retain it for future reference in case of an inquiry. Calibration providers (whether accredited or not) that do not provide “as found – as left” data should probably be avoided. One does not know if they provided a legitimate calibration or they “stickered” the calibrated item and produced a generic certificate.

Other laboratories complying with ANSI Z540-1 or ANSI Z540.3 requirements are also required to provide “as found – as left” data. Otherwise, they are not fully complying with Z540 requirements.

The September 2010 Quality Progress Measure for Measure column, “Calibration Evaluation,” discusses evaluating non-accredited calibration providers and what to look for when assessing them.

Dilip A Shah
ASQ CQE, CQA, CCT
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio
http://www.emc3solutions.com

For more on this topic, please visit ASQ’s website.

Gage R&R Study on a Torque Wrench

Gage R&R, Torque Wrence

Q: I need information on performing a Gage R&R on a torque wrench. We are using the wrench to check customer parts.

A: For reference on both variable and attribute Gage R & R techniques, a good source is the Automotive Industry Action Group (AIAG) Measurement Systems Analysis (MSA) publication.

The traditional torque wrench is a “generate” device in the sense that it generates a torque to tighten or loosen a fastener (a nut or a bolt, etc.). So, in a strict sense, it is not a “measurement” device. Therefore, both preset and settable torque wrenches are set to a torque value and then used to tighten a fastener or loosen a fastener. When loosening a fastener, it will determine how much torque is required to loosen the fastener. Usually, the clockwise motion is for tightening and counterclockwise motion is for loosening in a torque wrench.

To conduct a variable Gage R & R study on a torque wrench, we would need a “measurement” device which would be a torque checker with a capability to register peak (or breaking) torque. Many such devices are commercially available and if a facility is using torque wrenches, it is a good idea to have one of these to verify performance of torque wrenches. Such a device is usually calibrated (ensure traceable accredited calibration) and provides reference for proper working of torque wrenches.

Now, one would conduct a Gage R&R study using the typical format:

Two or more appraisers.
5 to 10 repeat measurements at a preset torque by each appraiser, replicated 2 to 3 or more times.

A word of caution on torque wrenches and setting up the Gage R&R:

The measurement is operator dependent, so operators need to be trained on proper toque wrench usage techniques.
Ensure that torque is set between every measurement in the settable torque wrench to simulate actual usage between repeated readings.
Ensure the number of repeated reading and replicated readings are the same for all appraisers.

The templates for data collection are available in spreadsheet format from commercial providers. Alternatively, one can design the template from the MSA publication referenced. The data would be analyzed using the guidelines from the MSA publication.

Good luck with the Gage R&R! It is a very useful and worthwhile exercise in understanding your measurement process.

Related Content:

To learn more about this topic, visit ASQ’s website.

ISO 17025 Clause 5.4.2 – Selection of Methods

Q: We are working with the Mexican Accreditation Entity (EMA) for certification to ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories. Clause 5.4.2 states: The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.

We are a testing laboratory and work with Method 21 – Determination of Volatile Organic Compound, EPA 40 CFR Ch.1 ( 01/07/04 Edition ) Test: Monitoring of Fugitive Emissions.

The question is: What would be the best way or a way to confirm the method? Or, to put it another way, how can we satisfy the requirements in clause 5.4.2 ?

A: The questioner is referring to clause 5.4.2 from ISO/IEC 17025:2005. An excerpt of this clause is below. Please refer to ISO/IEC 17025:2005 for the full clause.

5.4.2 Selection of methods

“…Methods published in international, regional or national standards shall preferably be used….. Laboratory-developed methods or methods adopted by the laboratory may also be used if they are appropriate for the intended use and if they are validated…. The customer shall be informed as to the method chosen. The laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations.…”

Since the questioner is using the published methods, there is no need for validation of the method unless the method is modified.

However, the proficiency of being able to apply the published method needs to be demonstrated. This can be demonstrated by a documented Gage R & R study, Analysis of Variance (ANOVA) or Design of Experiments (DOE) study as appropriate to show proficiency in being able to utilize the test method properly.

The results from these studies may also be used to estimate the uncertainty of measurement for the tests. Reporting uncertainty of measurement with both test and calibration results is a requirement in ISO/IEC 17025:2005. The ILAC P14 document is a good guidance document on reporting uncertainty.

For more on this topic, please visit ASQ’s website.

Calibration of AutoCAD Software

About ASQ's Ask the Standards Expert program and blog

Q: To what extent must an engineering firm, specializing in railway infrastructure and transportation, have its AutoCAD software “calibrated” or verified?

Also, what about software designed to calculate earthwork quantities for railway alignments laid out on topographic mapping for all levels of studies – pre-feasibility through preliminary engineering (not for final design, operation simulation and design dynamic system models)? This type of software is utilized by competent draft persons and engineers, but it is not verified prior to use or periodically calibrated.

We don’t confirm “the ability of computer software to satisfy the intended application…”

Your assistance or reference is appreciated

A: AutoCAD is considered “Commercial -Off-The-Shelf” (COTS) software. It is purchased without modification and cannot be modified by the end-user. A similar example would be Excel spreadsheet software. The COTS software by itself should be considered validated and used as is provided it is configured per the software manufacturer’s instructions.

The functionality of the software (distance, volume, formulae and other functions) is fit to be used as intended. If an application is created using COTS software (Excel Templates, AutoCAD applications), then it must be validated and records of validation must be kept.

It should also be noted that definitions of verification and validation are not clearly understood. So, I am repeating them here:

ISO/IEC Guide 99:2007—International vocabulary of metrology—Basic and general concepts and associated terms, defines these terms as:

Verification: provision of objective evidence that a given item fulfills specified requirements

Validation: verification, where the specified requirements are adequate for an intended use

Further explanation:

Validation is a quality assurance process of establishing evidence that provides a high degree of assurance that a product, service, or system accomplishes its intended requirements. This often involves acceptance of fitness for purpose with end users and other product stakeholders.

It is sometimes said that validation can be expressed by the query “Are you building the right product?” and verification by “Are you building it right?”

“Building the right thing” refers back to the user’s needs, while “Are we building the product right?” checks that the specifications are correctly implemented by the system. In some contexts, it is required to have written requirements for both as well as formal procedures or protocols for determining compliance.

For more on this topic, please visit ASQ’s website.

ISO 17025; Rounding Measurements

ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories

Q: At the lab I work for, certified to ISO 17025:2005 General requirements for the competence of testing and calibration laboratories, the documented quality assurance system does not allow the rounding of numbers. For example, the requirement for the weight of an adhesive material is 25 to 35 grams, and the actual weight is 24.6 grams.

The engineering member of the team feels this is acceptable because 25 grams is specified with two significant figures; 24.6 grams, expressed as two significant figures is 25 grams. If the intent was not to round off in the tenths place, the document would read “25.0” and rounding would be in the hundredths.

A: If the requirement (specification) is 25 to 35 grams, the need to specify accurately (24.6 grams) is not as critical and the number can be rounded to 25 grams. We would assume that the nominal desired value would be 30 grams. (Personal opinion: the 25 to 35 gram requirement is a fairly loose tolerance, but I do not know the application).

But, this raises more questions:

How was the weight measured? Was the reported value an average of repeated measurements? Was the measuring instrument capable of reading two or three significant digits? What was the measurement uncertainty of the measurement? Was the measurement uncertainty higher than the 25 to 35 grams requirement?

If the reported measurement was an average of n number of measurements made with a two significant digit measuring scale, the reported averaged is always carried to an extra significant digit. If it was three significant digits, then round to four significant digits.

If the measurement uncertainty was +/- 7 grams, the reported value could fall between 17.6 to 31.6 grams. This scenario would require a better measurement process with smaller measurement uncertainty.

For general number rounding conventions, NIST offers Publication SP811 (appendix B.7 on page 43) which provides a good reference. It can be downloaded as a free PDF.

Using the 10:1 Ratio Rule and the 4:1 Ratio Rule

Q: Can you explain when I should be using the 10:1 ratio rule and the 4:1 ratio rule within my calibration lab? We calibrate standards as well as manufacturing gages.

A: First, I will use the right nomenclature. What the user means is 10:1 and 4:1 Test Accuracy Ratio (TAR). That is, one uses standards 4 or 10 times as accurate as the Unit Under Test (UUT) to calibrate it with.

Unfortunately, the answer to the user’s question is NEVER if we were to use newer metrologically accepted practices.

The TAR is replaced by Test Uncertainty Ratio (TUR). The ANSI/NCSLI Z540.3:2006 definition of TUR is:

“The ratio of the span of the tolerance of a measurement quantity subject to calibration, to twice the 95% expanded uncertainty of the measurement process used for calibration.”

*NOTE: This applies to two-sided tolerances.

The TUR is represented as a mathematical equation below:

Test Uncertainty Ratio (TUR) represented as an equation

Because of advances in technology, one can purchase highly precise and accurate instrumentation at the end user level, it gets challenging to find standards 4 or 10 times as precise with which to calibrate it and maintain metrological traceability at the same time (definition per ISO Guide 99:2007, Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty).

Proper measurement uncertainty analysis of the UUT (including standards used with its uncertainty) identifies all the errors associated with the measurement process and ensures confidence that calibration is within the specification desired by the end user.

ISO/IEC 17025-2005: General requirements for the competence of testing and calibration laboratories, clause 5.10.4.2, third paragraph, also states that “When statements of compliance are made, the uncertainty of measurement shall be taken into account.”

This would also ensure confidence in the calibration employing the metrological and statistical practices recommended.

The other rule of thumb not to be confused in this discussion is to measure/calibrate with the right resolution. In the ASQ Quality Progress March 2011 Measure for Measure column, I wrote more on resolution with respect to specification and measurement uncertainty. The general rule of the thumb is if you want to measure/calibrate a 2-decimal place resolution device, you need at least 3-decimal place or higher resolution device.

This is a very good question posed and it is also unfortunately the most misunderstood practice among a lot of folks performing calibration.