Q: At the lab I work for, certified to ISO 17025:2005 General requirements for the competence of testing and calibration laboratories, the documented quality assurance system does not allow the rounding of numbers. For example, the requirement for the weight of an adhesive material is 25 to 35 grams, and the actual weight is 24.6 grams.
The engineering member of the team feels this is acceptable because 25 grams is specified with two significant figures; 24.6 grams, expressed as two significant figures is 25 grams. If the intent was not to round off in the tenths place, the document would read “25.0” and rounding would be in the hundredths.
A: If the requirement (specification) is 25 to 35 grams, the need to specify accurately (24.6 grams) is not as critical and the number can be rounded to 25 grams. We would assume that the nominal desired value would be 30 grams. (Personal opinion: the 25 to 35 gram requirement is a fairly loose tolerance, but I do not know the application).
But, this raises more questions:
How was the weight measured? Was the reported value an average of repeated measurements? Was the measuring instrument capable of reading two or three significant digits? What was the measurement uncertainty of the measurement? Was the measurement uncertainty higher than the 25 to 35 grams requirement?
If the reported measurement was an average of n number of measurements made with a two significant digit measuring scale, the reported averaged is always carried to an extra significant digit. If it was three significant digits, then round to four significant digits.
If the measurement uncertainty was +/- 7 grams, the reported value could fall between 17.6 to 31.6 grams. This scenario would require a better measurement process with smaller measurement uncertainty.
For general number rounding conventions, NIST offers Publication SP811 (appendix B.7 on page 43) which provides a good reference. It can be downloaded as a free PDF.
Dilip A Shah
ASQ CQE, CQA, CCT
President, E = mc3 Solutions
Chair, ASQ Measurement Quality Division (2012-2013)
Secretary and Member of the A2LA Board of Directors (2006-2014)
Medina, Ohio
http://www.emc3solutions.com