Category: Sampling

Determining Statistically Significant Sample Size

Sterile, Lab, Clean Room, Requirements, Standard

Question

I am developing an internal audit process within our supply chain to determine packaging and Finalizing SOP’s are being followed. I need to determine what will be the sample size needed to accurately represent the population. We are currently shipping out 650k cartons a day. How do I determine how many audits I need a day for statistical significance?

Answer

Statistical sampling theory shows that for large populations, the sample size is not a function of the population size, assuming all units in the population have an equal probability of being selected for the sample.  To ensure a representative sample, stratified random sampling is employed to represent in the audit sample. This method requires that each category (or stratum) is specified, and that none of them overlap (i.e., items to be audited must fall in only one category).  For example, you can break the packaging records in groups of 25,000 (26 stratum for 650,000 records), sampling 1/26th of the sample from each stratum.

To determine the sample size, we employ the binomial distribution where a records is either confirming or nonconforming. 

The basic formula for the binomial confidence interval is

For a given sample size (n) with a given number of defects (x), the probability of the sample coming from a population with probability (p) is given by the value alpha (a).  The above equation can be solved for probability (p) at a given a level or can be solved for a at a given population probability (p).

In other words, you specify the percent defective in the population you can accept.  The only when to ensure 0% defective is 100% sampling. You solve the equation for n by setting 1-alpha (1-a) equal to a high probability (i.e. 95%).  If you desire to accept zero (0) defects in the sample then set x equal to zero. In this case, the equation reduces to ln(1- a)/ln(1-p).

Hope this helps with the question.

Thanks

Steven

For more on this topic, please visit ASQ’s website.

Z 1.4 AQL Levels

Food safety testing, lab, standards

Question

I need help understanding the AQL values in the tables of ASQ Z1.4. They are defined in paragraph 4.5 as percentages or ratios, but there are some values that are less than 1 and greater than 100. How should these values be interpreted?  Since this standard is for attribute data, is there a standard for variable data?

Answer

A percentage can be from 0 to more than 100% depending on what the ratio represents.  First we need to define AQL.  Section 4.2 states “The AQL is the quality level that is the worst tolerable process average when a continuing series of lots is submitted for acceptance sampling.”  Therefore, an AQL of 0.65% means that on average we can accept 65 defects per 10,000 units in a lot.  The sampling plans with percentages greater than 100% are carried over from the MIL-STD-105 and are considered to be antiquated and not used any longer.

The ANSI standard for variable data sampling plans is ANSI/ASQ Z1.9.  It is based on probability of being outside the acceptance region.

Steven Walfish

For more on this topic, please visit ASQ’s website.

Zero Acceptance Number Sampling Plans

Airplane, aerospace, AS9100

Question

Regarding Nicholas Squeglia’s Zero Acceptance Number Sampling Plans, in the 4th edition for lot size 151-280 (1% AQL), a sample size of 20 is provided.  However, in the 5th edition, for the same lot size 151-280 and AQL of 1%, the sample size is 29. Which is correct – a sample size of 20 or 29?

Answer

In the 5th edition of Nicolas Squeglia’s book, he mentions on page xii the rationale of the change in sample sizes.  From  the 5th edition, “in the early 2000’s, a large aerospace manufacturer was given permission by ASQ to reproduce the c=0 sampling table.  They modified the table by changing several sample sizes, and for convenience it was therefore originally decided to carry those modifications into the fifth edition.”

Table 1a is the original tables (4th edition and previous) which has the sample size of 29.  Use this table unless otherwise specified by contract.

Table 1b is the modified table which has a sample size of 20.

Thanks

Steven Walfish

For more on this topic, please visit ASQ’s website.

Sampling Foils, Films, and Labels

Manufacturing, inspection, exclusions

Question

My question is about sampling aluminium foils, films used in packaging and sticker labels received in rolls which are wound around a core. I can decide to chose the number of rolls to sample from using the tables given in Z1.4, but how should I decide on the amount of stickers and aluminium foil and film to be sampled? I ask this question since it is practically impossible to sample from within a wound roll.
Answer

The ANSI Z1.4 and Z1.9 standards might be applicable when all units do not have the same probability of being selected.  Since you cannot sample units closer to the core, and defects would never be detected unless they occur at the end of the roll, I would recommend a different strategy, either using a vision system (100% inspection) or in process inspection.

If you want to use the standard, the sample size should be based on the number of samples, not the number of rolls.  For example, a roll with 5000 labels would be an N=5000 not N=1.

Steven Walfish

For more on this topic, please visit ASQ’s website.

ANSI Z1.4 Reduced Inspection

Sterile, Lab, Clean Room, Requirements, Standard

Question

If you have Ac=0 and Re=2 what do you do for 1? I have not used the reduced sampling before, so am curious what should be done in this instance.

Answer

If you review the footnotes for Table II-C of ANSI Z1.4, you will see that there is a note (†) that states: If the acceptance number has been exceeded , but the rejection number has not been reached, accept the lot, but reinstate normal inspection (see 10.1.4).  So in your case, with a single reject, you would accept and reinstate normal inspection.

Steven Walfish

For more information on this topic, please visit ASQ’s website.

Defective Parts Per Million (DPPM) Calculation

Chart, graph, sampling, plan, calculation, z1.4

Question

Recently, there is a debate in my organization about Defective Parts Per Million (DPPM) computation.

Camp 1 - DPPM = (No of parts rejected / No of parts inspected) * 1,000,000
Camp 2 - DPPM = (No of parts rejected / No of parts received) * 1,000,000

We perform sampling inspection based on AQL.
Camp 1 insists they are correct and likewise for Camp 2.  Which is correct or more appropriate to reflect supplier quality?

Answer

This is not an uncommon question. If you look at the standard, they define the % nonconforming as the number of parts nonconforming/number of parts inspected x 100. If you are looking at DPPM, instead of multiplying by 100, you put in 1,000,000.

This means that by your definition, Camp 1 is correct. This is also what was intended by the creators of the sampling scheme.

Jim Bossert
Sr Performance Improvement Specialist
JPS Hospital
ASQ Fellow, CQE, CQA, CMQ/OE, CSSBB, CSSMBB
Fort Worth, TX

For more on this topic, please visit ASQ’s website.

AQL Clarifications

Automotive inspection, TS 16949, IATF 16949

Question

I am confused about the values used for AQLs. For example in Table II-A the AQL values range from 0.010 to 1000. Where do these values come from and what do they mean?

The table states, “AQLs, in Percent Nonconforming Items and Nonconformities per 100 Items .” At first I thought the values were percentages, but how can you have more than 100, as in 100%, as the values go up to 1000? Also how can there be more than 100 nonconformities per 100 items, unless one part can have multiple nonconformities?

Just looking for clarification on the AQL numbers, what they mean, and how to interpret them.

Answer

Let’s start with the definition of Acceptable Quality Level (AQL).  From Z1.4, the AQL is the quality level that is the worst tolerable process average when a continuing series of lots is submitted for acceptance sampling.  Although individual lots with quality as bad as the AQL can be accepted with fairly high probability, the designation of an AQL does not suggest that this is necessarily a desirable quality level. The AQL is a parameter of the sampling scheme and should not be confused with a process average which describes the operating level of a manufacturing process. It is expected that the product quality level will be less than the AQL to avoid excessive non-accepted lots.

The columns with percentages greater than 100% should not be included in the standard, but remain as indication of how to interpret lots where the entire sample is defective.  It has some statistical relevance with use of the switching rules, but for the general practitioner, it should be ignored.

Hope this helps.

Steven Walfish

For more about this topic, please visit ASQ’s website.

DPMO

Inventory, Inspection, Review, Suppliers, Supplies

Question

My question concerns the process performance metric DPMO (defects per million opportunities). I want to use this to quantify a particular supplier’s performance. My question is, is the number of defects referred to in the calculation the number of defects produced by the supplier (in which case it would involve data I don’t have access to), or is it the number of defects experienced by the customer (which is us)? I of course can count the number of defects we receive from the supplier, but if this metric is supposed to be based on the number of defects produced by an organization, I would have no way of knowing how many defects are produced by the supplier’s process, but contained within the supplier’s facility. My hope is to be able to characterize the supplier’s process performance in terms of sigma level.

Answer

The DPMO metric is not usually considered a point estimate of the true percent defective in the lot (either at the supplier or customer site).  It is a relative performance metric used to equate the observed percent defective from a sample to defective units per million opportunities.  If a supplier culls out all the defective units before shipping to you (i.e. perfect inspection system), your internal DPMO would be 0, even if the supplier DPMO is high. If your goal is to characterize the supplier’s process performance in terms of sigma level, you would need their data, as the data you collect internally is just an estimate for the average outgoing quality from the supplier and not their process performance.

Steven Walfish

For more on this topic, please visit ASQ’s website.

Z1.4 Sample Size

Pharmaceutical sampling

Question

I am trying to determine the sampling size using my ANSI/ASQ Z1.4 table and I wanted to get some clarification. If I am using Table II A and my Sample Size Code letter is D, what would be my sample size? If it falls on an arrow does it mean that I have to change to the next sample size based on where the arrow points?

Answers

From Charlie Cianfrani:

If you are using Z1.4, your sample size is selected based on your lot size.  You would pick the AQL you need based on the risk you are willing to take for the process average of percent defective.  It is important to understand what you are doing when using sampling plans, what they are and the protection you are trying to ensure. Thus, the important step is to determine the AQL. Then you select the sample size to provide the level of protection you are striving to ensure. It is more important to understand the theory behind the tables than to mechanically use the tables.

From Fred Schenkelberg:

Use the sample size where the arrow points. In the 2008 and 2013 versions it explains this in section 9.4, “When no sampling plan is available for a given combination of AQL and code letter, the tables direct the user to a different letter. The sample size to be used is given by the new code letter, not by the original letter.”

From Steven Walfish:

The standard sample size for Code Letter D from IIA is a sample size of 8.  But depending on your AQL, a sample size of 8 would be inappropriate, so the standard has arrows to delineate alternative sample sizes to reach the target AQL.  So, you sample size and accept/reject values are changed.  For example, at an AQL of 0.25, you would move down to a sample size of 50, with an accept/reject of 0/1.  If the lot size is less than 50, you would need to do 100% inspection.  In other words, there is no sampling plan that can give an AQL of 0.25 without a minimum sample size of 50.

From James Werner:

Yes.  When using Z1.4 two items need to be known, lot size and the AQL (Acceptance Quality Limit).  You use Table I – Sample size code letters to determine the Sample size code letter based on the Lot or batch size.  In the question below that was determined to be “D”.  Next step is to use Table II-A to find the sample size related to the sample size code letter – D and the AQL.  On Table II-A go across the table’s row for letter D until it intersect the given AQL column heading.  If an arrow is in that intersection point, follow the arrow then go back to the sample size code letter column to find the actual sample size (if a up/down arrow is in there then you choose).

Example 1.  Code letter is D (as in the question below).  Let’s say the AQL is 0.25.  Starting at code letter D, move across that row until you intersect at the AQL 0.25 column.  There’s a down arrow this row/column intersection.  Follow the arrow downward until the “Ac Re” reads ” 0 1″.  Staying on this row go back to the Sample size code letter column and find Code Letter H and Sample size = 50.  This means for the lot size with code letter D and with an AQL of 0.25 the sample size = 50 and accept the entire lot if no nonconformances were found else reject the entire lot if 1 or more nonconformance were found in the sample.

Example 2.  Let’s say the Sample size code letter was determine from Table I to be “F”.  Looking at Table II-A; If the AQL = 0.65, then the sample size would be 20 and the lot would be accepted zero nonconformance.  But if the AQL = 0.15 then the sample size would be 80.

For more information on this topic, please visit ASQ’s website. 

Acceptance Sampling Inspection

Automotive inspection, TS 16949, IATF 16949

Question

We have an acceptance sampling inspection in place where we use the ANSI/ASQ Z1.4 -2013 standard under Normal Inspection, using General Inspection Level II to drive our samples size and accept, reject criteria. We do not uses switching rules as we have always found them too difficult to manage. I have two questions.

If I have one lot that fails acceptance sampling and I am trying to bound the issue is it suitable to bound it to the one affected lot if the lot before and after pass or do I need to carry out additional sampling.

My second question is if I have a batch that passes acceptance sampling but at a subsequent downstream process a defect being inspected for by the upstream acceptance sampling inspection is found how do I determine if the lot is acceptable? Do I trust the acceptance sampling inspection or react?

Answer

The first question is not an uncommon one and actually it is a good practice to isolate the lot and do 100% inspection of it.  That way you can estimate the % defective and if another failure occurs in the next 5 lots, then increase the sampling until you have some confidence that the supplier has fixed the problem.  Once that confidence is restored, then you go back to what you inspected originally.

The second question, is one that you have to understand how well do you follow the acceptance sampling process?  If your alpha level is at 95%, 5% of the time, you can accept a bad batch as good. That is the pure definition of the alpha risk.  If this failure falls within the 5%, your process is working and while you sort through the lot, and notify the supplier, it is not something that you over react to.

I hope this helps.

Jim

James Bossert, PhD, MBB, CQA, CQE, CqM/OE
Sr Performance Improvement Consultant

For more on this topic, please visit ASQ’s website.