Tag: sampling

Determining Statistically Significant Sample Size

Sterile, Lab, Clean Room, Requirements, Standard

Question

I am developing an internal audit process within our supply chain to determine packaging and Finalizing SOP’s are being followed. I need to determine what will be the sample size needed to accurately represent the population. We are currently shipping out 650k cartons a day. How do I determine how many audits I need a day for statistical significance?

Answer

Statistical sampling theory shows that for large populations, the sample size is not a function of the population size, assuming all units in the population have an equal probability of being selected for the sample.  To ensure a representative sample, stratified random sampling is employed to represent in the audit sample. This method requires that each category (or stratum) is specified, and that none of them overlap (i.e., items to be audited must fall in only one category).  For example, you can break the packaging records in groups of 25,000 (26 stratum for 650,000 records), sampling 1/26th of the sample from each stratum.

To determine the sample size, we employ the binomial distribution where a records is either confirming or nonconforming. 

The basic formula for the binomial confidence interval is

For a given sample size (n) with a given number of defects (x), the probability of the sample coming from a population with probability (p) is given by the value alpha (a).  The above equation can be solved for probability (p) at a given a level or can be solved for a at a given population probability (p).

In other words, you specify the percent defective in the population you can accept.  The only when to ensure 0% defective is 100% sampling. You solve the equation for n by setting 1-alpha (1-a) equal to a high probability (i.e. 95%).  If you desire to accept zero (0) defects in the sample then set x equal to zero. In this case, the equation reduces to ln(1- a)/ln(1-p).

Hope this helps with the question.

Thanks

Steven

For more on this topic, please visit ASQ’s website.

Z1.4 Inspection Requirements

Automotive inspection, TS 16949, IATF 16949

Question

Our customers require that we follow the ANSI Z1.4 standard for attribute sampling plans; however, it is not feasible to wait until lots are completed to perform inspections. Lots can be large and run for many days and waiting until lot completion to determine the sample size, based on the finished lot size, is too late because we will have missed our chance to correct any production issues that may result in defective parts. Another limitation is a lack of space to stage product while waiting for the final inspection of the completed production lots. Product is made as orders are received, and not typically stored as inventory, so our on-time delivery demands also hinder our ability to hold product for final inspections of completed production lots. Therefore, we are seeking guidance on a practical way to implement a in-process inspection during production that follows the ANSI Z1.4 standard.

Answer

Yes, you can sample as you produce to get to the sample size.  It is important that you keep track of your accept/reject items.  Since you know how long you are running the product, you can project the approximate lot size to get the sample size.  Work with your scheduler before the product starts so you can take samples early and continue on in the process.

In addition, if you have material changes as the product is running, I am sure that you are sampling then to make sure everything is set correctly, you can use those samples also.  As an example, let’s say your product is running 4 days and based on the projected lot size, you have a sample of 28 to take, you could take 7 samples each day spaced throughout the day or you take 10 samples the first day, 7 the second day, 6 the third day and 5 the last day of production.  You need to figure the right sequence that fits your history of the product.

Jim Bossert

For more on this topic, please visit ASQ’s website.

AQL Calculation Explanation

Reporting, best practices, non-compliance reporting

Question

Regarding part of your answer to a post found here, you state:

“The calculation of AQL is not dependent on lot size. In other words, a sample size of 315 gives a minimum AQL of 0.04, so a larger sample is required to estimate an AQL of 0.01.”

Can you explain for the non-statistical folks like me people how that math works? Specifically, I am wondering what the minimum sample size would be for an AQL of 0.25, when using Special Inspection level S2? Would it be a minimum of 50, no mater what the lot size is?

Answer

Acceptance sampling procedures were developed during the early 1920s at Western Electric Company and later formalized at Bell Telephone Laboratories where terms like producer’s risk and consumer’s risk were established.  Later, during World War II, sampling plans such as MIL-STD-105 were developed by Harold F. Dodge and others working with the Army Quartermaster Corps (Dodge, 1967).

Two special features were employed in order to gain agreement with the large body of military suppliers.  One was the use of the acceptable quality limit (AQL) as opposed to the RQL in presenting the plans.  The goal at the time was to focus on rewarding suppliers for production whose quality levels were considered good.  RQLs were recognized but not often brought to the surface during discussions. Also, at that time, the term “AQL” was deliberately vague or inexact.  It was a close approximation, not an exact probability statement.

The other feature was the practice of increasing sample sizes with increased lot sizes.  As noted in Section 3, in most situations, the lot size does not factor in plan construction (based on the binomial).  For many, however, this lacks intuitive appeal.  Therefore, in the development of MIL-STD-105 and its derivatives a deliberate increase in sample sizes for higher lot sizes was introduced, with corresponding increases in acceptance numbers for similar AQLs.  Clearly, this practice resulted in over-sampling and consequent increased inspection costs.  Government operatives believed that the increased sampling cost was of small consequence relative to the power to persuade.

For the binomial distribution you solve for the AQL that gives a high probability of passing.  Usually this probability is set at 95%.  For example if you have a sample size of 80 units with an accept/reject of 1, an AQL of 0.65% would have a 90% probability of passing the sampling plan.

You can use Excel to solve this with the function

=BINOMDIST(1,80,0.0065,1)

Hope this helps,

Steven Walfish

For more on this topic, please visit ASQ’s website.

Sampling Plan Review?

Chart, graph, sampling, plan, calculation, z1.4

Question

When following ANSI/ASQ Z1.4-2003 (R2018), if a product has been placed in a “reduced” sampling plan based on the previous 10 lots results, is it a requirement to convert back to a “normal” sampling plan on an annual basis, or should that decision remain based on supplier performance? I have been told that we should revert to normal sampling each year, but I do not see that in the AQL inspection manual.

Answer

The standard does not require annual (or periodic) review of the sampling plan.  The switching rules are time invariant, and reflects just the normal flow of lots, which can span more than a year.  Unless the supplier requires a change in the inspection level, the standard is silent on resetting to the normal level annually.

Steven Walfish

For more on this topic, please visit ASQ’s website.

Zero Acceptance Number Sampling Plans and the FDA

Pharmaceutical sampling

Question

There has been some debate over using the MIL-STD-1916 acceptance sampling plan over the ANSI/ASQ Z1.4-2003 (R2018) sampling plans.  The opinion is that the ANSI/ASQ Z1.4-2003 (R2018) is outdated and no longer an acceptable method of determining a qualification sample plan and the MIL-STD-1916 should be used in place of ANSI/ASQ Z1.4-2003 (R2018). Do you have information around this debate over which sampling plans are acceptable by the FDA?

Answer

FDA does not (and can not) tell you what sampling plan is to be used.  The FDA requirement is that the plan be statistically valid.  As long as you follow the regulation, you are meeting FDA requirements.

In medical device manufacturing the key point is to have the plan accept on zero defectives.  This point is not FDA but legalese.  It is based on past lawsuits.  The plan “Zero Acceptance Number Sampling Plans” by Nicholas L. Squeglia (available from ASQ) has been widely adopted for this reason.

ANSI/ASQ Z1.4 in not outdated and continues to be widely used.  It is the American National Standard Institute (ANSI) version of MIL-STD-105 which the government discontinued maintaining, allowing ANSI to maintain it along with many, many other MIL-STD’s as a government cost reduction.

MIL-STD-1916 can be used but it is not widely used because of its difficulty and practical use.

James Werner

For more about this topic, visit ASQ’s website.

Acceptance Sampling Inspection

Automotive inspection, TS 16949, IATF 16949

Question

We have an acceptance sampling inspection in place where we use the ANSI/ASQ Z1.4 -2013 standard under Normal Inspection, using General Inspection Level II to drive our samples size and accept, reject criteria. We do not uses switching rules as we have always found them too difficult to manage. I have two questions.

If I have one lot that fails acceptance sampling and I am trying to bound the issue is it suitable to bound it to the one affected lot if the lot before and after pass or do I need to carry out additional sampling.

My second question is if I have a batch that passes acceptance sampling but at a subsequent downstream process a defect being inspected for by the upstream acceptance sampling inspection is found how do I determine if the lot is acceptable? Do I trust the acceptance sampling inspection or react?

Answer

The first question is not an uncommon one and actually it is a good practice to isolate the lot and do 100% inspection of it.  That way you can estimate the % defective and if another failure occurs in the next 5 lots, then increase the sampling until you have some confidence that the supplier has fixed the problem.  Once that confidence is restored, then you go back to what you inspected originally.

The second question, is one that you have to understand how well do you follow the acceptance sampling process?  If your alpha level is at 95%, 5% of the time, you can accept a bad batch as good. That is the pure definition of the alpha risk.  If this failure falls within the 5%, your process is working and while you sort through the lot, and notify the supplier, it is not something that you over react to.

I hope this helps.

Jim

James Bossert, PhD, MBB, CQA, CQE, CqM/OE
Sr Performance Improvement Consultant

For more on this topic, please visit ASQ’s website.

Sampling and Daylight Savings Time

Pharmaceutical sampling

Question

We are wondering if there is any generally accepted procedure to account for seemingly duplicate sample times when operating over the time period when clocks ‘fall back’ one hour for the end of daylight savings time? Our standard practice is to analyze a chemistry sample every half hour, so we foresee two each of the 01:00, 01:30 & 02:00 AM samples this next Sunday morning. Please advise on any generally accepted practice to account for such seemingly duplicate samples.

Answer

I do not know an industry accepted standard for this yet.

If used for control charting, just plot as normal, noting when the sample was taken. For the second 2:30am sample, just note it was after the time change… continue monitoring the process as normal.

If used for lot sampling, analyze the results as normal.

If doing a daily average, then adjust the calculation for the two extra samples, i.e. divide by 26 instead of 24.

At most it may require a slight change to calculations based on the number of samples, otherwise it’s just not a big issue that may require at most a comment or note about the seemingly duplicate sample times or two missing times (in spring).

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

For more on this topic, please visit ASQ’s website.

Sample Size and Z1.4

Data review, data analysis, data migration

Question

My question is if I’m trying to determine the sample size of migrated data to see if it migrated correctly to the target database, is the Z1.4 table applicable to that?

The scenario is data is being transferred from an old system to a new system and I want to do a quality check on the data in the new database to make sure everything was transferred correctly. I’m hoping to use the Z1.4 table to determine the sample size if its applicable. Is it applicable and if not, do you know of other standards that I should be looking into that is more applicable?

Answer

The movement of a database from one system to another certainly may introduce errors and it may also carry over errors that already exist. In some cases the move may also find and repair errors, yet that generally is done by design.

So, let’s say it’s just a move and you are checking for any new errors that are introduced.

Since you have access to the entire population, the database, in a before (old system) and after the move (new system) and I’m assuming you do not want to check every entry, instead just a sample, then I would recommend using an hypothesis test approach rather than a lot sampling approach.

A hypothesis test based on the binomial distribution may be appropriate as you are checking field entries to determine if they are correct or not (pass/fail).

You can set a threshold defect rate that you want to check the new system is at least this good or better, or you can measure the old system and compare to the new system – it should be equal to the old system as null hypothesis.

You can find a bit more information about a p-test in a good stats book or online at a short tutorial I wrote at https://creprep.wordpress.com/2013/06/01/hypothesis-tests-for-proportion/

The Z1.4 standard would require you to artificially define a lot or consider the entire database as one lot. The standard lot testing approach does not provide the control and statistical power of hypothesis testing, thus my recommendation. With the p-test you can define the confidence, defect rate to detect, and sample size to fit your needs concerning ability to make measurements, cost, and risk.

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

For more on this topic, please visit ASQ’s website.

Sampling Schemes

Inventory, Inspection, Review, Suppliers, Supplies

Question

Is there a sampling plan for determining the number of cases to pull in a batch from which you perform the ANSI/ASQ sampling of individual products?  For example: you receive 550 cases with 145 product vials/case.  Is it proper to sample a total of 500 vials from 25 cases (using square root of n+1) or would applying the ANSI/ASQ single level II be more appropriate?  We would then need to pull 500 vials from 80 cases.  Or is there a better statistical method?

Answer

There are two ways to answer this. One is to follow the standard and take samples from 80 cases until you get 500. It is assumed that the samples are random so that you do not always take the samples from the same location in the case.  That is following the standard.

The second is that you take a sample from 25 cases in a random manner.  That is fine also.  There are no standards for sampling from cases so either way will work.  Years ago, I developed a sampling scheme similar to what is proposed at the employer I was working with at the time.  Sometimes you have to be creative.

Jim Bossert

SVP Process Design Manger, Process Optimization
Bank of America
ASQ Fellow, CQE, CQA, CMQ/OE, CSSBB, CSSMBB
Fort Worth, TX

For more on this topic, please visit ASQ’s website.

Sample Size

Manufacturing, inspection, exclusions

Question

If we have a lot size of 27 and we are using a normal inspection level II with an AQL of 2.5. What is the sample size?

Answer

Assuming an attribute is being measured, we use ANSI ASQ Z1.4.2013 to find the sample size.

Given a lot size of 27 we first find in Table I. Sample Size Code Letter that Code letter D represents the sampling plan code letter for lot sizes between 26 and 50 for normal sampling (General Inspection Level II).

The move to Table II-A Single sampling plans for normal inspection to find the row for code letter D and under column for ASQ 2.5 find an up arrow. This indicates that we should use the code letter C which suggests a sampling plan of 5 samples and accept the lot if there are zero defect and reject the lot with one or more rejects.

Hope that helps.

Cheers,

Fred

Fred Schenkelberg
Reliability Engineering and Management Consultant
FMS Reliability
(408) 710-8248
fms@fmsreliability.com
www.fmsreliability.com
@fmsreliability

For more on this topic, please visit ASQ’s website.