© 2002 By CRC Press LLC 20 Multiple Paired Comparisons of k Averages KEY WORDS data snooping, data dredging, Dunnett’s procedure, multiple comparisons, sliding refer- ence distribution, studentized range , t -tests, Tukey’s procedure. The problem of comparing several averages arises in many contexts: compare five bioassay treatments against a control, compare four new polymers for sludge conditioning, or compare eight new combina- tions of media for treating odorous ventilation air. One multiple paired comparison problem is to compare all possible pairs of k treatments. Another is to compare k – 1 treatments with a control. Knowing how to do a t -test may tempt us to compare several combinations of treatments using a series of paired t -tests. If there are k treatments, the number of pair-wise comparisons that could be made is k ( k – 1) / 2. For k = 4, there are 6 possible combinations, for k = 5 there are 10, for k = 10 there are 45, and for k = 15 there are 105. Checking 5, 10, 45, or even 105 combinations is manageable but not recommended. Statisticians call this data snooping (Sokal and Rohlf, 1969) or data dredging (Tukey, 1991). We need to understand why data snooping is dangerous. Suppose, to take a not too extreme example, that we have 15 different treatments. The number of possible pair-wise comparisons that could be made is 15(15 – 1) / 2 = 105. If, before the results are known, we make one selected comparison using a t -test with a 100 α % = 5% error rate, there is a 5% chance of reaching the wrong decision each time we repeat the data collection experiment for those two treatments. If, however, several pairs of treatments are tested for possible differences using this procedure, the error rate will be larger than the expected 5% rate. Imagine that a two-sample t -test is used to compare the largest of the 15 average values against the smallest. The null hypothesis that this difference, the largest of all the 105 possible pair-wise differences, is likely to be rejected almost every time the experiment is repeated, instead of just at the 5% rate that would apply to making one pair-wise comparison selected at random from among the 105 possible comparisons. The number of comparisons does not have to be large for problems to arise. If there are just three treatment methods and of the three averages, A is larger than B and C is slightly larger than A it is possible for the three possible t -tests to indicate that A gives higher results than B ( η A > η B ), A is not different from C ( η A = η C ), and B is not different from C ( η B = η C ). This apparent contradiction can happen because different variances are used to make the different comparisons. Analysis of variance (Chapter 21) eliminates this problem by using a common variance to make a single test of significance (using the F statistic). The multiple comparison test is similar to a t -test but an allowance is made in the error rate to keep the collective error rate at the stated level. This collective rate can be defined in two ways. Returning to the example of 15 treatments and 105 possible pair-wise comparisons, the probability of getting the wrong conclusion for a single randomly selected comparison is the individual error rate . The family error rate (also called the Bonferroni error rate ) is the chance of getting one or more of the 105 comparisons wrong in each repetition of data collection for all 15 treatments. The family error rate counts an error for each wrong comparison in each repetition of data collection for all 15 treatments. Thus, to make valid statistical comparisons, the individual per comparison error rate must be shrunk to keep the simultaneous family error rate at the desired level. y C y A y B >>(), L1592_Frame_C20 Page 169 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC Case Study: Measurements of Lead by Five Laboratories Five laboratories each made measurements of lead on ten replicate wastewater specimens. The data are given in Table 20.1 along with the means and variance for each laboratory. The ten possible comparisons of mean lead concentrations are given in Table 20.2. Laboratory 3 has the highest mean (4.46 µ g/L) and laboratory 4 has the lowest (3.12 µ g/L). Are the differences consistent with what one might expect from random sampling and measurement error, or can the differences be attributed to real differences in the performance of the laboratories? We will illustrate Tukey’s multiple t- test and Dunnett’s method of multiple comparisons with a control , with a minimal explanation of statistical theory. Tukey’s Paired Comparison Method A (1 – α )100% confidence interval for the true difference between the means of two treatments, say treatments i and j , is: TABLE 20.1 Ten Measurements of Lead Concentration ( µ g/L) Measured on Identical Wastewater Specimens by Five Laboratories Lab 1 Lab 2 Lab 3 Lab 4 Lab 5 3.4 4.5 5.3 3.2 3.3 3.0 3.7 4.7 3.4 2.4 3.4 3.8 3.6 3.1 2.7 5.0 3.9 5.0 3.0 3.2 5.1 4.3 3.6 3.9 3.3 5.5 3.9 4.5 2.0 2.9 5.4 4.1 4.6 1.9 4.4 4.2 4.0 5.3 2.7 3.4 3.8 3.0 3.9 3.8 4.8 4.2 4.5 4.1 4.2 3.0 Mean = 4.30 3.97 4.46 3.12 3.34 Variance = 0.82 0.19 0.41 0.58 0.54 TABLE 20.2 Ten Possible Differences of Means Between Five Laboratories Laboratory i (Average = ) Laboratory j 1 (4.30) 2 (3.97) 3 (4.46) 4 (3.12) 5 (3.34) 1 ———— 2 0.33 ——— 3 − 0.16 − 0.49 —— 4 1.18 0.85 1.34 — 5 0.96 0.63 1.12 − 0.22 — y s i 2 y i y j –() y i y i y j –()t ν , α /2 s pool 1 n i 1 n j +± L1592_Frame_C20 Page 170 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC where it is assumed that the two treatments have the same variance, which is estimated by pooling the two sample variances: The chance that the interval includes the true value for any single comparison is exactly 1 – α . But the chance that all possible k ( k – 1) / 2 intervals will simultaneously contain their true values is less than 1 – α . Tukey (1949) showed that the confidence interval for the difference in two means ( η i and η j ), taking into account that all possible comparisons of k treatments may be made, is given by: where q k , ν , α /2 is the upper significance level of the studentized range for k means and ν degrees of freedom in the estimate of the variance σ 2 . This formula is exact if the numbers of observations in all the averages are equal, and approximate if the k treatments have different numbers of observations. The value of is obtained by pooling sample variances over all k treatments: The size of the confidence interval is larger when q k , ν , α /2 is used than for the t statistic. This is because the studentized range allows for the possibility that any one of the k ( k – 1) / 2 possible pair-wise comparisons might be selected for the test. Critical values of q k , v , α /2 have been tabulated by Harter (1960) and may be found in the statistical tables of Rohlf and Sokal (1981) and Pearson and Hartley (1966). Table 20.3 gives a few values for computing the two-sided 95% confidence interval. Solution: Tukey’s Method For this example, k = 5, = 0.51, s pool = 0.71, ν = 50 – 5 = 45, and q 5,40,0.05/2 = 4.49. This gives the 95% confidence limits of: TABLE 20.3 Values of the Studentized Range Statistic q k, ν , α /2 for k(k – 1)/2 Two-Sided Comparisons for a Joint 95% Confidence Interval Where There are a Total of k Treatments k νν νν 23456810 5 4.47 5.56 6.26 6.78 7.19 7.82 8.29 10 3.73 4.47 4.94 5.29 5.56 5.97 6.29 15 3.52 4.18 4.59 4.89 5.12 5.47 5.74 20 3.43 4.05 4.43 4.70 4.91 5.24 5.48 30 3.34 3.92 4.27 4.52 4.72 5.02 5.24 60 3.25 3.80 4.12 4.36 4.54 4.81 5.01 ∞ 3.17 3.68 3.98 4.20 4.36 4.61 4.78 Note: Family error rate = 5%; α /2 = 0.05/2 = 0.025. Source: Harter, H. L. (1960). Annals Math. Stat., 31, 1122–1147. s pool 2 n i 1–()s i 2 n j 1–()s j 2 + n i n j 2–+ = y i y j – q k, ν , α /2 2 s pool 1 n i 1 n j +± s pool 2 s pool 2 s pool 2 n 1 1–()s 1 2 … n k 1–()s k 2 ++ n 1 … n k k–++ = s pool 2 y i y j –() 4.49 2 0.71() 1 10 1 10 +± L1592_Frame_C20 Page 171 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC and the difference in the true means is, with 95% confidence, within the interval: We can say, with a high degree of confidence, that any observed difference larger than 1.01 µ g/L or smaller than −1.01 µ g/L is not likely to be zero. We conclude that laboratories 3 and 1 are higher than 4 and that laboratory 3 is also different from laboratory 5. We cannot say which laboratory is correct, or which one is best, without knowing the true concentration of the test specimens. Dunnett’s Method for Multiple Comparisons with a Control In many experiments and monitoring programs, one experimental condition (treatment, location, etc.) is a standard or a control treatment. In bioassays, there is always an unexposed group of organisms that serve as a control. In river monitoring, one location above a waste outfall may serve as a control or reference station. Now, instead of k treatments to compare, there are only k – 1. And there is a strong likelihood that the control will be different from at least one of the other treatments. The quantities to be tested are the differences where is the observed average response for the control treatment. The (1 – α )100% confidence intervals for all k – 1 comparisons with the control are given by: This expression is similar to Tukey’s as used in the previous section except the quantity is replaced with Dunnett’s The value of s pool is obtained by pooling over all treatments. An abbreviated table for 95% confidence intervals is reproduced in Table 20.4. More extensive tables for one- and two-sided tests are found in Dunnett (1964). Solution: Dunnet’s Method Rather than create a new example we reconsider the data in Table 20.1 supposing that laboratory 2 is a reference (control) laboratory. Pooling sample variances over all five laboratories gives the estimated within-laboratory variance, and s pool = 0.71. For k – 1 = 4 treatments to be compared with the control and ν = 45 degrees of freedom, the value of t 4,45,0.05 /2 = 2.55 is found in Table 20.4. The 95% TABLE 20.4 Table of t k–1, ν ,0.05 /2 for k – 1 Two-Sided Comparisons for a Joint 95% Confidence Level Where There are a Total of k Treatments, One of Which is a Control k – 1 = Number of Treatments Excluding the Control νν νν 2345 6810 5 3.03 3.29 3.48 3.62 3.73 3.90 4.03 10 2.57 2.76 2.89 2.99 3.07 3.19 3.29 15 2.44 2.61 2.73 2.82 2.89 3.00 3.08 20 2.38 2.54 2.65 2.73 2.80 2.90 2.98 30 2.32 2.47 2.58 2.66 2.72 2.82 2.89 60 2.27 2.41 2.51 2.58 2.64 2.73 2.80 ∞ 2.21 2.35 2.44 2.51 2.57 2.65 2.72 Source: Dunnett, C. W. (1964). Biometrics, 20, 482–491. 1.01– y i ≤ y j 1.01≤– y i y c ,– y c y i y c –()t k–1, ν , α /2 s pool 1 n i 1 n c +± q k, ν , α /2 / 2 t k–1, ν , α /2 . s pool 2 0.51= L1592_Frame_C20 Page 172 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC confidence limits are: We can say with 95% confidence that any observed difference greater than 0.81 or smaller than −0.81 is unlikely to be zero. The four comparisons with laboratory 2 shown in Table 20.5 indicate that the measurements from laboratory 4 are smaller than those of the control laboratory. Comments Box et al. (1978) describe yet another way of making multiple comparisons. The simple idea is that if k treatment averages had the same mean, they would appear to be k observations from the same, nearly normal distribution with standard deviation The plausibility of this outcome is examined graphically by constructing such a normal reference distribution and superimposing upon it a dot diagram of the k average values. The reference distribution is then moved along the horizontal axis to see if there is a way to locate it so that all the observed averages appear to be typical random values selected from it. This sliding reference distribution is a “…rough method for making what are called multiple comparisons.” The Tukey and Dunnett methods are more formal ways of making these comparisons. Dunnett (1955) discussed the allocation of observations between the control group and the other p = k – 1 treatment groups. For practical purposes, if the experimenter is working with a joint confidence level in the neighborhood of 95% or greater, then the experiment should be designed so that approximately, where n c is the number of observations on the control and n is the number on each of the p noncontrol treatments. Thus, for an experiment that compares four treatments to a control, p = 4 and n c is approximately 2n. References Box, G. E. P., W. G. Hunter, and J. S. Hunter (1978). Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, New York, Wiley Interscience. Dunnett, C. W. (1955). “Multiple Comparison Procedure for Comparing Several Treatments with a Control,” J. Am. Stat. Assoc., 50, 1096–1121. Dunnett, C. W. (1964). “New Tables for Multiple Comparisons with a Control,” Biometrics, 20, 482–491. Harter, H. L. (1960). “Tables of Range and Studentized Range,” Annals Math. Stat., 31, 1122–1147. Pearson, E. S. and H. O. Hartley (1966). Biometrika Tables for Statisticians, Vol. 1, 3rd ed., Cambridge, England, Cambridge University Press. Rohlf, F. J. and R. R. Sokal (1981). Statistical Tables, 2nd ed., New York, W. H. Freeman & Co. Sokal, R. R. and F. J. Rohlf (1969). Biometry: The Principles and Practice of Statistics in Biological Research, New York, W. H. Freeman and Co. Tukey, J. W. (1949). “Comparing Individual Means in the Analysis of Variance,” Biometrics, 5, 99. Tukey, J. W. (1991). “The Philosophy of Multiple Comparisons,” Stat. Sci., 6(6), 100–116. TABLE 20.5 Comparing Four Laboratories with a Reference Laboratory Laboratory Control 1 3 4 5 Average 3.97 4.30 4.46 3.12 3.34 Difference — 0.33 0.49 –0.85 –0.63y i y c –() y i y c – 2.55 0.71()± 1 10 1 10 + 0.81– y i y c – 0.81≤≤ σ / n. n c /np= L1592_Frame_C20 Page 173 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC Exercises 20.1 Storage of Soil Samples. The concentration of benzene ( µ g/g) in soil was measured after being stored in sealed glass ampules for different times, as shown in the data below. Quantities given are average ± standard deviation, based on n = 3. Do the results indicate that storage time must be limited to avoid biodegradation? 20.2 Biomonitoring. The data below come from a biological monitoring test for chronic toxicity on fish larvae. The control is clean (tap) water. The other four conditions are tap water mixed with the indicated percentages of treated wastewater effluent. The lowest observed effect level (LOEL) is the lowest percentage of effluent that is statistically different from the control. What is the LOEL? 20.3 Biological Treatment. The data below show the results of applying four treatment conditions to remove a recalcitrant pollutant from contaminated groundwater. All treatments were rep- licated three times. The “Controls” were done using microorganisms that have been inhibited with respect to biodegrading the contaminant. The “Bioreactor” uses organisms that are expected to actively degrade the contaminant. If the contaminant is not biodegraded, it could be removed by chemical degradation, volatilization, sorption, etc. Is biodegradation a signif- icant factor in removing the contaminant? Day 0 Day 5 Day 11 Day 20 6.1 ± 0.7 5.9 ± 0.2 6.2 ± 0.2 5.7 ± 0.2 Source: Hewitt, A. D. et al. (1995). Am. Anal. Lab., Feb., p. 26. Percentage Effluent Replicate Control 1.0 3.2 10.0 32.0 1 1.017 1.157 0.998 0.837 0.715 2 0.745 0.914 0.793 0.935 0.907 3 0.862 0.992 1.021 0.839 1.044 Mean 0.875 1.021 0.937 0.882 0.889 Variance 0.0186 0.0154 0.0158 0.0031 0.0273 Condition T 0 T 1 T 2 T 3 Control 1220 1090 695 575 1300 854 780 580 1380 1056 688 495 Bioreactor 1327 982 550 325 1320 865 674 310 1253 803 666 465 Source: Dobbins, D. C. (1994). J. Air & Waste Mgmt. Assoc., 44, 1226–1229. L1592_Frame_C20 Page 174 Tuesday, December 18, 2001 1:53 PM © 2002 By CRC Press LLC 21 Tolerance Intervals and Prediction Intervals KEY WORDS confidence interval, coverage, groundwater monitoring, interval estimate, lognormal distribution, mean, normal distribution, point estimate, precision, prediction interval, random sampling, random variation, spare parts inventory, standard deviation, tolerance coefficient, tolerance interval, transformation, variance, water quality monitoring. Often we are interested more in an interval estimate of a parameter than in a point estimate. When told that the average efficiency of a sample of eight pumps was 88.3%, an engineer might say, “The point estimate of 88.3% is a concise summary of the results, but it provides no information about their precision.” The estimate based on the sample of 8 pumps may be quite different from the results if a different sample of 8 pumps were tested, or if 50 pumps were tested. Is the estimate 88.3 ± 1%, or 88.3 ± 5%? How good is 88.3% as an estimate of the efficiency of the next pump that will be delivered? Can we be reasonably confident that it will be within 1% or 10% of 88.3%? Understanding this uncertainty is as important as making the point estimate. The main goal of statistical analysis is to quantify these kinds of uncertainties, which are expressed as intervals. The choice of a statistical interval depends on the application and the needs of the problem. One must decide whether the main interest is in describing the population or process from which the sample has been selected or in predicting the results of a future sample from the same population. Confidence intervals enclose the population mean and tolerance intervals contain a specified proportion of a population. In contrast, intervals for a future sample mean and intervals to include all of m future observations are called prediction intervals because they deal with predicting (or containing) the results of a future sample from a previously sampled population (Hahn and Meeker, 1991). Confidence intervals were discussed in previous chapters. This chapter briefly considers tolerance intervals and prediction intervals. Tolerance Intervals A tolerance interval contains a specified proportion ( p ) of the units from the sampled population or process. For example, based upon a past sample of copper concentration measurements in sludge, we might wish to compute an interval to contain, with a specified degree of confidence, the concentration of at least 90% of the copper concentrations from the sampled process. The tolerance interval is constructed from the data using two coefficients, the coverage and the tolerance coefficient. The coverage is the proportion of the population ( p ) that an interval is supposed to contain. The tolerance coefficient is the degree of confidence with which the interval reaches the specified coverage. A tolerance interval with coverage of 95% and a tolerance coefficient of 90% will contain 95% of the population distribution with a confidence of 90%. The form of a two-sided tolerance interval is the same as a confidence interval: where the factor K 1 − α , p , n has a 100(1 − α )% confidence level and depends on n , the number of observations in the given sample. Table 21.1 gives the factors for two-sided 95% confidence intervals yK 1− α , p, n s± (t n−1, α /2 / n) L1592_frame_C21 Page 175 Tuesday, December 18, 2001 2:43 PM © 2002 By CRC Press LLC for the population mean η and values of K 1 − α , p , n for two-sided tolerance intervals to contain at least a specified proportion (coverage) of p = 0.90, 0.95, or 0.99 of the population at a 100(1 − α )% = 95% confidence level. Complete tables for one-sided and two-sided confidence intervals, tolerance intervals, and prediction intervals are given by Hahn and Meeker (1991) and Gibbons (1994). The factors in these tables were calculated assuming that the data are a random sample. Simple random sampling gives every possible sample of n units from the population the same probability of being selected. The assumption of random sampling is critical because the statistical intervals reflect only the randomness introduced by the sampling process. They do not take into account bias that might be introduced by a nonrandom sample. The use of these tables is illustrated by example. Example 21.1 A random sample of n = 5 observations yields the values and s = 1.18 µ g/L. The second row of Table 21.1 gives the needed factors for n = 5 1. The two-sided 95% confidence interval for the mean η of the population is: 28.4 ± 1.24(1.18) = [26.9, 29.9] The coefficient We are 95% confident that the interval 26.9 to 29.9 µ g/L contains the true (but unknown) mean concentration of the population. The 95% confidence describes the percentage of time that a claim of this type is correct. That is, 95% of intervals so constructed will contain the true mean concentration. 2. The two-sided 95% tolerance interval to contain at least 99% of the sampled population is: 28.4 ± 6.60(1.18) = [20.6, 36.2] The factor 6.60 is for n = 5, p = 0.99, and 1 − α = 0.95. We are 95% confident that the interval 20.6 to 36.2 µ g/L contains at least 99% of the population. This is called a 95% tolerance interval for 99% of the population. TABLE 21.1 Factors for Two-Sided 95% Confidence Intervals and Tolerance Intervals for the Mean of a Normal Distribution Confidence Intervals K 1−− −− αα αα , p , n for Tolerance Intervals n ( t n−− −− 1, αα αα // // 22 22 ) p == == 0.90 p == == 0.95 p == == 0.99 4 1.59 5.37 6.34 8.22 5 1.24 4.29 5.08 6.60 6 1.05 3.73 4.42 5.76 7 0.92 3.39 4.02 5.24 8 0.84 3.16 3.75 4.89 9 0.77 2.99 3.55 4.63 10 0.72 2.86 3.39 4.44 12 0.64 2.67 3.17 4.16 15 0.50 2.49 2.96 3.89 20 0.47 2.32 2.76 3.62 25 0.41 2.22 2.64 3.46 30 0.37 2.15 2.55 3.35 40 0.32 2.06 2.45 3.22 60 0.26 1.96 2.34 3.07 ∞ 0.00 1.64 1.96 2.58 Source: Hahn, G. J. (1970). J. Qual. Tech., 3, 18–22. // // n y 28.4 µ g/L= 1.24 t 4, 0.025 / 5.= L1592_frame_C21 Page 176 Tuesday, December 18, 2001 2:43 PM © 2002 By CRC Press LLC Prediction Intervals A prediction interval contains the expected results of a future sample to be obtained from a previously sampled population or process. Based upon a past sample of measurements, we might wish to construct a prediction interval to contain, with a specified degree of confidence: (1) the concentration of a randomly selected single future unit from the sampled population, (2) the concentrations for five future specimens, or (3) the average concentration of five future units. The form of a two-sided prediction interval is the same as a confidence interval or a tolerance interval: The factor K 1 − α , n has a 100(1 − α )% confidence level and depends on n , the number of observations in the given sample, and also on whether the prediction interval is to contain a single future value, several future values, or a future mean. Table 21.2 gives the factors to calculate (1) two-sided simultaneous prediction intervals to contain all of m future observations from the previously sampled normal population for m = 1, 2, 10, 20 and m = n ; and (2) two-sided prediction intervals to contain the mean of m = n future observations. The confidence level associated with these intervals is 95%. The two-sided (1 − α )100% prediction limit for the next single measurement of a normally distributed random variable is: where the t statistic is for n − 1 degrees of freedom, based on the sample of n measurements used to estimate the mean and standard deviation. For the one-sided upper (1 − α )100% confidence prediction limit use TABLE 21.2 Factors for Two-Sided 95% Prediction Intervals for a Normal Distribution Simultaneous Prediction Intervals to Contain All m Future Observations Prediction Intervals to Contain the Mean nm == == 1 m == == 2 m == == 5 m == == 10 m == == ∞∞ ∞∞ of n Future Observations 4 3.56 4.41 5.56 6.41 5.29 2.25 5 3.04 3.70 4.58 5.23 4.58 1.76 6 2.78 3.33 4.08 4.63 4.22 1.48 7 2.62 3.11 3.77 4.26 4.01 1.31 8 2.51 2.97 3.57 4.02 3.88 1.18 9 2.43 2.86 3.43 3.85 3.78 1.09 10 2.37 2.79 3.32 3.72 3.72 1.01 12 2.29 2.68 3.17 3.53 3.63 0.90 15 2.22 2.57 3.03 3.36 3.56 0.78 20 2.14 2.48 2.90 3.21 3.50 0.66 30 2.08 2.39 2.78 3.06 3.48 0.53 40 2.05 2.35 2.73 2.99 3.49 0.45 60 2.02 2.31 2.67 2.93 3.53 0.37 ∞ 1.96 2.24 2.57 2.80 ∞ 0.00 Source: Hahn, G. J. (1970). J. Qual. Tech., 3, 18–22. yK 1− α , n s± yt n−1, α /2 s 1 1 n +± y + t n−1, α s 1 1 n + . L1592_frame_C21 Page 177 Tuesday, December 18, 2001 2:43 PM © 2002 By CRC Press LLC Example 21.2 A random sample of n = 5 observations yields the values = 28.4 µ g/L and s = 1.18 µ g/L. An additional m = 10 specimens are to be taken at random from the same population. 1. Construct a two-sided (simultaneous) 95% prediction interval to contain the concentrations of all 10 additional specimens. For n = 5, m = 10, and α = 0.05, the factor is 5.23 from the second row of Table 21.2. The prediction interval is: 28.4 ± 5.23(1.18) = [22.2, 34.6] We are 95% confident that the concentration of all 10 specimens will be contained within the interval 22.2 to 34.6 µ g/L. 2. Construct a two-sided prediction interval to contain the mean of the concentration readings of five additional specimens randomly selected from the same population. For n = 5, m = 5, and 1 − α = 0.95, the factor is 1.76 and the interval is: 28.4 ± 1.76(1.18) = [26.3, 30.5] We are 95% confident that the mean of the readings of five additional concentrations will be in the interval 26.3 to 30.5 µ g/L. There are two sources of imprecision in statistical prediction. First, because the given data are limited, there is uncertainty with respect to the parameters of the previously sampled population. Second, there is random variation in the future sample. Say, for example, that the results of an initial sample of size n from a normal population with unknown mean η and unknown standard deviation σ are used to predict the value of a single future randomly selected observation from the same population. The mean of the initial sample is used to predict the future observation. Now where e, the random variation associated with the mean of the initial sample, is normally distributed with mean 0 and variance σ 2 /n. The future observation to be predicted is where e f is the random variation associated with the future observation, normally distributed with mean 0 and variance σ 2 . Thus, the prediction error is which has variance σ 2 + ( σ 2 /n). The length of the prediction interval to contain y f will be proportional to Increasing the initial sample will reduce the imprecision associated with the sample mean (i.e., σ 2 /n), but it will not reduce the sampling error in the estimate of the variation ( σ 2 ) associated with the future observations. Thus, an increase in the size of the initial sample size beyond the point where the inherent variation in the future sample tends to dominate will not materially reduce the length of the prediction interval. A confidence interval to contain a population parameter converges to a point as the sample size increases. A prediction interval converges to an interval. Thus, it is not possible to obtain a prediction interval consistently shorter than some limiting interval, no matter how large an initial sample is taken (Hahn and Meeker, 1991). Statistical Interval for the Standard Deviation of a Normal Distribution Confidence and prediction intervals for the standard deviation of a normal distribution can be calculated using factors from Table 21.3. The factors are based on the χ 2 distribution and are asymmetric. They are multipliers and the intervals have the form: [k 1 s, k 2 s] y y y η e+= , y f η e f ,+= y f y– e f e,–= σ 2 ( σ 2 /n)+. y L1592_frame_C21 Page 178 Tuesday, December 18, 2001 2:43 PM [...]... - +   m n 1/2 1 1 - = 33 2 .3 + 2.015 ( 39 .3 )  +   8 6 1/2 = 37 5.1 Thus, an upper 95% prediction bound for the total 8-year demand is 8 (37 5.1) = 30 01 bearings We are 95% confident that the total demand for the next 8 years will not exceed 30 01 bearings At the same time, if the manufacturer actually built 30 01 bearings, we would predict that the inventory would most likely last for 30 01 /33 2 .3. .. factors in 13 runs Two-level, 3- factor design in 8 runs (d) Composite two-level, 3- factor design in 15 runs FIGURE 22.6 Four possible experimental designs for studying three factors The worst is (a), the one-factor-at-a-time design (top left) (b) is a two-level, three-factor design in eight runs and can describe a smooth nonplanar surface The Box-Behnken design (c) and the composite two-level, three-factor... Temperature One-Factor-at-a Time Experiment Two-level Factorial Design Experiment FIGURE 22.5 Graphical demonstration of why one-factor-at-a-time (OFAT) experiments cannot estimate the two-factor interaction between temperature and pressure that is revealed by the two-level, two-factor design (a) Time (b) Optional center point Pressure Temperature One-factor-at-a time design in 13 runs (c) Box-Behnken design... Some of these are one-factor-at-a-time, paired comparison, two-level factorials, fractional factorials, Latin squares, Graeco-Latin squares, Box-Behnken, Plackett-Burman, and Taguchi designs An efficient design gives a lot of information for a little work A “botched” design gives very little information for a lot of work This chapter has the goal of convincing you that one-factor-at-a-time designs are... Distribution n= Factor 2 0.886 3 0.591 4 0.486 5 0. 430 6 0 .39 5 7 0 .37 0 n= Factor 11 0 .31 5 12 0 .30 7 13 0 .30 0 14 0.294 15 0.288 >15 0.250 8 0 .35 1 9 0 .33 7 10 0 .32 5 The following example illustrates that it is not always possible to achieve a stated objective by increasing the sample size This happens when the stated objective is inconsistent with statistical reality Example 23. 3 A system has been changed... of bearings that will meet, with 95% confidence, the demand for replacement bearings for at least 8 years The number of replacement bearings required in each of the past 6 years were: 282, 38 0, 31 8, 298, 36 8, and 34 8 © 2002 By CRC Press LLC L1592_frame_C21 Page 180 Tuesday, December 18, 2001 2: 43 PM For the given data, y = 33 2 .3 and s = 39 .3 Assume that the number of units sold per year follows a normal... 0. 73 0.77 1.00 3. 93 3.10 2.67 2.41 2. 23 2.01 1. 73 1.59 1 .38 1.29 1.00 Source: Hahn, G J and W Q Meeker (1991) Statistical Intervals: A Guide for Practitioners, New York, John Wiley Example 21 .3 A random sample of n = 5 observations yields the values y = 28.4 µg/L and σ = 1.18 µg/L 1 Using the factors in Table 21 .3, find a two-sided confidence interval for the standard deviation σ of the population For. .. 2: 43 PM TABLE 21 .3 Factors for Two-Sided 95% Statistical Intervals for a Standard Deviation of a Normal Distribution n 4 5 6 7 8 10 15 20 40 60 ∞ Confidence Intervals k1 k2 0.57 0.60 0.62 0.64 0.66 0.69 0. 73 0.76 0.82 0.85 1.00 3. 73 2.87 2.45 2.20 2.04 1. 83 1.58 1.46 1.28 1.22 1.00 Simultaneous Prediction Intervals to Contain All m = n Future Observations k1 k2 0.25 0 .32 0 .37 0.41 0.45 0.50 0.58 0. 63. .. = 33 2 .3 provides a prediction for the average yearly demand The demand for replacement bearings over 8 years is thus 8 (33 2 .3) = 2658 However, because of statistical variability in both the past and future yearly demands, the actual total would be expected to differ from this prediction A one-sided upper 95% prediction bound (y U ) for the mean of the yearly sales for the next m = 8 years is: 1 1 -. .. Building, New York, Wiley Interscience Colquhoun, D (1971) Lectures in Biostatistics, Oxford, England, Clarendon Press Czitrom, Veronica (1999) “One-Factor-at-a Time Versus Designed Experiments,” Am Stat., 53( 2), 126– 131 Joiner, B L (1981) “Lurking Variables: Some Examples,” Am Stat., 35 , 227– 233 Pushkarev et al (19 83) Treatment of Oil-Containing Wastewater, New York, Allerton Press Tennessee Valley Authority . Lab 3 Lab 4 Lab 5 3. 4 4.5 5 .3 3.2 3. 3 3. 0 3. 7 4.7 3. 4 2.4 3. 4 3. 8 3. 6 3. 1 2.7 5.0 3. 9 5.0 3. 0 3. 2 5.1 4 .3 3.6 3. 9 3. 3 5.5 3. 9 4.5 2.0 2.9 5.4 4.1 4.6 1.9 4.4 4.2 4.0 5 .3 2.7 3. 4 3. 8 3. 0 3. 9 3. 8. 1.09 10 2 .37 2.79 3. 32 3. 72 3. 72 1.01 12 2.29 2.68 3. 17 3. 53 3. 63 0.90 15 2.22 2.57 3. 03 3 .36 3. 56 0.78 20 2.14 2.48 2.90 3. 21 3. 50 0.66 30 2.08 2 .39 2.78 3. 06 3. 48 0. 53 40 2.05 2 .35 2. 73 2.99 3. 49. Control νν νν 234 5 6810 5 3. 03 3.29 3. 48 3. 62 3. 73 3.90 4. 03 10 2.57 2.76 2.89 2.99 3. 07 3. 19 3. 29 15 2.44 2.61 2. 73 2.82 2.89 3. 00 3. 08 20 2 .38 2.54 2.65 2. 73 2.80 2.90 2.98 30 2 .32 2.47 2.58