Fig. 2 Schematic of distribution of detection probabilities for cracks of fixed length Equation 1 implies that the POD(a) function is the curve through the averages of the individual density functions of the detection probabilities. This curve is the regression equation and provides the basis for testing assumptions about the applicability of various POD(a) models. In Ref 4, seven different functional forms were tested for applicability to available POD data, and it was concluded that the log-logistics (log odds) function best modeled the data and provided an acceptable model for the data sets of the study. Note that the log odds model is commonly used in the analysis of binary (hit/miss) data because of its analytical tractability and its close agreement with the cumulative log normal distribution (Ref 8). Two mathematically equivalent forms of the log odds model have subsequently been used. The earliest form is given by: (Eq 2) This parametrization can also be expressed as: (Eq 3) In the Eq 3 form, the log of the odds of the probability of detection (the left-hand side of Eq 3) is expressed as a linear function of ln (a) and is the source of the name of the log odds model. Note that given the results of a large number of independent inspections of a large number of cracks, the parameters of the model can be fit with a regression analysis. As an example, Fig. 3 shows Eq 3 fit to the data of Fig. 1. This regression approach will not be discussed further, because the maximum likelihood estimates (see the section "Analysis of Hit/Miss Data" in this article) can be applied to much smaller samples of inspection results and can give equivalent answers for large sample sizes. Fig. 3 Example linear relation between log odds of crack detection and log crack size Although the parametrizations of Eq 2 and 3 are sensible in terms of estimation through regression analyses, and are not easily interpretable in physical terms. A mathematically equivalent form of the log odds POD(a) model is given by (Ref 8): (Eq 4) In this form, μ= ln a 0.5 , where a 0.5 is the flaw size that is detected 50% of the time, that is, the median detectable crack size. The steepness of the POD(a) function is inversely proportional to ; that is, the smaller the value of σ, the steeper the POD(a) function. The parameters of Eq 2 and 4 are related by: (Eq 5) (Eq 6) The log odds POD(a) function is practically equivalent to a cumulative log normal distribution with the same parameters, μ and σ of Eq 4. Figure 4 compares the log odds and cumulative log normal distribution functions for μ= 0 and σ= 1. Equation 4 is the form of the log odds model that will be used in the section "Analysis of Hit/Miss Data" in this article. Fig. 4 Comparison of log odds and cumulative log normal models POD(a) From Signal Response Data. The NDE flaw indications are based on interpreting the response to a stimulus. In eddy current or ultrasonic systems, the response might be a peak voltage referenced to a calibration. In fluorescent penetrant inspections, the response would be a combination of brightness and size of the indication. Assume the response can be quantified and recorded in terms of a parameter, , that is correlated with flaw size. Then summarizes the information for determining if a positive flaw indication will be given. Only if exceeds a defined decision threshold, dec , will a positive indication be given. As an example of the concept, Table 2 summarizes the results of highly automated eddy current inspections of 28 cracks in flat plate specimens. The three data sets resulted from the use of three probes, with all other factors held constant. The values in Table 2 are the depth of each crack and the peak voltage in counts recorded by the system. Figure 5 shows a plot of the versus a data for probe A. No signal was recorded for 2 of the cracks, because their values were below the recording signal threshold, th . These points are indicated by a down arrow at th , indicating that the response was at an indeterminable value below the recording signal threshold. Similarly, for 5 of the cracks exceeded the saturation limit, sat , of the recording system. These points are indicated by an up arrow at sat , indicating that the response was at an indeterminable value above the recording saturation limit. In Fig. 5, the decision threshold is set at 250 counts. Only those cracks whose value is above 250 would have been flagged (detected). Table 2 Example of a summary data sheet of versus a data The example is based on eddy current inspections of flat plates. Crack depth Peak voltage in counts Crack identification mm in. Probe A Probe B Probe C 11 0.33 0.013 1052 884 1282 30 1.40 0.055 4095 4095 3831 42 0.38 0.015 1480 1182 1699 2 0.25 0.010 723 624 840 21 0.74 0.029 4095 4095 2249 13 0.48 0.019 2621 2401 1101 19 0.30 0.012 377 809 350 26 0.23 0.009 223 205 277 15 0.56 0.022 1654 3319 1289 29 1.65 0.065 4095 4095 2648 33 0.08 0.003 (a) (a) (a) 25 0.25 0.010 669 565 824 32 0.18 0.007 374 379 407 34 0.03 0.001 (a) (a) (a) 39 0.18 0.007 409 387 586 12 0.28 0.011 895 690 677 38 0.20 0.008 374 301 549 20 0.79 0.031 4095 4095 1778 28 0.23 0.009 638 454 782 27 0.15 0.006 533 385 631 1 0.08 0.003 150 136 135 35 0.28 0.011 749 660 989 40 0.20 0.008 433 378 591 31 0.36 0.014 879 888 1402 3 0.23 0.009 286 211 352 7 0.23 0.009 298 163 215 16 0.41 0.016 1171 1110 1628 37 2.54 0.100 4095 4095 4095 (a) Peak voltage below the recording level threshold Fig. 5 Example inspection signal response as a function of crack depth The POD(a) function can be obtained from the relation between and a. If g a ( ) represents the probability density of the values for fixed crack size a, then: (Eq 7) This calculation is illustrated in Fig. 6, in which the shaded area under the density functions represents the probability of detection. Fig. 6 Schematic of POD(a) calculation from versus a relation In general, the correlating function between and a defines the mean of g a ( ), that is: = (a) + (Eq 8) where (a) is the mean of g a ( ) and is a random error term accounting for the differences between and (a). The distributional properties of δ determine the probability density g a ( ) about μ (a), as will be shown. In the data analyzed to date, a linear relation between ln ( ) and ln (a) with normally distributed deviations has proved satisfactory (for example, Fig. 5). This model is expressed by: ln ( ) = 0 + 1 ln (a) + (Eq 9) where δ is normally distributed with zero mean and constant standard deviation, . Data have been observed that flatten at the large crack sizes. However, because the decision threshold was far below the non-linear range, restricting the range of cracks to smaller sizes permitted the application of Eq 9. The normality of has proved to be an acceptable assumption. Assuming that the versus a relation is modeled by Eq 9 and that is normally distributed with zero mean and standard deviation of , the POD(a) function is calculated as: (Eq 10) where is the standard normal distribution function. Using the symmetry properties of , Eq 10 can be reduced to: (Eq 11) Equation 11 is a cumulative log normal distribution function with mean and standard deviation of log crack length given by: (Eq 12) (Eq 13) In the section "Signal Response Analysis" in this article, maximum likelihood methods for estimating β 0 , β 1 , and σ from versus a data will be presented. Note that the values below the recording threshold and above the saturation limit must be properly accounted for in these analyses. Note also that data from multiple inspections of the same cracks require analysis methods that are dependent on the design of the reliability experiment. Methods for placing lower confidence bounds on the estimated POD(a) function using the sampling distributions of the maximum likelihood estimates of β 0 , β 1 , and are also included in the section "Signal Response Analysis." References cited in this section 3. W.H. Lewis, W.H. Sproat, B.D. Dodd, and J.M. Hamilton, "Reliability of Nondestructive Inspections Final Report," SA-ALC/MME 76-6-38-1, San Antonio Air Logistics Center, Kelly Air Force Base, Dec 1978 4. A.P. Berens and P.W. Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-81- 4160, Vol 1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 1981 5. A.P. Berens and P.W. Hovey, Statistical Methods for Estimating Crack Detection Probabili ties, in Probabilistic Fracture Mechanics and Fatigue Methods: Applications for Structural Design and Maintenance, STP 798, J.M. Bloom and J.C. Ekvall, Ed., American Society for Testing and Materials, 1983, p 79-94 6. D.E. Allison et al., "Cost/Risk Analysis for Disk Retirement Volume I," AFWAL-TR-83- 4089, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 1984 7. A.P. Berens and P.W. Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-84-4022, Air Force Wright-Aeronautical Laboratories, Wright- Patterson Air Force Base, April 1984 8. D.R. Cox, The Analysis of Binary Data, Methuen and Co., 1970 NDE Reliability Data Analysis Alan P. Berens, University of Dayton Research Institute Design of NDE Reliability Experiments An NDE reliability experiment comprises a test matrix of inspections on a set of specimens with known flaw locations and sizes. The specimens are inspected under conditions that simulate as closely as practical the actual application conditions. The experimental design determines the test matrix, and there are four major analysis concerns to be addressed in the experimental design. These are: • The method of controlling the factors to be evaluated in the experiment • The method of accounting for the uncontrolled factors in the experiment • The number of flawed and unflawed inspection sites • The sizes of the flaws in the specimens These topics are addressed in the following sections. Controlled and Uncontrolled Factors The primary objective of NDE reliability experiments has been to demonstrate efficacy for a particular application by estimating the POD(a) function and its lower 95% confidence bound. (Although NDE reliability experiments can also be conducted to optimize a system, analyses to meet this objective are beyond the scope of this article.) To demonstrate capability, it is assumed that the protocol for conducting the inspections is well defined for the application, that the inspection process is under control (hit/miss decisions are stable over time), and that all other factors introducing variability into the inspection decision will be representative of the application. The representativeness of these other factors can be ensured either by controlling the factors during the inspection or randomly sampling the factors to be used in the experiment. The methods of accounting for these factors are important aspects of the statistical design of the experiment and significantly influence the statistical properties of the estimates of the POD(a) function parameters. Of particular note in this regard is that k inspections on n flaws is not equivalent to inspections on n · k different flaws, even if the inspections are totally independent. The most important of the factors introducing variation are: • Differences in physical properties of cracks of nominally identical sizes • The basic repeatability of the magnitude of the NDE signal response when a specific crack is independently inspected by a single inspector using the same equipment • The summation of all the human factors associated with the particular inspectors in the population of interest • Differences introduced by changes in inspection hardware These factors must be addressed explicitly or implicitly in every NDE reliability experiment. In general, the specimens used in NDE reliability experiments are very expensive to obtain and characterize in terms of the sizes of the flaws in the specimens. Therefore, each experiment is based on one set of specimens containing flawed and unflawed inspection sites. Because the results are significantly influenced by the specimens, it must be assumed that the flaws are representative of those that will be present in the structural application. If other factors are to be included in the experiment, they will be based on repeated inspections of the same flaws. From a statistical viewpoint, this restriction on the experimental design limits the sample size to the number of flaws in the specimen set. Because different cracks of the same size can have significantly different crack detection probabilities, multiple inspections of the same crack provide information about the detection probability of only that crack. [...]... Retirement Volume I," AFWAL-TR-8 3-4 089, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 198 4 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-8 4-4 022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, April 198 4 D.R Cox, The Analysis of Binary Data, Methuen and Co., 197 0 A.P Berens and P.W... of Nondestructive Evaluation (NDE) Reliability on Aircraft Production Parts, Mater Eng., Vol 40, Aug 198 2, p 92 2-9 32 W.H Lewis, W.H Sproat, B.D Dodd, and J.M Hamilton, "Reliability of Nondestructive Inspections-Final Report," SA-ALC/MME 7 6-6 -3 8-1 , San Antonio Air Logistics Center, Kelly Air Force Base, Dec 197 8 A.P Berens and P.W Hovey, "Evaluation of NDE Reliability Characterization," AFWAL-TR-8 1-4 160,... NDE Reliability Characterization," AFWAL-TR-8 1-4 160, Vol 1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 198 1 9 A.P Berens and P.W Hovey, The Sample Size and Flaw Size Effects in NDI Reliability Experiments, in Review of Progress in Quantitative Nondestructive Evaluation 4B, D.O Thompson and D.E Chimenti, Ed., Plenum Press, 198 5 NDE Reliability Data Analysis Alan... Ci, Oj, Pk, and Rl are the random effects due to cracks, operators, probes, and repeats, respectively The random term, δ of Eq 9, is the sum of all random effects It can be assumed that the mean and variance of random effect X are zero and , respectively Then: = + + + + Therefore, β0 and β1 can be estimated from a regression analysis, and variance using the expected mean squares for the random effects... Continuous Random Variables, Technometrics, Vol 25 (No 1), Feb 198 3, p 7 7-8 6 R.C.H Cheng and T.C Iles, One Sided Confidence Bands for Cumulative Distribution Functions, Technometrics, Vol 32 (No 2), May 198 8, p 15 5-1 59 14 A Ralston, A First Course in Numerical Analysis, McGraw-Hill, 196 5 15 M Glaser, Regression Analysis With Dependent Variable Censored, Biometrics, Vol 21, June 196 5, p 30 0-3 07 16 S.R... capability References 1 S.T Rolfe and J.M Barson, Fracture and Fatigue Control in Structures: Application of Fracture Mechanics, Prentice-Hall, 197 7 2 R.B Thompson and T.A Gray, Use of Ultrasonic Models in the Design and Validation of New NDE Techniques, Philos Trans R Soc (London) A, Vol 320, 198 6, p 32 9-3 40 3 R.B Thompson and H.N.G Wadley, The Use of Elastic Wave-Material Structure Interaction Theories... 1, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Dec 198 1 A.P Berens and P.W Hovey, Statistical Methods for Estimating Crack Detection Probabilities, in Probabilistic Fracture Mechanics and Fatigue Methods: Applications for Structural Design and Maintenance, STP 798, J.M Bloom and J.C Ekvall, Ed., American Society for Testing and Materials, 198 3, p 7 9-9 4 D.E Allison et... crack and all detection probabilities are plotted at 0 or 1 Fig 9 Example fit of hit/miss POD(a) function and lower 95% confidence bound to observed detection probabilities (three inspections per crack) References cited in this section 7 A.P Berens and P.W Hovey, "Flaw Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-8 4-4 022, Air Force Wright-Aeronautical Laboratories, Wright-Patterson... Sample Size and Flaw Size Effects in NDI Reliability Experiments, in Review of Progress in Quantitative Nondestructive Evaluation 4B, D.O Thompson and D.E Chimenti, Ed., Plenum Press, 198 5 H Cramer, Mathematical Methods of Statistics, Princeton University Press, 194 6 J.F Lawless, Statistical Models and Methods for Lifetime Data, John Wiley & Sons, 198 2 R.C.H Cheng and T.C Iles, Confidence Bands for Cumulative... confidence bound is a five-step process: • The information matrix for the estimates of • The variance-covariance matrix of 0, 1, and is obtained by inverting the information matrix (Eq 20) The variance-covariance matrix of the estimates of and are calculated based on a first-order Taylor series expansion of the equations relating 0, 1, and to and (Eq 12 and 13) The information matrix for and is obtained by . "Reliability of Nondestructive Inspections Final Report," SA-ALC/MME 7 6-6 -3 8-1 , San Antonio Air Logistics Center, Kelly Air Force Base, Dec 197 8 4. A.P. Berens and P.W. Hovey, " ;Evaluation. Analysis for Disk Retirement Volume I," AFWAL-TR-8 3- 4089, Air Force Wright-Aeronautical Laboratories, Wright-Patterson Air Force Base, Feb 198 4 7. A.P. Berens and P.W. Hovey, "Flaw. Detection Reliability Criteria, Volume I Methods and Results," AFWAL-TR-8 4-4 022, Air Force Wright-Aeronautical Laboratories, Wright- Patterson Air Force Base, April 198 4 8. D.R. Cox, The