[...]... Type I errors, whereas a more stringent standard might lead to few Type I errors.1 A second type of error, referred to as Type II error, is also common in statistical hypothesis testing (J Cohen, 1994; Sedlmeier & Gigerenzer, 1989) A Type II error occurs when researchers conclude in favor of H0, when in fact Hl is true Statistical power analysis is concerned with Type II errors The power of a statistical. .. criteria is probably not the best explanation for low levels of statistical power The best explanation for the low levels of power observed in many areas of research is many studies use samples that are much too small to provide accurate and credible results Researchers routinely use samples of 20, 50, or 75 observations to make inferences about population parameters When sample results are unreliable,... one minus the probability of making a Type II error (i.e., if the probability of making a Type II error is b, power = 1 - b, or power is the probability that you will avoid a Type II error) Studies with high levels of statistical power will rarely fail to detect the effects of treatments If it 'It is important to note that Type I errors can only occur when the null hypothesis is actually true If the null... or whether they fall within the range of outcomes that might be produced by random sampling error For example, there are two possible interpretations of the findings in this study of reading instruction: 1 The difference between average scores from the two programs is so small that it might reasonably represent nothing more than sampling error 2 The difference between average scores from the two programs... also increases the statistical power of the study As Fig 1.1 shows, there is always a trade-off between Type I and Type II errors Making it very difficult to reject the null hypothesis minimizes Type I errors (incorrect rejections), but also increases the number of Type II errors That is, if the null is rarely rejected, sometimes sample results will be incorrectly dismissed as mere sampling error when... cures for errors in statistical inference Statistical power analysis (J Cohen, 1988; Kraemer & Thiemann, 1987; Lipsey, 1990) falls under this general heading Studies with too little statistical power can frequently lead to erroneous conclusions In particular, they will very often lead to the incorrect conclusion that findings reported in a particular study are not likely to be true in the broader population... the true effects of treatments Numerous authors have noted that procedures to control or minimize Type I errors can substantially reduce statistical power, and may cause more problems (i.e., Type II errors) than they solve (J Cohen, 1994; Sedlmeier & Gigerenzer, 1989) THE POWER OF STATISTICAL TESTS 15 Power Analysis and the General Linear Model The following chapters describe a simple and general model... necessary It is important to understand how each of the parameters involved is determined when conducting a power analysis Determining the Effect Size There is a built-in dilemma in power analysis In order to determine the statistical power of a study, the effect size must be known But if researchers already knew the exact strength of the effect of the particular treatment, intervention, and so forth,... sampling error likely to be present in different statistical procedures and tests, and thereby gaining some idea about the amount of risk involved in using a particular procedure THE POWER OF STATISTICAL TESTS 3 Statistical significance tests can be thought of as decision aids That is, these tests can help researchers draw conclusions about whether the findings of a particular study represent real population... the range of values to reasonably expect for some test statistic if the real effect of treatments is small, or medium, or large Figure 1.2 illustrates the key ideas in statistical power analysis Suppose researchers devise a new test statistic and use it to evaluate the 6-point difference in reading test scores described earlier The larger the difference between the two treatment groups, the larger the . edition, by Kevin R. Murphy and Brett Myors. Includes bibliographical references and index. ISBN 0-8 05 8-4 52 5-9 (cloth : alk. paper) ISBN 0-8 05 8-4 52 6-7 (pbk. : alk. paper) Copyright . the proba- bility of making a Type II error is b, power = 1 - b, or power is the prob- ability that you will avoid a Type II error). Studies with high levels of statistical . lead to more frequent Type I errors, whereas a more stringent standard might lead to few Type I errors. 1 A second type of error, referred to as Type II error, is also