Logical Extremes, Beta, and the Power of the Test

James V. Pinto, Pin Ng, and David S. Allen
Northern Arizona University

Journal of Statistics Education Volume 11, Number 1 (2003), www.amstat.org/publications/jse/v11n1/pinto.html

Copyright © 2003 by James V. Pinto, Pin Ng, and David S. Allen, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.


Key Words: Power function; Simulations; Type II error.

Abstract

There is a potential misuse of the power function under the logical extreme when the null hypothesis is true. The power function is defined to measure the probability of rejecting the null given any value of the parameter being tested. It can be used to obtain the power and the beta values only under the alternative hypothesis. When the null is true, the power function can be used to obtain the size of the test. The power and the probability of committing a Type II error are, however, undefined and, hence, the power function should not be used to obtain these values.

1. Introduction

It is not an easy or smooth task to calculate the power of a test for beginning statistics students. Until computer algorithms and simulations were designed to help in this process (see, for example, Doane, Mathieson, and Tracy 2001), the best one could do was to calculate a single point on the power function representing 1 - beta (as in Kvanli, Ravur, and Guynes 2000). Yu and Behrens (1994) illustrate the usefulness of simulations in determining the power of a test. It is possible to calculate every point on the power curve, but in-class demonstrations are usually limited to single points due to computational constraints and the repetitive nature of the process. Logical extremes, as detailed below, are usually ignored. The purpose of this note is to demonstrate that the logical extremes are nontrivial and that one of the extremes results in calculations of beta and the power of the test that are inconsistent with the common pedagogy. A detailed history of the nature of hypothesis testing, including a section of the power of the test in light of the Fisher approach versus the Neyman-Pearson approach, can be found in Lehmann (1993). He concluded that combining the best features of both could unify the two approaches. This historical debate will not be addressed in this paper.

2. Calculation Process

We will use the familiar test on the population mean of a normal distribution with a known variance to set up the stage for logical extremes. We assume that the data generating process is phiofmusigmasquared where phi is the probability density function of a normal distribution with an unknown mean mu and a known variance sigmasquared.

We first consider testing the simple null hypothesis H0: mu = mu0 = 1.5 against a two-sided composite alternative H1: mu neq mu0 assuming sigma = 0.1 and n = 36. In our example, under the null hypothesis, the sampling distribution of the sample mean is normal, centered at mu0 = 1.5, with a standard error of sigmasubxbar = sigma / sqrt( n ) = 0.1 / sqrt( 36 ). This is illustrated by the normal distribution on the left in Figure 1, which was generated using Visual Statistics (Doane, Mathieson, and Tracy 2001). The size of the z-test (the probability of committing a Type I error), alpha, is chosen to be 10%. Figure 1 also presents a second normal distribution on the right, that is the sampling distribution of the sample mean at the specific value of mu = 1.51 when the null hypothesis is false. The shaded area under this sampling distribution in the middle acceptance region measures the probability of committing a Type II error while the shaded areas in the rejection regions provide the power of the test, 1 - beta.


Figure 1

Figure 1. Beta and the Power of the Test for Two-Tailed Test.


The power function, defined as the probability of rejecting the null given a specific value of mu, for the z test is given by pi( mu) = ..., where Phi is the cumulative distribution function of the standard normal distribution. This yields 1 - beta = 0.16 on the power function at this single point of mu = 1.51. The entire power function is shown in Figure 2 and resembles an upside down normal distribution. It should be noted that the power function in this example approaches one asymptotically as we move away from 1.5 in either direction.


Figure 2

Figure 2. The Power Function for a Two-Tailed Test.


For the one-tailed test, we consider the composite null hypothesis H0: mu leq mu0 versus the composite alternative H1: mu > mu0. The sampling distributions of the sample mean under the null of mu = mu0 = 1.5 and the specific alternative of mu = 1.51. are illustrated, respectively, by the left and right distributions in Figure 3. The power function is given by pi( mu) = .... The power of the test at mu = 1.51. is 0.25 and the whole power function is illustrated in Figure 4. The power function in this example approaches one and zero asymptotically as we move away from 1.5 in the positive and negative directions, respectively.


Figure 3

Figure 3. Beta and the Power of the Test for One-Tail Test.


Figure 4

Figure 4. The Power Function for a One-Tail Test.


3. The Logical Extremes

The first logical extreme occurs when the two sampling distributions in both Figure 1 and 3 pull apart to a point where there is very little overlap between them. In this case the power of the test approaches 1 (as the two distributions in Figure 1 and Figure 3 separate), and the beta approaches zero. This can easily be seen in Figure 2 when mu moves away from mu0 = 1.5 in both directions and also in Figure 4 when mu moves away from mu0 = 1.5 to the right.

The second logical extreme occurs when mu approaches mu0 = 1.5. One subtle point that has not usually been made explicitly in the definition of the power function is the fact that the power function yields the power of a test only when the null hypothesis is false. Under the null hypothesis, the power function provides the probability of committing a Type I error instead of the power of the test. As presented in Figure 2, the power function yields the 10% size at mu = mu0 = 1.5 according to the accepted pedagogy (Freund and Walpole 1987).

This is where this logical extreme breaks down. In this case, it is incorrect to say that “the power of the test is 10% or the probability of committing a Type II error is 90%” since the power is defined as “the probability of rejecting a false null hypothesis” and a Type II error is defined to be “the failure to reject the false null hypothesis.” But there is no false null hypothesis when the null hypothesis is true at mu = mu0 = 1.5. beta should be undefined rather than having a maximum value of 90%. Thus, the contention that the power of the test is at a minimum of 10% under the null cannot be supported because there is no false null hypothesis when the null hypothesis is true at mu = mu0 = 1.5. Both the power and the probability of committing a Type II error are undefined under this logical extreme. Unfortunately, this is usually overlooked. For example Figure 5 is generated for mu = mu0 = 1.5 using Visual Statistics which labels the area under the “accept” region as “Type II” error and the area under the “Reject” regions as “Power.” This misconception is also apparent in the few current textbooks that attempt to address the power function. In Berenson, Levine and Krehbiel (2002, pp. CD9-6) they state “Interestingly, if the population mean is actually 368 grams, the power of the test is equal to alpha, the level of significance (which is 0.05 in this example), because the null hypothesis is actually true.” In Newbold, Carlson and Thorne (2003, p. 357), the authors make a similar statement that includes “The power of the test at mu0 equals alpha, ...”


Figure 5

Figure 5. Logical Extreme: Beta and the Power of the Test for Two-Tailed Test.


Similarly, the sampling distribution and the power function for the one-tailed test are presented in Figures 6 and 7.


Figure 6

Figure 6. Logical Extreme: Beta and the Power of the Test for One-Tail Test (Right).


Figure 7

Figure 7. Logical Extreme: The Power Function for a One-Tail Test (Right).


The power function in Figure 7 provides the probability of rejecting the null hypothesis over the whole range of values of mu. It can be used to obtain the power of the test only when mu > 1.5. The size of the test is defined as the supremum of the power function over mu leq 1.5. This yields a size of 10% according to accepted pedagogy (Mood, Graybill and Boes 1974; Tamhane and Dunlop 2000 ). The power and beta remain undefined under the null when mu leq 1.5.

4. Conclusions

There is a potential misuse of the power function under the logical extreme when the null hypothesis is true. The power function is defined to measure the probability of rejecting the null given any value of the parameter being tested. It can be used to obtain the power and the beta values only under the alternative hypothesis. When the null is true, the power function can be used to obtain the size of the test. The power and the probability of committing a Type II error are, however, undefined and, hence, the power function should not be used to obtain these values.


Acknowledgments

The authors wish to thank two anonymous referees for their detailed and useful comments. All remaining errors are the responsibility of the authors.


References

Berenson, M. L., Levine, D. M., and Krehbiel, T. C. (2002), Basic Business Statistics, (8th ed.), Upper Saddle River, NJ: Prentice Hall.

Doane, D. P., Mathieson, K., and Tracy, R. L. (2001), Visual Statistics 2.0, New York: McGraw-Hill/Irwin.

Freund, J. E., and Walpole, R. E. (1987), Mathematical Statistics, Englewoods Cliffs, NJ: Prentice Hall.

Kvanli, A. H., Pavur, R. J., and Guynes, C. S. (2000), Introduction to Business Statistics: A Computer Integrated, Data Analysis Approach, (5th ed.), Cincinnati, OH: South-Western College Publishing.

Lehmann, E. L., “The Fisher, Neyman-Pearson Theories of Testing Hypothesis: One Theory or Two?” Journal of the American Statistical Association, 88, 1242-1249.

Mood, A. M., Graybill, F. A., and Boes, D. C. (1974), Introduction to the Theory of Statistics, (3rd ed.), New York: McGraw-Hill.

Newbold, P., Carlson, W. L., and Thorne, B. (2003), Statistics for Business and Economics, (5th ed.), Upper Saddle River, NJ: Prentice Hall.

Tamhane, A. C., and Dunlop D. D. (2000), Statistics and Data Analysis from Elementary to Intermediate, Upper Saddle River, NJ: Prentice Hall.

Yu, C. H., and Behrens, J. T., (1994), “Identification of Misconceptions in Learning Statistical Power With Dynamic Graphics As a Remedial Tool,” American Statistical Association Proceedings of the Section on Statistical Education, 242-246.


James V. Pinto
College of Business Administration
Northern Arizona University
Flagstaff, AZ 86011
James.Pinto@nau.edu

Pin Ng
College of Business Administration
Northern Arizona University
Flagstaff, AZ 86011
Pin.Ng@nau.edu

David S. Allen
College of Business Administration
Northern Arizona University
Flagstaff, AZ 86011
David.Allen@nau.edu


Volume 11 (2003) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications