Stephen B. Vardeman

Iowa State University

Joanne R. Wendelberger

Los Alamos National Laboratory

Journal of Statistics Education Volume 13, Number 1 (2005), www.amstat.org/publications/jse/v13n1/vardeman.html

Copyright © 2005 by Stephen B. Vardeman and Joanne R. Wendelberger, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

**Key Words:** Heteroscedastic; Method of moments; One-way model; Two-factor hierarchical model;
Standard error of the mean; Variance component

**Lemma 1** If *Y _{1}, Y_{2}, ..., Y_{n}* are uncorrelated random variables with a common mean
(say ) and possibly different variances
, and

is their sample variance, then

**Proof:** First note that

Then observe that one may with no loss of generality assume that = 0.
(The *Y _{i}* and the
have the same sample variance, and if necessary one could replace the

A referee has suggested that in many classroom proofs of Lemma 1, it will be best
to write in the form
and further suggests that a good exercise will often be to ask students to redo the proof without simplifying to the
= 0 case. Notice that under the = 0
case assumption, the type of summation notation used may not be so important, in that in either notation it is immediate
from the fact that E*Y _{i}Y_{j}* = 0 for all
that . Not making use of the observation that one may reduce to the
= 0 case requires using the facts that
and , and being able to
count that there are terms in
in order to get the
necessary cancellation of squared means. How it is easiest for students to see the counting fact from the type of
summation notation used depends upon what has gone before in a course. In any case, we think that it is important to use
the device of reducing to = 0 in classroom proofs, not simply because it
is “elegant,” but more importantly because it foreshadows how the lemma can be applied in variance component estimation.
(See the use of the fact that sample variances are unchanged by the addition of a common value to each element of a “data
set” in our later discussion of estimation in an unbalanced two-factor nested design.)

Lemma 1 is very simple and arguably “obvious.” But it is not well known and provides a mathematically satisfying extension of the standard result. Further, it can be applied to good effect in important teaching and data analysis contexts.

Note, for example, that under the hypotheses of the lemma

So is potentially a sensible estimator of
(at least where the relative precisions of the *Y _{i}* are
unknown) and

functions as a standard error for in the potentially heteroscedastic case of the lemma as well as the more familiar iid situation. This is a kind of “robustness” result for the usual standard error of the sample mean and appears as Problem 2.2.3 on page 52 of Stapleton (1995) without explicit mention of Lemma 1. (This is the only reference known to the authors that even hints at Lemma 1.)

We proceed to illustrate that the lemma has important additional uses beyond this most obvious one.

But it is possible to use Lemma 1 to produce simple/from-first-principles estimators based on (even) unbalanced data under linear random effects models (and in the process demystify the problem of estimation in these models). This is because the lemma shows expected sample variances of appropriate sample average observations to be easily-identified linear combinations of variance components. We first illustrate in the general context of the one-way random effects model.

That is, suppose that for *i* = 1, 2, ..., *I,* and *j* = 1, 2, ..., *n _{i}*

for some constant, with mean 0 and variance , with mean 0 and variance , and all of the and uncorrelated. We may apply the foregoing to the uncorrelated sample means

that have

The unweighted mean of sample means

is an unbiased estimator of with

. | (1) |

If we write

by Lemma 1, this sample variance (of sample means) has expected value

. | (2) |

So in light of (1) and (2), a standard error for the unbiased estimator of is

. | (3) |

regardless of whether or not the data are balanced.

The authors’ original motivation for considering applications of Lemma 1 (and in
particular, standard error (3)) in the one-way context was a calibration problem
where represented a day-to-day variance component in the
measurement of a standard, represented a within-day variance
component, and constraints in the measurement process led to an error analysis based on the average values. The approach
was also applicable in another situation, where analysis of summary data was required, and the sample sizes (and individual
observations *X _{ij}*) were not available.

What is more, where the sample sizes and within-group sample variances are available, it is easy to use Lemma 1 to motivate simple estimators of the variance components. Let

be the usual pooled sample variance (or mean squared error). This has mean . In light of equation (2),

which suggests the simple estimators of variance components

. | (4) |

which appear, for example, in Rao (1997, page 20) and Cox and Solomon (2003, pages 74-76).

Figure 1

Figure 1: Schematic of a particular unbalanced two-factor hierarchical data structure

That is, with

suppose that

for some constant, the with mean 0 and variance , the with mean 0 and variance , the with mean 0 and variance , and all of the , , and uncorrelated. Let

and define sample means

and unweighted means of these

and the unweighted mean of these

We consider estimators of the variance components , , and based on the sample variances (of unweighted sample means)

and

To begin, as always, the usual pooled sample variance

serves as an unbiased estimator of . Note then that using the usual notation for averages of ’s

and that *S _{1}^{2}* is not only the sample variance of
and ,

Similarly,

So for any *c* between 0 and 1,

which then suggests that for such *c*, be estimated as

Finally, consider estimating . With the usual notation for averages of ’s and ,

So once more applying Lemma 1 (to the sample variance of uncorrelated variables with a common mean and ),

which in turn suggests the estimator

While our discussion has focused exclusively on moment results (and is thus not restricted to Gaussian models), there is
much traditional interest and a huge literature concerned with distributional (and inference) results when one adds
normality to the kind of assumptions we’ve made. Our reviewers have made several interesting points regarding connections
to that literature. If one adds (joint) normality to the assumptions of Lemma 1,
the resulting distribution for *S ^{2}* is not chi-squared, but rather that of a weighted average of
independent chi-square variables. On the other hand, under the normal one-way random effects model, our
is sometimes referred to as the unweighted mean square, and pages
68-73 of Burdick and Graybill (1992) argue that suitably scaled, it is
approximately chi-square. Further, this result has been used by
El-Bassiouni and Abelhafez (2000) to produce valid confidence intervals
for in this context. Finally, pages 98-106 of
Burdick and Graybill argue that in the normal version of the two-factor
nested design, provided

Financial support of the Deutsche Forschungsgemeinschaft (SFB 475, “Reduction of Complexity in Multivariate Data Structures”) through the University of Dortmund and of the Los Alamos National Laboratory Statistical Sciences Group is gratefully acknowledged by the first author.

Burdick, R.K. and Graybill, F.A. (1992), *Confidence Intervals on Variance Components*, New York: Marcel Dekker.

Casella, G. and Berger, R.L. (2002), *Statistical Inference*, Pacific Grove, California: Duxbury.

Cox, D.R. and Solomon, P.J. (2003), *Components of Variance*, New York: Chapman & Hall.

El-Bassiouni, M.Y. and Abdelhafez, M.E.M. (2000), “Interval estimation of the mean in a two-stage nested model,”
*Journal of Statistical Computation and Simulation*, 67 (4), pp. 333-350.

Hicks, C.R. and Turner, K.V. (1999), *Fundamental Concepts in the Design of Experiments, 5 ^{th} Ed.*,
Oxford: Oxford University Press.

Miller, I. and Miller, M. (2004), *John E. Freund’s Mathematical Statistics, 7 ^{th} Edition*,
Upper Saddle River, New Jersey: Prentice Hall.

Neter, J., Kutner, M.H., Wasserman, W., and Nachtsheim, C.J. (1996), *Applied Linear Statistical Models, 4 ^{th}
Edition*, Chicago: McGraw-Hill/Irwin.

Rao, P.S.R.S. (1997), *Variance Components Estimation*, New York: Chapman & Hall.

Stapleton, J.H. (1995), *Linear Statistical Models*, New York: John Wiley & Sons.

Wackerly, D.D., Mendenhall W., and Scheaffer, R.L. (2002), *Mathematical Statistics with Applications, 6 ^{th}
Edition*, Pacific Grove, California: Duxbury.

Wasserman, L. (2004), *All of Statistics: A Concise Course in Statistical Inference*, New York: Springer-Verlag.

Stephen B. Vardeman

Departments of Statistics and Industrial and Manufacturing Systems Engineering

Iowa State University

Ames, IA 50011-1210

U.S.A.
*vardeman@iastate.edu*

Joanne R. Wendelberger

Statistical Sciences Group

Los Alamos National Laboratory

Los Alamos, NM

U.S.A.
*joanne@lanl.gov*

Volume 13 (2005) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications