Subject: [isostat] Answers regarding the Wilson estimator
From: "Christopher J. Lacke"
Date: Mon, 06 Dec 2004 17:43:57 -0500
To: SIGMAASTATED@ENTERPRISE.MAA.ORG, isostat@oberlin.edu, lacke@rowan.edu

Hi all:

First, I'd like to thank those of you who took the time to respond. 
Second, before I post the responses, I'd like to clarify a couple of
points from my previous posting.

1.  I teach the Wilson Estimator.  It was amazing to see how many
people inferred that I don't teach it from the tone of my question.  As
a matter of fact, I have omitted the response (private, thank you) from
one person who lambasted my teaching skills because he/she thought that
I don't use it.

2.  My statement about the lack of use came from a survey of business,
biology, survey methodology, and psychology books that are being used at
Rowan this semester.  That said, maybe I should have reworded my
question to ask "What good is teaching a new method to students when our
colleagues in other fields are unaware of it?  How can we get them to
change their pedagogy to match ours?"  I have searched the method and I
have found it to be quite popular in practice, but not in the small
sample of pedagogy in the disciplines that use statistics.

3.  I also asked the question because it relates to the question of
teaching the z-interval and z-test for a population mean.  Some people
claim that since it's not very realistic in practice, we should just
teach the t procedures and approach the z-procedures through a
proportion. (I know that is a loaded question!)

Anyway, thanks for the dialectic and have a good evening.

Chris

From: 	"Patricia Humphrey" <phumphre@GeorgiaSouthern.edu>
To:	"Christopher J. Lacke" <lacke@rowan.edu>,
<SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu>
Date: 	12/6/04 10:58AM
Subject: 	Re: [isostat] Teaching a method that isn't used

FYI, Chris (and all) - although D and V do mention the Wilson
estimator, it
makes no difference in practice as long as the sample is large.  M&M in
the
fifth edition (available for review in Feb) are making a distinction -
use
p-hat for large samples, only use Wilson for small ones (where the np >
10 and n(1-p) > 10 fail.)

From: 	David Moore <dsmoore@stat.purdue.edu>
To:	<isostat@oberlin.edu>
Date: 	12/6/04 11:27AM
Subject: 	[isostat] "plus four" intervals for proportions


Gang,

The traditional CIs for proportions can't be trusted for the sample
sizes typically found in research studies. Period.  It's much worse
than monotonic improvement as n increases.  There are "lucky" and
"unlucky" combinations of n and p.  Here, for example, is a
quote from Brown/Cai/DasGupta Statistical Science 16 (2001) p102:

	 For instance, when $n$ is 100, the actual coverage probability
	 of the nominal 95\% standard interval is 0.952 if $p$ is
	 0.106, but only 0.911 if $p$ is 0.107.  The behavior of the
	 coverage probability can be even more erratic as a function of
	 $n$.  If the true $p$ is 0.5, the actual coverage of the
	 nominal 95\% interval is 0.953 at the rather small sample size
	 n = 14, but falls to 0.919 at the much larger sample size n =
	 40.

The traditional intervals are OK for the sample sizes typically used
in opinion polls -- but, of course, opinion polls don't use these
intervals because they use complex sampling designs.

The "plus four" intervals due to Alan Agresti and his students (I like
that name, as it describes what is needed in both one- and two-sample
cases: add 4 observations) are simple and give a remarkable
improvement.

And be careful about saying that the traditional and plus four
intervals don't differ by much.  Our goal in using a 95\% CI is
to cover the true parameter 95\% of the time.  Consistently
shrinking away from 0 and 1 by a small amount (which is what
plus four does) can have a large effect on coverage probability
even though the intervals aren't far from the traditional CIs
for most samples.

As a final comment, students tend to see the need for the plus four
estimates when shown examples with count of successes = 0.  We don't
really think phat = 0 is a plausible estimate.  At the sample sizes
common in research studies, counts zero do occur.  I like the
drug-sniffing rats example in BPS.  Here are two more examples (new
exercise settings for BPS).  One is from a thesis, the second from a
2003 issue of Science.

	We don't like to find broken crackers when we open the
	package.  How can makers reduce breaking?  One idea is to
	microwave the crackers for 30 seconds right after baking them.
	Breaks start as hairline cracks called ``checking.''  Assign 65
	newly-baked crackers to the microwave and another 65 to a
	control group that is not microwaved.  After one day, none of
	the microwave group and 16 of the control group show checking.

	Genetic influences on cancer can be studied by manipulating the
	genetic makeup of mice.  One of the processes that turn genes
	on or off (so to speak) in particular locations is called ``DNA
	methylation.''  Do low levels of this process help cause
	tumors?  Compare mice altered to have low levels with normal
	mice.  Of 33 mice with lowered levels of DNA methylation, 23
	developed tumors.  None of the control group of 18 normal mice
	developed tumors in the same time period.

It's hard to break with tradition, but we can't ignore research
results
that are now well-established.  It's either teach plus four (or some
slightly more accurate but much more complicated method) or restrict
inference for proportions to large samples.

Cheers,
David

From: 	Dean Nelson <den+@pitt.edu>
To:	"Christopher J. Lacke" <lacke@ROWAN.EDU>,
<SIGMAASTATED@ENTERPRISE.MAA.ORG>
Date: 	12/6/04 11:34AM
Subject: 	Re: [STAT] Teaching a method that isn't used

Chris,

I just encountered the Wilson estimator in my class this semester since
I 
just started using
Moore and McCabe.  I didn't teach it.  It causes a problem not using
it, 
because that is what
is in the book.   I made handouts for that section.  The reasons I
don't 
use it are:

1) I read the original article in the American Statistician in which
the 
Wilson estimator was
investigated for sample sizes of 10, 20, and 30.  Since as the sample
size 
increases, the
difference between the Wilson estimator and the ordinary proportion 
estimate is negligible, larger
sample sizes were not investigated.  It is an adjustment for small
samples.
2) I am teaching an introductory statisitcs course.  There are a lot of

improved methods for doing
hypothesis testing for small samples, and I don't teach any of them. I

don't think small sample
estimation is a necessary topic in an introductory course. I will get
to 
those in later
courses.
 3) The motivation for the estimator is not accessible to intro
students. 
I like to make them take
things for granted as little as possible.
4) If I were going to teach an improved method for comparing
proportions 
for small sample sizes,
I would simply teach a non-parametric exact test.   Computation of the

exact test is
available for sample sizes this small, and even with larger samples,
Monte 
Carlo estimates of
the exact test are very good.  The exact test is more intuitively 
accessible to students at an
introductory level than the rationale for the Wilson estimator.


Dean

Dean Nelson, Ph.D.
Assistant Professor of Statistics
University of Pittsburgh at Greensburg
den@pitt.edu
(724) 838-8044

Chris,
I'm not a statistician or even a teacher, so I don't know anything
about the
Wilson Estimator. But I think in this case the pertinent question to
ask it
"will the students learn something useful from learning the concept?" 
If WE
is not used but it reinforces introduces something that makes a later
concept easier to understand, then it is valuable to teach it. If the
only
reason to teach it is "because I had to learn it in school" then there
is no
reason to teach it.

Just my two cents.

Doug Hall

From: 	"Wetzel, Nathan" <nwetzel@uwsp.edu>
To:	"Christopher J. Lacke" <lacke@rowan.edu>
Date: 	12/6/04 12:02PM
Subject: 	RE: [isostat] Teaching a method that isn't used

Chris

I strongly recommend reading "Approximate is Better than Exact for
Interval Estimation of Binomial Proportions" by Agresti and Coull in
American Statistician, May 1998, vol 52, No 2, pages 119 - 126

This article gives the specifics but it the gist is: 
1) The typical intro stat book 95% confidence interval (called the
Wald
interval) has a coverage probability that is almost always less than
95%
for various values of p and n. 
2) The Exact 95% confidence interval (Minitab14 gives these values by
default) has coverage probability that is almost always greater than
95%.  
3) The Wilson 95% confidence interval (called the Adjusted Wald in the
article) has a coverage probability that appears to average slightly
more than 95%.  

Also, note that the Wilson interval does NOT require np>=10 ...

Finally, since Minitab's default is already not the same as the Wald
interval, I think we need to address the issue in class and discuss
the
advantages / disadvantages of the different methods. 

Nate
Dr. Nathan Wetzel
Associate Professor
Dept of Mathematics and Computing
UW - Stevens Point
Stevens Point, WI 54481
nwetzel@uwsp.edu 


From: 	Mary Parker <mparker@austincc.edu>
To:	"Christopher J. Lacke" <lacke@rowan.edu>
Date: 	12/6/04 12:12PM
Subject: 	Re: [STAT] Teaching a method that isn't used

Chris,

About why it can't be used in the accompanying hypothesis test -- I 
haven't looked up the references, but I think that the assumption
listed 
for confidence intervals (90% or greater) tells us that this is a
better 
approximation to the actual "best estimator" than the usual p-hat in
the 
tails of the distribution, but not necessarily a better approximation
in 
the non-tail areas.  And there isn't any very clean and easy way to
deal 
with that in giving instructions for a hypothesis test.

Personally I like to give students a glimpse of some of the theory and

places where current research is going on, to open their minds to the 
idea that this is a living field and not just a collection of formulas.

  But of course, too much of that can be discouraging to students so it

does take balance.

Mary

From: 	"Johanna Hardin" <Jo.Hardin@pomona.edu>
To:	"Christopher J. Lacke" <lacke@rowan.edu>,
<SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu>
Date: 	12/6/04 1:28PM
Subject: 	RE: [isostat] Teaching a method that isn't used

I wanted to point out that Rossman & Chance also deal with the Wilson
estimator in their new book, "Investigating Statistical Concepts,
Applications, and Methods."  They also have a really nice applet that
shows the lack of correct coverage which makes teaching the concept
*much* easier.  (The students actually understand why you're giving
them
two different formulas.)

The applet is available even if you don't use the book:
http://www.rossmanchance.com/iscat/applets/Confsim/Confsim.html

(note that in the applet it is called the "adjusted wald" method.)

-Jo

From: 	"Lachenbruch, Peter" <lachenbruch@cber.FDA.gov>
To:	"'Christopher J. Lacke'" <lacke@rowan.edu>,
<SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu>
Date: 	12/6/04 1:56PM
Subject: 	RE: [isostat] Teaching a method that isn't used

There were a number of articles recently that have discussed "better"
estimates (in the sense of not bouncing around with a single increase
in n).
Alan Agresti had a couple (I think)  in American Statistician, and
Larry
Brown had an article in Annals of Statistics.  

I have taught these as well as the usual estimator (y/n).

Peter A. Lachenbruch
Director, Division of Biostatistics
FDA/CBER/OBE
1401 Rockville Pike, HFM-215
Rockville, MD 20852
Tel.  (301) 827-3320
FAX (301) 827-5218

From: 	Luis Bernall <bernall@fiu.edu>
To:	"Christopher J. Lacke" <lacke@ROWAN.EDU>
Date: 	12/6/04 2:00PM
Subject: 	Re: [STAT] Teaching a method that isn't used

    This is a kind of question I find difficult to answer; meaning that

I don't know what to do about this problem myself.  Therefore I'll give

you a view of what is going on around here.
    The textbook we use (McClave & Sincich: "Statistics", Prentice
Hall) 
includes this method in the last two editions (8th & 9th; that is,
since 
2000)  In the 9th edition it is on page 311. Incidentally this is a 
Department selected textbook and I don't like it (neither do the
students)
    BUT: No one here is teaching this paragraph, because they claim
that 
it will make the life of the poor students even more difficult.  And I
am 
strongly tempted to include this method in my classes.
    I wish you the best luck.

    L E Bernal
    Florida International University

From: 	"Martha K. Smith" <mks@math.utexas.edu>
To:	"Christopher J. Lacke" <lacke@ROWAN.EDU>,
<SIGMAASTATED@ENTERPRISE.MAA.ORG>
Date: 	12/6/04 2:33PM
Subject: 	Re: [STAT] Teaching a method that isn't used

I haven't taught a course using Moore and McCabe since the 4th 
edition came out, so I might not be best qualified to comment, but my 
initial reaction is: Just because a better technique isn't commonly 
used is no reason not to teach it! Statistics is a constantly 
developing field. If we can do something better easily, let's do it! 
Ideally, what we should be doing (and I realize we don't all have the 
time and other resources to do it -- but maybe some  on this list do 
have the time to do this a little) is contacting the instructors of 
the courses that require stat and saying, "Hey, do you know that this 
estimate works better in these circumstances? We teach it in our 
introductory course and hope you will use it."
Of course, part of whether or not an instructor teaches the technique 
depends on the purposes of the particular courses. Some classes may 
just be trying to give the ideas. However, those courses should be 
using a textbook that's more suitable for that purpose than Moore and 
McCabe. In courses where students will be using statistics in later 
courses, l say let's try to give them the best techniques we can, as 
well, of course, as try to have them understand the concepts of 
statistical procedures.
Martha Smith

From: 	"Lachenbruch, Peter" <lachenbruch@cber.FDA.gov>
To:	"isostat (E-mail)" <isostat@oberlin.edu>
Date: 	12/6/04 3:30PM
Subject: 	[isostat] More on Wilson

I was mulling over the idea of the Wilson interval and happened to
think
that if one used a Beta prior with parameters 2 and 2, the posterior
distribution would have a=2+y  b=2+n-y with mean (y+2)/(n+4)  which is
the
Wilson estimator.  I checked the 95% credible interval and the Wilson
interval.  With y=10 and n=20, the Bayesian interval is 0.0306 to 0.694
and
the Wilson interval is 0.299 to 0.701.  You can play with this to see
how
they work out for other y and n

Peter A. Lachenbruch
Director, Division of Biostatistics
FDA/CBER/OBE
1401 Rockville Pike, HFM-215
Rockville, MD 20852
Tel.  (301) 827-3320
FAX (301) 827-5218

From: 	"Thielman, Loretta" <ThielmanL@uwstout.edu>
To:	"Christopher J. Lacke" <lacke@ROWAN.EDU>
Date: 	12/6/04 3:44PM
Subject: 	RE: [STAT] Teaching a method that isn't used

Chris,

I think it is a good idea to use the Wilson estimator for estimation
tasks. I know it seems a little clumsy but I think it is better than
the
usual sample proportion. It isn't necessary to use it in the
hypothesis
testing scenarios since the null hypothesis value for p also
determines
the standard deviation for the sample proportion. I think college
students can handle complexity of two different estimates, at least I
hope so!

Loretta Thielman

From: 	Michael Cohen <michael.cohen@bts.gov>
To:	<isostat@oberlin.edu>, <dsmoore@stat.purdue.edu>
Date: 	12/6/04 4:28PM
Subject: 	Re: [isostat] "plus four" intervals for proportions

I would like to reinforce David Moore's comments.  First of all, Wilson
and similar "non-traditional" intervals ARE used in practice.  Second of
all, even for $n$ much larger than 100, we are often interested in rare
events (small $p$).



Michael P. Cohen
Assistant Director for Survey Programs
Bureau of Transportation Statistics
400 Seventh Street SW #4432
Washington DC 20590 USA
phone 202-366-9949 fax 202-366-3385

Christopher Jay Lacke, Ph.D.
Associate Professor - Mathematics Department
Rowan University
Glassboro, NJ  08028
(856) 256-4500 x3897  (office)
(856) 256-4816  (fax)
http://www.rowan.edu/mars/depts/math/lacke/lacke1.html
------------------------------------------------------------------------------
Some circumstantial evidence is very strong, 
as when you find a trout in the milk. H.D. Thoreau
-------------------------------------------------------------------------------