Subject: [isostat] Answers regarding the Wilson estimator |

From: "Christopher J. Lacke" |

Date: Mon, 06 Dec 2004 17:43:57 -0500 |

To: SIGMAASTATED@ENTERPRISE.MAA.ORG, isostat@oberlin.edu, lacke@rowan.edu |

Hi all: First, I'd like to thank those of you who took the time to respond. Second, before I post the responses, I'd like to clarify a couple of points from my previous posting. 1. I teach the Wilson Estimator. It was amazing to see how many people inferred that I don't teach it from the tone of my question. As a matter of fact, I have omitted the response (private, thank you) from one person who lambasted my teaching skills because he/she thought that I don't use it. 2. My statement about the lack of use came from a survey of business, biology, survey methodology, and psychology books that are being used at Rowan this semester. That said, maybe I should have reworded my question to ask "What good is teaching a new method to students when our colleagues in other fields are unaware of it? How can we get them to change their pedagogy to match ours?" I have searched the method and I have found it to be quite popular in practice, but not in the small sample of pedagogy in the disciplines that use statistics. 3. I also asked the question because it relates to the question of teaching the z-interval and z-test for a population mean. Some people claim that since it's not very realistic in practice, we should just teach the t procedures and approach the z-procedures through a proportion. (I know that is a loaded question!) Anyway, thanks for the dialectic and have a good evening. Chris From: "Patricia Humphrey" <phumphre@GeorgiaSouthern.edu> To: "Christopher J. Lacke" <lacke@rowan.edu>, <SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu> Date: 12/6/04 10:58AM Subject: Re: [isostat] Teaching a method that isn't used FYI, Chris (and all) - although D and V do mention the Wilson estimator, it makes no difference in practice as long as the sample is large. M&M in the fifth edition (available for review in Feb) are making a distinction - use p-hat for large samples, only use Wilson for small ones (where the np > 10 and n(1-p) > 10 fail.) From: David Moore <dsmoore@stat.purdue.edu> To: <isostat@oberlin.edu> Date: 12/6/04 11:27AM Subject: [isostat] "plus four" intervals for proportions Gang, The traditional CIs for proportions can't be trusted for the sample sizes typically found in research studies. Period. It's much worse than monotonic improvement as n increases. There are "lucky" and "unlucky" combinations of n and p. Here, for example, is a quote from Brown/Cai/DasGupta Statistical Science 16 (2001) p102: For instance, when $n$ is 100, the actual coverage probability of the nominal 95\% standard interval is 0.952 if $p$ is 0.106, but only 0.911 if $p$ is 0.107. The behavior of the coverage probability can be even more erratic as a function of $n$. If the true $p$ is 0.5, the actual coverage of the nominal 95\% interval is 0.953 at the rather small sample size n = 14, but falls to 0.919 at the much larger sample size n = 40. The traditional intervals are OK for the sample sizes typically used in opinion polls -- but, of course, opinion polls don't use these intervals because they use complex sampling designs. The "plus four" intervals due to Alan Agresti and his students (I like that name, as it describes what is needed in both one- and two-sample cases: add 4 observations) are simple and give a remarkable improvement. And be careful about saying that the traditional and plus four intervals don't differ by much. Our goal in using a 95\% CI is to cover the true parameter 95\% of the time. Consistently shrinking away from 0 and 1 by a small amount (which is what plus four does) can have a large effect on coverage probability even though the intervals aren't far from the traditional CIs for most samples. As a final comment, students tend to see the need for the plus four estimates when shown examples with count of successes = 0. We don't really think phat = 0 is a plausible estimate. At the sample sizes common in research studies, counts zero do occur. I like the drug-sniffing rats example in BPS. Here are two more examples (new exercise settings for BPS). One is from a thesis, the second from a 2003 issue of Science. We don't like to find broken crackers when we open the package. How can makers reduce breaking? One idea is to microwave the crackers for 30 seconds right after baking them. Breaks start as hairline cracks called ``checking.'' Assign 65 newly-baked crackers to the microwave and another 65 to a control group that is not microwaved. After one day, none of the microwave group and 16 of the control group show checking. Genetic influences on cancer can be studied by manipulating the genetic makeup of mice. One of the processes that turn genes on or off (so to speak) in particular locations is called ``DNA methylation.'' Do low levels of this process help cause tumors? Compare mice altered to have low levels with normal mice. Of 33 mice with lowered levels of DNA methylation, 23 developed tumors. None of the control group of 18 normal mice developed tumors in the same time period. It's hard to break with tradition, but we can't ignore research results that are now well-established. It's either teach plus four (or some slightly more accurate but much more complicated method) or restrict inference for proportions to large samples. Cheers, David From: Dean Nelson <den+@pitt.edu> To: "Christopher J. Lacke" <lacke@ROWAN.EDU>, <SIGMAASTATED@ENTERPRISE.MAA.ORG> Date: 12/6/04 11:34AM Subject: Re: [STAT] Teaching a method that isn't used Chris, I just encountered the Wilson estimator in my class this semester since I just started using Moore and McCabe. I didn't teach it. It causes a problem not using it, because that is what is in the book. I made handouts for that section. The reasons I don't use it are: 1) I read the original article in the American Statistician in which the Wilson estimator was investigated for sample sizes of 10, 20, and 30. Since as the sample size increases, the difference between the Wilson estimator and the ordinary proportion estimate is negligible, larger sample sizes were not investigated. It is an adjustment for small samples. 2) I am teaching an introductory statisitcs course. There are a lot of improved methods for doing hypothesis testing for small samples, and I don't teach any of them. I don't think small sample estimation is a necessary topic in an introductory course. I will get to those in later courses. 3) The motivation for the estimator is not accessible to intro students. I like to make them take things for granted as little as possible. 4) If I were going to teach an improved method for comparing proportions for small sample sizes, I would simply teach a non-parametric exact test. Computation of the exact test is available for sample sizes this small, and even with larger samples, Monte Carlo estimates of the exact test are very good. The exact test is more intuitively accessible to students at an introductory level than the rationale for the Wilson estimator. Dean Dean Nelson, Ph.D. Assistant Professor of Statistics University of Pittsburgh at Greensburg den@pitt.edu (724) 838-8044 Chris, I'm not a statistician or even a teacher, so I don't know anything about the Wilson Estimator. But I think in this case the pertinent question to ask it "will the students learn something useful from learning the concept?" If WE is not used but it reinforces introduces something that makes a later concept easier to understand, then it is valuable to teach it. If the only reason to teach it is "because I had to learn it in school" then there is no reason to teach it. Just my two cents. Doug Hall From: "Wetzel, Nathan" <nwetzel@uwsp.edu> To: "Christopher J. Lacke" <lacke@rowan.edu> Date: 12/6/04 12:02PM Subject: RE: [isostat] Teaching a method that isn't used Chris I strongly recommend reading "Approximate is Better than Exact for Interval Estimation of Binomial Proportions" by Agresti and Coull in American Statistician, May 1998, vol 52, No 2, pages 119 - 126 This article gives the specifics but it the gist is: 1) The typical intro stat book 95% confidence interval (called the Wald interval) has a coverage probability that is almost always less than 95% for various values of p and n. 2) The Exact 95% confidence interval (Minitab14 gives these values by default) has coverage probability that is almost always greater than 95%. 3) The Wilson 95% confidence interval (called the Adjusted Wald in the article) has a coverage probability that appears to average slightly more than 95%. Also, note that the Wilson interval does NOT require np>=10 ... Finally, since Minitab's default is already not the same as the Wald interval, I think we need to address the issue in class and discuss the advantages / disadvantages of the different methods. Nate Dr. Nathan Wetzel Associate Professor Dept of Mathematics and Computing UW - Stevens Point Stevens Point, WI 54481 nwetzel@uwsp.edu From: Mary Parker <mparker@austincc.edu> To: "Christopher J. Lacke" <lacke@rowan.edu> Date: 12/6/04 12:12PM Subject: Re: [STAT] Teaching a method that isn't used Chris, About why it can't be used in the accompanying hypothesis test -- I haven't looked up the references, but I think that the assumption listed for confidence intervals (90% or greater) tells us that this is a better approximation to the actual "best estimator" than the usual p-hat in the tails of the distribution, but not necessarily a better approximation in the non-tail areas. And there isn't any very clean and easy way to deal with that in giving instructions for a hypothesis test. Personally I like to give students a glimpse of some of the theory and places where current research is going on, to open their minds to the idea that this is a living field and not just a collection of formulas. But of course, too much of that can be discouraging to students so it does take balance. Mary From: "Johanna Hardin" <Jo.Hardin@pomona.edu> To: "Christopher J. Lacke" <lacke@rowan.edu>, <SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu> Date: 12/6/04 1:28PM Subject: RE: [isostat] Teaching a method that isn't used I wanted to point out that Rossman & Chance also deal with the Wilson estimator in their new book, "Investigating Statistical Concepts, Applications, and Methods." They also have a really nice applet that shows the lack of correct coverage which makes teaching the concept*much*easier. (The students actually understand why you're giving them two different formulas.) The applet is available even if you don't use the book: http://www.rossmanchance.com/iscat/applets/Confsim/Confsim.html (note that in the applet it is called the "adjusted wald" method.) -Jo From: "Lachenbruch, Peter" <lachenbruch@cber.FDA.gov> To: "'Christopher J. Lacke'" <lacke@rowan.edu>, <SIGMAASTATED@ENTERPRISE.MAA.ORG>, <isostat@oberlin.edu> Date: 12/6/04 1:56PM Subject: RE: [isostat] Teaching a method that isn't used There were a number of articles recently that have discussed "better" estimates (in the sense of not bouncing around with a single increase in n). Alan Agresti had a couple (I think) in American Statistician, and Larry Brown had an article in Annals of Statistics. I have taught these as well as the usual estimator (y/n). Peter A. Lachenbruch Director, Division of Biostatistics FDA/CBER/OBE 1401 Rockville Pike, HFM-215 Rockville, MD 20852 Tel. (301) 827-3320 FAX (301) 827-5218 From: Luis Bernall <bernall@fiu.edu> To: "Christopher J. Lacke" <lacke@ROWAN.EDU> Date: 12/6/04 2:00PM Subject: Re: [STAT] Teaching a method that isn't used This is a kind of question I find difficult to answer; meaning that I don't know what to do about this problem myself. Therefore I'll give you a view of what is going on around here. The textbook we use (McClave & Sincich: "Statistics", Prentice Hall) includes this method in the last two editions (8th & 9th; that is, since 2000) In the 9th edition it is on page 311. Incidentally this is a Department selected textbook and I don't like it (neither do the students) BUT: No one here is teaching this paragraph, because they claim that it will make the life of the poor students even more difficult. And I am strongly tempted to include this method in my classes. I wish you the best luck. L E Bernal Florida International University From: "Martha K. Smith" <mks@math.utexas.edu> To: "Christopher J. Lacke" <lacke@ROWAN.EDU>, <SIGMAASTATED@ENTERPRISE.MAA.ORG> Date: 12/6/04 2:33PM Subject: Re: [STAT] Teaching a method that isn't used I haven't taught a course using Moore and McCabe since the 4th edition came out, so I might not be best qualified to comment, but my initial reaction is: Just because a better technique isn't commonly used is no reason not to teach it! Statistics is a constantly developing field. If we can do something better easily, let's do it! Ideally, what we should be doing (and I realize we don't all have the time and other resources to do it -- but maybe some on this list do have the time to do this a little) is contacting the instructors of the courses that require stat and saying, "Hey, do you know that this estimate works better in these circumstances? We teach it in our introductory course and hope you will use it." Of course, part of whether or not an instructor teaches the technique depends on the purposes of the particular courses. Some classes may just be trying to give the ideas. However, those courses should be using a textbook that's more suitable for that purpose than Moore and McCabe. In courses where students will be using statistics in later courses, l say let's try to give them the best techniques we can, as well, of course, as try to have them understand the concepts of statistical procedures. Martha Smith From: "Lachenbruch, Peter" <lachenbruch@cber.FDA.gov> To: "isostat (E-mail)" <isostat@oberlin.edu> Date: 12/6/04 3:30PM Subject: [isostat] More on Wilson I was mulling over the idea of the Wilson interval and happened to think that if one used a Beta prior with parameters 2 and 2, the posterior distribution would have a=2+y b=2+n-y with mean (y+2)/(n+4) which is the Wilson estimator. I checked the 95% credible interval and the Wilson interval. With y=10 and n=20, the Bayesian interval is 0.0306 to 0.694 and the Wilson interval is 0.299 to 0.701. You can play with this to see how they work out for other y and n Peter A. Lachenbruch Director, Division of Biostatistics FDA/CBER/OBE 1401 Rockville Pike, HFM-215 Rockville, MD 20852 Tel. (301) 827-3320 FAX (301) 827-5218 From: "Thielman, Loretta" <ThielmanL@uwstout.edu> To: "Christopher J. Lacke" <lacke@ROWAN.EDU> Date: 12/6/04 3:44PM Subject: RE: [STAT] Teaching a method that isn't used Chris, I think it is a good idea to use the Wilson estimator for estimation tasks. I know it seems a little clumsy but I think it is better than the usual sample proportion. It isn't necessary to use it in the hypothesis testing scenarios since the null hypothesis value for p also determines the standard deviation for the sample proportion. I think college students can handle complexity of two different estimates, at least I hope so! Loretta Thielman From: Michael Cohen <michael.cohen@bts.gov> To: <isostat@oberlin.edu>, <dsmoore@stat.purdue.edu> Date: 12/6/04 4:28PM Subject: Re: [isostat] "plus four" intervals for proportions I would like to reinforce David Moore's comments. First of all, Wilson and similar "non-traditional" intervals ARE used in practice. Second of all, even for $n$ much larger than 100, we are often interested in rare events (small $p$). Michael P. Cohen Assistant Director for Survey Programs Bureau of Transportation Statistics 400 Seventh Street SW #4432 Washington DC 20590 USA phone 202-366-9949 fax 202-366-3385 Christopher Jay Lacke, Ph.D. Associate Professor - Mathematics Department Rowan University Glassboro, NJ 08028 (856) 256-4500 x3897 (office) (856) 256-4816 (fax) http://www.rowan.edu/mars/depts/math/lacke/lacke1.html ------------------------------------------------------------------------------ Some circumstantial evidence is very strong, as when you find a trout in the milk. H.D. Thoreau -------------------------------------------------------------------------------