A Model of Classroom Research in Action: Developing Simulation Activities to Improve Students' Statistical Reasoning

Robert C. delMas and Joan Garfield
University of Minnesota

Beth L. Chance
University of the Pacific

Journal of Statistics Education v.7, n.3 (1999)

Copyright (c) 1999 by Robert C. delMas, Joan Garfield, and Beth L. Chance, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

Appendix B: Classification of Student Responses Into Four Reasoning Types

The two tables in Appendix B (Tables B.1 and B.2) present pretest and posttest results, respectively, for Problem 4 (see Figure 4 or Appendix A). The five most common reasoning types (or patterns) are indicated at the top of each column. The second line of each column heading indicates the graph choices that correspond to each reasoning type. For example, a student was placed in the "good" reasoning category in the third column if she chose either graph E or graph B for Part A and then chose graph C for Part B of Problem 4 (refer to Figure 4 or Appendix A to see the graphs). The label "Lg - Sm" refers to the larger to smaller variance reasoning type, while the "Sm - Lg" label refers to the smaller to larger variance reasoning type. As described earlier, there are two subcategories for larger to smaller reasoning.

The number of students who exhibited each pattern is given at the top of each column. The column counts do not add up to the total sample size because some students did not fall into one of the five most common categories (twelve students on the pretest and eight on the posttest). The values in each cell of the tables are the percentages of students who chose each reason relative to the total number of students who exhibited a reasoning type. Note, also, that the percentages in the columns can add to more than 100 because students were encouraged to provide multiple reasons for each sampling distribution choice.

The reasoning types match well with students' indicated reasons. Students who made the correct choice for Problem 4 were expected to select reasons 1 and 3 for Part A and reasons 1, 3, 8, 10, and 11 for Part B. With the exception of reason 10 on the pretest, two-thirds or more of the students who made the correct choice indicated the expected set of explanations, while the percentages of these students who selected other reasons were quite low. The tables also illustrate that the reasoning types do appear to indicate true differences in students' thinking. As an example, the distribution of percentages across the 14 different reasons is quite similar for the correct and good reasoning categories with one noticeable difference. All students who made a correct pair of choices selected reason 1 for Part A on both the pretest and posttest. Almost none of the students (7.7% on the pretest and 0% on the posttest) who chose a good pair of graphs indicated this reason for Part A. The sample size for Part A of Problem 4 was n = 4. Good reasoning on Part A, therefore, was exhibited by students who thought the sampling distribution for this small sample size would still have a shape similar to the population but with less variability (e.g., graph E). As expected, students who made a good choice tended to select reason 2 as a justification for their choice in Part A (61.5% on the pretest and 89.3% on the posttest). Reason 2 was rarely selected by students who chose the correct pair of graphs (0% and 11.1% on the pretest and posttest, respectively).

We also thought there might be two distinct types of larger to smaller reasoning among the students as illustrated by the two columns for this reasoning type in the tables. In the first subtype, students chose a histogram with variability similar to that of the population even though the sample size was greater than 1 (e.g., graph A for n = 4), but chose a graph with a normal distribution for Part B (e.g., graph C for n = 25). In the second subtype, students chose a graph that looked like the population for both sample sizes, although the histogram for the larger sample size had less variability. The percentage of students who chose each reason is similar for the two subtypes of larger to smaller reasoning, with the exception of reasons 1 (I expect the sampling distribution to be shaped like a NORMAL DISTRIBUTION) and 2 (I expect the sampling distribution to be shaped like the POPULATION) on Part B. Within the first subtype, almost all of the students selected reason 1 with few selecting reason 2 for Part B, while the opposite trend was evident in the second subtype. These observations hold for both the pretest and the posttest results. The pattern of percentages across the 14 reasons for these two subtypes of larger to smaller reasoning are also similar to the good reasoning patterns. They are distinguished from the good reasoning type by the higher percentage of students who selected reason 4 (I expect the sampling distribution to have MORE VARIABILITY than the POPULATION) for Part A on the posttest. This may indicate that students exhibiting larger to smaller reasoning misunderstand what is meant by variability in a distribution.

The last reasoning type, smaller to larger variability, represents a choice of graphs that clearly contradicts the Central Limit Theorem. This reasoning type was exhibited by a majority of students on the pretest, while its frequency declined considerably on the posttest (from 53% to 20%). Selection of reason 9 on Part B appears to distinguish this reasoning type from the others. Reason 9 states that the sampling distribution for the larger sample size will look more like the population than the sampling distribution for the smaller sample size. Less than half of these students chose reason 7, that the second sampling distribution would have more variability than the first, for Part B. This reasoning type may not reflect a belief that larger sample sizes produce sampling distributions with greater variability as much as an expectation that the larger the sample, the greater the similarity between the population and the sampling distribution. Most of these students may not be attending to the variability of the sample means, or may not have a good understanding of variability. Another possibility is that these students had similar expectations for the distribution of a sample drawn from a population and a distribution of the sample means. Students learn that, given two random samples, a larger sample will tend to be more representative of the population than a smaller one. If this expectation is transferred to distributions of sample means, the implication is that larger samples should produce sampling distributions that look more like the population.

There is one puzzling aspect of the results that relates to the last point. A considerable number of students selected reason 4 (the sampling distribution has more variability than the population) when the graph they chose arguably indicated either less or about the same amount of variability. This is surprising given that these students had completed more than half of the introductory course and should have been familiar with the concept of variability. While the assessment items do not offer information that can be used to understand what these students mean by variability, we plan to interview students as they work with the Sampling Distributions program and the problems in order to better understand their interpretations of terms such as "variability."

Table B.1. Percent of Winter 1997 Students (N = 89) Who Indicated Each Reason for a Given Response Pattern on Problem 4 on the Pretest

Table B.2. Percent of Winter 1997 Students (N = 89) Who Indicated Each Reason for a Given Response Pattern on Problem 4 on the Posttest

JSE Homepage | Subscription Information | Current Issue | JSE Archive (1993-1998) | Data Archive | Index | Search JSE | JSE Information Service | Editorial Board | Information for Authors | Contact JSE | ASA Publications