Statistical Consulting Courses for Undergraduates: Fortune or Folly?

KB Boomer, Neal Rogness, and Brian Jersky
/Bucknell University, Grand Valley State University, St. Mary's College of California

Journal of Statistics Education Volume 15, Number 3 (2007), www.amstat.org/publications/jse/v15n3/boomer.html

Copyright © 2007 by KB Boomer, Neal Rogness, and Brian Jersky all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the author and advance notification of the editor.


Key Words: undergraduate consulting; data-based applied courses; course development; pitfalls; minimum skills

Abstract

This article presents an overview of three undergraduate-level statistical consulting courses being taught at institutions of different size (small, medium, and large). Topics that will be discussed include the evolution of these courses, thoughts on what makes such courses successful, potential pitfalls to watch for, the necessary minimal skills students should have to be successful in the courses, and thoughts on where these courses should appear in a statistics curriculum. This paper will provide an overview of the similarities and differences in the way applied consulting courses are presented within the three undergraduate programs.

1. Introduction

Applied statistical consulting courses are found in many graduate programs in the U.S.  In contrast, relatively few programs include such courses at the undergraduate level. Three universities that successfully offer undergraduate statistical consulting courses include Pennsylvania State University (Pennsylvania), Grand Valley State University (Michigan), and Sonoma State University (California). These three universities represent a large, medium, and small campus, respectively (see Appendix A for a summary of key information about these three universities).

It is reasonable to question the wisdom of involving undergraduate statistics students in consulting opportunities.  As this paper will detail, these experiences can certainly pose challenges. However, they also provide students with opportunities to work with real data resulting from on-going scientific inquiry, use statistical software to answer client specific questions, and to learn how to communicate with non-statisticians through verbal exchanges and written work.  Based upon the collective experiences, the authors strongly believe that these consulting opportunities provide a rich source of learning for undergraduate students and that the pros far outweigh the cons. 

A statistical consulting course provides a way in which to partially meet the ASA Curriculum Guidelines for Undergraduate Programs in Statistical Science (ASA 2004). Specifically, this Guideline includes that, “Graduate should be expected to write clearly, to speak fluently, and to have developed skills in collaboration and teamwork and in organizing and managing projects.”  Further, the Guideline notes that, “Academic programs often fail to offer adequate preparation in these areas.”  As a practice, the Guideline recommends that, “When possible, the undergraduate experience should include an internship, a senior-level ‘capstone’ course, a consulting experience of some kind, or combination of these.”  In addition, a growing number of universities are requiring students to participate in a service component.  Statistical consulting projects with non-profit agencies can offer wonderful service experiences that are mutually beneficial for the students and the agencies.

2. Evolution of the Consulting Courses

The goal of an undergraduate applied consulting course can seem ambitious: the authors aim to provide undergraduate students with real data-based applied consulting experiences, while discussing the finer points of statistical consulting within the context of a writing-intensive class. The target audience varies across our three universities, ranging from statistics majors to statistically interested general students who have completed at least one introductory statistics course; for statistics majors, the course is required at all three universities. Non-statistics majors often come from businesses, economics, psychology and various other areas.

As shown in Appendix A, SSU has approximately 20 undergraduate statistics majors whereas PSU and GVSU have almost 60. If desired, such a consulting course could be offered by a mathematics department offering an emphasis in statistics, as is done at SSU. It is worth noting that, at GVSU, the undergraduate consulting course predates the organization of its statistics department and the consulting course predates the creation of the statistics major itself. The number of students in the consulting class varies from three to 18, depending on the year and the university. The undergraduate-level consulting courses represent relatively new additions to the curriculum (GVSU: 1989, PSU: 1997, SSU: 1994), mirroring the rise in the number of graduate-level consulting courses at U.S. universities. The undergraduate consulting course has been taught 16 times at SSU, 18 times at GVSU, and 11 times at PSU. PSU and SSU offer the course once annually. While most often taught as a winter semester course at GVSU, the course is now being offered both fall and winter semesters effective fall 2006. 

The three courses described here are innovative in that they bring the applied consulting course experience of graduate programs to the undergraduate student. At each university, students actively interact with clients. Depending on the university, client projects include long-term and short-term consulting services. At SSU, the consulting course has been offered in two varieties in different years. The first variety is to offer the students a semester-long project, which they work on from the beginning of the semester until the end, and occasionally beyond. The second variety is to line up clients who need short-term statistical advice on a specific aspect of a problem. At both GVSU and PSU, the students work on one long-term consulting project from the beginning of the semester until the end.

Each author has been actively involved in running a statistical consulting center at one of the universities discussed herein.  These centers offer free consulting to students and faculty, and can help form a pool of clients for the undergraduate classes. At both PSU and SSU, approximately half of the clients in the consulting courses are students and half are faculty/staff.  Clients in the course at GVSU have been primarily faculty/staff. Certainly, a consulting center allows for students in the consulting course to have richer experiences. For instance, at GVSU, students in the consulting course are required to observe and comment upon two consulting sessions, which gives them exposure to additional clients and consulting projects. However, it is possible to offer such a course independent of a consulting center. In fact, the consulting course at GVSU was offered 12 times prior to the formation of the consulting center.  Projects can be solicited from colleagues or from the community as service-learning experiences.  An example of a flyer used at GVSU to solicit projects from across the university can be found in Appendix B

3. Prerequisites and Course Content

The prerequisites for these three consulting courses generally include an intermediate-level linear models class. For the consulting course at SSU, only one statistics course is required, and it can be taken as a general education elementary statistics course. The more advanced student will have taken statistical inference and probability theory. The GVSU course has Introductory Applied Statistics or Probability and Statistics and Intermediate Applied Statistics as prerequisites. Introductory Applied Statistics uses the statistical package SPSS and Intermediate Applied Statistics is where students learn more advanced techniques and are first introduced to SAS programming. The faculty at GVSU is currently in discussion about increasing the prerequisites for the consulting course. The prerequisites for the PSU course are Intermediate Applied Statistics, Applied Regression, and Introduction to Statistical Program Packages, where SAS is introduced.  The students will have used Minitab in previous applied classes.

Both PSU and GVSU have writing intensive general education requirements. At PSU, undergraduates must take one writing-intensive course and the consulting course fulfills this requirement. At GVSU, students must take two Supplemental Writing Skills courses, and the consulting course can fulfill one of these. Communication is vital within consulting, which includes verbal, written, and non-verbal communications. In consulting classes, students need to be able to clearly define the statistical questions and discuss the methods to be used for analysis in non-technical terms. Thus, writing a consulting report requires the student to digest the statistical terminology presented in the classroom into words of their own (Radke-Sharpe, 1991). This is a great learning opportunity for students serving as active statisticians to practice such crucial skills. While there is a strong emphasis on writing, each of the courses is first and foremost a statistics course.

Although developed independently, the course content for the three courses is similar.  Each course begins by introducing the process of consulting and honing such skills as listening, questioning, and reflecting. Topics include client expectations, client meetings, and formulating the client question into statistical terms. The authors have found it useful to devote time discussing the kinds of questions to ask clients often using previous client files or guest clients for these purposes. This background in non-technical aspects gives the students critical skills that will be used throughout the semester and beyond. At GVSU, students read and discuss a selection of articles on statistical consulting early in the semester before interacting with their clients (Derr and Stinnett, 1994; Kirk, 1991; Zahn and Boroto, 1983).

The middle third of the semester focuses more on technical skills including a review of SPSS and SAS and identification of appropriate techniques to apply. Classes at this point are driven by the needs of the current projects with students asking for lectures on methods they need to analyze their client’s data; hence, it becomes student-motivated learning at the undergraduate level. During this time, an entire class period may be spent trouble-shooting one or more projects.

The last third of the semester tends to be devoted to presentation of the results. For each class, a thorough consulting report is required. Drafts are submitted and graded/commented upon throughout the semester, with the final report due at the end of the semester. When students work in groups, the authors recommend having each student do an assessment of each member’s involvement in the client’s project. At PSU and SSU, students give in-class oral presentations on their projects to which clients can be invited. At GVSU, each student is required to do a presentation (either oral or poster) about his or her consulting experience at the University’s annual Student Scholarship Day or at a Statistics Department seminar. 

Students prepare writing assignments based on a variety of readings. Electronic databases such as JSTOR provide a wealth of related articles while introducing research skills. Textbooks that have been used in these courses include Andrews et al. (1998), Chatfield (1995), Cody (1999), Derr (2000), and Lang et al. (2006).

Class assignments are typically directly related to the consulting projects. The final recommendation report is built upon earlier assignments. For example, one assignment at PSU due at the eighth week of term is to complete an exploratory data analysis involving at least two dependent and four independent variables from the consulting projects. The inference report (due at 12 weeks) is then appended to the previous EDA report, with instructor modifications incorporated. A similar plan is used at GVSU with the first draft focusing on summarizing the goals and methodology of the study; drafts two and three then focus on EDA and on inferential results with conclusions, respectively. Other assignments used at the three universities include maintaining a research journal, writing meetings summaries, interviewing practicing statistical consultants, and designing their own consulting offices.

The semester grades for the consulting courses are slightly higher than for other statistics classes.  The reasons are many. The authors work very closely with the students, guiding them along the way. Journals and class discussions help guarantee that the student is progressing in the correct direction. Further, multiple revisions on the cumulative final report give the students ample guidance as to what is expected from them. Hence, it is not uncommon to see more A/A- grades than lower grades, although C grades and lower have been assigned.

4. What Makes a Course Successful

A successful course requires many components to come together. While this is true for any course, there are some unique requirements for consulting classes.

Early on, the professor must decide on a format for the course, whether students will work in groups, pairs, or individually and whether to focus on short-term or long-term consulting projects.

At SSU students are typically divided into groups of three or four students to make three groups. The students are grouped by level of expertise, so that beginning students work with other beginners on less advanced topics, and advanced students work on advanced topics. This approach is adopted due to the heterogeneity in the class group and allows one client project to be split into appropriate groups. At GVSU, students have the benefit of group and individual work. When students are paired, the goal has been to pair a more advanced student with a less advanced student. At PSU, students work individually with clients. The objective of individual work at PSU is to foster independent thinking and responsibility for one’s work. A downfall of working with only one or a few clients is that the student ends up gaining limited experience (i.e., only applying one or two statistical techniques to real-world data). However, by spending class time trouble-shooting a project, the rest of the class serve as “mini consultants” to that project’s consultant. 

Once the format has been decided, appropriate consulting opportunities must then be found. Based upon the experience of the authors, it is recommended that considerable time be devoted before the semester begins conversing with a potential client so as to gain a clear understanding of the project prior to accepting it. Although identifying suitable projects is time consuming and increases the amount of work the professor must do before the start of the semester, as noted by Singer et al. (1990), the payoff is well worth the effort. When selecting projects, a useful guideline is that for an undergraduate consulting project, the professor should have a clear idea of appropriate analyses from a client’s abstract. This means that the professor should have a clear idea of what the client requires, whether the data are in sufficient order to allow that to be done by relatively unsophisticated students, and what the students will need to review or understand in order to produce the graphs, tables, analyses and reports that the client expects. On the other hand, for a graduate consulting project perhaps only a sense of the type of analysis required is needed, since students can be trusted to carry out that analysis accurately and completely. This information also aids in reasonably matching student to projects. Apart from the fact that students will work hard on projects that interest them, students will have difficulty with projects where clients are not clear in their expectations. It is important that clients and students understand that the process of consulting in this environment is primarily educational in purpose. See Appendix C for the project information sheet used at GVSU.  Some factors to consider in deciding if a project is acceptable include the project timeline (e.g., Is the client willing to receive the final report at the end of the semester? Will the data be provided in time for data cleaning and analyses to be comfortably done?), the willingness of the client to use this as an educational experience for the student, and the anticipated analyses (e.g., Will the methods be accessible for the students?).

Next, the knowledge of the students must be accurately judged as quickly as possible. The authors recommend meeting personally with each student during office hours or collecting extensive information from each student regarding prior coursework.   Particularly, in the situation in which there may be only one prerequisite, such as with the SSU course, it is vitally important to tailor the projects to the students. With the more advanced students, it is also useful to identify the weaker versus stronger students. This avoids overwhelming some while challenging all. In addition to student backgrounds, other useful factors to consider when assigning a project are the availability for students to physically meet with clients and the student’s interest in each project. The latter can be accomplished by having students indicate their top five choices when given brief overviews of each project. It is necessary to emphasize explicitly to students that their responsibility to complete a project is somewhat higher than their responsibility to complete homework in a regular class, say, since it affects themselves, the client, the instructor, and the school.

Throughout the semester, class time needs to be carefully monitored, so that each group is offered roughly equivalent time. If it is easy for students to see that their professor is more interested in a complicated time series analysis problem than a data presentation problem, those students involved in the ‘easier’ work will stop taking an interest. In each course, the authors believe that student dialog is crucial; the courses are somewhat loosely structured to encourage discussions related to client projects. The most valuable lectures seem to be those that stem from an answer to the question, “Does anyone have a question about his or her consulting project you would like to ask?” Before this can occur, however, the professor needs to spend time making sure the students perceive the class as a truly safe environment in which to ask questions.

In some class periods the entire time is spent trouble-shooting projects. The students are encouraged to address their requests for help to the entire class. While the professor may facilitate the discussion, the objective is to involve all of the students in the dialog. Most of the time, the authors have no trouble eliciting good advice from class members. However, there is absolutely nothing worse than having a non-responsive class; this is compounded when it is a consulting class in which the students are intended to be active participants.

The experiences of the authors have shown that each student tends to think that he or she has the worst project in class.  When students hear about the projects of other students and the difficulties these other students have encountered, it helps each student to “normalize” his or her own experience. Interestingly, this sharing of perceived problems makes it easier for students to make progress on their projects. Like all beginning consultants, students are often unable to start or make progress on their projects until the details are all worked out. This makes it easy to never start, since details are almost never all worked out!

This course format requires that the professor must be able to deal with a wide variety of issues in statistics, often on the fly. The norm of more traditional classes, in which expectations for material to be covered are set a priori, often do not apply to consulting courses. While the professor may plan for a particular topic, he or she should also be flexible and willing to spend time on issues and methods that arise from students’ consulting projects. Many students have had the experience in other classes of seeing everything fitted into, say, an ANOVA framework, and they need to see that the problem determines the solution, not the other way round.   

The authors are divided on whether to recommend lecturing to the classroom on a topic useful to only a few students. One view is that this introduces students to new areas such as logistic regression and ANCOVA so that they can understand the work of their peers and will retain a general overview of the methods. This approach may be more suitable when the class consists of advanced statistics students. The other view is that non-relevant or unnecessary topics and advanced methods may intimidate the students, negatively affecting student motivation. However, whether in lecture format or discussed in individual meetings, the identification of topics by students makes them take responsibility for their learning, an essential skill for future researchers.

Further, it is recommended that each group’s work be monitored weekly; in addition, each student’s work within a group must be monitored, to avoid the “free rider” problem. Students (and faculty) often procrastinate, and this must be discouraged.  To avoid the procrastination problem, it is important to have students regularly report on the status of the projects.  One means of doing this is to have the students maintain and submit weekly graded journals. The journals are used as a record of any discussions students have had with their client, peers, and/or us regarding the project (including any agreed upon work) as well as what has been accomplished. The students can be asked to respond to regular questions such as “Discuss with appropriate detail what activity you’ve done this week on your faculty/staff project”, “What challenges are you currently facing in your project?”, and “Provide a summary of your results and outline what analyses remain.” Journals allow the professor to keep informed, to become aware of problems in a timely fashion, and to intervene when needed. It is also recommend that professors contact each client at least once midway through the semester. This is useful to identify concerns or problems while there is still time to intervene.

The successful consulting course also has motivated students. In many cases, these consulting courses will be the first time students have worked with real clients (and data) outside the safe context of textbooks with neatly divided topics. Some students will come to class excited to rise to the challenge of working in this consulting setting. Others will be more hesitant, often lacking confidence in their own statistical and reasoning abilities (Spurrier, 2001). Confidence however can be nurtured through the semester in the ways that the course professor interacts with the students via written and verbal feedback.  As opposed to a traditional course in which the textbook problem has a correct answer and students each complete the same assignment, in a consulting course, different projects bring in different statistical components.  Equivalently, students each bring different strengths and weakness to the class. 

Professors may take the roles of teacher, consultant, and cheerleader in a consulting course. In grading papers, it is important to provide positive feedback and shift from solely identifying spelling and grammatical errors. It has been said that writing is an act of confidence and consulting courses are often a student’s first introduction to technical writing. A good reference is Daiker (1989). 

Encouragement is also needed as frustration arises in the data analysis phase, when models do not fit and SAS code does not work. This is the opportunity to remind the students of how much they have learned, not of how many failures they have had. This in turn raises their confidence. Students new to consulting often see the task as formidable and they can easily become overwhelmed. Confidence can also be built by having students work on small pieces at a time (e.g., weekly mini goals) which accumulate throughout the semester until a satisfying outcome is achieved. 

As another way of motivating, one author has his students review papers that were written by former students.  The former students were instructed, “To write a paper describing your major consulting experience (faculty project) this semester. You should first briefly describe what the project attempted to accomplish. Then address the following points: (1) what you learned regarding statistical consulting, (2) an overview of your experiences including what went well and what might have gone better, and (3) your suggestions for how best to proceed for someone who will just be beginning a similar consulting experience.” The current students are instructed to read these papers and to summarize what the former students collectively learned about statistical consulting from their experiences and the recommendations they made on how to proceed once a consulting project is assigned. The students seem to “take to heart” the words of wisdom provided by their former peers.

The authors recommend, when schedules allow, for the professor to attend the first or second meeting between the student and the client. This allows the professor to gain a better understanding of the project for when questions later arise.  Still, it is important for the student consultant to be responsible for running for session, with the professor injecting comments and questions were appropriate. If this does not happen, the students will defer to the professor and not take responsibility for their projects.

5. Pitfalls

One should take care to not accept projects that are too complicated for undergraduates. This requires considerable project screening on the front end by the professor. Appropriate projects are crucial to the success of the course, and these can be difficult to find.  It is easy to find projects that are too difficult. It is also surprisingly easy to find projects that are too easy, and provide no challenge or excitement. It seems that it is to some extent a matter of luck and to a great extent a matter of advance preparation and screening. Assigning students to a project that is too challenging will only frustrate all parties involved. It is better to abandon a project that is not appropriate than to keep going on it, only to discourage the students even further. If a project does not work out, the displaced student can be assigned to assist on another project, by perhaps expanding the original focus so the student can perform some independent analyses. It is also recommended to avoid projects that yet require considerable data entry. Despite a client’s best intentions, if data entry does not occur in a timely fashion, the time frame for completion of the project greatly diminishes.

From a customer management point of view, the statistical clients should be forewarned that problems may arise, and that they should build this in to their expectations. The authors suggest using a consulting agreement form that identifies tasks to be completed, start and end dates, and specifies client obligations, such as being available to answer consultant questions and providing information on the data necessary for proper analysis. By defining the tasks, it is clear to all parties when the analyses are complete and can help avoid the situation in which a client subsequently wants further analyses done. It is the experience of the authors that clients who understand the consulting relationship at the onset are more likely to remain active participants. If the clients cannot accept the expectations for them and for the students, they need to be offered other alternatives before trouble starts. See Appendix D for the agreement form used at GVSU.

A related pitfall occurs when students do not finish a project in a satisfactory manner, which does happen. Since 1998, the PSU course has had seven students (out of 104) fail the course and two withdraw mid-semester. Five of the students who failed stopped coming to class mid-semester but were unable to formally withdraw (i.e. did not have any drop credits remaining). An agreement form does help prepare the client for this situation. As the clients are often themselves faculty members, they understand the reality that sometimes students drop courses mid-semester or fail the courses. At PSU, no additional follow-up is done when the student stops working on the project mid-semester. However, when the analysis is not completed accurately, it may be possible to reassign the project to another student or for the faculty member to simply complete the analysis.

Another potential problem is a lack of appropriate projects. If it appears as though there will not be enough projects to go around, the professor may have to search for additional projects. In this case, prior clients are often more than willing to serve as repeat clients if asked. This has been done at PSU, with clients eager to have assistance with another dataset.  This enables class to have unique projects, contributing to the sense that the students’ work is contributing to on-going research. However, at the instructor’s discretion, projects could be repeated from year to year, as this would still give students experience with real data. Administrative offices on campus often have data waiting to be analyzed and can serve as another source of last-minute projects. Interestingly, we have observed that the largest university (PSU) seems to have the most difficulty finding projects. Other departments tend to teach their own statistics classes and there are multiple centers on campus offering statistical assistance. Perhaps at smaller universities it is easier to find projects because there is only one such center. At SSU, a service learning approach is used in which projects are found from the neighboring community.

On the rare occasion that there are more viable projects than there are students in the course, one or more projects can be turned into a class-wide experience. The client can be asked to come to class for a group consultation and the students are then involved in all subsequent analyses and recommendations for the client. This can also provide a good means for modeling the consulting process throughout the semester.

In addition, one should devote considerable time to helping students identify the appropriate techniques for a given situation.  Do not expect that students have retained the information from prior courses or that they have begun to make connections between courses. Students find it helpful if a systematic approach is employed to identify these techniques (e.g., flowcharts or worksheets that ask a series of questions about the variables and the data). One such useful reference is the Andrews et al. text.

The professor needs to keep track of the students’ progress. It is easy to miss critical steps with only casual feedback from students that the project is going well and the client is happy. In one author’s early experiences, students and clients worked with less direct supervision. The client in this case was an undergraduate student in another department, who was intimidated by her advisor. The advisor had explained mediating versus moderating variables. The student client, not understanding these concepts and unwilling/unable to ask her advisor for help, attempted to explain it to the consulting student. By the time the work was examined closely, the damage was already done; an incorrect model was used, the semester was over, and the client’s advisor was not satisfied. A detailed journal, reviewed weekly by the professor, helps to keep projects on track.

The unique nature of each consulting project makes these classes more time consuming to teach than the typical statistics course, both due to the time it takes to work through a solution and because the students’ work should be graded by the professor, not a graduate student. The fact that there are no “right” answers to these client questions makes it impossible to grade according to an answer key. However, at PSU, a teaching assistant is available to work with students in implementing professor recommendations for a particular problem. SSU’s course, with a mix of statistical sophistication among the students, offers a built-in class assistant for the less experienced students. In developing a new consulting course, it would be beneficial to discuss the expected teaching load in light of the extra effort required for an involved consulting course requiring individual mentoring. 

In the three courses described, the authors do not receive additional release time for teaching our consulting courses; however, doing so was also linked to the authors’ duties within the respective consulting centers. If developing a new course, one might attempt to equate the time required to that required for a standard course. For example, at PSU, 15 students in a 500 level two credit graduate consulting practicum (different than the course described herein) is equivalent, in terms of teaching load, to a 500 level three credit course of up to 30 students. Thus, any number of students greater than 15 yet less than 30 in this graduate consulting practicum is counted as two courses in the professor’s teaching load. An interested reader may be able to use this as a guideline when proposing a new course to be developed. GVSU caps enrollment for the consulting course at 15 students per semester.

As previously mentioned, the class sizes for these consulting courses have ranged from three to eighteen. Based upon the experiences of the authors, the maximum number of projects that can reasonably be overseen by one professor in a consulting course is ten. If the desire is to provide each student with a unique experience, then the course capacity needs to be set accordingly. Otherwise, it will be necessary to pair students on projects. 

6. Minimal Skills

Undergraduate students can have successful, confidence-building consulting experiences with relatively minimal prior skills, as long as the projects are not overly challenging. Students will need to have good writing skills in order to prepare a written report for the client(s). For instance, at GVSU, students will typically have had a freshman- and a junior-level writing course and one other Supplemental Writing Skill course prior to taking the consulting course. As this is the first time students will likely be required to write a technical report, the authors have found it useful to spend time discussing the organization and requirements of technical reports. Students can further be introduced to technical writing by reviewing previous consulting reports.

With respect to statistical background, advanced students should have had at least two applied classes prior to the consulting class.  These courses should cover the more commonly used introductory-level topics.  Further, these students should have had one or more classes that introduce SAS and at least one other commonly used statistical package (e.g., Minitab or SPSS). However, students can perform astonishingly well even with only one semester of statistics, as has been seen at SSU. Of course, the students who take the class are self-selected, so they are usually motivated and keen. It is very important to provide positive, yet constructive, feedback, so that their expectations are met, and they are not put off the subject. It goes without saying, however, that more prepared students often learn more specific subject matter.

Ideally, a class that helps students develop oral communication skills would be also be beneficial. Students at PSU and GVSU are not required to take such a course. As such, time is spent nurturing these skills as they apply to statistical consulting.

7.  Reasonable Expectations of Students

At the end of the semester, we find that students often comment that the consulting course is one of the most challenging classes they have taken. At the same time, these students often identify the course as one of the most rewarding classes.  It is reasonable to expect that students will lack confidence in the beginning of the statistical consulting process.  As previously mentioned, care should be taken to avoid overly challenging projects. Further, students should be guided through the process using a structured approach that requires students to demonstrate progress on a regular basis. An area that the students will find to be very, if not the most, challenging is identifying an appropriate statistical technique to use in a given situation.  Providing the students with numerous scenarios will help to boost their confidence in this area.

At all three universities we use feedback from the clients in assessing student performance and client satisfaction.  This can take the form of a simple question such as “How effective has this consultation been for you” or more complex, as in a survey asking about the student’s professionalism, ability to understand the client’s question, and quality of the finished consulting project. At GVSU, the consulting contract at the beginning of the semester stipulates that the client complete an evaluation at the end of the semester (see Appendix E). Such an assessment was suggested by a client at PSU with the objective of making the students more responsive to the clients. The challenge with this system is that not all clients do complete the feedback form, and of the completed forms, some arrive before semester grades are entered and some after that point. Regardless of when received, the client feedback generally indicates that the clients appreciated the opportunity to work with our students and that in many cases, clients report learning more about statistics as a result of their participation.

8. Conclusion

The authors collectively agree that undergraduate students can effectively be involved in statistical consulting experiences.  These experiences can provide a rich learning environment for students. The authors have all seen course evaluation comments in which students comment that, despite the often heavy workload, the course was one of the most enjoyable classes taken and that the actual consulting project helped them understand the roles and responsibilities of the parties. 

It may be helpful for the reader if some exemplary undergraduate statistical consulting projects are provided. The Faculty Teaching and Learning Center at GVSU conducted a survey to better understand faculty perceptions of the Center. The student assigned to this project and the professor met with the Director of the Center in order to understand her project needs. The student then continued to regularly meet with the client on her own throughout the semester. The analyses done by the student for this project were all descriptive and involved a combination of bar charts, pie charts, and cross tables to present the results. The Final report was well received by the client. Another more advanced exemplary project completed at PSU explored factors affecting social responsibility among high school students. The student conducted reliability analyses on several factors as part of the exploratory data analysis and analyzed a logistic regression model and two general linear mixed model in SAS. These two projects demonstrate the range of projects that can be completed. The ideal project for any given student will be dependent upon his or her statistical background and the availability of suitable projects.

Yet, as pointed out, involving undergraduate students in statistical consulting experiences is not without challenges. As an example, a project was accepted and assigned to a student with the assumption that regression modeling would be employed.  As it turned out, once the client’s objectives were better drawn out well into the semester, it became apparent that factor analysis and discriminant analysis would be needed in order to answer the client’s questions. Since both techniques were completely unfamiliar to the student, the project required considerable more time for both the student and the professor.

Still, the authors strongly feel that any cons are far outweighed by pros when it comes to involving undergraduate students in statistical consulting experiences making them experiences of great “fortune” for all parties involved.


Appendix A:  Key Program and University Information

Interested persons are encouraged to contact the authors for supplemental course information.

Grand Valley State University

Location: Three campuses near or in Grand Rapids, Michigan


Size:  24,000 students (fall 2006)
Primary focus:Teaching; Active scholarship to enhance teaching
Number of undergraduate statistics majors:  58
Number of full-time faculty who are statisticians:  13
Statistics-related degrees offered:  Statistics major; Statistics minor; Applied statistics minor

Overview of undergraduate statistics major curriculum

  • 29 hours in statistics (including 6 hours of electives)

    o Required

    § STA 215 (Introductory Applied Statistics, 3 cr.) or STA 312 (Probability and Statistics, 3 cr.)

    § STA 216 (Intermediate Applied Statistics, 3 cr.)

    § STA 311 (Introduction to Survey Sampling, 3 cr.)

    § STA 315 (Design of Experiments, 3 cr.)

    § STA 319 (Statistics Project, 3 cr.) *

    § STA 412 (Mathematical Statistics I, 4 cr.)

    § STA 415 (Mathematical Statistics II, 4 cr.)

    o Elective courses include:  Applied Regression Analysis, Biostatistics, Multivariate Analysis, Nonparametrics, and Quality Contro

  • 16 hours in mathematics and computer science (Calculus I and II, Linear Algebra, Computer Science I)

  • 6 hours of application courses (from a pre-approved list of courses throughout the university that have a significant statistics or research focus)

    * Consulting course


     


    STA 319 at Grand Valley State University

    Title:  Statistics Project
    Credits: 3 semester hours
    Terms offered: Fall and winter semesters
    Catalog description:
    Students will learn a systematic approach to statistical consulting, how to communicate about statistics with non-mathematical audiences, and develop the ability to apply appropriate statistical techniques to research questions. Actual experience with current university and industry projects and statistical analysis packages such as SAS/SPSS is given.
    Prerequisites: STA 216 (Intermediate Applied Statistics)
    Enrollment range:  11 to 18
    Textbooks/resources used:

    1        Required

    § Selecting Statistical Techniques for Social Science Data: A Guide for SAS Users (1998).  Andrews et al.  SAS Institute, Cary NC

    § Cody’s Data Cleaning Techniques Using SAS Software (1999).  Cody.  SAS Institute, Cary NC

    § How to Report Statistics in Medicine:  Annotated Guidelines for Authors, Editors, and Reviewers, 2nd (2006).  Lang & Secic.  American College of Physicians, Philadelphia, PA

    2        Optional

    § Blair Handbook (1996).  Fulwiler & Hayakawa.  Prentice Hall, Englewood Cliffs, NJ


    Number of consulting experiences in which a student is typically involved:  one involving a faculty/staff client
    General topics covered in course (listed in order of coverage):
    Communication skills as applied to statistical consulting; Systematic approach to statistical consulting; Responsibilities of client and consultant; Non-statistical aspects of consulting; Ethics in statistical consulting; Writing technical reports; Review of SPSS and SAS; Review of statistics including how to select the appropriate technique; Project problem shooting; Preparation for SSD presentation.
    Course requirements/assignments:
    (1) Three writing assignments: 
    (a) reactions to letters written by prior STA 319 students about their consulting experiences; (b) reflection on a journal article related to statistical consulting; (c) final report for major consulting client; (2) Weekly journal; (3) Semester exam; (5) In-class exercises and reflections; (4) SSD presentation


    Additional comments:  This course also serves as a Supplemental Writing Skills class (all GVSU students must complete two SWS courses); as such it is writing intensive (with revisions).  All students must talk about their consulting experiences at the university-wide Student Scholarship Day (SSD) in April.


    Pennsylvania State University

    Location: Main campus at University Park, State College, PA; 21 other campuses throughout Pennsylvania
    Size:  40,000 students
    Primary focus: Teaching, Research, Service
    Number of undergraduate statistics majors: 30
    Number of full-time faculty who are statisticians: 30
    Statistics-related degrees offered:  Statistics major; Statistics minor; Graduate student option; Statistics and Computing option; Applied Statistics option; Masters of Applied Statistics; Masters of Science; Ph.D.

    Overview of undergraduate statistics major curriculum

  • 43 to 45 hours in statistics

    o 19 required credits in statistics

    § STAT 220 (Basic Statistics for Quantitative Students, 3 cr)

    § STAT 414 (Introduction to Probability Theory, 3 cr)

    § STAT 415 (Introduction to Mathematical Statistics, 3 cr)

    § STAT 460 (Intermediate Applied Statistics, 3 cr)

    § STAT 462 (Applied Regression Analysis, 3 cr)

    § STAT 470W (Problem Solving and Communication in Applied Statistics, 3 cr)*

    § STAT 480 (Introduction to Statistical Program Packages, 1 cr)

    o 12-14 hours in mathematics (Calculus I and II, Calculus of Several Variables, Matrices)

    o 3 credits in computer programming

    o Elective courses include: numerical calculations, computational statistics, nonparametric statistics, stochastic modeling, survey sampling, time series (12-14 cr)

    *Consulting course, satisfies Writing across the Curriculum requirement

     


    STAT470W at Penn State University

    Title:  Problem Solving and Communication in Applied Statistics
    Credits: 3
    Terms offered: Spring
    Catalog description: Provide problem solving and communication skills through development of writing ability, interaction with peers and the Statistical Consulting Center, and oral presentations.


    Prerequisites: Stat460 (Intermediate Applied Statistics), Stat462 (Applied Regression Analysis), Stat480 (Introduction to Statistical Program Packages)
    Enrollment range:  7 to 14
    Textbooks/resources used:  Readings are obtained from JSTOR, including:

  • Chatfield, C, 1991, “Avoiding Statistical Pitfalls”, Statistical Science, 6: 240-252.

  • Royall, R.M. 1991, “Ethics and Statistics in Randomized Clinical Trials”, Statistical Science, 6: 52-62.

  • Pearce, R.A. 1979, “Experimental Design: R.A. Fisher and some modern rivals”, Statistician, 28:153-161.

    Statistical Consulting Center client files are also used as case studies.
    Number of consulting experiences in which a student is typically involved: One extensive long-term client, who may be an undergraduate Honors student completing a thesis (47.5%), a graduate student (5%), or a faculty member (47.5%).
    General topics covered in course (listed in order of coverage):
    Introduction to consulting, client meetings,  Chatfield’s Avoiding Pitfalls, randomization in study design, DOE, development and analysis of surveys, EDA, special topics lectures (repeated measures, PROC MIXED, logistic regression, time series), computer labs to complete data analysis
    Course requirements/assignments:

    (1) Class attendance and participation (10%);
    (2) Five writing assignments (25%) based on journal articles and existing Statistical Consulting Center client files;
    (3) Class presentations (15%): (a) client proposal and research questions (February), (b) EDA analysis (March), (c) final presentation (April);
    (4) EDA homework problem sets (15%);
    (5) Semester project (35%): (a) weekly journal, (b) Proposal and research questions report, (c) EDA report, (d) Inference report, (e) peer-review, (f) final report.
    Additional comments: This course satisfies the University requirement as a writing-intensive class.


    Sonoma State University


    Location:  In the wine country 45 miles North of the Golden Gate Bridge, California
    Size:  8,000 students (Spring 2002)
    Primary focus:  Teaching; Research (especially with undergraduates)
    Number of undergraduate statistics majors:  20 (out of 110 mathematics majors)
    Number of faculty who are statisticians: 4.5 FTEF
    Statistics-related degrees offered:  Statistics concentration within a Mathematics major

    Overview of undergraduate statistics concentration curriculum

  • 53 hours in mathematics and statistics

    o Required Core classes (classes are 3 units unless indicated otherwise)

    § MATH 161: Calculus I (4 units)

    § PHYSICS 114: Introductory Calculus based Physics (4 units)

    § MATH 181: Statistical Programming (SAS, SPSS) (2 units)

    § MATH 211: Calculus II (4 units)

    § MATH 220: Introduction to Higher Mathematics (Proof class)

    § MATH 241: Calculus III (Linear algebra and differential equations) (4 units)

    § MATH 261: Calculus IV (4 units)

    § MATH 322: Linear Algebra

    § MATH 340: Real Analysis (4 units)

    § MATH 345: Probability Theory

    § MATH 365: Statistical Inference I

    § MATH 367: Statistical Consulting (2 units, taken twice)

    § MATH 465: Statistical Inference II (4 units)

    § MATH 441: Operations Research

    § MATH 470: Mathematical Modeling



    MATH 367 at Sonoma State University

    Title: Statistics Consulting
    Credits:  semester hours
    Terms offered:  Every Semester
    Catalog description:
    This course is a blend of theoretical and practical aspects of statistical consulting. Students learn how to consult with professionals in various fields, find creative statistical solutions to real-world problems and present results in written and oral form. Students also learn about library research and statistical software packages. This course may be repeated once for a total of 4 units.
    Prerequisites: MATH 165 (Elementary Statistics) or MATH 250 (Probability and Statistics) or MATH 365 (Statistical Inference I) or consent of instructor
    Enrollment range:  5 to 15 (6 is typical)
    Textbooks/resources used:

  • Required

    § Problem Solving: A Statistician’s Guide, 2nd edition; Chris Chatfield, Chapman Hall, 1995

    § Other reference books as appropriate.

    Number of consulting experiences in which a student is typically involved:
    If the course is offered in the ‘major project’ mode, one, with multiple meetings. If the course is offered in ‘multiple consultations’ mode, usually 6.
    Description of the class:
    The class is offered in two modes. Students should take one of each type if they are repeating the class for credit.
    Mode A: Major Project Mode. Here, the students will work in a group to solve a multi-step, major project. Depending on the number of students in the class, there will be one to four such projects. Each project entails multiple statistical topics, and will require additional reading, class time, and research.
    Mode B: Multiple Consultations Mode: Here, the students will meet every 2 weeks with clients, and will write up papers for the clients, as well as presenting their recommendations to the clients orally. Each session will be limited to one particular statistical aspect of the project under consideration.
    In both cases, their will be homework assigned, which counts 25% of the final grade of the class. This HW will deal with technical aspects of consulting, such as interviewing techniques, ethical consulting, oral and written presentations, and so on.
    The remainder of the grade will come from either the major project report (written and oral), or the bi-weekly consulting reports (written and oral).


    Appendix B:  GVSU Flyer Used to Solicit Projects

    S t a t i s t i c s ?!      

    A r g h h h !!!

    Perhaps you’ve been collecting data but are just not sure what to do with it or how to best analyze the data. Perhaps you have a research project that you’ve started but just doesn’t seem to get finished. Does this sound like you? 

    If so, the Department of Statistics is offering STA 319 (Statistics Project) during the fall semester 2006. STA 319 is taken by undergraduate students who are working on either a statistics major or an applied statistics minor. The course objectives include providing the students with an opportunity to gain experience in:

  • statistical consulting,
  • manipulating data via a computer,
  • applying the appropriate statistical technique for a give situation,
  • correctly interpreting the results, and
  • communicating the findings in clear, non-mathematical terms.

    I am soliciting the GVSU faculty and staff for projects consistent with these objectives. If you request help, you should be willing to regularly meet and work one-on-one with the student assigned to your project. The students in class are undergraduate students and, for many, this is the first time they are serving in a consulting capacity. The students are familiar with a variety of procedures including chi-square, t-tests, regression, ANOVA, and nonparametric tests. They are familiar to a lesser extent with advanced techniques such as factor and discriminant analyses. I will supervise the work being done by the student, and I will provide the necessary technical guidance. However, the primary consultation will be between you and the student. By the end of the semester, the student consultant will provide you with a written summary of his or her involvement in the study.

    To request the services offered by this course, please send Neal Rogness (rognessn@gvsu.edu) an email message indicating that you are interested in statistical assistance via STA 319. Please send the email by [date] (and the sooner the better); you will be informed by the middle of September if I am able to assign a student to your project. Whether your request can be honored depends upon a number of factors such as enrollment in STA 319, the number of requests for help that I receive, and the complexity of the assistance you request.

    Should you have any questions, please contact Neal Rogness via email. THANKS FOR YOUR HELP!!!


    Appendix C:  GVSU project information sheet

    Thank you for responding to my request for projects in STA 319 for fall semester. I would not be able offer the students in STA 319 the richness of a statistical consulting experience without folks like yourself. As a reminder, the students in STA 319 are undergraduate students. For most, if not all, of these students, this will be the first time he or she has provided statistical consulting. While I will provide technical assistance for the students, the primary consultation will be between you and the student. 

    What follows is a 17-item questionnaire. Your collective responses will help me match an appropriate student with your project (assuming the enrollment in STA 319 allows me to fulfill your request for help). Also, since I will share this information with the student, it will also help him or her formulate questions to ask at the initial consulting session.

    I would appreciate receiving this information as an email attachment no later than [date] (however, as always, the sooner the better). Should you have any questions or concerns, please contact me at 331-2447 or email me at rognessn@gvsu.edu.

    If you have multiple potential projects, please complete a separate form for each project.

     

    * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

       

    1. Faculty name:

    2. Department:

    3. Campus address:

    4. Campus phone number(s):

    5. Describe the primary goals/objectives of your research project (use as much space as needed):

    6. Being as specific as possible, please indicate the type of research and/or statistical help you desire (use as much space as needed):

    7. Does your study involve any sensitive data that might limit the involvement of a student in your project?

    _____  No.

    _____  Yes. Please explain:

    8. Has data collection with respect to this study been completed?

    _____ No.  Please skip to Question 12

    _____ Yes.  Please continue with Question 9

    9. Approximately how large is your sample:

    10. Approximately how many pieces of information did you collect from each object/person (or how many variables do you have in your study):

    11. Have your data been computer entered?

        

    _____ Yes.  In what format?

    _____ Excel

    _____ SPSS

    _____ Text file

    _____ Other (please identify):

                _____ No.  By what date do you realistically expect to have data entry

                                  finished: 

                   Please skip to Question 16

        

     

     

    ANSWER QUESTIONS 12 - 15 IF YOU SAID "No" TO QUESTION 8

    12. By what date do you realistically expect to complete data collection:

    13. Approximately how large of a sample do you anticipate:

    14. Approximately how many pieces of information do you plan to collect from each object/peson (or how many variable do you anticipate having in your study)?

    15. Do you have a strong preference for using a particular statistical software package?

    _____ No.

    _____ Yes. If yes, please indicate your preference (e.g., SAS, SPSS, etc.):

     

    16. With respect to timing, I prefer that the student be given until the end of November to complete all analyses and have preliminary results.  Do you have an earlier deadline by when this project MUST be done?

    _____ No

    _____ Yes.  If so, please specify your deadline date: 

    17. I will require the student to regularly be in contact with you as a consultant, to give you progress reports, and to share with you preliminary drafts of his or her report.  To help accomplish these goals, your availability is crucial.  Do you agree to make yourself accessible to the student throughout the semester?

                _____  No

                _____  Yes.     Please list the times for each weekday during the fall 2006
                                        semester when you anticipate being accessible and could meet
                                        with the student assigned to your project.

                                        Monday:


                                        Tuesday:


                                        Wednesday:


                                        Thursday:


                                        Friday:



    Appendix D:  GVSU Client – Consultant Agreement Form

    Student(s)_________________________________________________________________, hereafter

    called the consultant(s), has/have been assigned to the research project of faculty/staff member

    ___________________________________________________, hereafter called the client. This relationship is

    formed to provide the students with experience as statistical consultants. The consultant(s) agree

    to perform the following responsibilities:

  • with respect to the project, complete the following:

  • communicate with the client on a regular basis (as agreed to by the consultants and the client)
  • complete the analyses by ___________________________________
  • provide the client with a written report summarizing the involvement of the consultant(s) with the project by the end of April unless otherwise noted
  • other (as specified)

    The client agrees to perform the following duties:

  • be available to regularly meet with the consultant(s) and to serve as a mentor concerning the research process (this includes making sure that the consultant(s) clearly understand the objectives of the study)

  • provide the consultant(s) with the complete data no later than ____________________ (if the data have not already been collected and provided)
  • evaluate the consultant(s) (individually, if two consultants) toward the end of the semester regarding his or her involvement with the project. This evaluation will be used in determining the student's overall semester grade.
  • allow the consultant(s) to give a presentation at Student Scholarship Day; the focus of the presentation will be on what the consultant(s) learned about being statistical consultants relative to this project and may also include a brief summary of the findings.

  • other (as Specified):

    ________________________________                    ________________________________     

    Signature of consultant                                               Signature of client   

     

    ________________________________                    ________________________________                    
    Signature of STA 319 instructor                         Date of agreement

     

    NOTE: Unforeseen circumstance may dictate a change in the agreed upon responsibilities. Such change(s) must be communicated to, and agreed upon by, all interested parties. This agreement should not be interpreted as a legally binding contract.


    Appendix E:  GVSU Client Evaluation form

    As part of the semester grade for the STA 319 student assigned to your project, I would like for you to provide feedback concerning this consulting experience. Please indicate the extent to which you agree with each of the following items by circling the appropriate numerical value. Please answer each question based upon your experiences to date. I would appreciate receiving your honest assessment. However, please also keep in mind that this was the first experience the student had in providing statistical consulting.

  • Client(s): ______________________________________________________________________

     

    Statistical consultant (student) being evaluated: _______________________________________

     

            ______________________________________________________________________

                0                      1                      2                      3                      4                      5

                Strongly                                                                                                 Strongly

                disagree                                                                                                    agree           

     

     1. The student was dependable in keeping appointments.                         NA   0   1   2   3   4   5

     

     2. The student came prepared for meetings.                                           NA   0   1   2   3   4   5

     

     3. The student conducted his or her work in a timely fashion.                  NA   0   1   2   3   4   5

     

     4. The student conducted him or herself in a professional manner.           NA   0   1   2   3   4   5

     

     5. The student was eager to assist with the project.                                NA   0   1   2   3   4   5                         

     6. The student achieved a good understanding of what                            NA   0   1   2   3   4   5

         this research project was trying to accomplish.

     

     7. The student kept me/us regularly informed of the progress                  NA   0   1   2   3   4   5

         being made with the project analyses.

     

     8. The student was knowledgeable about statistics as                              NA   0   1   2   3   4   5

          applied to this project.

     

     9. The student clearly explained the project’s statistical aspects.              NA   0   1   2   3   4   5

     

    10. Overall, the student did a good job as a statistical                               NA   0   1   2   3   4   5

           consultant on this project.

     

    11.  Which of the following letter grades do you feel BEST reflects the overall performance of

            the student with respect to the consulting experience?  (circle one)

     

    A         A-        B+        B          B-        C+        C          C-        D         F

  •  

    Please complete & return this form to Neal Rogness in MAK 1133 by [date].

     


    References

    American Statistical Association (2004), “Curriculum Guidelines for Undergraduate Programs in Statistical Science”, www.amstat.org/education/index.cfm?fuseaction=Curriculum_Guidelines

    Andrews, F. M., Klem, L., O'Malley, P. M., Rodgers, W. L., Welch, K. B., and Davidson, T. N. (1998), Selecting Statistical Techniques for Social Science Data:  A Guide for SAS Users, Cary, NC: SAS Institute.

    Chatfield, C. (1995), Problem Solving: A statistician’s guide, New York: Chapman & Hall.

    Cody, R. (1999), Cody’s Data Cleaning Techniques Using SAS Software, Cary, NC: SAS Institute.

    Daiker, D.A. (1989), “Learning to Praise,” in Writing and Responses: Theory, Practice, and Research, eds. C.M. Anson, Urbana, IL: National Council of Teachers of English.

    Derr, J. (2000), Statistical Consulting: A guide to effective communication, Pacific Grove: Duxbury.

    Derr, J. and Stinnett, S. (1994), “The Interesting Life of a Practicing Statistician”, STATS, Spring.

    Kirk, R. (1991), “Statistical Consulting in a University: Dealing with People and Other Challenges”, The American Statistician, 45, 28 – 34.

    Lang, T. A., and Secic, M. (2006), How to Report Statistics in Medicine:  Annotated Guidelines for Authors, Editors, and Reviewers, 2nd, Philadelphia, PA: American College of Physicians.

    Radke-Sharpe, N. (1991), "Writing As a Component of Statistics Education"; The American Statistician, 45, 292-293.

    Singer, J.D. and Willett, J.B. (1990), “Improving the Teaching of Applied Statistics: Putting the Data Back Into Data Analysis,” The American Statistician, 44, 223-230.

    Spurrier, J.D. (2001) “A Capstone Course for Undergraduate Statistics Majors”, Journal of Statistics Education [Online], 9(1), (www.amstat.org/publications/jse/v9n1/spurrier.html)

    Zahn, D. and Boroto, D. (1983), “Being Responsible for the Success of Consulting Sessions”, American Statistical Association 1983 Proceedings on Statistical Education, 107 – 108.


    KB Boomer
    376 Olin Building
    Bucknell University
    Lewisburg, PA 17837
    U.S.A.
    KMB031@bucknell.edu

    Neal Rogness
    1133 Mackinac Hall
    Grand Valley State University
    Allendale, MI 49401
    U.S.A.
    rognessn@gvsu.edu

    Brian Jersky
    School of Science
    St. Mary's College of California
    Moraga, CA 94575-4507
    U.S.A.
    brian.jersky1@stmarys-ca.edu

    An earlier version of this paper was presented at the 2002 Joint Statistical Meetings in New York.

     


    Volume 15 (2007) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications