W. Robert Stephenson
Iowa State University
Journal of Statistics Education Volume 9, Number 3 (2001)
Copyright © 2001 by W. Robert Stephenson, all rights reserved.
This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the author and advance notification of the editor.
Key Words: Assessment; Distance education; Industrial statistics; Videotape.
In 1993 the Statistics Department at Iowa State University entered into a collaborative agreement with General Motors to develop and deliver a new sequence of courses titled "Applied Statistics for Industry." This paper describes the development and content of these courses as well as their method of delivery. In order to accommodate on campus students as well as students at a distance, the course is presented live at Iowa State University and by videotape delay at General Motors Technical Education sites in Michigan, Ohio, Arizona and Mexico, and across the country at sites of other partner industries. Some of the differences between a statistics course taught in the traditional campus setting and a statistics course taught at a distance will be highlighted. Since there are two audiences (on campus and off campus), several compromises are made in how the course is conducted. These compromises, and their possible effects on students in both environments, are discussed. A summary of how on and off campus students did in these courses over the past five years is included.
Distance education is not a new idea, either in general or in statistics. In the U.S., one can trace distance education back to its roots in correspondence courses as early as the late 1800's. There were some attempts at providing education over the radio in the 1920's and on TV as early as the 1930's. In the 1960's distance education, and statistics education, came together on the Continental Classroom (see Moore 1993). Since that time introductory statistics courses have been developed for the Annenberg/Corporation for Public Broadcasting Project and entire masters degree programs in statistics offered to off campus students by schools such as Colorado State University, University of Florida, and Iowa State University.
The Department of Statistics at Iowa State University has a long history of providing statistics courses to varied groups of students. In order to better serve the campus community, courses in applied statistics have been tailored to meet the specific needs of students in agriculture, the social sciences, business, and engineering. In 1993 a steering committee of statisticians and engineers from General Motors (GM) contacted the Department to see if we would develop a statistics course to meet their needs. In addition to developing the course, GM wanted the Department to deliver the course to GM Technical Education Program sites in Michigan and around the world.
GM, like many other companies, must compete in a global market to produce the highest quality products at the lowest costs. The GM steering committee recognized the value of statistical thinking and statistical methods in meeting this competition. The steering committee's vision was to have a two semester sequence of courses on statistical practice in industry. This sequence would not be a standard course in probability and statistics. Of course, students would be exposed to statistical methods and how to apply them. More importantly, students would gain an appreciation for statistical thinking. Statistical thinking and statistical methods would provide the framework and tools for solving problems and building knowledge. Ultimately, students completing the courses would be able to use data to make better decisions, solve problems, build knowledge and continuously improve the processes and products they deal with in their day-to-day work. The steering committee recognized that although problem solving is important, one will never get ahead by simply fixing mistakes. Therefore, the course would stress that the fundamental purpose of the collection and analysis of data and of statistical thinking is to build knowledge about processes and products. Increased knowledge not only helps solve problems when they arise but may also help anticipate and avoid problems in the future.
Given the steering committee's vision, what is the reality of the courses? In Section 2 the content of the two course sequence, Applied Statistics for Industry, that was developed to meet the needs of General Motors is presented. The delivery method is discussed in Section 3. Since the primary audience views the course via videotape, the format and style of the course is very different from the typical on campus course. The unique challenges encountered in having an off campus and an on campus audience are discussed in Section 4. Finally, the success of students, both on and off campus, is evaluated. Additionally, student ratings of the courses' content and delivery are summarized.
Applied Statistics for Industry I and II (STAT 495-496) has two general purposes. The first is as a stand-alone course in the practice of statistics. The second is to provide a bridge, or feeder, course to the Master of Science degree program at Iowa State University. STAT 495-496, is an overview of important statistical ideas and methods pertinent to industry. It exposes students to specific topics in quality philosophy, process monitoring, collection and analysis of data (especially designed experiments) and reliability. The emphasis is on the application and practice, not the theory, of statistics. To that end the collection and analysis of data provide a central theme.
In terms of goals and objectives, students should come away from the courses with:
Applied Statistics for Industry I
Applied Statistics for Industry II
Both courses revolve around the central theme of statistical thinking. Statistical thinking has two fundamental elements:
Throughout the courses we return to these two elements. We demonstrate how statistical methods help to understand and quantify variability. We reinforce the fact that a single experiment or set of data does not give us complete knowledge. Instead we focus on how knowledge is built in increments.
A major theme running through both courses is the connection between the scientific method and statistical thinking and practice. With statistics, as with the scientific method, we build knowledge either by informed observation or direct experimentation. Applied Statistics for Industry I (STAT 495) looks primarily at informed observation. That is, being in the right place at the right time with the right tools. When learning about an existing process, several topics in statistics provide the right tools for informed observation. The major topics in Applied Statistics for Industry I (STAT 495) are:
Although building knowledge is central, the connection between increased knowledge and improvement of processes and products is key. The reduction of variability is the means by which improvements in quality are made. One must understand and quantify variability before one can reduce it. The goal, then, in STAT 495 is to understand processes and quantify variability.
In order to assess how well students understand the material presented in STAT 495, we require an individual, or team, project. The project involves the describing, observing, collecting data, monitoring and making suggestions for improvement for a process the students encounter in their work or in their day-to-day life.
Applied Statistics for Industry II (STAT 496) looks more carefully at direct experimentation. Rather than simply observe and monitor processes, we look at what can be done to change them. We discuss how statistical ideas can help in the efficient collection and analysis of data. Again the connection between statistical thinking and practice and quality improvement is explored. The major topics in Applied Statistics for Industry II (STAT 496) are:
An important part of this course is a team project on the design and analysis of an experiment. Several times this project has involved the design of a paper helicopter. Among the factors investigated were: wing length, tail length, launch height, number of paper clips attached to tail, body folds, winglets, and launch position (wings up or down). Each team had to plan and carry out an experiment to learn about the effects of each factor on the response variable, flight time. After the data collection and analysis was completed, specifications for several of the variables and the flight time were given. Students had to use their data analysis to come up with optimum settings for the other variables, construct test models and verify that their models would meet the specifications consistently. A final written report, suitable for presentation to a company executive, was submitted at the end of the semester. The report was evaluated on the completeness and efficiency of the experiment, the accuracy and appropriateness of the analysis and the quality of the written presentation.
When the course was first offered, there was no official text. Instead a recommended set of readings was given. Additionally, each student received a handbook of materials. Students requested that there be some text for the courses. The past three times the courses have been offered we have used the text by DeVor, Chang, and Sutherland (1992) in addition to the readings and handbook materials.
The students are evaluated on the basis of their performance on two midterm examinations, a final examination, and homework assignments. STAT 495 includes the individual/team project involving the description and monitoring of a process, and the experimental design project is assigned in STAT 496. Samples of homework assignments, exams and instructions for the projects can be found at www.public.iastate.edu/~wrstephe/.
General Motors Technical Education Program offers a wide variety of courses to GM employees by means of videotape delay. A videotape is made during a live class presentation on campus. The videotape is then shipped to the off campus sites where it is viewed. This creates a one week delay between the on and off campus students throughout the semester. Technical Education Sites across Michigan and in various locations around the world, facilitate the videotape offerings. Videotape is preferred over a live-link format because of the flexibility the tapes allow the GM employees. Students at GM can keep up with the courses even if work obligations, such as travel and special projects, interrupt the scheduled class meeting times. In addition to the students at GM, we offer the courses via videotape delay to other partner industries; 3M and Rockwell-Collins, as well as through the Engineering Distance Education program at Iowa State University.
The courses are filmed in front of a live audience. In order to attract an audience similar to those at GM, the on campus courses were first offered in a three hour session on Tuesday evenings. During the 1994-95 school year there were approximately 15 students on campus for each of the two courses. These students were primarily engineers and managers from local industry or graduate students in engineering fields at Iowa State University. There were over 50 off campus students taking STAT 495 in Fall of 1994 and 40 off campus students taking STAT 496 in Spring 1995. Rather than make one three hour videotape, two videotapes lasting from one hour to one hour and 30 minutes each are made each week. The students at GM have a three hour period scheduled each week where they can view the videotapes as a group. They can also check out the videotapes to view them on their own. Many GM sites have a course moderator who is available to answer student questions on the videotapes or homework. The moderator is also available to monitor examinations. Other off campus students have differing experiences. Occasionally there is only one person at a particular site.
Since the off campus students, especially those at GM, are the primary audience, it is very important that the course presentation is compatible with the videotape format. What is appropriate for a classroom equipped with a chalk board and an overhead projector is not appropriate for a videotape course. For example, most people make overhead slides in a portrait mode, 8 1/2 inch horizontal dimension and 11 inch vertical dimension. For videotape the preferred dimensions are six inch horizontal by four inch vertical, a landscape mode. Using a chalk, or white board is problematic. In order to capture you, the presenter, and the board, the writing on the board usually becomes unreadable on the videotape. Additionally, it is more difficult for a camera to follow a moving object like the presenter as he/she moves around a classroom. Since every word and action is captured on videotape, preparation is key to a smooth presentation.
Rather than make my usual set of handwritten notes supplemented with data sets and graphs, I use the Microsoft POWERPOINT software program to prepare material for the courses. For me, a POWERPOINT presentation is an extended outline, or skeleton, of the material to be covered. Additional details and discussion are added during the class. Making a POWERPOINT presentation for each of the videotapes has several advantages.
There are some drawbacks to the POWERPOINT software. For example, tables and statistical graphs are problematic. I can produce better quality graphs and tables using other software. I can copy and paste these tables and graphs into a POWERPOINT slide but they tend to be too small, especially when the slides are reduced to three per page handouts. Instead, I tend to use the overhead camera in the videotape classroom to present original copies of tables and graphs to the class. In some ways this can be an advantage since it breaks up the presentation and jogs students in the live audience (and I assume those watching the videotape) back to attention. Of course if the computer goes down, so does the POWERPOINT presentation. A backup hard copy of the handouts and the overhead camera can save the day should this occur.
The room where the courses are videotaped is a technology enhanced classroom. There are several cameras (presenter, audience, overhead) plus computer, audio and video feeds. A student video engineer assists with transitions between the various cameras, computers, etc. The engineer can also insert a picture-in-picture so that the camera image of the presenter appears in the corner of the POWERPOINT slide or graph.
Putting together a one to one and one half hour presentation can be time consuming. If you are working from already existing lecture notes you need to condense those into the POWERPOINT presentation and augment them with graphs, computer output, etc. Putting equations into the presentation using the Microsoft equation editor is not difficult but it can be time consuming. Altogether, plan to spend anywhere from one to three times the length of the presentation on preparation. Of course, as you get more familiar with the software you will spend less time.
Rather than simply talking to the videotape camera, I feel that teaching to a live audience provides a more natural presentation. This combination of a live audience and the constraints produced by having to videotape presents some unique situations and challenges. One is tempted to teach to the live audience as they provide the instructor with the most immediate feedback. However, the students watching the videotapes might have different backgrounds and expectations. Since the off campus students are working full time as engineers and managers it is appropriate to recruit a live audience with similar characteristics. In the 1994-95 school year there was partial success in that 65% of the live audience came from local industry. The other 35% were graduate students in engineering related fields. In the past few years, the live audience has become almost 100% graduate students. Students from local industry who could drive to campus to take the course now opt (there is an additional cost for off campus delivery) for the videotape version delivered to their home or office.
The conflict between covering the material on the syllabus and responding to the questions and concerns of the students is complicated by the distance and delay experienced by the off campus students. Are the questions from the live audience of concern to the videotape students or should they be answered off camera? When the questions deal with course logistics rather than content they may be better answered separately for the two groups. Students in the live audience appear to be reluctant to ask questions during the class. They may feel that they are interrupting the videotaping process and are more comfortable asking their questions off camera. The classroom is set up so that a student in the live audience can key a microphone and have the question recorded on the videotape. If the student forgets to key the microphone there is a segment of the videotape with no sound. If the instructor does not repeat the question, the answer has no context for the videotape audience. It becomes necessary to remind students to use the microphone whenever they are about to ask a question. The only opportunity for off campus students to ask questions of the instructor is during telephone office hours. When distance students take advantage of this I try to re-cap their questions at the beginning of the next class period. In Spring 1995 the students at one GM site used a speaker phone so that everyone at the site could ask questions and hear my responses. Other means of communication, e.g. fax and electronic mail are used when available. When the course was first developed, GM students did not have access to external email or the Internet because of security concerns. Now, email is the preferred means of communication. It is advantageous if all students have access to electronic mail so that questions and answers can be posted to all students in the class.
Because of the reluctance on the part of a live audience to ask questions it is very difficult to stimulate class participation. Several times during each course I have activities that are done during class time. The activities are videotaped but it is more important for all students to do the activity than to watch others do it. The big problem with class activities is anticipating what will and will not work with the live audience as well as the remote audience. Each year I try to include more activities in these courses. Again, it is the practice of statistics that we are aiming for and it is worthwhile to take some of the class time to do that practice.
The choice of what statistical software to use is an important one. We require that all students have access to MINITAB for the Applied Statistics for Industry courses. We chose MINITAB because it was the standard package for our on campus introductory statistics courses, it had the functionality needed, and it was cost effective. Through a University wide site license students (on and off campus) could purchase a full version of MINITAB at a very low cost. Now, for a modest semester cost, you can down load a version from an Internet site.
The coordination of the distance course is quite time consuming. The actual videotaping and distribution of the tapes is handled by the university's engineering extension office. In order to accommodate the delay produced by mailing the videotapes, the off campus students are one week behind the on campus students. Further delay is encountered returning the graded assignments and exams. These delays cause problems with going over exams and homework. Off campus students may not get their graded work back before they view the videotape that discusses the solutions. Without their corrected work in front of them, these discussions are often difficult to follow. To overcome these problems, solutions to homework are sent to off campus sites or posted on the course Website. Sample exams and project write-ups are also posted. The videotapes are kept on site during the course of the semester, students can always go back and view those discussions once their corrected homework is returned.
Since all of the off campus students are working full time, there are many individual difficulties that arise during the course of a semester. Off campus students often fall behind because of work related travel and/or projects. I am flexible with dates for turning in assignments and taking exams. It has been my experience that most of the off campus students make up the missed material very quickly.
The delays caused by having students at a distance adds an additional two weeks onto the semester in terms of grading final exams, projects and submitting course grades. Because of semester break and vacation or holiday plans this means that off campus students may not receive their course grades until three or four weeks after taking the final exam.
There is no question that distance learning is the buzz word of the new century. It is important to ask the questions: "do off campus students perform similarly to those on campus?" and "do off campus students have similar satisfaction with the course content and delivery as those on campus?" Although the original intent was to attract individuals from local industries for the on campus sections, these sections have become almost exclusively graduate students in engineering. Thus the distinction between on and off campus is confounded with the distinction between full time graduate students and full time employees.
The two semester sequence of courses, Applied Statistics for Industry has been offered for five consecutive academic years (Fall 1994 through Spring 1999). During that time, 280 off campus and 132 on campus students have completed a course. Some students have taken both courses while others have taken only one (usually the first semester course STAT 495).
As mentioned earlier, students are assessed on the basis of performance on homework, three exams and a project. Homework, exams and projects are graded according to rubrics to maintain consistency. This is especially important given the delay between grading on campus and off campus papers. Grades are assigned from A through F with +/- grades possible. The overall performance in terms of grades is summarized in two ways. First, a grade point average (gpa) for on and off campus students is computed for each semester. This average is based on: A = 4.00, A- = 3.67, B+ = 3.33, B = 3.00, B- = 2.67, C+ = 2.33, C = 2.00, C- = 1.67, D+ = 1.33, D = 1.00, D- = 0.67, F = 0.00. The averages are given in Table 1 and plotted in Figure 1. Although the on campus students have slightly higher grade point averages in most semesters, this could be due to random variation.
Table 1. Grade Point Averages for On and Off Campus Students
Figure 1. Grade Point Averages Fall 1994 through Spring 1999
Additionally, the number of on and off campus students earning various letter grades is summarized in Table 2.
Table 2. Letter Grades for On and Off Campus Students
The value of the statistic for testing independence of letter grade and on/off campus status is 3.07 with five degrees of freedom. This corresponds to a P-value of 0.689. It appears that letter grades are independent of whether one is an on or off campus student. Although performance is quite similar, there is one difference I have observed. When it comes to the group projects, the off campus students often have much more interesting topics because they choose to investigate things that they are dealing with in their jobs.
At the end of each semester course/instructor evaluations are administered to both on and off campus students. Most of the questions on the Iowa State University form ask for evaluation of the instructor. However, starting in Fall 1996, there is one question that asks for the students' overall rating of the course. This rating is on a five point scale with one being "Very Poor" and five being "Very Good." Table 3 summarizes the ratings for students enrolled since Fall 1996.
Table 3. Overall Rating of the Course
Although there appears to be a somewhat sizeable difference between the average rating of off campus students (4.17) when compared to the average rating of on campus students (4.40), this can be accounted for by the low off campus ratings for the 1996-97 school year. In Fall 1996 we opened up the course to industrial sites other than those affiliated with the General Motors Technical Education Program. There were over 50 off campus students in Fall 1996. Besides the larger number of students, there were some logistical problems getting the videotapes, homework, etc. to the new sites in a timely fashion. Also, many off campus students had difficulty obtaining access to the software used in the course. With smaller class sizes and adjustments to our handling of the new off campus sites, as well as overcoming the software difficulties, the students' evaluation of the courses has been quite similar for on and off campus students.
In summary, there was very little difference between on and off campus students in terms of their performance in the courses. The off campus students tended to rate the course about the same as the on campus students except in those semesters where the number of off campus students was quite large and there were logistical problems with the delivery of the course or with the computing. These findings agree with those of Merisotis and Phipps (1999) who summarized the results of about 40 studies that looked at student outcomes and attitudes in the context of distance learning. They report "... most of these studies conclude that, regardless of the technology used, distance-learning courses compare favorably with classroom-based instruction and enjoy high student satisfaction." (p. 13)
Presenting an applied statistics course via videotape has several advantages and disadvantages. One advantage is that the videotape course can reach a much wider audience than can be serviced by traditional on campus courses. The on campus students appreciate the fact that all the classes are on videotape and can be viewed at the media reserve room at the campus library. They have the same opportunity as the distance students to "time shift" the viewing of the course material. The videotape format provides greater flexibility than a live-link format at a specified time each week. The videotape format puts a premium on careful preparation. This means more work for the instructor but ultimately a better presentation. With the appropriate presentation software, a presentation that is compatible with the video format can be easily converted into handouts for the students.
On the down side, there is not as much interaction, in the classroom, or with the off campus students, as there is with the more traditional courses. The videotape classroom puts many constraints on the instructor. The instructor is not free to roam about the classroom, to work at the board, or to react to the needs of the students watching the videotape. There is not as much eye contact with the students in the live audience since they are often watching the video monitors. A major challenge is keeping the audience's attention for a three hour block of time. Even when it is split into two chunks, with frequent breaks, it is a long time to keep focused on any subject. The off campus students have the advantage of being able to shut off the videotape and take a break whenever they wish. The students on campus can tune out as well but they will miss something. One of the biggest challenges is simply keeping up with the logistics of two sets of students who are a week apart in the course material.
Over the five years that these courses have been offered both the on campus and remote site students performed comparably. My original expectation was that the remote site students would be at a disadvantage. They did not have the same access to the instructor as on campus students. However, their motivation and commitment may have been strong enough to overcome the challenges of learning statistics at a distance.
The logistics of facilitating the production and viewing of a videotape course can be daunting. The GM Technical Education Program has a great deal of experience with this format and has been very helpful. On campus, the engineering extension program and personnel, especially the media technical specialist, Paul Jewell, the student video engineers, and the secretary Kristy Goodale have made teaching Applied Statistics for Industry at a distance a reality.
Part of this paper appears in the American Statistical Association 1995 Proceedings of the Section on Statistical Education.
DeVor, R. E., Chang, T., and Sutherland, J. W. (1992), Statistical Quality Design and Control, New York: Macmillan Publishing Co.
Merisotis, J. P., and Phipps, R. A. (1999), "What's the Difference? Outcomes of Distance vs. Traditional Classroom-based Learning," Change, May/June, pp. 13-17.
Moore, D. S. (1993), "A Generation of Statistics Education:
An Interview with Frederick Mosteller," Journal of Statistics Education [Online], 1(1).
W. Robert Stephenson
Dept. of Statistics
327 Snedecor Hall
Iowa State University
Ames, IA 50011-1210
phone: (515) 294-7805
quad fax: (515) 294-4040}