The
system of instruction introduced by Fred Keller (1968) and subsequently
known as the Keller Plan has been the basis for 21 offerings
since 1976 of an introductory statistics course that I direct
within a Department of Psychology.
The course is a full-year course with a typical enrolment
each year of 120 students.
It is one of several courses which satisfy the Department’s
requirement that all Honours BA students complete introductory
statistics. Many
of the students enrolled in the course have considerable anxiety
about their ability to learn statistics and mention the structure
and support available in the Keller Plan format as features
that attracted them to it.
In this article I wish to describe the course, accenting
those features to which I attribute its notable success in facilitating
student learning. However,
the system is complex and presents many challenges to instructors
seeking to design a course around it.
I will, therefore, also mention these difficulties and
some of the means by which I have confronted them.
The Keller
Plan
The
primary features of the system as Keller (1968) described
it (also referred to as the Personalized System of Instruction,
PSI or Individualized Instruction) are:
- an
emphasis on written materials that are divided into small
units called modules
- a
clear statement of the objectives of each module
- a
requirement that students attain mastery of the objectives
as evidenced by their performance on criteria-referenced
unit quizzes that may be repeated if necessary
- infrequent
lectures and demonstrations, presented primarily for motivational
purposes
- undergraduate
students called “proctors,” “tutors” or “teaching assistants.”
These students administer quizzes, provide immediate
feedback on quizzes, and offer tutoring and social support.
The
central unifying aspect of Keller’s proposal was the application
of Skinnerian behaviour analysis principles to university
instruction. However, I have not found these principles essential
to the conduct of the course, and I have not used them to
guide my design and modifications over the years.
Rather I have proceeded more pragmatically, discussing
problems and potential modifications with my students and
teaching assistants, and then implementing the proposals we
find most practical. The Keller Plan was the subject of considerable
research in the 70’s (for reviews of this research see Kulik,
Kulik & Carmichael, 1974 and Kulik, Kulik and Cohen,
1979) and was even the topic of a journal, the “Journal of
Personalized Instruction”.
Despite extremely favourable reports of the format’s
value for learning, the Keller Plan is infrequently cited
today, and I am unfamiliar with any other courses currently
offered with this format as a basis.
Gallup
(1995), a strong advocate of the system, provides an interesting
personal account of the difficulties he encountered in his
attempt to maintain Keller Plan offerings over the years and
the reasons he eventually gave up that attempt.
I make no attempt in this article to prove the efficacy
of the Keller Plan, nor to argue that its complexity can be
easily overcome. My purpose is to document my experience with
a structure that I feel has enhanced my teaching over a lengthy
period, and I hope the example may encourage others seeking
to enhance their own teaching.
Course
Description
A
detailed description of the current offering of the course
is available in the 2000/01
Course Outline, the outline distributed to all students
entering the course.
Here I will describe in a more general way the features
that I have retained from the Keller Plan and the modifications
that I have introduced over the years. Those features I have retained relatively unchanged are:
the division of the course material into modules with
extensive printed material for each, mastery quizzes for the
regular modules, self-paced study of the modules, and tutorial
assistance from undergraduate tutors (TAs) in a teaching lab
devoted to the course.
The
self-pacing has worked less and less well through the years
and I have had to set deadlines for quizzes.
Other modifications include:
the addition of periodic review modules with non-mastery
quizzes, the inclusion of regular lectures and problem sessions
given by me, and the removal of TAs from any involvement in
the evaluation of students with a shifting of their focus
to tutoring and advising students.
I will make a few comments on each of these modifications
in the following section, but understanding them will require
first some attention to the importance of the teaching lab
and the interactions that occur there.
Physically
the teaching lab has tables with seating for about 20 in a
central area surrounded by a dozen small cubicles with space
for one person. The lab is open to students for three or four
hours each day, and one or more of the undergraduate TAs is
available in the lab each of those hours. Students come to
the lab to write their quizzes and to seek help with course
material from the TAs or from other students. Students write
their quizzes independently when they are ready for them,
provided the deadline for the quiz has not passed.
In the lab the TAs hand out and receive the students’
quiz sheets; students work on the quizzes individually in
the cubicles. Because
students may consult their texts, notes and calculators freely
during the quizzes, monitoring of the students in the cubicles
is not a concern.
Quizzes
are not marked immediately, but the results are posted in
the lab the following day along with an indication of the
objectives addressed by questions answered incorrectly.
The marked quizzes themselves are only available for
viewing at fixed times just before review exams.
The quizzes consist of ten multiple-choice questions
and students must answer eight correctly to satisfy the mastery
requirement. Those who do not achieve eight correct must return
to the lab for another attempt to demonstrate mastery; however
students are not allowed to make this second attempt without
turning in an acceptably completed assignment for the module.
The acceptability of the assignments as well as the
quizzes themselves are evaluated by a paid graduate assistant.
If necessary, students
may make repeated attempts to demonstrate their mastery of
a module. Multiple
forms of each quiz are prepared, thus reducing the likelihood
a student will encounter the same questions when repeating
an unsuccessful quiz.
In
the central area of the lab students can seek assistance from
the TAs, study course materials or prepare assignments, either
alone or in groups. Prior
to attempting the quiz on a module, students are encouraged
to complete the associated assignment and practice quiz and
then to review these with a TA in the lab.
The lab has become very popular for these purposes,
and many students build specific times into their schedules
to work there. The atmosphere is supportive and formalizes
the kind of assistance which students sometimes try to arrange
amongst themselves in other courses.
The fact that students write their quizzes in the lab
probably enhances the use of the TAs considerably.
In the process of writing their quizzes students quickly
become familiar with both the TAs and the location.
They see students asking questions of the TAs and others,
and the advantages of the assistance offered there soon become
clear.
Modified
Components
The
details of the weighting of the various assignments, quizzes
and exams for the computation of a final grade can be found
in the 2000/01
Course Outline. The original Keller plan imagined students
simply continuing to master units indefinitely, eventually
completing all units successfully and earning a grade of A.
In practice, matters
have been much more complicated, and the current implementation
attempts to balance many factors; for example, the advantages
of self-pacing with those of a structured learning environment.
In this section I would like to comment further on
aspects of the Keller Plan that my experience has led me to
modify.
Deadlines
In
the first year of the course students were permitted complete
self pacing. Their mark was based on the number of modules
they were able to master by the end of the course plus their
final examination performance. However, many students left
numerous modules until the final week or two of the course
and they were generally unsuccessful, both with the modules
and the final exam. Many other students, recognizing a bit
earlier the inevitability of such an outcome, withdrew from
the course. In fact, nearly half the students enrolled in
that first offering of the course dropped out without credit,
about double the percentage typical for our Department’s statistics
courses (20 to 30%). As a result, self-pacing has been restricted
in subsequent offerings by establishing deadlines for completion
of the regular module mastery quizzes.
Students are permitted now to attempt a quiz anytime
prior to the deadline, but those who have not been successful
by the deadline for that quiz receive no credit for the module.
The structure provided by deadlines has greatly reduced
procrastination and the drop rate to the more typical level
(though I have never been able to reduce the drop rate below
that level).
Of
course, some students have difficulty meeting the deadlines.
For several years I tried to encourage such students
to keep learning by frequently granting them extensions of
deadlines. Unfortunately, these extensions were often not
satisfactory as, once behind, many students stayed behind,
found the lectures less and less useful, etc.
Instead of granting extensions, a grace period was
introduced in 1995/96 and has worked quite well since then
to offer some flexibility to all students. During the grace
period, the seven days immediately following a quiz deadline,
students may still write a quiz, without penalty, and without
seeking special permission.
There is a catch however.
Students who wait until the grace period to attempt
a quiz are on their own in the sense that if some misfortune
prevents them from writing the quiz in the grace period, they
can not seek any further extension because of that misfortune.
Of course, some students begin to view the end of the
grace periods as the “real” deadlines and plan their work
to those dates, and by the final weeks of the course relatively
few are completing quizzes before the originally established
deadlines. But the concept of a grace period has been a very
successful means of balancing flexibility and structure in
a manner that assists students to complete their work in a
timely manner. Interestingly, while I used to agonize over
the appropriateness of each extension request, the grace period
concept has allowed me to feel relatively comfortable denying
most requests—though I do continue to grant extensions when
circumstances seem to warrant them.
Review
Modules
By
design each module of a Keller Plan course deals with a relatively
narrow set of topics.
In the early years of the course I saw that mastery
of the separate modules did not guarantee that students would
learn to integrate the topics of the separate modules.
Thus I introduced review modules emphasizing synthesis
of concepts and the practical use of them in reading published
papers with statistical content.
The course kit contains several papers associated with
the topics of the regular modules; these papers are reviewed
in the review modules and questions about them are included
on the review exams.
Four review modules are spaced evenly through the course
and students take an exam associated with each.
Unlike the quizzes, which are always in multiple-choice
format, these exams require written responses and problem
solutions. Also
the mastery requirement does not apply to them; they are offered
at fixed times and may each be written only once.
The rationale for this limitation is simply pragmatic—the
resources of the course and the time available for completing
the course are insufficient to proceed otherwise.
The review modules have improved students’ preparedness
for the final exam in comparison to the early years of the
course, and many students mention the comprehension benefits
that come from seeing the statistical concepts used in a realistic
context.
Lectures
and Problem Sessions
One
of the aims of the course is to have “everything in print,”
but this does not mean that class meetings would be superfluous.
Because comprehensive text material is available, the
lectures do not have to cover all the all the topics in the
modules but can focus on difficult topics and on giving an
overview of each module.
The lectures are quite interactive, featuring work
with real data sets and samples as well as many demonstrations.
They also aim to help students integrate the modules
and prepare for the review exams.
There are two lecture periods per week.
The first, a two-hour meeting, provides a general discussion
of the current module.
The second, a one-hour meeting, is a problem session
during which the current assignment is discussed.
Students are expected to bring questions about the
assignment to that class and to have read the published paper
associated with the module. Discussion of the paper is intended to help students learn
how to read such papers, how to extract the most important
ideas, and how to read the graphs and tables in them.
Undergraduate
Teaching Assistants
Many of the students need a lot of help to complete
the modules successfully and since they are proceeding at
their own pace, it is important that help be available on
a regular basis. It
is obviously not possible to provide such extensive help alone.
I select about a dozen undergraduate students each
year to serve as teaching assistants.
These undergraduates are not paid for this work but
register in an independent study course for which they receive
credit. They
are selected on the basis of their excellent work in a statistics
course the previous year and their promise as teachers.
Their independent study course, Teaching
Statistics: 1 and 2, consists of further improving their
knowledge of statistics and developing their skills as teachers.
They are expected to formulate a set of personal goals
at the beginning of the term and each meets with me at the
end to discuss what has been achieved, essentially a self-evaluation.
The grade they receive is based on the quality of their
work. Given the
careful preselection, I make the assumption that all TAs will
earn a grade of A. Prompt
feedback is provided to the very few who appear not to be
meeting the expected standard.
Consistent with the print intensive Keller Plan a TA
Handbook is prepared for the TAs addressing expectations
for them and providing supporting material.
The
TAs do learn a great deal.
While their administration of the quizzes is vital
to the course structure, their learning is primarily in the
context of tutoring and preparing for it.
We meet together as a group for two hours each week
to discuss the content of upcoming modules, review the associated
assignments and prepare for the most likely student questions.
The TAs each spend three hours per week in the lab
responding to questions on the practice quizzes which accompany
each module (but not on the actual quizzes), on the assignments
and on the material generally.
They are not expected to “know everything” and are
encouraged to help students find their own answers when possible.
A common theme in TA reports of their own learning
is the fact that participating in these discussions with students
and preparing for them has led to a much deeper level of understanding
than they had anticipated with respect to concepts they thought
they had already learned.
Do
the TAs facilitate student learning?
The students certainly believe they do, giving high
marks to this aspect of the course.
My own impression is that being students themselves,
the TAs are able to empathize well with the statistics students
and thereby appreciate and respond well to their questions.
Many students who would feel awkward about asking questions
in class or who feel their questions are too elementary to
bring there are willing to approach a TA for assistance.
More surprising to me than the quality of their tutoring
is the extent to which the TAs have been willing to invest
themselves in the creation and development of resources to
assist the students.
Several of these have been incorporated into the regular
material of the course, for example a math preparation module
by Linda Di Francesco; the Mathematics Preparation Manual
by Alison Longhorn and Darryle-Anne Zink; Discover Graphing!,
a graphing manual by Deborah Buehler; and Steps to
Statistics, a set of general tips for students by Rose
Monacelli. The
TAs offer an Orientation Workshop and a Calculator Workshop
in the first weeks of the course and it is common for one
or more TAs to construct and offer materials or workshops
on other topics during the year.
The weekly meetings often feature lively discussions
of suggestions for improving not only the tutoring, but also
the course in general.
The
Critical Features?
The
Keller Plan is a complex system and a good deal of commitment
to it is necessary to implement and maintain it in its full
form. Such commitment
does not come without cost, and I have sometimes speculated
whether my own commitment is not instead a single-mindedness
that has come at the expense of broader teaching possibilities.
While I am satisfied with my choice, I am not surprised
that others might ask which features of the Keller Plan are
most important for its success, seeking to gain the benefits
of those features without the complexities of the system as
a whole. I do
think it is fair to say that the course has been successful.
It is well-known and popular with the students, despite
the feared subject matter.
Students appreciate the great amount of help available
and the flexibility in writing quizzes.
I believe all the features described above have contributed
to the success of my course, but which do I believe are the
most critical? Perhaps
the difficulty of implementing these critical features explains
the apparent demise of the Keller Plan.
- First
and foremost, I believe the course could only have carried
on with the superb set of quizzes created by Elke Weber
from 1979 to 1982.
She started her work as an undergraduate and completed
them while a graduate student at Harvard.
(She is now a full professor at Columbia University.)
These multiple-choice quizzes with many parallel
forms are truly exceptional.
Each question is based on one of the objectives for
the module. These objectives, and a practice quiz, appear in Weber’s
study guide. The
questions were derived in part from a large set of actual
answers to earlier completion-form quizzes, but, more important,
were created following a rational model of how students
answer these questions.
Each wrong answer is based on a particular misconception
or error. Only
by having quizzes of sufficient difficulty and depth can
a university-level Keller course be justified. I was fortunate that Elke Weber was around when I started
the course and my textbook.
- As
already noted above, I make a distinction between regular
modules and review modules.
If a course has only regular modules, it is too easy,
and students aren’t being challenged by the depth and breadth
of review modules. A university course has to have more
than just regular modules, even with the exceptional quizzes
that Weber created.
- The
course materials are very extensive and thorough.
They include the textbook I wrote (Herzberg, 1983),
Weber’s study guide (Weber, 1983), and a 200+ page course
kit that I revise each year.
- The
TAs are very carefully selected and I keep in close touch
with them throughout the course.
I observe them in the lab and meet with them weekly.
In short, they have supervision together with considerable
independence. They
do not work in a prescribed mode in the lab, but rather
use their intelligence and creativity to develop new teaching
strategies on their own or by observing other TAs.
It is always very satisfying for me to work with
the TAs in my course.
The 267 TAs I have had in the past twenty-five years
are among the finest people I have known.
Eighty of them returned for a reunion in May, 1999,
and it was very moving to hear many of them speak of their
work in the course as a highlight of their undergraduate
programme. I
must say that they have been my teacher as much as I have
been theirs.
I
acknowledge some disappointment that the course and, perhaps,
the Keller Plan itself are not likely to continue beyond my
retirement in a few years.
I will be very pleased if my account encourages some
instructors to explore seriously the features of the Keller
Plan which, in my career, I have found so beneficial to my
students.
References
Gallup,
H. Personalized
System of Instruction: Behavior modification in education.
A presentation to the Lafayette College Psychology Club, Easton,
PA., April 11 (1995).
Herzberg,
P.A. Principles of Statistics. New York: Wiley.
Reprinted (1989) Malabar, FL: Krieger.
Keller, F.S. (1968). Goodbye, Teacher...Journal of Applied Behavioral
Analysis, 1 (1), pp. 79-89.
Kulik,
J.A., Kulik, C., & Carmichael, K. (1974). The Keller
Plan in Science Teaching. Science, 183, pp. 379-383
Kulik,
J.A., Kulik, C., & Cohen, P.A. A meta-analysis of outcome
studies of Keller's personalized system of instruction.
American Psychologist. 34, (1979), pp. 307-318.
Weber,
E.U. (1983). Study guide for principles of statistics.
New York: Wiley. Reprinted (1989) Malabar, FL: Krieger.
Acknowledgment
I
wish to thank Ron Sheese for proposing this article and for
his substantial work on it. His contributions to the course
described in the article extend back to 1976. He approached
me that spring, saying the Keller Plan that he had explored
in his introductory course appeared to be especially suitable
for a statistics course. We developed the course together
in the first few years. Thanks to Ron, I found my calling—teaching
a statistics course following the Keller Plan model.
|
