If you reply to this very long (48 kB) post
please don't hit the reply button unless you
prune the copy of this post that may appear in
your reply down to a few relevant lines,
otherwise the entire already archived post may be
needlessly resent to subscribers.

ABSTRACT: I respond to criticisms of my 14 Feb
2007 post "Re: study after study shows children
learn better with hands-on experience than with
direct instruction" by Derek Muller and John
Sweller. Three main points are: (1) even though
the "direct instruction"(DI) advocated by Klahr
and Nigam (KN) and Kirschner, Sweller, and Clark
(KSC) is similar in many respects to
"interactive engagement" methods found by physics
education researchers to be relatively effective
in promoting students' conceptual understanding,
there is still a danger that the work of
KN/KSC will be misinterpreted as supporting the
drill & practice, teach 'em the facts,
passive-student lecture pedagogy of DI
extremists, as has already occurred for the
research of KN as discussed by Klahr and Li. (2)
If Sweller is correct in his claim that
(paraphrasing) "students who study worked
examples or partially worked examples of problems
perform better on transfer tests than students
who work through the problems themselves," then
the Hellers' "Cooperative Group Problem Solving"
program and Problem-Based Learning programs
generally need to be revamped so that students
consider primarily worked or partially worked
problems. (3) I contest the implication of
Kirschner, Sweller, and Clark that (a) all
Constructivist, Discovery, Problem-Based,
Experiential, and Inquiry-Based Teaching" is
"minimally guided" and (b) constructivist-based
teaching has failed.

In response to my PhysLrnR post of 14 Feb 2007
titled "Re: study after study shows children
learn better with hands-on experience than with
direct instruction" [Hake (2007a)], Derek Muller
(2007) responded as follows [my inserts at ". . .
. . . .[[insert]]. . . . . .":

"About Hake's post, I think two points need
making. . . [[changing Muller's 1, 2 to I, II] to
allow standard outlining format]. . . .
I. Some of the quotes lack empirical support, and
II. The language Hake uses surrounding direct
instruction (and Direct Instruction) and
interactive engagement aggravate the
communication failure that he seems to abhor."

Considering I and II in order:
I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I
I. Some of the quotes lack empirical support.
[[Muller wrote:]]
"I forwarded Hake's post to John Sweller. . . . .
. . . .
[[<http://education.arts.unsw.edu.au/staff/staff.php?first=John&last=Sweller>
, <http://en.wikipedia.org/wiki/John_Sweller>,
<http://en.wikipedia.org/wiki/Cognitive_load>]].
. . . . and I include some excerpts from his
response below. . . .[ [Sections A - D below.]]

AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  [[Sweller wrote]]:
"From the Panzer report, Hake quotes:
>Smartboards and wireless technology are just as commonplace in school
>libraries as classrooms, said library director Sylvia Rockwood....
>"STUDY AFTER STUDY SHOWS THAT CHILDREN LEARN BETTER WITH MORE
>HANDS-ON LESSONS THAN DIRECT INSTRUCTION," Rockwood said.
First, hype has always accompanied new
technologies like smartboards to the classroom -
but technology has a track record of failing to
live up to expectations (Clark 1983, Cuban 1986,
Clark 1994). A recent review of smartboards
found little solid supporting research (Smith et
al. 2005)."

Three points:
1. Rockwood's claim that "Study after study shows
that children learn better with more hands-on
lessons than direct instruction" is meaningless
without operational definitions of "hands-on" and
"direct instruction." For my guesses as to what
various factions mean by:

(a) "direct instruction" see Hake (2004a, 2004b, 2007c);

(b) "hands-on" instruction see Hake (2004a, 2004c).

2. I have not seen Smith et al. (2005), but the
author's abstract at <http://tinyurl.com/3ywovd>
states: "Although the literature reviewed is
overwhelmingly positive about the impact and the
potential of Interactive White Boards, it is
primarily based on the views of teachers and
pupils. There is insufficient evidence to
identify the actual impact of such technologies
upon learning either in terms of classroom
interaction or upon attainment and achievement."

Unfortunately most educational research yields
insufficient evidence to identify the actual
impact of any pedagogical method on student
learning. As stated in the abstract of "Should
We Measure Change? Yes!" [Hake (2007b)]:

"Formative pre/post testing is being successfully
employed to improve the effectiveness of courses
in undergraduate astronomy, economics, biology,
chemistry, economics, geoscience, engineering,
and physics. But such testing is still anathema
to many members of the
psychology-education-psychometric (PEP)
community. I argue that this irrational bias
impedes a much needed enhancement of student
learning in higher education."

  I wonder if the literature review of Smith et
al. (2005) included the Arizona State University
Modeling Program. Judging from its pre-to-post
test gains [Hake 1998b, Table Ic] the ASU program
is relatively effective in increasing students'
conceptual understanding of Newtonian mechanics,
even despite the use of white boards, a primitive
form of smartboards.

3. I have not seen Clark (1983, 1994), but in
"The No Significant Difference Debate" [Hake
(2004d)], I wrote [bracketed by lines "HHHHHH. .
. . ."; FOR THE REFERENCES SEE Hake (2002)]:

HHHHHHHHHHHHHHHHHHHHH
I get the impression that those involved in "The
No Significant Difference Debate" (primarily
education specialists and psychologists) are
oblivious of the effective use of computers and
media in physics education, e.g.:

(a) computerized classroom communication systems
[see, e.g., Dufresne et al. 1996, Mazur 1997,
Abrahamson 1998, Burnstein & Lederman 2001,
Better Education 2001];

(b) properly implemented microcomputer-based labs (Thornton and Sokoloff 1990);

(c) interactive computer animations for use AFTER
hands- and minds-on experiments and Socratic
dialogue (Hake 2001a);

(d) computer implemented tutorials (Reif & Scott 1999);

(e) "Just-In-Time Teaching" (Novak et al. 1998, 1999; Gavrin 2001).

For the above references see Hake (2002), Lesson
#2 "The use of Interactive Engagement and/or
high-tech methods, by themselves, does NOT INSURE
superior student learning"
HHHHHHHHHHHHHHHHHHHHH


BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[[Sweller wrote]]:
  "Second, contrasting hands on with direct
instruction excludes the possibility that one
might make use of the other - direct instruction
with hands on experimentation as is Klahr and Li
(2005), or hands on lessons that incorporate
teacher led introductions and scaffolding.
Furthermore, maybe Rockwood meant direct
instruction as in lecturing, rather than as
envisioned by Kirschner, Sweller, and Clark
(2006) or Klahr and Nigam (2004)."

Here Sweller is contesting Rockwood and Panzer
(2006), not Hake. In "IA1" above, I make
essentially the same point that Sweller makes -
Rockwood's statement is meaningless without
operational definitions of "hands-on" and "direct
instruction." BTW: I think Sweller might have
meant to write ". . . .contrasting hands on with
direct instruction excludes the possibility that
one might make use of the other - direct
instruction with hands on experimentation as is
Klahr and Nigam (2004). . . . . .."


CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
[[Sweller wrote]]:
"Another quote that should not go unexamined is from Joe Layng (2007):
>For example, the "worked example" effect is limited to only the problems
>worked. No general problem solving skills are learned and students seldom
>become independent problem solvers, and this applies to even high
>performing students.
The research on this topic has shown that
students who study worked examples or completion
problems (partially worked examples) perform
better on transfer tests (these are new,
different problems - not the ones studied) than
students who work through the problems themselves
[Sweller et al. (1998), Sweller (1988)]."

I have not seen Sweller et al. (1998) or Sweller
(1988) but if the remarkable finding that:

  "students who study worked examples or
completion problems (partially worked examples)
perform better on transfer tests (these are new,
different problems - not the ones studied) than
students who work through the problems themselves"

is, in fact, correct then it would seem that programs featuring:

(a) "Problem Based Leaning" [see e.g. UDL (2007),
Woods (1998, 2005), PBL (2007], and

(b) Pat & Ken Heller's [UMPERG (2007)]
"Cooperative Group Problem Solving" and
"Context Rich Problems,"

need to be revamped so that students consider
primarily worked or partially worked problems.


DDDDDDDDDDDDDDDDDDDDDDDDDDDDD
[Muller writes] "Sweller's argument (which I
paraphrase here) is that general problem solving
skills as such cannot be taught or learnt. The
contention is that Chess masters are not better
than novices because they have learned some
fundamental conceptual rules governing effective
play but because they have seen many many board
configurations . . . [[see e.g., Chase & Simon
(1973), de Groot (1978), Ericsson & Smith (1991),
Ericsson et al. (1993), Charness et al. (1996),
and Ericsson (1996a)]]. . . . They draw on this
wealth of domain specific knowledge to make
decisions about what moves to play. You may
contend that this analogy is not a good one for
physics problem solving - I include Sweller's own
explanation in the post-script."

For a more complete explanation of Sweller's work
on problem solving than Muller's (2007)
post-script (containing Sweller's criticism of
attempts to teach *general* - not physics -
problem solving skills), see Cooper (1998) and
the section "Worked examples" in Kirschner et al.
(2006).

Regarding chess expertise not being a good
analogy for physics problem solving, in an
Educational Research issue devoted to expertise
[EdRes (2003)], psychologist Robert Sternberg
(2003a) wrote:

"Ericsson (1996) and Ericsson, Krampe, and
Tesch-Römer (1993) emphasize the role of
deliberate practice in acquiring expertise. Such
practice is indeed important in many fields,
especially in performance-based domains such as
music, athletics, or chess. It appears, however,
to be necessary but not sufficient in other kinds
of domains. Becoming an expert physicist,
composer, or teacher, for example, seems to
require a blend of creative (generate ideas),
analytical (evaluate the ideas), and practical
thinking (make the ideas work and convince
others of their worth) that goes substantially
beyond deliberate practice [Sternberg (2003b)]."


II-II-II-II-II-II-II-II-
II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II
II. "The language Hake uses surrounding direct
instruction (and Direct Instruction) and
interactive engagement aggravate the
communication failure that he seems to abhor."

Three points A, B, C below:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
[[Muller writes]]:
"My second point is that Hake seems to play into
the communication failure rather than trying to
ameliorate it. He quotes his abstract . . .
.[[Who's "he"? What "abstract"? A reference
freak's definitive wording might have been: "Hake
(2007a) quotes the abstract of Hake (2007d)]]
>I argue that (a) KSC's "direct instruction" approximates the guided
>"interactive engagement," found to be relatively effective in promoting
>student learning by physics education researchers

[[Sweller writes]]
  "Since we also believe "direct instruction". . .
.[[as defined by Kirschner et al. (2006) and
called by Hake "interactive engagement" (IE)]]. .
. . is effective, whatever it is called, there
seems no dispute. We only have a dispute with
people who expect learners to work it all out by
themselves instead of being shown."

"Instead of being shown"? I don't think the
"guided construction" of most IE methods is the
same as "being shown." This is reminiscent of
the passage in Kirschner et al. (2006):

"Direct instructional guidance is defined as
providing information that fully explains the
concepts and procedures students are required to
learn as well as learning strategy support that
is compatible with human cognitive architecture.
Learning, in turn, is defined as a change in
long-term memory."

In "Re: 'Why Minimal Guidance During
Instruction Does Not Work' " [Hake (2007d)], I
commented on the above passage as follows "It
would be a great boon to introductory physics
instructors if Kirchner et al. could tell them
how to "provide information that fully explains
the concepts and procedures" relative to e.g.,
Newton's Second Law of Motion." A copy of Hake
(2007d) was sent to Richard Clark, but so far
I've received no response to the above request.

David Meltzer (2007) has also questioned what
Sweller means by "being shown," writing:

MMMMMMMMMMMMMMMMM
I want to focus here only on two excerpts from
Sweller's remarks as quoted by Muller, as follows:
"Sweller responds: 'We only have a dispute with
people who expect learners to work it all out by
themselves instead of being shown.'"
"Sweller responds: 'Given our emphasis on worked
examples and, from recall, no mention of lectures
(which are only one form of direct
instruction)...'"

So Sweller emphasizes the terms "worked examples" and "being shown."

In the paper by Kirschner, Sweller, and Clark, it
is stated that "[McCray et al. (2003)]...amply
document the lack of evidence for unguided
approaches and the benefits of more strongly
guided instruction." The report of McCray et al.
in turn states that "Effective instructional
strategies...for producing conceptual
understanding for most students require
situations that demand active intellectual
engagement, such as tutorials, small group
learning, hands-on activities, case studies, and
problem-solving exercises with appropriate
scaffolding...(i.e., support and guidance in
learning specific concepts or tasks)."

So one must presume that by "worked examples,"
"being shown," and "more strongly guided"
instruction, Sweller intends to include or imply
"situations that demand active intellectual
engagement, such as tutorials, small group
learning, hands-on activities, case studies, and
problem-solving exercises with appropriate
scaffolding." I don't know for certain whether he
actually DOES intend to imply those things, but
it seems that would be a consistent
interpretation of the various statements.
MMMMMMMMMMMMMMMMM

A similar questioning of Kirschner et al. (2006)
by Meltzer was included in Hake (2007d). A copy
was sent to Richard Clark, but so far Clark has
not responded.


BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[[Muller writes]]
"Hake continues. . . [[it's not clear what Muller
regards Hake as continuing *from*, but Muller
proceeds to quote from the abstract of "Re:
study after study shows children learn better
with hands-on experience than with direct
instruction" [Hake (2007a)]]. . . . . .:
'Will articles such as that by Kirschner et al.
(2006) and Klahr and Nigam (2004) serve to move
American education more towards direct
instruction, i.e., passive student lecturing?'

Here Hake is equating direct instruction (after
citing KSC/KN) with passive student lecturing -
even though just a second ago, he was equating
KSC/KN's direct instruction with interactive
engagement! It seems like Hake agrees with
KSC/KN - Hakes considers their methods to be
interactive engagement - yet Hake fears KSC/KN
will be misinterpreted leading to passive student
lecturing. Why doesn't he make this clear
instead of contributing to the
misinterpretations?"

Judging from Muller's comments, despite my
continual harping on the widespread
misinterpretation of Klahr & Nigam (2004), I
should have been more explicit and stated
something like:

I FEAR THAT KIRSCHNER ET AL. (2006) AND KLAHR AND
NIGAM (2004) WILL BE MISINTERPRETED, AND THUS
SERVE TO MOVE AMERICAN EDUCATION MORE TOWARDS
PASSIVE STUDENT LECTURING.

In fact, as indicated in Hake (2005) and Klahr
and Li (2005), the research of Klahr and Nigam
(2004) has already been widely misinterpreted as
favoring passive student lecturing.

In Hake (2005) I listed such misinterpretation as
being one of the major reasons why NCLB may
promote what most people [but not Kirschner et
al. (2006) or Klahr and Nigam (2004)] call direct
instruction, vis., passive student lecturing. I
wrote [SEE THAT ARTICLE FOR REFERENCES OTHER THAN
Klahr and Nigam (2004)]:

HHHHHHHHHHHHHHHHHHHHH
IV. BUT THE RESEARCH (SECTIONS II & III) SHOWING
THE SUPERIORITY OF INTERACTIVE ENGAGEMENT (IE)
AND GUIDED INQUIRY (GI) METHODS TO DIRECT SCIENCE
INSTRUCTION IS IGNORED

A. In California
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

B. In the U.S.
There are at least seven reasons why Direct
Science Instruction (DSI) threatens to
predominate nationally under the aegis of the No
Child Left Behind Act:
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
2. The heavily publicized [Adelson (2004),
Begley (2004 a,b), Cavenaugh (2004 a,b ), Tweed
(2004a,b), USDE (2004)] research of Klahr and
Nigam (KN) (2004) is widely misinterpreted as
demonstrating the general superiority of direct
instruction.

Many DI enthusiasts [e.g., Mathematically Correct
(2005), Bishop (2004)] misleadingly imply that
KN's research demonstrates the superiority of
their favored drill and practice methods over any
other type of science instruction. Commenting on
Klahr & Nigam, Bishop (2004) opines:
"Deliberate, direct instruction is more
effective yet again. Surprise, surprise.
Replicate a replicable experiment and you get the
same results. It's the scientific approach."

But the "guided inquiry" and "interactive
engagement" methods of Sections II and III above
have little to do with KN's strawman of EXTREME
Discovery Learning (EDL), in which there is
almost no teacher guidance. KN showed, not
surprisingly, that EDL is inferior to "direct
instruction" for increasing third and fourth grade
children's effective use of the control of
variables strategy, a so-called "process skill."
It might be interesting for Klahr & Nigam to
extend their study to more guided forms of
"discovery learning" and to children's
acquisition of "operative knowledge" [Arons
(1983)].

Consistent with the above, Adelson (2004) wrote:
"Psychologist Rich Shavelson, professor of
education and (by courtesy) psychology at
Stanford University, notes that totally unguided
discovery of the type used in [KN's] study is
rarely used in the classroom." Still, he says,
"This study uses a strong research design. I'd
like to see a replication with [the more typical]
guided discovery. Plus, the extent to which
results would travel to classrooms with varying
teacher quality, opportunity to learn, et cetera,
has yet to be found out."

Klahr himself is not the DSI radical painted by
DI zealots such as Bishop (2004) and
Mathematically Correct (2005)]. Cavenaugh (2004a)
wrote: "While David Klahr . . . . believes that
complex science lessons often require a more
direct type of instruction, he also cautions
against too rigid an adherence to either method
by teachers or administrators. 'It depends on
what's being taught,' Mr. Klahr, a psychology
professor at Carnegie Mellon, said in an
interview."

I agree with David Klahr's caution that the
appropriate method of science instruction depends
on what's being taught. Teachers, to be
effective, need to use different approaches
(e.g., didactic lectures, coaching, collaborative
discussions, and Socratic dialogue) to fit the
classroom occasions and diverse natures of their
students. Each
method has its strengths and weaknesses for each
type of student, but in the hands of a *skilled
teacher* each can be made to compliment the other
methods so as to advance *every* student's
learning. A skilled teacher might *lecture* on
material that can be rote memorized, *coach*
skills such as typing or playing a musical
instrument, and use *Socratic dialogue or
collaborative discussions* (or some other
"interactive engagement" method) to induce
students to construct their conceptual
understanding of difficult counter-intuitive
material such as Newton's Laws.
HHHHHHHHHHHHHHHHHHHHH

In my view, the danger of misinterpretation is
enhanced in the case of Kirschner et al. (2006)
by their seemingly non sequitur title "Why
Minimal Guidance During Instruction Does Not
Work: An Analysis of the Failure of
Constructivist, Discovery, Problem-Based,
Experiential, and Inquiry-Based Teaching."

Since when has all Constructivist, Discovery,
Problem-Based, Experiential, and Inquiry-Based
Teaching" been "minimally guided"?

Since when has there been a failure of constructivist-based teaching?

In my opinion, IE methods [evidently known as
"direct instruction methods" by Kirschner et al.
(2006) and Klahr and Nigam (2004)] are, for the
most part, consistent with the "Knowledge-Based
Constructivism" espoused by cognitive scientists
Resnick & Hall (1998) who wrote [bracketed by
lines R&H-R&H-R&H. . ."; *emphasis* in the
original]:

R&H-R&H-R&H-R&H-R&H-R&H
Since about 1960, beginning with the publication
of Newell and Simon's (1972) landmark studies of
human problem solving, a body of
cognitive-science research has focused on the
nature of the mental processes involved in
thinking and learning. Hundreds of scholars have
been involved, using varied methods and examining
cognitive processes in people of all ages and
social conditions. Despite the variety of
approaches and the many theoretical differences
among congnitive scientists, it is possible to
outline a few important points of fundamental
agreement that we can take as the new core theory
of learning [Resnick (1987), Bruer (1993)].

Broadly speaking, cognitive science confirms
Piaget's claim that people must *construct* their
understanding; they do not simply register what
the world shows or tells them, as a camera or a
tape recorder does. To "know" something, indeed
even to memorize effectively, people must build a
mental representation that imposes order and
coherence on experience and information. Learning
is interpretive and inferential; it involves
active processes of reasoning and a kind of
'talking back' to the world - not just taking it
as it comes. Competent learners engage,
furthermore, in a great deal of self-management
of their cognitive processes, that is, in forms
of cognition known as *metacognitive* and *self
monitoring*.

This much sounds like the child-centered, process
theories of education. Early on, however,
cognitive scientists found that they could not
account for problem solving and learning without
attending to what people already *knew*. . . . .
In every field of thought, cognitive scientists
found that knowledge is essential to thinking and
acquiring new knowledge - in other words to
learning. . . The repeated findings about the
centrality of knowledge in learning make perfect
sense for a constructivist theory of learning,
because one has to have something with which to
construct. But they turn out to be almost as
challenging to Piagetian or Deweyan theories of
pedagogy as to Thorndikean ones. This is because
they insist that knowledge - *correct* knowledge
- is essential at every point in learning. And
they make it impossible to suggest that education
for the information age should not trouble itself
with facts and information, but only with
processes of learning and thinking. What we know
now is that just facts alone do not constitute
true knowledge and thinking power, so thinking
processes cannot proceed without something to
think about. Knowledge is in again, but alongside
thinking, indeed, intertwined with it, not
instead of thinking. So although it is essential
for children to have the experience of
discovering and inventing, their experience must
be of one of disciplined invention, that is, by
established processes of reasoning and logic.

[The above advocated] *Knowledge-based
Constructivism*, taken seriously, points to a
position that can moderate the century-long
polarity between passive drill pedagogies and
child-centered discovery pedagogies.
R&H-R&H-R&H-R&H-R&H-R&H


CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
[[Sweller wrote]]:
"The suggestion that "direct instruction" is
linguistically equivalent to "long lectures" is
novel. Given our emphasis on worked examples and,
from recall, no mention of lectures (which are
only one form of direct instruction), I doubt any
reader really believes we were talking of long
lectures. In any case, if it is all a linguistic
misunderstanding, it can be easily cleared up and
we presumably all agree."

I searched Kischner et al. (2006) for "lecturers" and obtained one hit:

KSC-KSC-KSC-KSC-KSC-KSC-KSC
In an attempt to rescue medical students from
lectures and memory-based recall exams,
approximately 60 medical schools in North America
have adopted PBL in the past two decades. This
variant of constructivist instruction with
minimal guidance, introduced at the McMaster
University School of Medicine in 1969, asks
medical students to work in groups to diagnose
and suggest treatment for common patient
symptoms. PBL student groups are supervised by a
clinical faculty member who is directed not to
solve problems for the students but instead to
offer alternatives and suggest sources of
information.

The best known survey of the comparisons of PBL
with conventional medical school instruction was
conducted by Albanese and Mitchell (1993). Their
meta-analysis of the English language literature
of the effectiveness of PBL produced a number of
negative findings concerning its impact,
including lower basic science exam scores, no
differences in residency selections, and more
study hours each day. They reported that although
PBL students receive better scores for their
clinical performance, they also order
significantly more unnecessary tests at a much
higher cost per patient with less benefit. There
was an indication in their review that increased
clinical practice evaluation scores may have been
due to the fact the PBL students are required to
spend more time in clinical settings.
KSC-KSC-KSC-KSC-KSC-KSC-KSC

For more positive views of PBL scan Woods (2005)
and the material at the University of Delaware
Problem-Based Learning Site
<http://www.udel.edu/pbl/>; for example listed at
<http://www.udel.edu/pbl/articles.html> is Duch
et al. (2001).


In any case, no one that I know of has set
"direct instruction" equal to "long lectures."

Judging from Sweller's quotes as relayed by
Muller, Sweller has not scanned my critique "Re:
'Why Minimal Guidance During Instruction Does
Not Work' " [Hake (2007d)] of Kirschner, Sweller,
& Clark (2006). Had he done so would have found
my guesses at what various factions mean by the
ambiguous term "direct instruction." There I
guessed that most physics education researchers
set "direct instruction" equal to traditional
PASSIVE student lectures, recipe labs, and
algorithmic problem sets.

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<[log in to unmask]>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

"Education is the acquisition of the art of the utilization of
knowledge. This an art very difficult to impart. We must beware of
what I will call 'inert ideas' that is to say, ideas that are merely
received into the mind without being utilized or tested or thrown
into fresh combinations."
    Alfred North Whitehead (1967) in "The Aims of Education."

REFERENCES
Albanese, M., & Mitchell, S. (1993).
Problem-based learning: A review of the
literature on its outcomes and implementation
issues. AcademicMedicine 68: 52-81.

Bruer, J.T. 1993. "Schools for Thought: A
Science of Learning in the Classroom." MIT Press,
information at
<http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=5900>.

Chase, W.G. & H.A. Simon. 1973. "Perception in
chess," Cognitive Psychology 4 (1): 55-81.

Charness, N., R.T. Krampe, & U. Mayr. 1996. 'The
role of practice and coaching in entrepreneurial
skill domains: An international comparison of
life-span chess skill acquisition,' in in
Ericsson (1996, pp. 51-80).

Clark, R. E. 1983. "Reconsidering Research on
Learning from Media," Review of Educational
Research 53(4): 445-459.

Clark, R. E. 1994. "Media Will Never Influence
Learning," Educational Technology, Research and
Development 42(2): 21-29.

Cooper, G. 1998. "Research into Cognitive Load
Theory and Instructional Design at UNSW," online
at
<http://educationnew.arts.unsw.edu.au/staff/sweller/clt/>.

Cuban, L. 1986. "Teachers and machines : the
classroom use of technology since 1920."
Teachers College Press.

de Groot, A. 1978. "Thought and Choice in
Chess." Mouton. This ground breaking research
was originally published in 1946.

Duch, B., S. Gron, and D. Allen. 2001. "The Power
of Problem-Based Learning." Stylus Publishing,
information at
<http://www.styluspub.com/Books/BookDetail.aspx?productID=44647>.

EdRes. 2003. AERA Educational Researcher,
November theme issue on expertise, online at
<http://www.aera.net/publications/?id=400>

Ericsson, K.A. & J. Smith, eds. 1991. "Toward a
General Theory of Expertise: Prospects and
Limits." Cambridge University Press, information
at
<http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521406123>.

Ericsson, K.A., R.T. Krampe, & C. Tesch-Römer.
1993. "The role of deliberate practice in the
acquisition of expert performance," Psychological
Review 100: 363-406.

Ericsson, K.A. 1996a. 'The acquisition of expert
performance: An introduction to some of the
issues," in Ericsson (1996b, pp. 1-50).

Ericsson, K.A. ed. 1996b. "The Road to
Excellence: The Acquisition of Expert Performance
in the Arts and Sciences, Sports, and Games."
Erlbaum, information at
<http://tinyurl.com/2fwb75>. See also Ericsson's
list of publications at
<http://www.psy.fsu.edu/> // "People" // "Faculty
Listing" // "K. Anders Ericsson" // "Dr.
Ericsson's Home Page" (where "//" means "click
on").

Ericsson, K.A. 2002. A brief summary and
description of the Acquisition of Expert
Performance and how this is explained by
Deliberate Practice is online at
<http://www.psy.fsu.edu/faculty/ericsson/ericsson.exp.perf.html>

Hake, R.R. 1998a. "Interactive-engagement vs
traditional methods: A six thousand-student
survey of mechanics test data for introductory
physics courses," Am. J. Phys. 66: 64-74; online
at
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf>
(84 kB).

Hake, R.R. 1998b. "Interactive-engagement methods
in introductory mechanics courses," online at
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf>
(108 kB) - a crucial companion paper to Hake
(1998a).

Hake, R.R. 2002. "Lessons from the physics
education reform effort," Ecology and Society
5(2): 28; online at
<http://www.ecologyandsociety.org/vol5/iss2/art28/>.
Ecology and Society(formerly Conservation
Ecology) is a free online "peer-reviewed journal
of integrative science and fundamental policy
research" with about 11,000 subscribers in about
108 countries.

Hake, R.R. 2004a. "Will the NCLB Tend to
Propagate California's Direct Science Instruction
Throughout the Entire Nation?" online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0501&L=pod&O=D&P=12783>.
Post of 14
Jan 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J, AERA-K, AERA-L, APPhysics,
ASSESS, Biopi-L, Chemed-L, Edstat-L, EvalTalk,
Math-Learn, Math-Teach, Phys-L, Physhare,
PhysLrnR, POD, & STLHE-L.

Hake, R.R. 2004b. "Re: Back to Basics vs. Hands-On Instruction" online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0402&L=pod&P=R17377>.
Post of 29 Feb 2004 17:57:25-0800 to AERA-K,
AP-Physics, Biopi-L, Chemed-L, EvalTalk,
Math-Learn, Math-Teach, Phys-L, PhysLrnR,
Physhare, PHYSOC, and POD.

Hake, R.R. 2004c. "Re: Back to Basics vs. Hands-On Instruction" online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0402&L=pod&P=R13581>.
Post 24 Feb 2004 17:45:19-0800 to AERA-K,
AP-Physics, Biopi-L, Chemed-L, FYA-List,
Math-Learn, Math- Teach, Phys-L, PhysLrnR,
Physhare, and POD.

Hake, R.R. 2004d. "The No Significant Difference Debate," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0401&L=pod&P=R6235&I=-3>.
Post of 22 Jan 2004 09:11:06-0800 to PhysLrnR,
POD, and STLHE-L.

Hake, R.R. 2005. "Will the No Child Left Behind
Act Promote Direct Instruction of Science?" Am.
Phys. Soc. 50: 851 (2005); APS March Meeting, Los
Angles, CA. 21-25 March; online at
<http://www.physics.indiana.edu/~hake/WillNCLBPromoteDSI-3.pdf> (256 kB).

Hake, R.R. 2006. "Possible Palliatives for the
Paralyzing Pre/Post Paranoia that Plagues Some
PEP's" [PEP's = Psychometricians, Education
specialists, and Psychologists], Journal of
MultiDisciplinary Evaluation, Number 6, November,
online at
<http://evaluation.wmich.edu/jmde/JMDE_Num006.html>.

Hake, R.R. 2007a. "Re: study after study shows
children learn better with hands-on experience
than with direct instruction," online at
<http://tinyurl.com/2tzsah>. Post 14 Feb 2007
16:21:34-0800 to PhysLrnR and ScListserv.

Hake, R.R. 2007b. "Should We Measure Change?
Yes!" download directly by clicking on
<http://www.physics.indiana.edu/~hake/MeasChangeS.pdf>
(2.5 MB). Failure to access that URL probably
means that a new version (T, U, V, W. . .) has
been placed online - it can be accessed as ref.
43 at <http://www.physics.indiana.edu/~hake>. To
appear as a chapter in "Evaluation of Teaching
and Student Learning in Higher Education," a
Monograph of the American Evaluation Association
<http://www.eval.org/>. A severely truncated
version appears at Hake (2006).

Hake, R.R. 2007c." Kirschner, Sweller, & Clark
Revisited," online at
<http://tinyurl.com/36klac>. Post of 18 Jan 2007
21:47:28-0800to PhysLrnR and MathLearn.

Hake, R.R. 2007d. "Re: 'Why Minimal Guidance
During Instruction Does Not Work', " online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0701&L=pod&O=D&P=20140>.
Post of 19-20 Jan 2007 to AERA-A, AERA-B, AERA-C,
AERA-J, AERA-K, AERA-L, ASSESS, EdResMeth,
ITForum (rejected), PBL, POD, RUME, MathTalk,
STLHE-L, TIPS, PsychTeacher (rejected),
TeachingEdPsych.

Kirschner, P. A., J. Sweller, & R.E. Clark. 2006.
"Why Minimal Guidance During Instruction Does
Not Work: An Analysis of the Failure of
Constructivist, Discovery, Problem-Based,
Experiential, and Inquiry-Based Teaching."
Educational Psychologist 41(2): 75-86;
online at
<http://www.cogtech.usc.edu/publications/kirschner_Sweller_Clark.pdf>
(176 kB).

Klahr, D. & M. Nigam. 2004. "The equivalence of
learning paths in early science instruction:
effects of direct instruction and discovery
learning," Psychological Science 15(10): 661-667;
online at
<http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>,
click on ""Cognition & Instruction." For a
discussion of widespread misinterpretation of
this paper see, e.g., "Will the No Child Left
Behind Act Promote Direct Instruction of
Science?" [Hake (2005)], and "Cognitive Research
and Elementary Science Instruction: From the
Laboratory, to the Classroom, and Back" [Klahr &
Li (2005)].

Klahr, D. & J. Li. 2005. "Cognitive Research and
Elementary Science Instruction: From the
Laboratory, to the Classroom, and Back," Journal
of Science Education and Technology 14(2):
217-238; online at
<http://www.psy.cmu.edu/faculty/klahr/personal/pdf/Klahr_Li_2005.pdf>
(536 kB).

Layng, T.V.J. 2007. "Re: study after study shows
children learn better with hands-on experience
than with direct instruction," ScListserv post of
1 Feb 2007 08:59:15-0800; online at
<http://tinyurl.com/35f8w9>.

McCray, R.A., R.L. DeHaan, J.A. Schuck, eds.
2003. Improving Undergraduate Instruction in
Science, Technology, Engineering, and
Mathematics: Report of a Workshop, Committee on
Undergraduate STEM Instruction, National Research
Council, National Academy Press; online at
<http://www.nap.edu/catalog/10711.html>. For
references to Hake's criticism of this report see
Hake (2007d).

Meltzer, D. 2007. Re: study after study shows
children learn better with hands-on experience
than. . . . .[with direct instruction]. . .,"
PhysLrnR post of 15 Feb 2007 23:37:19-0800;
online at <http://tinyurl.com/ys8h4e>.

Muller, D. 2007. "Re: study after study shows
children learn better with hands-on experience
than . . . . .[with direct instruction]. . .,"
PhysLrnR post of 15 Feb 2007 21:52:20-0700;
online at <http://tinyurl.com/yoo42h>.

Newell, A. & H.A. Simon. 1972. "Human Problem Solving." Prentice Hall.

Panzer, T. 2006. "Students get hi-tech advantage," PublicOpinion, online at
<http://www.publicopiniononline.com/localnews/ci_5115719>.

PBL. 2007. Problem Based Learning Discussion
Group with archives at
<http://www.jiscmail.ac.uk/lists/pbl.html>.

Resnick, L.B. 1987. "Education and Learning to
Think," National Academy Press, online at
<http://www.nap.edu/openbook/0309037859/html/>.

Resnick, L.B. and Hall, M.W. 1998. "Learning
Organizations for Sustainable Education Reform,"
Daedalus 127(4): 89-118.

Smith, H. J., S. Higgins, K. Wall, & J. Miller.
2005. "Interactive whiteboards: boon or
bandwagon? A critical review of the literature,"
Journal of Computer Assisted Learning 21: 91-101.
An author's abstract is online at
<http://tinyurl.com/3ywovd>.

Sternberg, R.J. 2003a. "What Is an "Expert
Student?" Educational Researcher 32(8): 5-9,
online at <http://tinyurl.com/ojh9x>, one of the
articles in EdRes (2003).

Sternberg, R. J. 2003b. "Wisdom, intelligence,
and creativity synthesized." Cambridge University
Press, information at
<http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521802383>.

Sweller, J. 1988. "Cognitive load during problem
solving: Effects on Learning," Cognitive Science
12(2): 257-285.

Sweller, J., J.J.G. van Merrienboer, & F. G. W.
C. Paas. 1998. "Cognitive architecture and
instructional design." Educational Psychology
Review 10(3): 251-296.

UDL. 2007. University of Delaware Problem-Based
Learning Site <http://www.udel.edu/pbl/>.

UMPERD. 2007. University of Minnesota Physics
Education Research and Development on line at
<http://groups.physics.umn.edu/physed/index.html>.
Especially "Cooperative Group Problems Solving"
<http://groups.physics.umn.edu/physed/Research/CGPS/CGPSintro.htm>
and "Context Rich Problems"
<http://groups.physics.umn.edu/physed/Research/CRP/crintro.html>.

Whitehead, A.N. 1967. "Aims of Education, " Free Press, Reissue edition.

Woods, D.R. 1998. "The MPS Program: The McMaster
Problem Solving Program," online at
<http://www.chemeng.mcmaster.ca/MPS/default1.htm>.
For a more recent publication by Don Woods, see,
e.g., Woods (2005).

Woods, D.R. 2005. "Problem-based Learning,
especially in the context of large classes" at
<http://chemeng.mcmaster.ca/pbl/pbl.htm>