If you reply to this very long (48 kB) post 
please don't hit the reply button unless you 
prune the copy of this post that may appear in 
your reply down to a few relevant lines, 
otherwise the entire already archived post may be 
needlessly resent to subscribers.

ABSTRACT: I respond to criticisms of my 14 Feb 
2007 post "Re: study after study shows children 
learn better with hands-on experience than with 
direct instruction" by Derek Muller and John 
Sweller.  Three main points are: (1) even though 
the "direct instruction"(DI) advocated by Klahr 
and Nigam (KN) and Kirschner, Sweller, and Clark 
(KSC)  is similar in many respects to 
"interactive engagement" methods found by physics 
education researchers to be relatively effective 
in promoting students' conceptual understanding, 
there is still a danger that    the work of 
KN/KSC will be misinterpreted as supporting the 
drill & practice, teach 'em the facts, 
passive-student lecture pedagogy of DI 
extremists, as has already occurred for the 
research of KN as discussed by Klahr and Li.  (2) 
If Sweller is correct in his claim that 
(paraphrasing) "students who study worked 
examples or partially worked examples of problems 
perform better on transfer tests than students 
who work through the problems  themselves," then 
the Hellers' "Cooperative Group Problem Solving" 
program and Problem-Based Learning programs 
generally need to be revamped so that students 
consider primarily worked or partially worked 
problems.  (3) I contest the implication of 
Kirschner, Sweller, and Clark that (a) all 
Constructivist, Discovery, Problem-Based, 
Experiential, and Inquiry-Based Teaching" is 
"minimally guided" and (b) constructivist-based 
teaching has failed.

In response to my PhysLrnR post of 14 Feb 2007 
titled  "Re: study after study shows children 
learn better with hands-on experience than with 
direct instruction" [Hake (2007a)], Derek Muller 
(2007) responded as follows [my inserts at ". . . 
. . . .[[insert]]. . . . . .":

"About Hake's post, I think two points need 
making. . . [[changing Muller's 1, 2 to I, II] to 
allow standard outlining format]. . . .
I. Some of the quotes lack empirical support, and
II. The language Hake uses surrounding direct 
instruction (and Direct Instruction) and 
interactive engagement aggravate the 
communication failure that he seems to abhor."

Considering I and II in order:
I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I-I
I. Some of the quotes lack empirical support.
[[Muller wrote:]]
"I forwarded Hake's post to John Sweller. . . . . 
. . . . 
[[<http://education.arts.unsw.edu.au/staff/staff.php?first=John&last=Sweller> 
, <http://en.wikipedia.org/wiki/John_Sweller>, 
<http://en.wikipedia.org/wiki/Cognitive_load>]]. 
. .  . . and I include some excerpts from his 
response below. . . .[ [Sections A - D below.]]

AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  [[Sweller  wrote]]:
"From the Panzer report, Hake quotes:
>Smartboards and wireless technology are just as commonplace in school
>libraries as classrooms, said library director Sylvia Rockwood....
>"STUDY AFTER STUDY SHOWS THAT CHILDREN LEARN BETTER WITH MORE
>HANDS-ON LESSONS THAN DIRECT INSTRUCTION," Rockwood said.
First, hype has always accompanied new 
technologies like smartboards to the classroom - 
but technology has a track record of failing to 
live up to expectations (Clark 1983, Cuban 1986, 
Clark 1994).  A recent review of smartboards 
found little solid supporting research (Smith et 
al.  2005)."

Three points:
1. Rockwood's claim that "Study after study shows 
that children learn better with more hands-on 
lessons than direct instruction" is meaningless 
without operational definitions of "hands-on" and 
"direct instruction."  For my guesses as to what 
various factions mean by:

(a)  "direct instruction" see Hake (2004a, 2004b, 2007c); 

(b) "hands-on" instruction see Hake (2004a, 2004c).

2. I have not seen Smith et al. (2005), but the 
author's abstract at <http://tinyurl.com/3ywovd> 
states:  "Although the literature reviewed is 
overwhelmingly positive about the impact and the 
potential of Interactive White Boards, it is 
primarily based on the views of teachers and 
pupils. There is insufficient evidence to 
identify the actual impact of such technologies 
upon learning either in terms of classroom 
interaction or upon attainment and achievement."

Unfortunately most educational research yields 
insufficient evidence to identify the actual 
impact of any pedagogical method on student 
learning.  As stated in the abstract of "Should 
We Measure Change? Yes!" [Hake (2007b)]:

"Formative pre/post testing is being successfully 
employed to improve the effectiveness of courses 
in undergraduate astronomy, economics, biology, 
chemistry, economics, geoscience, engineering, 
and physics. But such testing is still anathema 
to many members of the 
psychology-education-psychometric (PEP) 
community. I argue that this irrational bias 
impedes a much needed enhancement of student 
learning in higher education."

  I wonder if the literature review of Smith et 
al. (2005) included the Arizona State University 
Modeling Program. Judging from its pre-to-post 
test gains [Hake 1998b, Table Ic] the ASU program 
is relatively effective in increasing students' 
conceptual understanding of Newtonian mechanics, 
even despite the use of white boards, a primitive 
form of smartboards.

3. I have not seen Clark (1983, 1994), but in 
"The No Significant Difference Debate" [Hake 
(2004d)], I wrote [bracketed by lines "HHHHHH. . 
. . ."; FOR THE REFERENCES SEE Hake (2002)]:

HHHHHHHHHHHHHHHHHHHHH
I get the impression that those involved in "The 
No Significant Difference Debate" (primarily 
education specialists and psychologists) are 
oblivious of the effective use of computers and 
media in physics education, e.g.:

(a) computerized classroom communication systems 
[see, e.g., Dufresne et al. 1996, Mazur 1997, 
Abrahamson 1998, Burnstein & Lederman 2001, 
Better Education 2001];

(b) properly implemented microcomputer-based labs (Thornton and Sokoloff 1990);

(c) interactive computer animations for use AFTER 
hands- and minds-on experiments and Socratic 
dialogue (Hake 2001a);

(d) computer implemented tutorials (Reif & Scott 1999);

(e) "Just-In-Time Teaching" (Novak et al. 1998, 1999; Gavrin 2001).

For the above references see Hake (2002),  Lesson 
#2 "The use of Interactive Engagement and/or 
high-tech methods, by themselves, does NOT INSURE 
superior student learning"
HHHHHHHHHHHHHHHHHHHHH


BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[[Sweller  wrote]]:
  "Second, contrasting hands on with direct 
instruction excludes the possibility that one 
might make use of the other - direct instruction 
with hands on experimentation as is Klahr and Li 
(2005), or hands on lessons that incorporate 
teacher led introductions and scaffolding. 
Furthermore, maybe Rockwood meant direct 
instruction as in lecturing, rather than as 
envisioned by Kirschner, Sweller, and Clark 
(2006) or Klahr and Nigam (2004)."

Here Sweller is contesting Rockwood and Panzer 
(2006), not Hake.  In "IA1" above, I make 
essentially the same point that Sweller makes - 
Rockwood's statement is meaningless without 
operational definitions of "hands-on" and "direct 
instruction."  BTW:  I think Sweller might have 
meant to write ". . . .contrasting hands on with 
direct instruction excludes the possibility that 
one might make use of the other - direct 
instruction with hands on experimentation as is 
Klahr and Nigam (2004). . . . . .."


CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
[[Sweller  wrote]]:
"Another quote that should not go unexamined is from Joe Layng (2007):
>For example, the "worked example" effect is limited to only the problems
>worked. No general problem solving skills are learned and students seldom
>become independent problem solvers, and this applies to even high
>performing students.
The research on this topic has shown that 
students who study worked examples or completion 
problems (partially worked examples) perform 
better on transfer tests (these are new, 
different problems - not the ones studied) than 
students who work through the problems themselves 
[Sweller et al. (1998), Sweller (1988)]."

I have not seen Sweller et al. (1998) or Sweller 
(1988) but if the remarkable finding that:

  "students who study worked examples or 
completion problems (partially worked examples) 
perform better on transfer tests (these are new, 
different problems - not the ones studied) than 
students who work through the problems themselves"

is, in fact,  correct then it would seem that programs featuring:

(a) "Problem Based Leaning" [see e.g. UDL (2007), 
Woods (1998, 2005), PBL (2007], and

(b) Pat & Ken Heller's [UMPERG (2007)] 
"Cooperative Group Problem Solving"  and 
"Context Rich Problems,"

need to be revamped so that students consider 
primarily worked or partially worked problems.


DDDDDDDDDDDDDDDDDDDDDDDDDDDDD
[Muller writes] "Sweller's argument (which I 
paraphrase here) is that general problem solving
skills as such cannot be taught or learnt.  The 
contention is that Chess masters are not better 
than novices because they have learned some 
fundamental conceptual rules governing effective 
play but because they have seen many many board 
configurations . . . [[see e.g., Chase & Simon 
(1973), de Groot (1978), Ericsson & Smith (1991), 
Ericsson et al. (1993), Charness et al. (1996), 
and Ericsson (1996a)]]. . . . They draw on this 
wealth of domain specific knowledge to make 
decisions about what moves to play.  You may 
contend that this analogy is not a good one for 
physics problem solving - I include Sweller's own 
explanation in the post-script."

For a more complete explanation of Sweller's work 
on problem solving than Muller's (2007) 
post-script (containing Sweller's criticism of 
attempts to teach *general* - not physics - 
problem solving skills), see Cooper (1998) and 
the section "Worked examples" in Kirschner et al. 
(2006).

Regarding chess expertise not being a good 
analogy for physics problem solving, in an 
Educational Research issue devoted to expertise 
[EdRes (2003)],  psychologist Robert Sternberg 
(2003a) wrote:

"Ericsson (1996) and Ericsson, Krampe, and 
Tesch-Römer (1993) emphasize the role of 
deliberate practice in acquiring expertise. Such 
practice is indeed important in many fields, 
especially in performance-based domains such as 
music, athletics, or chess. It appears, however, 
to be necessary but not sufficient in other kinds 
of domains. Becoming an expert physicist, 
composer, or teacher, for example, seems to 
require a blend of creative (generate ideas), 
analytical (evaluate the ideas), and practical 
thinking (make the ideas work and convince
others of their worth) that goes substantially 
beyond deliberate practice [Sternberg (2003b)]."


II-II-II-II-II-II-II-II- 
II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II-II
II. "The language Hake uses surrounding direct 
instruction (and Direct Instruction) and 
interactive engagement aggravate the 
communication failure that he seems to abhor."

Three points A, B, C below:

AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
[[Muller writes]]:
"My second point is that Hake seems to play into 
the communication failure rather than trying to 
ameliorate it.  He quotes his abstract . . . 
.[[Who's "he"? What "abstract"? A reference 
freak's definitive wording might have been: "Hake 
(2007a) quotes the abstract of Hake (2007d)]]
>I argue that (a) KSC's "direct instruction" approximates the guided
>"interactive engagement," found to be relatively effective in promoting
>student learning by physics education researchers

[[Sweller writes]]
  "Since we also believe "direct instruction". . . 
.[[as defined by Kirschner et al. (2006) and 
called by Hake "interactive engagement" (IE)]]. . 
. . is effective, whatever it is called, there 
seems no dispute. We only have a dispute with 
people who expect learners to work it all out by 
themselves instead of being shown."

"Instead of being shown"?  I don't think the 
"guided construction" of most IE methods is the 
same as "being shown."  This is reminiscent of 
the passage in Kirschner et al. (2006):

"Direct instructional guidance is defined as 
providing information that fully explains the 
concepts and procedures students are required to 
learn as well as learning strategy support that 
is compatible with human cognitive architecture. 
Learning, in turn, is defined as a change in 
long-term memory."

In  "Re: 'Why Minimal Guidance  During 
Instruction Does Not Work' " [Hake (2007d)], I 
commented on the above passage as follows  "It 
would be a great boon to introductory physics 
instructors if Kirchner et al. could tell them 
how to "provide information that fully explains 
the concepts and procedures" relative to e.g., 
Newton's Second Law of Motion."  A copy of Hake 
(2007d) was sent to Richard Clark, but so far 
I've received no response to the above request.

David Meltzer (2007) has also questioned what 
Sweller means by "being shown," writing:

MMMMMMMMMMMMMMMMM
I want to focus here only on two excerpts from 
Sweller's remarks as quoted by Muller, as follows:
"Sweller responds: 'We only have a dispute with 
people who expect learners to work it all out by 
themselves instead of being shown.'"
"Sweller responds: 'Given our emphasis on worked 
examples and, from recall, no mention of lectures 
(which are only one form of direct 
instruction)...'"

So Sweller emphasizes the terms "worked examples" and "being shown."

In the paper by Kirschner, Sweller, and Clark, it 
is stated that "[McCray et al. (2003)]...amply 
document the lack of evidence for unguided 
approaches and the benefits of more strongly 
guided instruction." The report of McCray et al. 
in turn states that "Effective instructional 
strategies...for producing conceptual 
understanding for most students require 
situations that demand active intellectual 
engagement, such as tutorials, small group 
learning, hands-on activities, case studies, and 
problem-solving exercises with appropriate 
scaffolding...(i.e., support and guidance in 
learning specific concepts or tasks)."

So one must presume that by "worked examples," 
"being shown," and "more strongly guided" 
instruction, Sweller intends to include or imply 
"situations that demand active intellectual 
engagement, such as tutorials, small group 
learning, hands-on activities, case studies, and 
problem-solving exercises with appropriate 
scaffolding." I don't know for certain whether he 
actually DOES intend to imply those things, but 
it seems that would be a consistent 
interpretation of the various statements.
MMMMMMMMMMMMMMMMM

A similar questioning of Kirschner et al. (2006) 
by Meltzer was included in Hake (2007d). A copy 
was sent to Richard Clark, but so far Clark has 
not responded.


BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
[[Muller writes]]
"Hake continues. . . [[it's not clear what Muller 
regards Hake as continuing *from*, but Muller 
proceeds to quote from the abstract of  "Re: 
study after study shows children learn better 
with hands-on experience than with direct 
instruction" [Hake (2007a)]]. . . . . .:
'Will articles such as that by Kirschner et al. 
(2006) and Klahr and Nigam (2004) serve to move 
American education more towards direct 
instruction, i.e., passive student lecturing?'

Here Hake is equating direct instruction (after 
citing KSC/KN) with passive student lecturing - 
even though just a second ago, he was equating 
KSC/KN's direct instruction with interactive 
engagement!  It seems like Hake agrees with 
KSC/KN - Hakes considers their methods to be 
interactive engagement - yet Hake fears KSC/KN 
will be misinterpreted leading to passive student 
lecturing.  Why doesn't he make this clear 
instead of contributing to the 
misinterpretations?"

Judging from Muller's comments, despite my 
continual harping on the widespread 
misinterpretation of Klahr & Nigam (2004), I 
should have been more explicit and stated 
something like:

I FEAR THAT KIRSCHNER ET AL. (2006) AND KLAHR AND 
NIGAM (2004) WILL BE MISINTERPRETED, AND THUS 
SERVE TO MOVE AMERICAN EDUCATION MORE TOWARDS 
PASSIVE STUDENT LECTURING.

In fact,  as indicated in Hake (2005) and Klahr 
and Li (2005),  the research of Klahr and Nigam 
(2004) has already been widely misinterpreted as 
favoring passive student lecturing.

In Hake (2005) I listed such misinterpretation as 
being one of the major reasons why NCLB may 
promote what most people [but not Kirschner et 
al. (2006) or Klahr and Nigam (2004)] call direct 
instruction, vis., passive student lecturing. I 
wrote [SEE THAT ARTICLE FOR REFERENCES OTHER THAN 
Klahr and Nigam (2004)]: 

HHHHHHHHHHHHHHHHHHHHH
IV. BUT THE RESEARCH (SECTIONS II & III) SHOWING 
THE SUPERIORITY OF INTERACTIVE ENGAGEMENT (IE) 
AND GUIDED INQUIRY (GI) METHODS TO DIRECT SCIENCE 
INSTRUCTION IS IGNORED

A. In California
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .

B. In the U.S.
There are at least seven reasons why Direct 
Science Instruction (DSI) threatens to 
predominate nationally under the aegis of the No 
Child Left Behind Act:
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
2. The heavily publicized [Adelson (2004), 
Begley (2004  a,b),  Cavenaugh (2004 a,b ), Tweed 
(2004a,b), USDE (2004)] research of Klahr and 
Nigam (KN) (2004) is widely misinterpreted as 
demonstrating the general superiority of direct 
instruction.

Many DI enthusiasts [e.g., Mathematically Correct 
(2005), Bishop (2004)] misleadingly imply that 
KN's research demonstrates the superiority of 
their favored drill and practice methods over any 
other type of science instruction. Commenting on 
Klahr & Nigam, Bishop (2004) opines: 
"Deliberate, direct instruction is more
effective yet again.  Surprise, surprise. 
Replicate a replicable experiment and you get the 
same results. It's the scientific approach."

But the "guided inquiry" and "interactive 
engagement" methods of Sections II and III above 
have little to do with KN's strawman of EXTREME 
Discovery Learning (EDL), in which there is 
almost no teacher guidance. KN showed, not 
surprisingly, that EDL is inferior to "direct 
instruction" for increasing third and fourth grade
children's effective use of the control of 
variables strategy, a so-called "process skill." 
It might be interesting for Klahr & Nigam to 
extend their study to more guided forms of 
"discovery learning" and to children's 
acquisition of "operative knowledge" [Arons 
(1983)].

Consistent with the above, Adelson (2004) wrote: 
"Psychologist Rich Shavelson, professor of 
education and (by courtesy) psychology at 
Stanford University, notes that totally unguided 
discovery of the type used in [KN's] study is 
rarely used in the classroom." Still, he says, 
"This study uses a strong research design. I'd 
like to see a replication with [the more typical] 
guided discovery. Plus, the extent to which 
results would travel to classrooms with varying 
teacher quality, opportunity to learn, et cetera, 
has yet to be found out."

Klahr himself is not the DSI radical painted by 
DI zealots such as Bishop (2004) and 
Mathematically Correct (2005)]. Cavenaugh (2004a) 
wrote: "While David Klahr . . . . believes that 
complex science lessons often require a more 
direct type of instruction, he also cautions 
against too rigid an adherence to either method 
by teachers or administrators. 'It depends on 
what's being taught,' Mr. Klahr, a psychology 
professor at Carnegie Mellon, said in an 
interview."

I agree with David Klahr's caution that the 
appropriate method of science instruction depends 
on what's being taught. Teachers, to be 
effective, need to use different approaches 
(e.g., didactic lectures, coaching, collaborative 
discussions, and Socratic dialogue) to fit the 
classroom occasions and diverse natures of their 
students. Each
method has its strengths and weaknesses for each 
type of student, but in the hands of a *skilled 
teacher* each can be made to compliment the other 
methods so as to advance *every* student's 
learning. A skilled teacher might *lecture* on 
material that can be rote memorized, *coach* 
skills such as typing or playing a musical 
instrument, and use *Socratic dialogue or 
collaborative discussions* (or some other 
"interactive engagement" method) to induce 
students to construct their conceptual 
understanding of difficult counter-intuitive 
material such as Newton's Laws.
HHHHHHHHHHHHHHHHHHHHH

In my view, the danger of misinterpretation is 
enhanced in the case of Kirschner et al. (2006) 
by their seemingly non sequitur title "Why 
Minimal  Guidance During Instruction Does Not 
Work: An Analysis of the Failure  of 
Constructivist, Discovery, Problem-Based, 
Experiential, and  Inquiry-Based Teaching."

Since when has all Constructivist, Discovery, 
Problem-Based, Experiential, and Inquiry-Based 
Teaching" been "minimally guided"?

Since when has there been a failure of constructivist-based teaching?  

In my opinion, IE methods [evidently known as 
"direct instruction methods" by Kirschner et al. 
(2006) and Klahr and Nigam (2004)] are, for the 
most part, consistent with the "Knowledge-Based 
Constructivism" espoused by cognitive scientists 
Resnick & Hall (1998) who wrote [bracketed by 
lines R&H-R&H-R&H. . ."; *emphasis* in the 
original]:

R&H-R&H-R&H-R&H-R&H-R&H
Since about 1960, beginning with the publication 
of Newell and Simon's (1972) landmark studies of 
human problem solving, a body of 
cognitive-science research has focused on the 
nature of the mental processes involved in 
thinking and learning.  Hundreds of scholars have 
been involved, using varied methods and examining 
cognitive processes in people of all ages and 
social conditions. Despite the variety of 
approaches and the many theoretical differences 
among congnitive scientists, it is possible to 
outline a few important points of fundamental 
agreement that we can take as the new core theory 
of learning [Resnick (1987), Bruer (1993)].

Broadly speaking, cognitive science confirms 
Piaget's claim that people must *construct* their 
understanding; they do not simply register what 
the world shows or tells them, as a camera or a 
tape recorder does. To "know" something, indeed 
even to memorize effectively, people must build a 
mental representation that imposes order and 
coherence on experience and information. Learning 
is interpretive and inferential; it involves 
active processes of reasoning and a kind of 
'talking back' to the world - not just taking it 
as it comes. Competent learners engage, 
furthermore, in a great deal of self-management 
of their cognitive processes, that is, in forms 
of cognition known as *metacognitive* and *self 
monitoring*.

This much sounds like the child-centered, process 
theories of education. Early on, however, 
cognitive scientists found that they could not 
account for problem solving and learning without 
attending to what people already *knew*. . . . . 
In every field of thought, cognitive scientists 
found that knowledge is essential to thinking and 
acquiring new knowledge - in other words to 
learning. . . The repeated findings about the 
centrality of knowledge in learning make perfect 
sense for a constructivist theory of learning, 
because one has to have something with which to 
construct. But they turn out to be almost as 
challenging to Piagetian or Deweyan theories of 
pedagogy as to Thorndikean ones. This is because 
they insist that knowledge - *correct* knowledge 
- is essential at every point in learning. And 
they make it impossible to suggest that education 
for the information age should not trouble itself 
with facts and information, but only with 
processes of learning and thinking. What we know 
now is that just facts alone do not constitute 
true knowledge and thinking power, so thinking 
processes cannot proceed without something to 
think about. Knowledge is in again, but alongside 
thinking, indeed, intertwined with it, not 
instead of thinking. So although it is essential 
for children to have the experience of 
discovering and inventing, their experience must 
be of one of disciplined invention, that is, by 
established processes of reasoning and logic.

[The above advocated] *Knowledge-based 
Constructivism*, taken seriously, points to a 
position that can moderate the century-long 
polarity between passive drill pedagogies and 
child-centered discovery pedagogies.
R&H-R&H-R&H-R&H-R&H-R&H


CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
[[Sweller wrote]]:
"The suggestion that "direct instruction" is 
linguistically equivalent to "long lectures" is 
novel. Given our emphasis on worked examples and, 
from recall, no mention of lectures (which are 
only one form of direct instruction), I doubt any 
reader really believes we were talking of long 
lectures. In any case, if it is all a linguistic 
misunderstanding, it can be easily cleared up and 
we presumably all agree."

I searched Kischner et al. (2006) for "lecturers" and obtained one hit:

KSC-KSC-KSC-KSC-KSC-KSC-KSC
In an attempt to rescue medical students from 
lectures and memory-based recall exams, 
approximately 60 medical schools in North America 
have adopted PBL in the past two decades. This 
variant of constructivist instruction with 
minimal guidance, introduced at the McMaster 
University School of Medicine in 1969, asks 
medical students to work in groups to diagnose 
and suggest treatment for common patient 
symptoms. PBL student groups are supervised by a 
clinical faculty member who is directed not to 
solve problems for the students but instead to 
offer alternatives and suggest sources of 
information.

The best known survey of the comparisons of PBL 
with conventional medical school instruction was 
conducted by Albanese and Mitchell (1993). Their 
meta-analysis of the English language literature 
of the effectiveness of PBL produced a number of 
negative findings concerning its impact, 
including lower basic science exam scores, no 
differences in residency selections, and more 
study hours each day. They reported that although 
PBL students receive better scores for their 
clinical performance, they also order 
significantly more unnecessary tests at a much 
higher cost per patient with less benefit. There 
was an indication in their review that increased 
clinical practice evaluation scores may have been 
due to the fact the PBL students are required to 
spend more time in clinical settings.
KSC-KSC-KSC-KSC-KSC-KSC-KSC

For more positive views of PBL scan Woods (2005) 
and the material at the University of Delaware 
Problem-Based Learning Site 
<http://www.udel.edu/pbl/>; for example listed at 
<http://www.udel.edu/pbl/articles.html> is Duch 
et al. (2001).


In any case, no one that I know of has set 
"direct instruction" equal to "long lectures."

Judging from Sweller's quotes as relayed by 
Muller, Sweller has not scanned my critique "Re: 
'Why Minimal Guidance  During Instruction Does 
Not Work' " [Hake (2007d)] of Kirschner, Sweller, 
& Clark (2006).  Had he done so would have found 
my guesses at what various factions mean by the 
ambiguous term "direct instruction." There I 
guessed that most physics education researchers 
set "direct instruction" equal to traditional 
PASSIVE student lectures, recipe labs, and 
algorithmic problem sets.

Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<[log in to unmask]>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>

"Education is the acquisition of the art of the utilization of
knowledge. This an art very difficult to impart. We must beware of
what I will call 'inert ideas' that is to say, ideas that are merely
received into the mind without being utilized or tested or thrown
into fresh combinations."
    Alfred North Whitehead (1967) in "The Aims of Education."

REFERENCES
Albanese, M., & Mitchell, S. (1993). 
Problem-based learning: A review of the 
literature on its outcomes and implementation 
issues. AcademicMedicine 68: 52-81.

Bruer, J.T.  1993.  "Schools for Thought:  A 
Science of Learning in the Classroom." MIT Press, 
information at 
<http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=5900>.

Chase, W.G. & H.A. Simon. 1973. "Perception in 
chess," Cognitive Psychology 4 (1): 55-81.

Charness, N., R.T. Krampe, & U. Mayr. 1996. 'The 
role of practice and coaching in entrepreneurial 
skill domains: An international comparison of 
life-span chess skill acquisition,' in in 
Ericsson (1996, pp. 51-80).

Clark, R. E. 1983. "Reconsidering Research on 
Learning from Media," Review of Educational 
Research 53(4): 445-459.

Clark, R. E. 1994. "Media Will Never Influence 
Learning," Educational Technology, Research and 
Development 42(2): 21-29.

Cooper, G. 1998. "Research into Cognitive Load 
Theory and Instructional Design at UNSW," online 
at 
<http://educationnew.arts.unsw.edu.au/staff/sweller/clt/>. 

Cuban, L. 1986. "Teachers and machines : the 
classroom use of technology  since 1920." 
Teachers College Press.

de Groot, A. 1978.  "Thought and Choice in 
Chess."  Mouton.  This ground breaking research 
was originally published in 1946.

Duch, B., S. Gron, and D. Allen. 2001. "The Power 
of Problem-Based Learning." Stylus Publishing, 
information at 
<http://www.styluspub.com/Books/BookDetail.aspx?productID=44647>.

EdRes. 2003. AERA Educational Researcher, 
November theme issue on expertise, online at 
<http://www.aera.net/publications/?id=400>

Ericsson, K.A. &  J. Smith, eds. 1991. "Toward a 
General Theory of Expertise:  Prospects and 
Limits."  Cambridge University Press, information 
at 
<http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521406123>.

Ericsson, K.A.,  R.T. Krampe, &  C. Tesch-Römer. 
1993. "The role of deliberate practice in the 
acquisition of expert performance," Psychological 
Review 100: 363-406.

Ericsson, K.A. 1996a.  'The acquisition of expert 
performance: An introduction to some of the 
issues," in Ericsson (1996b, pp. 1-50).

Ericsson, K.A. ed.  1996b.  "The Road to 
Excellence: The Acquisition of Expert Performance 
in the Arts and Sciences, Sports, and Games." 
Erlbaum, information at 
<http://tinyurl.com/2fwb75>. See also Ericsson's 
list of publications at 
<http://www.psy.fsu.edu/> // "People" // "Faculty 
Listing" // "K. Anders Ericsson" // "Dr. 
Ericsson's Home Page"  (where "//" means "click 
on").

Ericsson, K.A. 2002. A brief summary and 
description of the Acquisition of Expert 
Performance and how this is explained by 
Deliberate Practice is online at 
<http://www.psy.fsu.edu/faculty/ericsson/ericsson.exp.perf.html>

Hake, R.R. 1998a. "Interactive-engagement vs 
traditional methods: A six thousand-student 
survey of mechanics test data for introductory 
physics courses," Am. J. Phys. 66: 64-74; online 
at 
<http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> 
(84 kB).

Hake, R.R. 1998b. "Interactive-engagement methods 
in introductory mechanics courses," online at 
<http://www.physics.indiana.edu/~sdi/IEM-2b.pdf> 
(108 kB) - a crucial companion paper to Hake 
(1998a).

Hake, R.R. 2002. "Lessons from the physics 
education reform effort," Ecology and Society 
5(2): 28; online at 
<http://www.ecologyandsociety.org/vol5/iss2/art28/>. 
Ecology and Society(formerly Conservation 
Ecology) is a free online "peer-reviewed journal 
of integrative science and fundamental policy 
research" with about 11,000 subscribers in about 
108 countries.

Hake, R.R. 2004a. "Will the NCLB Tend to 
Propagate California's Direct Science Instruction 
Throughout the Entire Nation?" online at 
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0501&L=pod&O=D&P=12783>. 
Post of 14
Jan 2005 to AERA-C, AERA-D, AERA-G, AERA-H, AERA-J, AERA-K, AERA-L, APPhysics,
ASSESS, Biopi-L, Chemed-L, Edstat-L, EvalTalk, 
Math-Learn, Math-Teach, Phys-L, Physhare, 
PhysLrnR, POD, & STLHE-L.

Hake, R.R. 2004b. "Re: Back to Basics vs. Hands-On Instruction" online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0402&L=pod&P=R17377>. 
Post of 29 Feb 2004 17:57:25-0800 to AERA-K, 
AP-Physics, Biopi-L, Chemed-L, EvalTalk, 
Math-Learn, Math-Teach, Phys-L, PhysLrnR, 
Physhare, PHYSOC, and POD.

Hake, R.R. 2004c. "Re: Back to Basics vs. Hands-On Instruction" online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0402&L=pod&P=R13581>. 
Post 24 Feb 2004 17:45:19-0800 to AERA-K, 
AP-Physics, Biopi-L, Chemed-L, FYA-List, 
Math-Learn, Math- Teach, Phys-L, PhysLrnR, 
Physhare, and POD.

Hake, R.R. 2004d.  "The No Significant Difference Debate," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0401&L=pod&P=R6235&I=-3>. 
Post of 22 Jan 2004 09:11:06-0800 to PhysLrnR, 
POD, and STLHE-L.

Hake, R.R. 2005. "Will the No Child Left Behind 
Act Promote Direct Instruction of Science?" Am. 
Phys. Soc. 50: 851 (2005); APS March Meeting, Los 
Angles, CA. 21-25 March; online at
<http://www.physics.indiana.edu/~hake/WillNCLBPromoteDSI-3.pdf> (256 kB).

Hake, R.R. 2006. "Possible Palliatives for the 
Paralyzing Pre/Post Paranoia that Plagues Some 
PEP's"  [PEP's = Psychometricians,  Education 
specialists, and Psychologists], Journal of 
MultiDisciplinary Evaluation, Number 6, November, 
online at 
<http://evaluation.wmich.edu/jmde/JMDE_Num006.html>.

Hake, R.R. 2007a. "Re: study after study shows 
children learn better with hands-on experience 
than with direct instruction," online at 
<http://tinyurl.com/2tzsah>. Post 14 Feb 2007 
16:21:34-0800 to PhysLrnR and ScListserv.

Hake, R.R. 2007b. "Should We Measure Change? 
Yes!" download directly by clicking on 
<http://www.physics.indiana.edu/~hake/MeasChangeS.pdf> 
(2.5 MB). Failure to access that URL probably 
means that a new version (T, U, V, W. . .) has 
been placed online - it can be accessed as ref. 
43 at <http://www.physics.indiana.edu/~hake>. To 
appear as a chapter in "Evaluation of Teaching 
and Student Learning in Higher  Education," a 
Monograph of the American Evaluation Association 
<http://www.eval.org/>.  A severely truncated 
version appears at Hake (2006).

Hake, R.R. 2007c." Kirschner, Sweller, & Clark 
Revisited," online at 
<http://tinyurl.com/36klac>. Post of 18 Jan 2007 
21:47:28-0800to PhysLrnR and MathLearn.

Hake, R.R. 2007d.  "Re: 'Why Minimal Guidance 
During Instruction Does Not Work', " online at 
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0701&L=pod&O=D&P=20140>. 
Post of 19-20 Jan 2007 to AERA-A, AERA-B, AERA-C, 
AERA-J, AERA-K, AERA-L, ASSESS, EdResMeth, 
ITForum (rejected), PBL, POD, RUME, MathTalk, 
STLHE-L, TIPS, PsychTeacher (rejected), 
TeachingEdPsych.

Kirschner, P. A., J. Sweller, & R.E. Clark. 2006. 
"Why Minimal  Guidance During Instruction Does 
Not Work: An Analysis of the Failure  of 
Constructivist, Discovery, Problem-Based, 
Experiential, and  Inquiry-Based Teaching." 
Educational Psychologist 41(2): 75-86;
online at 
<http://www.cogtech.usc.edu/publications/kirschner_Sweller_Clark.pdf> 
(176 kB).

Klahr, D. & M. Nigam. 2004. "The equivalence of 
learning paths in early science instruction: 
effects of direct instruction and discovery 
learning," Psychological Science 15(10): 661-667; 
online at 
<http://www.psy.cmu.edu/faculty/klahr/personal/pubs.htm>, 
click on ""Cognition & Instruction." For a 
discussion of widespread misinterpretation of 
this paper see, e.g., "Will the No Child Left 
Behind Act Promote Direct Instruction of 
Science?" [Hake (2005)], and "Cognitive Research 
and  Elementary Science  Instruction: From the 
Laboratory, to the Classroom, and Back" [Klahr & 
Li (2005)].

Klahr, D. & J. Li. 2005. "Cognitive Research and 
Elementary Science  Instruction: From the 
Laboratory, to the Classroom, and Back,"  Journal 
of Science Education and Technology 14(2): 
217-238; online at 
<http://www.psy.cmu.edu/faculty/klahr/personal/pdf/Klahr_Li_2005.pdf> 
(536 kB).

Layng, T.V.J. 2007. "Re: study after study shows 
children learn better with hands-on experience 
than with direct instruction," ScListserv post of 
1 Feb 2007 08:59:15-0800; online at 
<http://tinyurl.com/35f8w9>.

McCray, R.A., R.L. DeHaan, J.A. Schuck, eds. 
2003. Improving Undergraduate Instruction in 
Science, Technology, Engineering, and 
Mathematics: Report of a Workshop, Committee on 
Undergraduate STEM Instruction, National Research 
Council, National Academy Press; online at 
<http://www.nap.edu/catalog/10711.html>.  For 
references to Hake's criticism of this report see 
Hake (2007d). 

Meltzer, D. 2007. Re: study after study shows 
children learn better with hands-on experience 
than. . . . .[with direct instruction]. . .," 
PhysLrnR post of 15 Feb 2007 23:37:19-0800; 
online at <http://tinyurl.com/ys8h4e>.

Muller, D. 2007. "Re: study after study shows 
children learn better with hands-on experience 
than . . . . .[with direct instruction]. . .," 
PhysLrnR post of 15 Feb 2007 21:52:20-0700; 
online at <http://tinyurl.com/yoo42h>.

Newell, A. & H.A. Simon. 1972. "Human Problem Solving." Prentice Hall. 

Panzer, T. 2006. "Students get hi-tech  advantage," PublicOpinion, online at
<http://www.publicopiniononline.com/localnews/ci_5115719>.

PBL. 2007. Problem Based Learning Discussion 
Group with archives at 
<http://www.jiscmail.ac.uk/lists/pbl.html>.

Resnick, L.B. 1987. "Education  and Learning to 
Think," National  Academy Press, online at
<http://www.nap.edu/openbook/0309037859/html/>.

Resnick, L.B. and Hall, M.W. 1998. "Learning 
Organizations for Sustainable Education Reform," 
Daedalus 127(4): 89-118.

Smith, H. J., S. Higgins, K. Wall, & J. Miller. 
2005. "Interactive whiteboards: boon or 
bandwagon? A critical review of the literature," 
Journal of Computer Assisted Learning 21: 91-101. 
An author's abstract is online at 
<http://tinyurl.com/3ywovd>.

Sternberg, R.J. 2003a. "What Is an "Expert 
Student?" Educational Researcher 32(8): 5-9, 
online at <http://tinyurl.com/ojh9x>, one of the 
articles in EdRes (2003).

Sternberg, R. J. 2003b. "Wisdom, intelligence, 
and creativity synthesized." Cambridge University 
Press, information at 
<http://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521802383>.

Sweller, J. 1988. "Cognitive load during problem 
solving: Effects on Learning,"  Cognitive Science 
12(2): 257-285.

Sweller, J.,  J.J.G. van Merrienboer, &  F. G. W. 
C. Paas. 1998.  "Cognitive architecture and 
instructional design."  Educational Psychology 
Review 10(3): 251-296.

UDL. 2007. University of Delaware Problem-Based 
Learning Site <http://www.udel.edu/pbl/>.

UMPERD. 2007. University of Minnesota Physics 
Education Research and Development on line at 
<http://groups.physics.umn.edu/physed/index.html>. 
Especially "Cooperative Group Problems Solving" 
<http://groups.physics.umn.edu/physed/Research/CGPS/CGPSintro.htm> 
and "Context Rich Problems" 
<http://groups.physics.umn.edu/physed/Research/CRP/crintro.html>.

Whitehead, A.N. 1967. "Aims of Education, " Free Press, Reissue edition.

Woods, D.R. 1998. "The MPS Program: The McMaster 
Problem Solving Program," online at
<http://www.chemeng.mcmaster.ca/MPS/default1.htm>. 
For a more recent publication by Don Woods, see, 
e.g., Woods (2005).

Woods, D.R. 2005. "Problem-based Learning, 
especially in the context of large classes" at
<http://chemeng.mcmaster.ca/pbl/pbl.htm>