.
Date: Thu, 17 Jun 2010 15:34:11 -0700
From: Richard Hake <[log in to unmask]>
Reply-To: [log in to unmask]
To: [log in to unmask]
Cc: [log in to unmask], [log in to unmask]
Subject: [Net-Gold] Is Psychometrics Pathological Science?
If you reply to this long (11 kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
*********************************************
ABSTRACT: PhysLrnR's Noah Podolefsky (2010) cited "Is Psychometrics
Pathological Science?'' [Michelle (2008)] and "Beyond romantic
versus sceptic: a microanalysis of conceptual change in kinematics"
[Roschelle (1998)] to clarify his questions about "whether the FCI
(and other instruments) are 'quantitative'." In my opinion, neither
of above references nor Robert Mislevy's "sociocognitive" perspective
contradict the premise that the *normalized* pre-to posttest gain on
the Force Concept Inventory is a valid and consistently reliable
quantitative gauge of the effectiveness of an introductory mechanics
course in promoting students conceptual understanding of Newtonian
mechanics.
*********************************************
Noah Podolefsky (2010) in his PhysLrnR post "Re: Is a Valid and
Reliable Concept Test as Impossible as Perpetual Motion? #2"
references "Is Psychometrics Pathological Science?'' [Michelle
(2008)]; "Beyond romantic versus sceptic: a microanalysis of
conceptual change in kinematics" [Roschelle (1998); and the tenuously
related "Rasch model based analysis of the Force Concept Inventory"
to clarify his questions about "whether the FCI (and other
instruments) are 'quantitative'."
IMHO none of the above references contradict the premise that the
*normalized* pre-to posttest gain on the Force Concept Inventory is a
valid and consistently reliable quantitative gauge of the
effectiveness of an introductory mechanics course in promoting
students conceptual understanding of Newtonian mechanics, as argued
in e.g., "Design-Based Research in Physics Education Research: A
Review" [Hake (2008)] and "Should We Measure Change? Yes!"[Hake
(2010)].
Similarly, in "How Cognitive Science Challenges the Educational
Measurement Tradition," psychometric guru Robert Mislevy (2008)
surveyed the work of Michell and others from what he called a
"sociocognitive" perspective i.e., integration of individual,
situative, and social perspectives on cognition.
But, again, IMHO, Mislevy's survey, just as his earlier criticism of
the "normalized gain" on the basis that it's is not "grounded in the
framework of probability-based reasoning" [Mislevy (2006)], fails to
contradict the premise that the *normalized* pre-to posttest gain on
the Force Concept Inventory is a valid and consistently reliable
quantitative gauge of the effectiveness of an introductory mechanics
course in promoting students conceptual understanding of Newtonian
mechanics.
Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which Recognize the
Invention of the Internet (PEDARRII)
<[log in to unmask]>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>
<http://HakesEdStuff.blogspot.com>
<http://iub.academia.edu/RichardHake>
REFERENCES [URL's shortened by <http://bit.ly/> and
<http://tinyurl.com/>; all URL's accessed on 17 June 2010]
Atkinson, D., E. Churchill, T. Nishino, & H. Okada. 2007. "Alignment
and Interaction in a Sociocognitive Approach to Second Language
Acquisition," Modern Language Journal 91:169-188; abstract online at
<http://bit.ly/bdXDxg>.
Hake, R.R. 2006. "Possible Palliatives for the Paralyzing Pre/Post
Paranoia that Plagues Some PEP's," Journal of MultiDisciplinary
Evaluation, Number 6, November, online at
<http://bit.ly/caWtWl>. This even despite the admirable
anti-alliteration advice at psychologist Donald Zimmerman's site
<http://mypage.direct.ca/z/zimmerma/> to "Always assiduously and
attentively avoid awful, awkward, atrocious, appalling, artificial,
affected alliteration."
Hake, R.R. 2008. "Design-Based Research in Physics Education
Research: A Review," in "Handbook of Design Research Methods in
Education: Innovations in Science, Technology, Engineering, and
Mathematics Learning and Teaching" [Kelly, Lesh, & Baek (2008)] - A
pre- publication version of Hake's chapter is online as a 1.1 MB pdf
at <http://bit.ly/9kORMZ>.
Hake, R.R. 2009. "Should We Measure Change? Yes!" online as a 2.5 MB pdf at
<http://bit.ly/d6WVKO> (2.5 MB) and as ref. 43 at
<http://www.physics.indiana.edu/~hake>. To appear as a chapter in
"Evaluation of Teaching and Student Learning in Higher Education"
[Hake (in preparation)]. A severely truncated version is online at
Hake (2006).
Halloun, I., R.R. Hake, E.P. Mosca, & D. Hestenes. 1995. Force
Concept Inventory (Revised), online (password protected) at
<http://modeling.asu.edu/R&E/Research.html>, scroll down to
"Evaluation Instruments." Currently available in 19 languages:
Arabic, Chinese, Czech, English, Finnish, French, French (Canadian),
German, Greek, Italian, Japanese, Malaysian, Persian, Portuguese,
Russian, Spanish, Slovak, Swedish, & Turkish.
Hestenes, D., M. Wells and G. Swackhamer. 1992. Force Concept
Inventory, Phys. Teach. 30, 141; online (but without the test itself)
at <http://modeling.asu.edu/R&E/Research.html>, Tables I and II,
revised for the 1995 version [Halloun et al. (2005)] are online at
<http://modeling.asu.edu/R&E/Research.html>, directly below the first
reference under "Articles about the FCI."
Kelly, A.E., R.A. Lesh, & J.Y. Baek. 2008. "Handbook of Design
Research Methods in Education: Innovations in Science, Technology,
Engineering, and Mathematics Learning and Teaching." Routledge.
Publisher's information at <http://tinyurl.com/4eazqs>; Amazon.com
information at <http://tinyurl.com/5n4vvo>.
Michell, J. 2005. "The logic of measurement: A realist overview,"
Measurement 38(4): 285-294; abstract online at <http://bit.ly/9RXSHD>.
Michell, J. 2008. "Is Psychometrics Pathological Science?"
Measurement: Interdisciplinary Research & Perspective 6 (1 & 2),
Issue 1 & 2: 7-24; abstract online at <http://bit.ly/cjZCgI>. See
also Michell (2005).
Mislevy, R. 2006: (a) "On approaches to assessing change," and (b)
"Clarification"; both online at <http://bit.ly/98WXLP>.
Mislevy, R. 2008. "How Cognitive Science Challenges the Educational
Measurement Tradition," unpublished? Online as a 250 kB pdf at
<http://bit.ly/b6213i>.
Planinic, M., L. Ivanjek, & A. Susac. 2010. "Rasch model based
analysis of the Force Concept Inventory" Phys. Rev. ST Physics Ed.
Research 6, 010103; online at <http://bit.ly/bVNMK2>.
Podolefsky, N. 2010. "Re: Is a Valid and Reliable Concept Test as
Impossible as Perpetual Motion? PhysLrnR post of 16 Jun 2010
16:32:31-0600; online at <http://bit.ly/9jC0jo>. To access the
archives of PhysLnR one needs to subscribe, but that takes only a few
minutes by clicking on
<http://listserv.boisestate.edu/archives/physlrnr.html> and then
clicking on "Join or leave the list (or change settings)." If you're
busy, then subscribe using the "NOMAIL" option under "Miscellaneous."
Then, as a subscriber, you may access the archives and/or post
messages at any time, while receiving NO MAIL from the list!
Roschelle, J. 1998. "Beyond romantic versus sceptic: a microanalysis
of conceptual change in kinematics," International Journal of Science
Education 20(9): 1025-1042; abstract online at
<http://bit.ly/cIb1Wm>. The full article is evidently not *free*
online. But judging from its abstract, similar points are made in the
free online article by Smith et al. (1995).
Smith, J.P., A.A. diSessa. & J. Roschelle. 1995. "Misconceptions
Reconceived: A Constructivist Analysis of Knowledge in Transition,"
Journal of the Learning Sciences, 3(2); online as a 246 kB pdf at
<http://bit.ly/9tWD06>.
.
|