Just out:
Branch, M. (2014). Malignant side effects of null-hypothesis significance testing. Theory and Psychology, 24, 2, 256-277.
Abstract
Six decades-worth of published information has shown irrefutably that null-hypothesis significance tests (NHSTs) provide no information about the reliability of research outcomes. Nevertheless, they are still the core of editorial decision-making in Psychology. Two reasons appear to contribute to the continuing practice. One, survey information suggests that a majority of psychological researchers incorrectly believe that p values provide information about reliability of results. Two, a position sometimes taken is that using them to make decisions has been essentially benign. The mistaken belief has been pointed out many times, so it is briefly covered because of the apparent persistence of the misunderstanding. The idea that NHSTs have been benign is challenged by seven “side-effects” that continue to retard effective development of psychological science. The article concludes with both a few suggestions about possible alternatives and a challenge to psychological researchers to develop new methods that actually assess the reliability of research findings.
The seven side-effects:
1. NHST promotes aimless, non-cumulative, non-integrated science
2. NHST promotes “sizeless” science
3. NHST blunts social processes that underlie successful science
4. NHST is essentially a fool’s errand
5. NHST promotes confusion of actuarial and behavioral science
6. NHST impedes the publication of “negative” results
7. NHST inhibits the range of experimentation
An interesting paragraph (p. 272-273):
“Psychology as a scientific discipline can be seen as wallowing, perhaps slowly disintegrating. The American Psychological Association currently has 59 divisions, most of which are completely independent of one another scientifically. They share no core of knowledge (except, ironically, how to employ NHSTs), the kind of knowledge that is generated by cumulative, evolving science. The typical introductory text has about 20– 25 chapters, each of which can be read pretty much independently of any of the others. The order of topics, which is dictated more by tradition than logic (if the organization mimicked that of other, more mature and integrated sciences, it is likely that the basic psychological phenomena would appear first, and the reductionistic analyses of them would occur later), does not generally reflect any accumulation, refinement, or integration of knowledge. Instead, a student comes away with the view that there are many interesting things that psychologists study, but that they are pretty much unrelated. In my own department, which I believe to be typical in its training of students, there is not a single, substantive psychological fact or set of facts that every graduate student must know. A student can complete our graduate program without learning anything at all about basic learning processes, or basic sensory and perceptual processes, or memorial and other cognitive processes, or developmental processes, or social processes, or approaches to personality, and so on. Students, as in most graduate programs, can pick and choose among a few courses on those (and other) topics to provide them presumed breadth. But the only training every student must have is in NHST. I sometimes like to say, only partly in jest, that current graduate training in Psychology emphasizes learning a set of methods from which no basic facts (that is, facts that every psychologist should know) have emerged! ”
This NHST saga is gradually moving from being an ‘abstract’ issue to one that will impact personal scientific integrity and externally-adjudged professional competence.
Regards .. Paul
Chief Research Scientist
Cognadev.com
__________________________________________________________________________________
W: www.cognadev.com
W: www.pbarrett.net
E: [log in to unmask]
M: +64-(0)21-415625