 |
 |
Steve Simon does not have to apologize for not being a pure relativist; nor is such skepticism proof of the post-modernist denounciation of objective science. He can go back to a much older, and completely respectable source, Bayes' theorem. This fundamental theorem of how to combine probabilities is the quantitative expression of the time-honored scientific maxim that outrageous claims require overwhelming proof. In Bayesian terms this means a very low prior probability requires a very high probability from experimental evidence to produce an acceptable new probability. The equipoise situation of exactly 50% probability (an uninformative prior) that is assumed with a conventional statistical significance test for a typical medical or scientific experimental hypothesis, that is supported by plausible potential mechanisms and previous data, is simply not appropriate or accepted by most scientists for the absurd claims of homeopathy, cold fusion, ESP, astrology, etc., that are supported by no plausible mechanisms and by previously fraudulent data. This higher level of evidence demanded for these hairbrained hypotheses seems unfair to the proponents. Being convinced, they have very high prior probabilities that can be confirmed with their underwhelming data. But the overwhelming-proof maxim contributes positively to the skeptical, conservative nature of science. To this extent science is prejudiced (prior prejudice is the essence of a low prior probability), but not immovable. Skeptical science has been presented with, and appropriately moved by, the overwhelming evidence for countless paradigm shifts, from Copernicus, to Darwin, to Einstein, to Bohr. Science is not fair in the relativist's sense, rather it is practical. Moderate evidence is acceptable for believable claims. For seemingly absurd claims, we need remarkable data (not merely a "mirror image") to get our attention. If science could be swayed easily by some fanatic's claim of divine revelation, or some crackpot's fishy small-time experiment, then science would be no different than, well, religion or quackery. As for the post-modernists, the lack of pure relativism in science does not derive from simple subjectivity. Rather it derives objectively from a very long history of learning how to separate the wheat from the chaffe. The apparent subjectivity implied by Bayes' theorem, is not the simplistic kind decried by post-modernists and quacks. It is the deep and unavoidable kind implied by Einstein's theory of relativity, Hiesenberg's uncertainty principle, and Godel's proof of the incompleteness of all closed systems. And it is equally uncomfortable and incomprehensible for simple minds. David L. Doggett, Ph.D. Senior Medical Research Analyst Health Technology Assessment and Information Services ECRI, a non-profit health services research organization 5200 Butler Pike Plymouth Meeting, Pennsylvania 19462, U.S.A. Phone: (610) 825-6000 x5509 FAX: (610) 834-1275 http://www.ecri.org <http://www.ecri.org/> e-mail: [log in to unmask] -----Original Message----- From: Simon, Steve, PhD [mailto:[log in to unmask]] Sent: Wednesday, October 17, 2001 7:47 PM To: [log in to unmask] Subject: Re: theory or evidence? Arash Rashidian writes: >I came across a paper by Vandenbroucke and de Craen in Ann >Intern Med. 2001; 135:507-513. I thought it might be of interest of >the group. >In their paper they have used the long lasting arguments on >'homeopathy' as an excuse to provide "a mirror image for >scientific reasoning in conventional medicine'. I feel it is a >very readable piece of work. I did enjoy it, and I agree with the authors basic premise. But the authors also left out an important issue which needs greater discussion, I believe, in the Evidence Based Medicine literature. This is the problem of biased assimilation effect. This is the tendency to look harder for flaws in papers that we disagree with and to overlook flaws in papers that we agree with. This has been demonstrated empirically (see MacCoun 1998 for several good examples). I see it most often when dealing with research that is highly emotional and where opinions are very strongly held. Gun control proponents are highly critical of John Lott's research that claims that concealed carry laws reduce crime and at the same time will cite much of the public health literature that claims that having a gun in the home increases your risk of injury and death. Opponents of gun control praise Lott's research and criticize the public health literature. Both sets of research are based on weak observational designs and have similar flaws and shortcomings. There is similar polarization about the relationship between IQ and heredity. One side claims that half the research is bad and cites the other half as proof. The opposite side will do the same, but will reverse the two halves. Is it possible that we are more critical of homeopathy research not because it is bad research, but because it supports a viewpoint that we disagree with? Are proponents of homeopathy too ready to overlook the gaps in the research? I don't want to sound like one of the post-modernist thinkers, but I do believe that it is very hard to be truly objective in evaluating research. I've noticed disagreement even on something as basic as whether a given article provides supportive evidence of or refutes homeopathy (I don't have the citation handy). And I've noticed all too often the following request: "the conclusions of this paper are all wrong--help me find the flaw in their reasoning." What do others think about this? MacCoun, R. (1998). "Biases in the interpretation and use of research results." Annu Rev Psychol 49: 259-87. Steve Simon, [log in to unmask], Standard Disclaimer. The STATS web page has moved to http://www.childrens-mercy.org/stats