Hi All,

It is true that once a belief is established, when a study comes out with contradictory findings, it is often attacked.  On the other hand, a number of studies with unexpected results (eg hormone replacement and cardiovascular disease) have been rightly hugely influential.  Prior beliefs are important (we call them hypotheses sometimes!) but so too are objective studies that confirm or refute them.  It is particularly challenging to sort those out  when we are comparing observational studies as is the case with this controversy.

But Willett said this:  “the fundamental reason is that the authors did not adequately separate people who are lean because they are ill from those who lean because they are active and healthy”

That seems a critically important methodological issue if it is true.

It seems usually wise to replicate and/or scrutinize studies that have surprising conclusions.

Disclosure: I recently wrote about how we should be careful re Confirmation Bias and not blindly attack studies that counter prior beliefs here http://ebm.bmj.com/content/18/2/43.full (and reproduced below).

Best

Rich

Richard Saitz MD, MPH, FACP, FASAM

Editor, Evidence-Based Medicine

Professor of Medicine & Epidemiology

Boston University Schools of Medicine & Public Health

Boston Medical Center

ebm.bmj.com

Evid Based Med 2013;18:43 doi:10.1136/eb-2013-101221

·         Editorial

Accentuating the positive

1.     Richard Saitz

+Author Affiliations

1.        Section of General Internal Medicine, Boston University & Boston Medical Center, Boston, Massachusetts, USA

1.     ^{Correspondence to}: Richard Saitz
Section of General Internal Medicine, Boston University and Boston Medical Center, Boston, Massachusetts 02118, USA; [log in to unmask]

You've got to accentuate the positiveEliminate the negativeAnd latch on to the affirmativeDon't mess with Mister In-BetweenJohnny MercerAc-Cent-Tchu-Ate The Positive, 1944

Although he was probably not thinking about medical scientists, lyricist Johnny Mercer's words described what we may be observing in the medical literature. In F1000 Research, Senn1raises doubt regarding the conclusions of observational studies that suggest major medical journals are not biased in favour of publishing positive studies. He points out that the observation of similar acceptance rates for manuscripts could even be indicative of the presence of a bias, if submitted negative studies tend to be of higher quality than positive studies, a situation for which he finds some evidence.

Senn's thought experiment is useful and a novel, clear-thinking way of understanding the issues, though the conclusion is not surprising in that it is consistent with what many authors believe. It serves as a plausible explanation for the data reported in studies of publication bias that he cites.

The larger context of bias towards positive studies is that of basic human psychology, and the recognition of confirmation bias.2 Confirmation bias occurs when observers favour data consistent with their beliefs. The bias is alive and well in medicine and is probably not discussed enough.

Studies that counter strongly held beliefs often attract not only criticism but also vitriolic comments. Two examples will suffice. A study in the New England Journal of Medicinesuggested that breast cancer screening was associated with a great deal of overdiagnosis and a very small effect on death from breast cancer.3 A radiologist and advocate of screening mammography called the study ‘junk science.’4 A review in Annals of Internal Medicine cited negative studies of screening and brief intervention for unhealthy alcohol use in hospitals, critically appraising a proposed performance measure's evidence base.5 Advocates for the practice commented on the review and its cited negative studies by simply asserting that ‘treatment works’, and describing the findings as ‘counterintuitive’, and ‘the death of medical common sense’.6 ,7 Although positive studies are also criticised, it seems less automatic when the intervention is believed to work. Further, the raw emotion that negative studies that counter strongly held beliefs seem to generate is remarkable, particularly when it comes from medical scientists. That confirmation bias may arise from deep psychological traits suggests we must be particularly cautious about it.

Fortunately, however, common sense and evidence-based medicine are not dead. And hopefully raising the topics of publication bias and its cousin, confirmation bias, will stimulate recognition of these biases and debate about how to handle them. I trust we will not apply Mr Mercer's advice blindly when we appraise the medical literature.

Footnotes

·         Competing interests None.

References

1.     Senn S

. Misunderstanding publication bias: editors are not blameless after all. [v1; ref status: indexed, http://f1000r.es/YvAwwD] F1000Research 2012;1:59 (doi: 10.3410/f1000reserch.1-59.v1) accessed January 20, 2013.

2.      

1.     Nickerson RS

. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Genet Psychol 1998;2:175–220.

[CrossRef]

2.     Bleyer A, 

3.     Welch G

. Effect of three decades of screening mammography on breast-cancer incidence. N Engl J Med 2012;367:1998–2005.

[CrossRef][Medline][Web of Science]

4.     Kotz D

. 1 Million women ‘overdiagnosed’ with breast cancer due to mammograms, controversial study suggests. Boston Globe, 21 November 2012.

Search Google Scholar

5.     Saitz R

. Candidate performance measures for screening for, assessing and treating unhealthy substance use in hospitals: advocacy or evidence-based practice? Ann Intern Med 2010;153:40–3.

[Medline]

3.      

1.     Gentilello L, 

2.     Goplerud E

. Letter in response to candidate performance measures for screening for, assessing and treating unhealthy substance use in hospitals: advocacy or evidence-based practice? Ann Intern Med 2011;154:73.

[Medline]

4.      

1.     Madras BK

. Letter in response to candidate performance measures for screening for, assessing and treating unhealthy substance use in hospitals: advocacy or evidence-based practice? Ann Intern Med 2011;154:72–3.

[Medline]

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Anoop B
Sent: Sunday, May 05, 2013 1:10 PM
To: [log in to unmask]
Subject: Re: All-Cause Mortality With Overweight and Obesity

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________