Aside from concerns about power, the studies may be 'good quality' in some respects, but if they are not pre-specified in a protocol- which they are definitely not in many fields, then anything is possible.
Another big reason for non-replication.
The culture of transparency needs to change, EBM has made huge advances, but many other fields lag behind. Good for Kahneman drawing the attention of a wider field to this, along with Ioannidis, etc
Frances
Frances Gardner,
Professor of Child and Family Psychology
Fellow of Wolfson College
Centre for Evidence-Based Intervention
Department of Social Policy & Intervention
University of Oxford, 32 Wellington Square, Oxford OX1 2ER, UK
tel:44-1865-270325 / 270334 email: [log in to unmask]
https://www.spi.ox.ac.uk/people/profile/gardner.html
publications: http://scholar.google.co.uk/citations?hl=en&user=srRcFJgAAAAJ&view_op=list_works
*** please note that this email address has now expired: [log in to unmask] ****
please use: [log in to unmask] or [log in to unmask]
________________________________________
From: Evidence based health (EBH) [[log in to unmask]] on behalf of Djulbegovic, Benjamin [[log in to unmask]]
Sent: 22 February 2017 12:24
To: [log in to unmask]
Subject: Re: Nobel Prize winner admits mistakes
Anoop, thanks for sending this - it takes greatness of the Nobelist to reach this stage of reflection; science, education and society would be so much better off if all of us emulate this behavior.
But , to answer your question, in general, the effects of intervention should be accepted (as closer to the "truth") when its effects are greater than combined effects of bias and random error. Even well designed small studies are prone to random error, which is what Kahneman is alluding in his comments.
Hope this helps
Ben
Sent from my iPad (please excuse typos )
> On Feb 21, 2017, at 7:57 PM, Anoop Balachandran <[log in to unmask]> wrote:
>
> A Nobel Prize-winning researcher has admitted on a blog that he relied on weak studies in a chapter of his bestselling book.
>
> http://retractionwatch.com/2017/02/20/placed-much-faith-underpowered-studies-nobel-prize-winner-admits-mistakes/
>
> What I don't understand is how can small sample studies be erroneous if they are of high quality?I agree the CI 's will be wide, but that shouldn't make it wrong. Any thought?
>
> Thanks
|