I am certainly not proposing that we do nothing to improve the reporting of pharma trials. Jeremy is no doubt unaware of the proposal that I made at my inaugural lecture, Figures, Figurers and Pharmaceuticals at University College London in 1997 (which is understandable and forgivable) but which I have made in print elsewhere since, that data from clinical trials should be made publicly available(1). I also think and have said so in print that the situation on authorship of pharma trials needs to be improved (1,6) but that many academic lab heads should also ask themselves whether their presence on publications is justified.
As regards trusting results, my solution is that everybody (including but not restricted to the pharma industry) should be much more open about sharing data (This is one of the excellent things about Cochrane but it comes with a warning: you should pay careful attention to the data Cochrane provides but you should be critical about the results of any analyses.) This includes being open with patients and healthy volunteers(3,4). Partly under pressure from others, pharma is taking steps in the right direction but academia should not be complacent. I think that many academics are blissfully unaware as to what changes in their own practice will be necessary (2,5,7).
Stephen
References
1. Senn, S. J. (2002), "Authorship of Drug Industry Trials," Pharmaceutical Statistics, 1, 5-7.
2. Senn, S. J. (2006), "Sharp Tongues and Bitter Pills," Significance, 3, 123-125.
3 Senn. S.J. (1997), "Are Placebo Run Ins Justified?," BRITISH MEDICAL JOURNAL, 314, 1191-1193.
4. Working Party on Statistical Issues in First-in-Man Studies. (2007), "Statistical Issues in First-in-Man Studies," Journal of the Royal Statistical Society, Series A, 170, 517-579
5. Senn, S. J. (2002), "Maintaining the Integrity of the Scientific Record. Scientific Standards Observed by Medical Journals Can Still Be Improved," BRITISH MEDICAL JOURNAL, 324, 169.
6. Senn, S. J. (2002), "Rules for Publication of Industry-Sponsored Clinical Trials," in Rules for Publication of Industry Sponsored Trials, eds. G. Stock and M. Lessl, Berlin: Ernst Schering Research Foundation.
7. Senn, S. J. (2005), "Bitter Pills and Puffed Trials," PLoS Medicine, 2.
Stephen Senn
Professor of Statistics
School of Mathematics and Statistics
Direct line: +44 (0)141 330 5141
Fax: +44 (0)141 330 4814
Private Webpage: http://www.senns.demon.co.uk/home.html
University of Glasgow
15 University Gardens
Glasgow G12 8QW
The University of Glasgow, charity number SC004401
-----Original Message-----
From: Jeremy Howick [mailto:[log in to unmask]]
Sent: 12 August 2011 11:34
To: Stephen Senn; [log in to unmask]
Subject: Senn's further fallacy
Stephen is correct that it is, in a sense, unfair to target one source of
bias (industry bias) when there are others. At the same time, the fallacy
I pointed out stands: TWO WRONGS DO NOT MAKE A RIGHT.
Moreover, Stephen's analogy between sources of bias (from industry, from
political points of view, etc.) and races, besides being polemical, does
not hold up. The police (and society at large) DO IN FACT wonder why
certain socio-economic and ethnic groups are more likely to commit crime.
They take steps to address these problems by implementing programs in
relevant areas. THEY ALSO DO OTHER THINGS IN OTHER AREAS!
Here is another fallacy:
(1) It is unfair to target industry when there are other sources of bias
(2) therefore, let's do nothing.
Industry bias is a problem that SUPERCEDES THE PROBLEM OF "checking
outputs". The different results obtained in studies funded by industry and
those not funded by industry COULD NOT BE DETECTED BY "CHECKING OUTPUTS".
Therefore SOMETHING ELSE NEEDS TO BE DONE.
Jeremy
* please note my email address is now [log in to unmask]
--
Jeremy Howick PhD, MSc, PGCert, DipSoc, BA
MRC/ESRC Postdoctoral Fellow
Centre for Evidence-Based Medicine
University of Oxford
Oxford OX3 7LF
United Kingdom
www.cebm.net
www.primarycare.ox.ac.uk/dept_staff/jeremy-howick/
eu.wiley.com/WileyCDA/WileyTitle/productCd-140519667X,descCd-authorInfo.htm
l
On 12/08/2011 11:07, "Stephen Senn" <[log in to unmask]> wrote:
>The logical fallacy is surely Jeremy's. If a chief of police announced
>that we should do something about black crime and a journalist present
>said surely this is a dangerous policy because by concentrating on crimes
>perpetrated by one ethinc subgroup rather than by type of crime you could
>miss many serious offences, for example offences committed by whites, we
>surely would not accept as a reply 'you have to start somewhere'.
>
>The point I and others have been making is that many meta-analyses are
>misleading and this is a problem that has to be tackled in terms of
>checking output rather than motive. For example I have found Cochrane
>meta-analyses in which the same data have been counted twice. This it
>seems to me is a problem even though the authors are pure as driven snow.
>
>Regards
>Stephen
>
>
>Stephen Senn
>
>Professor of Statistics
>School of Mathematics and Statistics
>Direct line: +44 (0)141 330 5141
>Fax: +44 (0)141 330 4814
>Private Webpage: http://www.senns.demon.co.uk/home.html
>
>University of Glasgow
>15 University Gardens
>Glasgow G12 8QW
>
>The University of Glasgow, charity number SC004401
>________________________________________
>From: Evidence based health (EBH) [[log in to unmask]]
>On Behalf Of Jeremy Howick [[log in to unmask]]
>Sent: 12 August 2011 10:38
>To: [log in to unmask]
>Subject: Ben Djulblegovic's evidence can resolve the debate over declared
>conflicts of interest
>
>Dear All,
>
>It strikes me that both Stephen Senn and Amy Price are committing the
>logical fallacy. It does not follow from:
>
>(1) industry bias is not the only bias
>
>to:
>
>(2) we should do nothing to reduce industry bias.
>
>That would be like saying, "allocation bias is not the only kind of bias,
>so we shouldn't require that reports of randomized trials report whether
>allocation was randomized and concealed".
>
>Reducing one major source of bias is better than not reducing any sources
>of bias.
>
>As an Evidence-Based discussion group, it is also useful for the debate
>to be based on evidence: there is a large body of evidence that drug
>sponsored trials provide misleading results:
>
>Bero, L., F. Oostvogel, P. Bacchetti, and K. Lee. 2007. Factors
>associated with findings of published trials of drug-drug comparisons:
>why some statins appear more efficacious than others. PLoS Med 4 (6):e184.
>Jorgensen, A. W., J. Hilden, and P. C. Gotzsche. 2006. Cochrane reviews
>compared with industry supported meta-analyses and other meta-analyses of
>the same drugs: systematic review. BMJ 333 (7572):782.
>Leopold, S. S., W. J. Warme, E. Fritz Braunlich, and S. Shott. 2003.
>Association between funding source and study outcome in orthopaedic
>research. Clin Orthop Relat Res (415):293-301.
>Lexchin, J., L. A. Bero, B. Djulbegovic, and O. Clark. 2003.
>Pharmaceutical industry sponsorship and research outcome and quality:
>systematic review. BMJ 326 (7400):1167-70.
>Schulz, K. F., D. G. Altman, and D. Moher. 2010. CONSORT 2010 statement:
>updated guidelines for reporting parallel group randomised trials. PLoS
>Med 7 (3):e1000251.
>Yaphe, J., R. Edman, B. Knishkowy, and J. Herman.2001. The association
>between funding by commercial interests and study outcome in randomized
>controlled drug trials. Fam Pract 18 (6):565-8.
>To cite just one example from a common and thoughtful contributor to this
>list (Ben Djulblegovic), "Research funded by drug companies was less
>likely to be published than research funded by other sources. Studies
>sponsored by pharmaceutical companies were more likely to have outcomes
>favouring the sponsor than were studies with other sponsors (odds ratio
>4.05; 95% confidence interval 2.98 to 5.51; 18 comparisons)."
>(http://www.ncbi.nlm.nih.gov/pmc/articles/PMC156458/)
>
>Jeremy
>
>ps: Stephen Send should be applauded and I'm being completely sincere
>for being so open about his list of potential conflicts of interest with
>industry (he has consulted for over 50 pharmaceutical companies and
>receive grants from a dozen of them:
>www.senns.demon.co.uk/Declaration_Interest.htm)
>
>* please note my email address is now [log in to unmask]
>--
>
>Jeremy Howick PhD, MSc, PGCert, DipSoc, BA
>MRC/ESRC Postdoctoral Fellow
>Centre for Evidence-Based Medicine
>University of Oxford
>Oxford OX3 7LF
>United Kingdom
>www.cebm.net
>www.primarycare.ox.ac.uk/dept_staff/jeremy-howick/
>eu.wiley.com/WileyCDA/WileyTitle/productCd-140519667X,descCd-authorInfo.ht
>ml
>
>From: Ash Paul <[log in to unmask]<mailto:[log in to unmask]>>
>Reply-To: Ash Paul <[log in to unmask]<mailto:[log in to unmask]>>
>Date: Fri, 12 Aug 2011 08:28:42 +0100
>To:
>"[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAI
>L.AC.UK>"
><[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAI
>L.AC.UK>>
>Subject: Re: Turning the tide on conflicts of interest
>
>Dear Jacob,
>In a tangential but related development, here's the press release issued
>today by PLOS Medical Journals:
>Authorship Rules For Medical Journals Flouted By Pharma Industry
>http://www.redorbit.com/news/science/2093981/authorship_rules_for_medical_
>journals_flouted_by_pharma_industry/index.html?
>There's also a reference to the following two PLOS articles:
>http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.10
>01072
>and
>http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.10
>01071
>Regards,
>Ash
>
>
>Ash
>Dr Ash Paul
>Medical Director
>NHS Bedfordshire
>21 Kimbolton Road
>Bedford
>MK40 2AW
>Tel no: 01234897224
>Email: [log in to unmask]<mailto:[log in to unmask]>
>
>
>
>From: Jacob Puliyel <[log in to unmask]<mailto:[log in to unmask]>>
>To:
>[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAIL
>.AC.UK>
>Sent: Friday, 12 August 2011, 6:00
>Subject: Re: Turning the tide on conflicts of interest
>
>Dear Jo and Others on the List
>Jo you wrote
>The reader should always know if and how writers could benefit but I am
>slightly nervous about the use of a total ban. This ban would really only
>target writers who have already declared their conflicting interests. How
>do we know who else has ties to industry if they do not declare them, or
>if they actively conceal these.
>
>Are we allowing the best to be the enemy of the good?
>
>We can tackle undeclared conflicts as the next step. Let us deal with the
>obvious declared conflicts of interest first.
>
>I see little point in just publishing the declaration of conflict of
>interests. It is editorial cop out.
>By publishing declarations of conflicts of interests, the Editor is
>saying simply, "This author has declared that he has conflicts of
>interest when writing this review/editorial. With all the resources I
>have at my disposal, I am unable to state that this has/has not biased
>his/her judgment. You, as the reader, must make that judgment. If you
>think that the conflict of interest has biased him/her, please disregard
>this review/editorial that we are publishing."
>
>Jacob Puliyel
>Head of Pediatrics
>St Stephens Hospital
>Delhi
>[log in to unmask]<mailto:[log in to unmask]>
>
>
>On Fri, Aug 12, 2011 at 7:42 AM, Dr. Amy Price
><[log in to unmask]<mailto:[log in to unmask]>> wrote:
>I would be willing to submit a rapid response as below...please edit me
>as I am new at this and would only want to say what would accurately
>represent the issue. I suggest we all send in something as the more rapid
>responses that are well written the better it will be. Perhaps this could
>be followed up with a paper dealing with this question to the BMJ by
>those most qualified to do so?
>
>My vote is no because a ban does not solve the core problems and it is
>like shutting the barn door after the cows have left the premises.
>
>Publication bias, (file drawer effect) happens when research results are
>suppressed/disregarded. In meta analysis sources of bias are not
>controlled by the method. Exclusive reliance on published studies could
>artificially inflate study effect size as could the quality or emphasis
>of studies chosen. One way to mitigate this could be for journals to
>insist that all studies rejected and accepted for review be disclosed
>upon request to any subscriber so the study could be replicated. This
>could include pharmaceutical industry/medical device raw data.
>
>Deliberate publication bias can be the result of financial, legal,
>political or cultural agendas and the use of influence to bias the study
>by Œcherry picking¹ where studies failing to produce the desired bias are
>rejected regardless of validity. A declaration of conflict of interest
>and a subsequent ban is not going to solve the problem. The Simpson's
>paradox/ Yule-Simpson effect may erroneously reverse success rates when
>groups are combined with inappropriate causal interpretation of frequency
>data, systematic formal analysis without publication bias and public
>statistical awareness of this element may mitigate this issue.
>
>Unified rejection from the scientific community, mandatory journal
>article retractions and withdrawal of funding for meta
>analysis/systematic review authors and organization who consistently show
>bias would be more effective than epidemiologists grandstanding by
>sending cherry picking/junk science letters to the editor without doing a
>shred of work other than citing their own papers in the complaint.
>
>Other strategies include choosing studies from journals respected for
>their objectivity and training students to follow suit.
>Universal/standardised methods with agreement on coding, weighting, and
>combining results whether by a fixed or random model could mitigate
>unintentional bias as could adding in quality unpublished research even
>with non significant results such as is available from universities and
>laboratories. Sometimes what doesn¹t show significance and why is just as
>important as studies that push all the right numbers buttons.
>
>For example recently a study on iPS cells show by DNA that these are not
>replicated stem cells capable of function but rather in the cytoskeleton
>there are critical regulatory factors missing. I asked several scientists
>in this field did you not even look? The answer was yes but the findings
>were not publishable in our eyes. I asked what about cultured stem cells
>and was told that this was covered by using stem like cells rather than
>stem cells in publishing. What this shows me is that although those in
>the industry could navigate to truth it is highly unlikely that those
>outside of the field even when well educated could do so.
>
>Language matters. This is a serious question for concern in this age of
>multi-disciplinary research. In conclusion ethics, diligence, and the
>standards of Evidence Based Medicine matter. Let each of us do our part
>to uphold them and celebrate with support those who are standing with us
>to fight this good fight.
>
>Best Regards,
>Amy
>
>Amy Price
>Http://empower2go.org<http://empower2go.org/>
>Building Brain Potential
>
>
>
>
>
>From: Evidence based health (EBH)
>[mailto:[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@
>JISCMAIL.AC.UK>] On Behalf Of Michael Bennett
>Sent: 11 August 2011 08:10 PM
>To:
>[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAIL
>.AC.UK>
>Subject: Re: Turning the tide on conflicts of interest
>
>I too would say no to this proposition Š after much introspection and
>consideration. Jo and Stephen make very good points and I will not rehash
>those.
>
>I would be much more excited by the proposition that journals no longer
>publish clinical trials (full reports of all data and outcomes published
>on-line), but concentrate on evaluating those trials using the
>appropriate methodologies of which this list is so aware. I think this
>was suggested by Smith? Will try and dig out the reference..
>
>Mike
>
>Mike Bennett
>
>From: Evidence based health (EBH)
>[mailto:[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@
>JISCMAIL.AC.UK>] On Behalf Of Stephen Senn
>Sent: Thursday, 11 August 2011 22:08
>To:
>[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAIL
>.AC.UK>
>Subject: Re: Turning the tide on conflicts of interest
>
>As a sinner with many ties to industry (I maintain a declaration of
>interest at http://www.senns.demon.co.uk/Declaration_Interest.htm ) I
>would say "no" to Ash Paul's question. Part of the problem with this
>approach is that it does not go far enough. "Trust nobody" should be the
>motto not "mistrust some". It is checkability that matters. I agree with
>Jo that others have axes to grind. Epidemiologists, just like journalists
>(remember them and phone hacking?) further their careers by bringing bad
>news.
>
>As an instance of the dangers of concentrating on the pharma dimension
>consider the chapter "Is Mainstream Medicine Evil? " in Ben Goldacre's
>otherwise excellent Bad Science. He describes a number of ways that you
>can massage your results without acknowledging that the drug regulatory
>framework with its emphasis on declared protocols and statistical
>analysis plans, not to mention re-analysis of results by regulators has a
>much tighter control of this than does medical publishing.
>
>Coincidentally, this morning on the Allstat list, Allun Reese drew
>attention to work by Keith Baggerly and Kevin Coombes and their work on
>forensic bioinformatics and how they have discovered that it is often
>very difficult to reproduce published results.
>
>I usually add to my declaration of interest, 'the author is an academic
>whose career is furthered by publishing' but journals often remove it.
>
>By the way, next year is the centenary of Piltdown Man.
>
>Stephen
>
>
>Stephen Senn
>Professor of Statistics
>School of Mathematics and Statistics
>
>Direct line: +44 (0)141 330 5141
>Fax: +44 (0)141 330 4814
>Private Webpage: http://www.senns.demon.co.uk/home.html
>
>University of Glasgow
>15 University Gardens
>Glasgow G12 8QW
>
>The University of Glasgow, charity number SC004401
>
>From: Evidence based health (EBH)
>[mailto:[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@
>JISCMAIL.AC.UK>] On Behalf Of jo kirkpatrick
>Sent: 11 August 2011 12:35
>To:
>[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAIL
>.AC.UK>
>Subject: Re: Turning the tide on conflicts of interest
>
>Dear Ash and EBH
>I completely agree with Ash and Fiona Godlee regarding COI. I also agree
>that there should be absolute openness and transparency over any ties to
>industry; and I assume this includes commerce and any potential source of
>benefits. The reader should always know if and how writers could benefit
>but I am slightly nervous about the use of a total ban. This ban would
>really only target writers who have already declared their conflicting
>interests. How do we know who else has ties to industry if they do not
>declare them, or if they actively conceal these.
>
>However it isn't only money, how can we tell what other axes writers
>might have to grind apart from financial gain? Eg If you agree with my
>dodgy findings, I will agree with yours. What does the term 'ties to
>industry' actually mean it is a bit ambiguous so exactly how would we
>define this? If it is too broad it could include anyone with any remote
>tie to any industry, and if it is too narrow people will wriggle through.
>So where would a ban start and end? Is there not a better way to deal
>with the whole COI problem? In the meanwhile I will continue to apply a
>large pinch of salt when I am reading reviews and reports of findings.
>
>Best wishes ,Jo
>
>From: Ash Paul <[log in to unmask]<mailto:[log in to unmask]>>
>To:
>[log in to unmask]<mailto:EVIDENCE-BASED-HEALTH@JISCMAIL
>.AC.UK>
>Sent: Thu, 11 August, 2011 8:34:15
>Subject: Turning the tide on conflicts of interest
>Dear colleagues,
>Fiona Godlee has written a great Editorial in this week's BMJ:
>Turning the Tide on Conflicts of Interest
>Cite this as: BMJ 2011;343:d5147
>At the end of the Editorial, she has posed this question to everybody:
>..... should the BMJ repeat the NEJM¹s experiment and ban editorials and
>clinical reviews from authors with ties to industry? I¹d welcome your
>views in rapid responses.
>
>Could I please request members of this group to give their views to the
>BMJ in rapid responses. COI issues are inextricably linked to the
>ultimate success or failure of EBM, and we, in this group, should have
>the moral courage to make our voices heard to the BMJ.
>
>Regards,
>
>Ash
>Dr Ash Paul
>Medical Director
>NHS Bedfordshire
>21 Kimbolton Road
>Bedford
>MK40 2AW
>Tel no: 01234897224
>Email: [log in to unmask]<mailto:[log in to unmask]>
>
>
>
>
>
|