The way I see it is that we currently have two systems
1. A regulatory system whereby all data that will be collected
are declared beforehand and are provided to the regulator where they are
scrutinised by dedicated professionals. The regulator is mandated to make a
decision on behalf of society. This involves a small cast of players: a few
sponsors and a few regulators.
2. A system whereby results of clinical trials are published in
journals with no systematic control as to what gets into the public arena. Many
of the analyses are amateurish with, for example, persons who don't even understand
the basics of randomisation or statistical theory involved in analysing and interpreting
data. However this system, whatever its weaknesses, has the advantage of being
far more visible to society at large and stimulating an ongoing debate of
results, involving a large number of persons. This evolving debate is the way
we increase our understanding.
System 2 is open to manipulation and is indeed sometimes
distorted by commercial interest. System 1 is insulated far too much from
public debate and there is the potential danger that decisions made by the
regulator are only capable of being scrutinised by the sponsor. (Although, the FDA
in particular, tries to make sure that this is not the case.)
My proposal, however, is not regarding the reform of system 2.
In fact I think that he whole idea of publishing results of clinical trials in
journals is out of date. It serves two classes of persons: those who work in
pharma marketing and academics wishing to polish their CVs. It's system 1 I wish
to reform, since I regard 2 as being incapable of delivering checkability. My
suggestion as to how the situation should evolve is the following
The regulatory system becomes the means of publishing the results
of trials. The reason that sponsors gain huge sums of money for selling pills
is because the drug development process adds a huge amount of information at great
cost on dose and effects etc. It is this information, as much as the molecule
itself that provides value. We should have an attitude as regards drug
licensing which is the same as we have regarding patenting: you can either keep
your invention secret or you can patent it but a patent requires that you place
in the public domain how it works.
We implement this by requiring electronic publishing of the results
as part and parcel of the dossier submitted to the FDA. The sponsor states as
part of the submission, how the data will be made available (including what guarantees
will provided for confidentiality) and no marketing of any kind is allowed
until the data are published in such a repository to a standard deemed
acceptable by the regulator. In fact one way that this could work is that the drug
application consists of a first draft deposit of data and sponsor's
interpretation in an electronic regulatory 'journal' (of course if this were
ever printed it would be huge). Only when the deposit has been revised as
requested by the editor (the FDA etc) and the reviewers, does the regulator
reveal if approval is granted or not. The deposit remains online whether or not
approval has been granted. A condition of getting an IND is that publication in
the elctronic depository will take place.
The logical extension of this, in my opinion is that we should
stop publishing primary results of clinical trials in journals (at least as
regards trials in phase I-III). Journals should be reserved for publishing debates,
opinions, re-evaluations, medical hypotheses etc and basically as a means by
which the wider community can discuss the results that are provided as part of
the regulatory process.
I have put it elsewher this way: we are moving from an era of private
data and public analysis to an era of public data and private analysis. In the
past the researcher kept his or her data under wraps and made his or her
preferred analysis public. In the future we will share data and ther will be dozens
of different analyses.
Stephen
Stephen Senn
Professor of Statistics
School of Mathematics and Statistics
University of Glasgow
15 University Gardens
Glasgow G12 8QW
The University of Glasgow, charity number SC004401
It is important to avoid rhetoric and maintain rigor if the debate
is going to advance to solving our common goal of improving trial quality.
- It does not follow from "checkability [I]s being more important than motive when
judging quality" that we cannot address both.
- Checkability doesn't explain why industry
sponsored trials – that pass the same 'checks' as non industry sponsored
trials, are more likely to yield a positive result (for just one example
see: http://www.ncbi.nlm.nih.gov/pubmed/17493109)
- The choice between an "aircaft built by Boeing for profit and flown by British
Airways for profit" or
"a plane bulit by
staff in the Department of Aeronautics at your local university and flown
by decicated and enthusiastic amateurs?" is a
false dichotomy. The choice is not between industry sponsored
professionals and university funded amateurs, but between two groups of
professionals – one group sponsored by industry, and the other not. But
let's continue with the airline analogy since Stephen brought it up. Of
the seven airlines awarded 5-star rating by Skytrax (Qatar, Cathay
Pacific, Kingfisher, Asiana Airlines, Malaysia Airlines, Singapore
Airlines, and Hainan Airlines), Qatar, Malaysia are wholly owned by
government, and Asiana was initially owned in full (and now 50% owned) by
the South Korean government.
I have flown on Malaysian Airlines, and found it more than
adequate.
Jeremy Howick PhD, MSc, PGCert, DipSoc, BA
MRC/ESRC Postdoctoral Fellow
Centre for Evidence-Based Medicine
Dear Jo,
You raise some interesting points but I am not sure that they
are relevant to this list. The reason that I have a statement of religion on my
declaration is the same as the reason that I have a statement of politics and a
statement of nationality etc it is because I cannot claim to be unifluenced by
such things. If motives are important (and remember that I have stressed
checkability as being more important than motive when judging quality) then my
conclusion was that people need to know such things.
For example, I have been asked to give advice on a meta-analysis
of the effect of previous abortion on subsequent pregnancy. I don't thing my
views on the statistics of this are affected by my religious views but
somepeopls may think they are, so I maintain a full list of interest in which I
try to state everything.
In the end, I maintain that regulation can work provided it is
implemented properly. This is the way I often put it 'which would you rather
take for your next trip to Amercica? An aircaft built by Boeing for
profit and flown by British Airways for profit or a plane bulit by staff in the
Department of Aeronautics at your local university and flown by decicated and
enthusiastic amateurs?'
Stephen
Stephen Senn
Professor of Statistics
School of Mathematics and Statistics
University of Glasgow
15 University Gardens
Glasgow G12 8QW
The University of Glasgow, charity number SC004401
I just wanted to say how much I
agreed with your post below on COIs, but I was too bogged down with a final
assignment to join in. I have read your declaration and I see we agree on many
other things. I have a question. What would you say to people like my
stepson who scoff at my belief in God, as the Creator? My belief is
based on the evidence that is all around me and what I have learned
about the universe and the brain. Because people have heard
about the big bang and read Dawkins they claim that science has 'Proved'
that God does not exist! I have tried to explain the difference between
evidence and proof, which is purely a mathematical and legal
concept that is meaningless in science. I have tried to explain
that science can only provide evidence it can not provide proof,
which is inferred or not by the amount of evidence on either side.
I have tried pointing out that
these theories only show a tiny part of the puzzle about how the universe was
created. They don't explain how positive and negative energies defied all the
laws of physics to come together and cause the bang unless something more
powerful made this possible. Darwin does not explain how life began
or appeared he only explains how it might have evolved but not how the process
was started. Darwin was a Reverend, biology was really only a hobby, I
think he would have been horrified that his theories were seen now as
evidence of the non-existance of God.
I am going on holiday for a week
tomorrow so I am trying to keep out of any EBH debates until I get back LOL. I
will have my £20 Android phone, which will keep me
in the loop but I am not very goodwriting with it
yet.
As a sinner with many ties to
industry (I maintain a declaration of interest at http://www.senns.demon.co.uk/Declaration_Interest.htm
) I would say "no" to Ash Paul's question. Part of the problem
with this approach is that it does not go far enough. "Trust nobody"
should be the motto not "mistrust some". It is checkability that
matters. I agree with Jo that others have axes to grind. Epidemiologists, just
like journalists (remember them and phone hacking?) further their careers by
bringing bad news.
As an instance of the dangers
of concentrating on the pharma dimension consider the chapter "Is
Mainstream Medicine Evil? " in Ben Goldacre's otherwise excellent Bad
Science. He describes a number ofways that you can massage your results without
acknowledging that the drug regulatory framework with its emphasis on declared
protocols and statistical analysis plans, not to mention re-analysis of results
by regulators has a much tighter control of this than does medical publishing.
Coincidentally, this morning on
the Allstat list, Allun Reese drew attention to work by Keith Baggerly and
Kevin Coombes and their work on forensic bioinformatics and how they have
discovered that it is often very difficult to reproduce published results.
I usually add to my declaration
of interest, 'the author is an academic whose career is furthered by
publishing' but journals often remove it.
By the way, next year is the
centenary of Piltdown Man.
Stephen
Stephen Senn
Professor of Statistics
School of Mathematics and Statistics
University of Glasgow
15 University Gardens
Glasgow G12 8QW
The University of Glasgow, charity number SC004401
I completely agree with Ash and
Fiona Godlee regarding COI. I also agree that there should be absolute openness
and transparency over any ties to industry; and I assume this includes
commerce and any potential source of benefits. The reader should
always know if and how writers could benefit but I am slightly
nervous about the use of a total ban. This ban would really
only target writers who have already declared their conflicting
interests. How do we know who else has ties to
industry if they do notdeclare them, or if they actively conceal these.
However it
isn't only money, how can we tell what other axes
writers might have to grind apart from financial gain? Eg If you agree with
my dodgy findings, I will agree with yours. What does the term
'ties to industry' actually mean it is a bit ambiguous so
exactly how would we
define this? If it is too broad it could include anyone with
any remote tie to any industry, and if it is too narrow people will
wriggle through. So where would a ban start and end? Is
there not a better way to deal with the whole COI
problem? In the meanwhile I will continue to apply a large pinch of salt
when I am reading reviews and reports of findings.
Fiona Godlee has written a great Editorial in this week's BMJ:
Turning the Tide on Conflicts of Interest
Cite this as: BMJ 2011;343:d5147
At the end of the Editorial, she has posed this question to
everybody: ..... should
the BMJ repeat the NEJM’s experiment and ban editorials and
clinical reviews from authors with ties to industry? I’d
welcome your views in rapid responses.
Could I please request members of this group to give their views
to the BMJ in rapid responses. COI issues are inextricably linked to the
ultimate success or failure of EBM, and we, in this
group, should have the moral courage to make our voices heard to the
BMJ.
Dr Ash Paul
Medical Director
NHS Bedfordshire