Dear Craig,
It made me think of the following article
Redefining the ‘E’ in EBM
Tom Jefferson, Lars Jørgensen
https://ebm.bmj.com/content/ebmed/23/2/46.full.pdf
This was published in April 2018
The authors suggested looking for regulatory data and compiling an index to gain insight into missing and what the limits of the reviews are. and to prevent Garbage In Garbage Out “synthesised garbage with a nice Cochrane logo on it.”
Some quotes from the article, which also refers to Prof. Peter Gøtzsche‘s dogged insistence.
“In the last decade evidence has accumulated across a spectrum of different interventions that journal publications cannot be trusted.”
“ When some of us started looking at the alternative sources of evidence for our Cochrane review of neuramin- idase inhibitors for influenza nearly a decade ago, we discovered that below every 10-page trial report lies a far deeper and more complex web of data and information needing attention. That is, if the trial was published in the first place.
The first problem is sheer bulk. For every page of journal article, there may be up to 8000 pages of regulatory data on the same clinical trial. We call it a compression factor.
The next problem is bias.“
“We reasoned that even the most faithful servant of evidence would not be able to publish a 10-pager based on a regulatory report without a radical selection of information and data. As we have no idea what the criteria for choosing which plum to publish are, this introduces unfathomable bias. Sometimes the bias is so bad that it distorts single trial reports, but it also distorts the findings of systematic reviews, as our neuraminidase inhibitor story shows.”
“The catalyst to this change was the Nordic Cochrane Centre’s dogged insistence of access and the European Union Ombudsman’s support which ultimately led the European Medicines Agency to change its policies.”
“The thing about clinical study reports is not just that they provide missing information on such pedestrian details like what’s in a placebo or even what it looks like. They also provide sufficient data for carrying out stratified analyses, and more often report patient-relevant outcomes lay bare the shabby way in which we currently look at and analyse harms.
So, should we ignore evidence from journal articles? If steps are not taken urgently to address the situation, then ‘probably’ would be our answer. By the law of Garbage In Garbage Out, whatever we produce in our reviews will be systematically assembled and synthesised garbage with a nice Cochrane logo on it.”
The authors then explain how to put the E in EBM with the example of HPV vaccines:
“How do we redefine the ‘E’ paradigm? We recently published an index of all prospective comparative studies on human papillomavirus vaccines we could find.”
(Think Alltrials)
“Clinical study reports are still commercial documents, but they are written for the wise and cannot, and should not, omit anything (although they may have internal inconsistencies. If there are distortions, they may be approved or overlooked by regulators.
Indexing, at least for now, is more resource-intensive than electronic database searches. The two can be run together and perhaps should be combined. But looking for regulatory data and compiling an index will give us a very good idea of what we are missing and what the limits of our reviews are.“
The authors finish with:
“Ethics and Evidence both begin with ‘E’.
Wouter
Dr W Havinga, Locum GP (family doctor)
########################################################################
To unsubscribe from the EVIDENCE-BASED-HEALTH list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=EVIDENCE-BASED-HEALTH&A=1
|