Thanks to all for this beautiful summer thread.

I stand on the same line as Stephen Senn

I agree that the problem must be tackled, but first we need see what the
problem is.

We are talking about narrative reviews.
These are commentaries which need to be written by experts in the field.
To be an expert of research, someone must have practice of research.
For a number of reasons (normative, financial and practical), it is
almost impossible nowadays to conduct independent research
Therefore, to become an expert one must somehow depend on industry funds.

The choice then is between having reviews written by somebody with some
linkage (and dependency) from the industry, having them written by
someone who has little idea of what is writing about, on having no
reviews at all.

I clearly prefer the first hypothesis. I also agree that full disclosure
is imperative and that it is not enough.

There is another fallacy that hasn't be mentioned: the "ad hominem"
fallacy. That is, to judge *who* say something or *why*, rather than the
*arguments* that s/he put forwards.

To judge the plausibility and the fairness of the arguments is the job
of editors and of their choice of reviewers and peer reviewers.

  Even the worst criminals have the right to expose their point of view.
That has to be carefully scrutinized, of course, but it still must be
listened and then could be accepted or rejected by the judge an jury,
who are responsible for protecting the community from the consequences
of accepting false arguments,

What is the duty of the judge in courts is the duty of editors and peer
reviewers in the publishing business (with different tools: I am net
saying that authors are criminals).
  The poor quality of reporting, the unfairness of reviews, the lack of
presenting contrasting points of view in current publications is not
just the fault of writers: it is a consequence of the acquiescence of
most editors towards the industry (large sponsored trials and reviews
from "opinion leaders" do increase the Impact Factor, and large Impact
Factors increase publisher's satisfaction) and their batteries of
low-quality and sponsored peer reviewers.

Yesterday I was reading the report of a publicly funded (if scientists
are biased by money, then public millions should have the same effect of
private money, and producers of CT scanners and healthcare organizations
also have financial interests), large trial on lung cancer screening
just published on the NEJM. This is a very important study with
important results, which will be widely read and quoted for years.
Still, the lack of proper review and the number of reporting flaws is
impressive: their primary outcomes are survival and incidence of cancer,
yet they fail to present proper actuarial curves. Cases occurring after
missing a screening are combined with those occurring after the
screening period (rather than considered as interval cancers), there is
no mention of the time when they started to count cancers as occurring
after the screening period rather than interval cancers. They fail to
give an estimate of potential overdiagnosis (at least 30% of cancers
detected by CT screening, according to my raw computation), just to name
a few.
If I would write a letter about it, I would complain more with the
reviewers than with the authors.


Therefore, it is my opinion that the editor of the BMJ is simply giving
up her duty and responsibility, proposing to refuse to judge what she is
supposed to judge.

It is this the main field of the battle, and the only measure that I can
propose, in addition to exercising judgment, is to abandon anonymous
peer review (would it be any better a football game with an anonymous
arbiter?) in favor of open review and to disclose the conflicts of
interests of peer reviewers as well as of the authors. But no rule can
succeed by itself, unless associated to a strong commitment to improve
the quality of the process of peer review of commentaries as well as of
original research.

As proposed by Stephen, this approach would have the advantage of non
being limited to face commercial conflicts of interests, but also those
arising from competition for public funds or from careers in the health
systems, or any other conflict or flaw.


This, and not the "ad hominem" fallacy should be in my opinion editor's
response to the problems posed by the conflicts of interest.


regards,
Piersante Sestini

On 11/08/12 14:02, Stephen Senn wrote:
> I am certainly not proposing that we do nothing to improve the reporting of pharma trials. Jeremy is no doubt unaware of the proposal that I made at my inaugural lecture, Figures, Figurers and Pharmaceuticals at University College London in 1997 (which is understandable and forgivable) but which I have made in print elsewhere since, that data from clinical trials should be made publicly available(1). I also think and have said so in print that the situation on authorship of pharma trials needs to be improved (1,6) but that many academic lab heads should also ask themselves whether their presence on publications is justified.
>
> As regards trusting results, my solution is that everybody (including but not restricted to the pharma industry) should be much more open about sharing data (This is one of the excellent things about Cochrane but it comes with a warning: you should pay careful attention to the data Cochrane provides but you should be critical about the results of any analyses.) This includes being open with patients and healthy volunteers(3,4). Partly under pressure from others, pharma is taking steps in the right direction but academia should not be complacent. I think that many academics are blissfully unaware as to what changes in their own practice will be necessary (2,5,7).
> Stephen
>
> References
> 1. Senn, S. J. (2002), "Authorship of Drug Industry Trials," Pharmaceutical Statistics, 1, 5-7.
> 2. Senn, S. J. (2006), "Sharp Tongues and Bitter Pills," Significance, 3, 123-125.
> 3 Senn. S.J. (1997), "Are Placebo Run Ins Justified?," BRITISH MEDICAL JOURNAL, 314, 1191-1193.
> 4. Working Party on Statistical Issues in First-in-Man Studies. (2007), "Statistical Issues in First-in-Man Studies," Journal of the Royal Statistical Society, Series A, 170, 517-579
> 5. Senn, S. J. (2002), "Maintaining the Integrity of the Scientific Record. Scientific Standards Observed by Medical Journals Can Still Be Improved," BRITISH MEDICAL JOURNAL, 324, 169.
> 6. Senn, S. J. (2002), "Rules for Publication of Industry-Sponsored Clinical Trials," in Rules for Publication of Industry Sponsored Trials, eds. G. Stock and M. Lessl, Berlin: Ernst Schering Research Foundation.
> 7. Senn, S. J. (2005), "Bitter Pills and Puffed Trials," PLoS Medicine, 2.
>