If I may, I would like to add my five cents to this debate on the feasibility of publishing an SR along with a trial. Without going into too much detail, my research activities include both clinical trials and systematic reviews and I am caught up in this exact dilemma at the moment. Assuming that there are no funds to hire a group to undertake a systematic review on your behalf (which usually range between $5,000 - $50,000 and have surpassed the half million dollar mark) depending on the complexity, number of anticipated trials, volume of data extracted and analyzed, and form of dissemination, then we have several issues to consider:

1)      Does the clinical team also have the skills, knowledge and experience, and available resources to undertake a proper systematic review of the literature?

2)      Does the clinical team have the time, energy and motivation to undertake a proper systematic review of the literature?

3)      Will the return on invest be worthwhile for the team to undertake a proper systematic review of the literature at this time? In other words, will this get accepted in a well-read journal or be more of the same since the evidence is probably not markedly affected by the new evidence.

Now the issue of handing the data to another team that can do the proper systematic review should not be an issue as all reports of trials published today should have enough information to allow any interested groups in extracting the needed data to allow them to update their systematic reviews. Cochrane asks us to update our reviews every two years or whenever new evidence may potentially change the conclusions of our reviews. Even so, it is difficult to keep up. Personally, I have worked on over 40 systematic reviews over the years. The only ones that are updated on a regular basis (to the best of my abilities) are the Cochrane reviews. With the other reviews, either they were performed on a consultancy basis, for a specific government report, are too overwhelming to update on a regular basis, or are no longer part of my research agenda. Since my work is published, anyone else can freely pick up where I left off, but in reality it doesn’t happen because preparing a proper systematic review of the literature is not like writing a narrative review. You can’t sit down with your thoughts and good cup of coffee and let you fingers do the walking. In many cases, it takes weeks and weeks (if not months and years) of painstaking effort by a dedicated team of professionals to bring together something that is meaningful.

Now what is feasible, in my mind and I have suggested this of trialists who try to put a fast and dirty update to a previously published review (that they obviously didn’t do in the first place) to spice up the results of their clinical trials is to add the results of their trial to the previously published review and clearly state that if this was  the ONLY new evidence, then here is what would have changed. But for us to ask every clinician to learn how to perform a proper systematic review in order to publish their results is a bit unrealistic. If go down this path then what about cost-effectiveness and other economic analyses. Surely you as a clinician would also like to know how this trial will affect the long-term costs of care for the patient and the system as a whole. And list of demands can go on and on…

My five (maybe six) cents…

Ahmed

Rich, I see it operationally very simple: a RCT is submitted for publication. The authors indicate a) this is the first RCT addressing a particular question- there is no SR; b) the authors (who presumably justified conducting their RCT based on the existing SR), do indicate that their RCT adds to knowledge base summarized in the existing SR. The editors request from the authors to update the SR or give data to the authors of the existing SR to prepare the updated SR to coincide with publication of the RCT. If this is routinely done, then we are talking of updating SRs with data from 1 RCT only (in most cases), which can be done in real time.  Too naïve?

Best

ben

Hi Ben,

I don’t see an informal review update as the same thing as a bonafide peer reviewed systematic review.  That is why I am still uncertain about this recommendation.  If what we mean is a back of the envelope “I checked Medline and there isn’t anything new” then I agree it is do-able.  But I think a true systematic review is something else.  Even an update of  Cochrane review is a big deal.  So maybe that is where we are (I am) getting stuck in this discussion.  If the standard is to check and make sure there isn’t a new trial published or to discuss one that has been published, I think there would be no argument.

Best

Rich

Colleagues,

While difficulty and practicality may not be acceptable excuses, they are reality, that we can all work towards improving.

 

The current situation is that trials may be funded but that funding currently does not include a systematic review (I am talking about the one that would be done after the trial is done to accompany the publication of the trial)(and lets talk about foundation and government sponsored ones not industry, just for the sake of a cleaner argument-no pun intended). 

 

So if today, the BMJ required trialists to do systematic reviews at the time they are reporting their trial results, that would, at a minimum, delay publication—which one can argue would be harmful and not fair to taxpayers who want to know the trial result soon, assuming the trial needed to be done in the first place.  More likely it would quash publication for a while until the review could be done properly, and with current reality, it would likely be infeasible.  The authors would likely find another outlet for publication.  If all journals joined, then the results would come out elsewhere, without peer review.  Taxpayers would again get the short end of the stick.

 

In an ideal world, it would be great if funding of a trial came with simultaneous funding of another team to do a systematic review that would be up to date and written up the moment a trial result is ready for publication.  So I ask, what do we do in the meantime, from a practical perspective?

 

Best
Rich

 

 

Richard Saitz MD, MPH, FACP, FASAM

Editor, Evidence-Based Medicine

 

Professor of Medicine & Epidemiology

Boston University Schools of Medicine & Public Health

Boston Medical Center

 

801 Massachusetts Avenue, 2^nd Floor

Boston, MA 02118

617 414 7744 (direct)

617 414 4676 (fax)

ebm.bmj.com

 

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Jeremy Howick
Sent: Monday, October 15, 2012 5:44 AM
To: [log in to unmask]
Subject: Re: effects of HRT on CVD events RCT

 

Dear All,

 

It is great that Iain made it clear where he stands (I was never in doubt), however the need for systematic reviews transcends personal views. Before we pay taxes to support a new trial, we need to know whether the question has been answered. Failure to do so can literally cause deaths (http://www.jhu.edu/jhumag/0202web/trials.html). Not putting research into context also causes confusion:

 

 - What do I do as a patient: follow the results of this trial, or other evidence?

 - What does the clinician do: follow the results of this trial, or other evidence?

 - What do do policy makers do: follow the results of this trial, or other evidence?

 

It may turn out, of course, that there is no straightforward answer and that more research (or scepticism) is required. But even to reach the conclusion that more research is required, we must consider all relevant studies! So there is no excuse for not doing a systematic review, and putting the research into context. If a journal's aim is to help patients and doctors they need to take this into account.

 

Trish is correct to note that expertise is required to conduct systematic reviews. At the same time 

 

Best wishes,

 

Jeremy

 

ps: It is also very curious that the trial was published in 2012 when it seemed to be completed in 2003.

 

 

* please note my email address is now [log in to unmask]

--

 

Jeremy Howick PhD

Department of Primary Care Health Sciences

Centre for Evidence-Based Medicine

University of Oxford

Oxford OX3 7LF

United Kingdom

www.cebm.net

www.primarycare.ox.ac.uk/dept_staff/jeremy-howick/

eu.wiley.com/WileyCDA/WileyTitle/productCd-140519667X,descCd-authorInfo.html

 

From: Trish Groves <[log in to unmask]>
Reply-To: Trish Groves <[log in to unmask]>
Date: Monday, 15 October 2012 09:52
To: "[log in to unmask]" <[log in to unmask]>
Subject: Re: effects of HRT on CVD events RCT

 

Thanks, Iain
I agree.
It'll need triallists with the time and expertise to do systematic reviews, trial funders willing to fund those reviews, and ethics committees insisting that triallists do a new systematic review if none exists.
Best wishes
Trish
Dr Trish Groves, Deputy editor BMJ, and Editor-in-Chief BMJ Open
bmj.com
bmjopen.bmj.com
Twitter @trished

(By Blackberry: please forgive any typos)

---

  From: "Iain Chalmers" [[log in to unmask]]
  Sent: 15/10/2012 09:04 CET
  To: "Djulbegovic, Benjamin" <[log in to unmask]>; Trish Groves; "EVIDENCE-BASED-HEALTH" <[log in to unmask]>
  Subject: RE: effects of HRT on CVD events RCT

 

Ben asks me to clarify my position.  As a tax-paying funder of research and a user of research reports I want to be assured that plans for a particular piece of new research have taken systematic account of what can be known from relevant previous research, and I want to know what contribution new findings make to the totality of relevant information on the question addressed. Those proposing and reporting new research need to fulfil these scientific, moral and economic responsibilities.  They may be able to do this by reference to relevant systematic reviews done by others – particularly now that they are helped by PubMedHealth and PROSPERO – but if not, then they need to do the jobs themselves.

 

Trish writes that “doing a systematic review takes expertise that not all trialists have, and reporting one properly takes a whole paper.”  These are not acceptable excuses for the damage to patients and volunteers (let alone the scientific, moral and economic consequences) that can result from failures to assess what is already known.  Putting the issue in the ‘too difficult’ box is not a defensible response. As in so many other respects, science needs to change in ways that take account of these sometimes very dangerous deficiencies in the ways we operate.

 

Best wishes, Iain

    

---

From: Djulbegovic, Benjamin [mailto:[log in to unmask]]
Sent: 13 October 2012 16:25
To: Trish Groves; EVIDENCE-BASED-HEALTH
Cc: Iain Chalmers
Subject: RE: effects of HRT on CVD events RCT

 

Hi Trish,

Thanks for this reply.

I am aware of logistical complexities of mandating doing a SR/MA from the trialists. I am also not sure how well the Lancet policy is enforced- I have certainly seen the papers in which the authors did perform SR/MA and those where only cursory reference to a “systematic” review process had been made.

 

However, if the goal of a RCT is to inform physicians’ decision-making, it is not clear to me how it is possible to “discuss [the] completed trial in the light of the totality of existing evidence without doing “your own systematic review"? After the BMJ published this latest HRT trial, what should I, as a practitioner do? (Re-)start prescribing HRT (with a goal to prevent heart disease, osteoporosis, improve life expectancy etc)? The only way I can really make this decision is to understand the results of this trial after it was pooled with other similar trials (assessing bias & getting estimate of the impact of random error, of course, remains integral part of this process).  

I am cc my reply to Iain as it appears that I have misunderstood his writings.

Many thanks for willingness to engage in an open dialogue with the authors and representatives of the public (which is how I see my role). I hope other editors will follow your example.

Ben

Ps re the HRT paper, I do suggest that you commission a rapid response/review (SR/MA) from Klim or his colleagues to settle the question.

From: Trish Groves [mailto:[log in to unmask]]
Sent: Saturday, October 13, 2012 10:31 AM
To: Djulbegovic, Benjamin; EVIDENCE-BASED-HEALTH
Subject: Re: effects of HRT on CVD events RCT

 

Hi Ben
Thanks for the prompt.

This is a good question, but a much misunderstood one.

Doing a systematic review takes expertise that not all triallists have, and reporting one properly takes a whole paper.

So insisting on doing and then reporting a new systematic review in the discussion section of every RCT papers is neither feasible or desirable - what use is a systematic review that's mentioned in a couple of sentences and is impossible to evaluate?

If you ask Iain directly (which I've done), he says it's not what he ever intended.
He meant that triallists should ensure before writing their protocol, let alone their paper, that the trial will add to what's known (by searching properly for relevant systematic reviews) and should discuss their completed trial in the light of the totality of existing evidence.
That's not the same at all as "do your own systematic review."

The Lancet's instructions for authors asks for a section in the discussion that puts the results into context and says, for RCTs, "authors are invited [ie not mandated] to either report their own up to date systematic review or cite a recent systematic review of other trials".  
Most Lancet authors take the second option, and that's fine.

At the BMJ we ask for a structured discussion in every paper:
www.bmj.com/about-bmj/resources-authors/article-types/research

This includes a subsection on "strengths and weaknesses in relation to other studies, discussing important differences in results".

Every paper also has a box stating what was already known and what the study adds.

For the record, the Schierbeck et al study you're discussing did that, citing previous RCTs and meta-analyses.

Anyway, do please bring to bmj.com (via Rapid Responses) the debate that's taking place on this listserve. We will, of course, ask the authors to respond in turn.

Best wishes
Trish

Dr Trish Groves, Deputy editor BMJ, and Editor-in-Chief BMJ Open
bmj.com
bmjopen.bmj.com
Twitter @trished

(By Blackberry: please forgive any typos)

---

  From: "Djulbegovic, Benjamin" [[log in to unmask]]
  Sent: 13/10/2012 13:48 GMT
  To: [log in to unmask]
  Subject: Re: effects of HRT on CVD events RCT

 

Indeed, Klim

So, what happens if the latest study is pooled with all other (eligible) studies? I believe that some journals mandate ( Lancet, not sure about BMJ and other journals) placing the findings of the latest study in the context of the totality of existing evidence. Iain Chalmers & Mike Clarke ( cc on this reply) have convincingly argued that all research should start with research synthesis ( systematic review to justify uncertainties to undertake a new clinical trial) and finish with systematic review ( placing findings of the study in the context of all available evidence). If this is not done, (reigniting old) confusion is bound to happen.

 

Because some of the members of this group are editors, I want to directly invite them to respond to this challenge: why their journals have not adopted the Lancet policy of mandating systematic review from all authors submitting a clinical trial report ( at least RCT) to their journals? Specifically, it would be good to hear from the BMJ editors: was this oversight or they do not think such policy should be put in place? 

In a new age of transparency and democratization of scientific discourse, I hope we hear from the editors as this is really important issue.

Thanks 

Ben

 

On Oct 13, 2012, at 4:52 AM, "Klim McPherson" <[log in to unmask]> wrote:

Thanks Anoop,

 

My rapid response:

 

Schierbeck and colleagues suggested that “initiation of hormone replacement therapy in women early after menopause significantly reduces the risk of the combined endpoint of mortality, myocardial infarction, or heart failure” [1]. They invoke the age hypothesis, which remains one of the abiding critiques of the WHI study, namely that women well past the menopause in WHI responded differently to exogenous hormones than do symptomatic menopausal women, with respect to cardiovascular risk.

 

All such arguments have perforce to ignore the only double blind placebo controlled studies of such symptomatic women, a synthesis of which was published in the BMJ [2] and then the Lancet [3] including unpublished studies (a process resisted by the manufacturers) and summarised in the BMJ [4].  The evidence from a synthesis of some 200-efficacy studies submitted for licensing of HRT products strongly suggests that the risk is the same as found among older women in WHI as among women with menopausal symptoms. Purveyors of the age hypothesis need to take on board all of the evidence, especially from well-controlled RCT’s.

 

Yours,

Klim McPherson

New College

Oxford OX1 3BN

 

References

 

Schierbeck LL, Rejnmark L, Tofteng CL, Stilgren L, Eiken P, Mosekilde L, Køber L, Jensen JE. Effect of hormone replacement therapy on cardiovascular events in recently postmenopausal women: randomised trial. BMJ 2012; 345: e6409.

 

Hemminki E, McPherson K. Impact of postmenopausal hormone therapy on cardiovascular events and cancer: Pooled data from clinical trials.  BMJ; 1997; 315:149-153.

 

 

Hemminki, E, McPherson K.  The value of drug-licensing documents in studying the effect of postmenopausal hormone therapy on cardiovascular disease.  Lancet, 2000; 355: 566– 8

 

McPherson K, Hemminki E. Synthesising licensing data to assess drug safety. BMJ, Feb 2004; 328: 518-520

From: Anoop Balachandran <[log in to unmask]>
Reply-To: Anoop Balachandran <[log in to unmask]>
Date: Friday, 12 October 2012 13:45
To: "[log in to unmask]" <[log in to unmask]>
Subject: Re: effects of HRT on CVD events RCT

 

My comments:

1. The sample size is small. In the WHI study, they had 17,000 in one arm and 10,000 in other arm. So the CI's were very wide. 

2 No placebo-controlled group. 

 

here is a rapid response for that article in BMJ: 

 

Schierbeck and colleagues suggested that “initiation of hormone replacement therapy in women early after menopause significantly reduces the risk of the combined endpoint of mortality, myocardial infarction, or heart failure” [1]. However, after a closer look at their study, the results do not appear so straightforward.

Indeed, after a mean duration of 10.1 years of randomised treatment in 1006 women, the primary endpoint occurred in 17 fewer subjects (33 vs 16) in treated group than in control group. However, women in the control group were 0.47 years (about 5.7 months) older than those in the treated group [1].

In the cardiovascular secondary prevention trial Cholesterol and Recurrent Events (CARE) [2], patients were randomized to 40 mg daily of the cholesterol-lowering drug pravastatin or placebo. In the 4159 participants, statin therapy was associated with 97 fewer revascularizations [2]. It has been calculated that the average delay of revascularization was 0.09 years (33 days) over 5 years [3]. Although it is difficult to make a similar calculation for the study by Schierbeck and colleagues [1], it seems unlikely that the projected delay on 10 years of the events defined in the primary endpoint would be longer. In other words, although the authors adjusted the composite endpoint’s results for age, the apparent significant beneficial results may have been seriously flawed.

Furthermore, the assumption that heart protection in fertile women is mediated by premenopausal levels of female hormones is not consistent with epidemiological findings that premenopausal hysterectomy essentially cancels the protection, even in cases with preservation of functioning ovaries [4]. On the contrary, these data [4] suggest that an intact uterus has an important role in the protection of premenopausal women, and likely this is related to the beneficial effect of iron depletion in menstruating women [5] (i.e., the iron hypothesis suggested by Sullivan more than 30 years ago [6]).

If you torture numbers enough they will say anything you want.

Luca Mascitelli, MD
Comando Brigata alpina “Julia”, Udine 33100, Italy

Mark R. Goldstein, MD, FACP
NCH Healthcare Group, Naples, FL, USA

REFERENCES

1. Schierbeck LL, Rejnmark L, Tofteng CL, Stilgren L, Eiken P, Mosekilde L, Køber L, Jensen JE. Effect of hormone replacement therapy on cardiovascular events in recently postmenopausal women: randomised trial. BMJ 2012; 345: e6409.

2. Sacks FM, Pfeffer MA, Moye LA, et al, for the Cholesterol and Recurrent Events Trial Investigators. The effect of pravastatin on coronary events after myocardial infarction in patients with average cholesterol levels. N Engl J Med 1996; 335: 1001–9.

3. Bassan M, Panush N. Treating hypercholesterolemia—without the hype. Am J Cardiol 1997; 79: 1001–3.

4. Kannel WB, Levy D. Menopause, hormones, and cardiovascular vulnerability in women. Arch Intern Med 2004; 164: 479-81.

5. Mascitelli L, Goldstein MR, Pezzetta F. Explaining sex difference in coronary heart disease: is it time to shift from the oestrogen hypothesis to the iron hypothesis? J Cardiovasc Med (Hagerstown) 2011; 12: 64-5.

6. Sullivan JL. Iron and the sex difference in heart disease risk. Lancet 1981; 1: 1293-4.

 

On Thu, Oct 11, 2012 at 9:57 AM, healingjia Price <[log in to unmask]> wrote:

As one who enjoyed profound and immediate benefits I would like to still see this replicated as a blind, industry free trial and also wonder if strategic genomic analysis would be helpful. The 8% rate is a concern what was exclusion criteria?

Amy

Amy Price 

Empower 2 Go 

Building Brain Potential

Http://empower2go.org

Sent from my iPad

On 11 Oct 2012, at 02:38 PM, "Klim McPherson" <[log in to unmask]> wrote:

I would be fascinated to know the dispassionate opinion of this group on a quite large trial of HRT v nothing which appears to negate a great deal of evidence from other RCT's on the effect of HRT on CVD and breast cancer in particular.

 

It is not blind (does that matter and how much). It is funded by HRT Pharma, most of the PI's have advisory roles with the companies. 8% of eligible patients were recruited into the trial ………..

 

Where should it sit in the panoply of evidence on the safety of HRT ??

 

This is an important question because of the unambiguous and profound benefits of HRT for menopausal symptoms but particularly bone density loss, and hence the 'need' to downplay possible harms.

 

 

http://www.bmj.com/content/345/bmj.e6409

 

 

Your opinions would be greatly valued.

 

 

 

Klim McPherson MA Phd FFPH FMedSci

Visiting Professor of Public Health Epidemiology

Nuffield Dept Obstetrics & Gynaecology

Emeritus Fellow of New College  

University of Oxford

Mobile 07711335993

 

 

 

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

_______________________________________________________________________
The BMJ Group is one of the world's most trusted providers of medical information for doctors, researchers, health care workers and patients group.bmj.com. This email and any attachments are confidential. If you have received this email in error, please delete it and kindly notify us. If the email contains personal views then the BMJ Group accepts no responsibility for these statements. The recipient should check this email and attachments for viruses because the BMJ Group accepts no liability for any damage caused by viruses. Emails sent or received by the BMJ Group may be monitored for size, traffic, distribution and content. BMJ Publishing Group Limited trading as BMJ Group. A private limited company, registered in England and Wales under registration number 03102371. Registered office: BMA House, Tavistock Square, London WC1H 9JR, UK.
_______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

_______________________________________________________________________
The BMJ Group is one of the world's most trusted providers of medical information for doctors, researchers, health care workers and patients group.bmj.com. This email and any attachments are confidential. If you have received this email in error, please delete it and kindly notify us. If the email contains personal views then the BMJ Group accepts no responsibility for these statements. The recipient should check this email and attachments for viruses because the BMJ Group accepts no liability for any damage caused by viruses. Emails sent or received by the BMJ Group may be monitored for size, traffic, distribution and content. BMJ Publishing Group Limited trading as BMJ Group. A private limited company, registered in England and Wales under registration number 03102371. Registered office: BMA House, Tavistock Square, London WC1H 9JR, UK.
_______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________