Hi all,
 Here is an objective example of the value of RCTs compare to epidemiologic studies,


Hormone Replacement Therapy in Epidemiologic Studies and Randomized Clinical Trials- Are We Checkmate?

Michels, Karin B.

From the Department of Epidemiology, Harvard School of Public Health, and the Obstetrics and Gynecology Epidemiology Center, Brigham and Women's Hospital and Harvard Medical School, Boston, MA. Address correspondence to: Karin B. Michels, Department of Epidemiology, Harvard School of Public Health, 677 Huntington Ave., Boston, MA 02115; [log in to unmask]


Epidemiologists and clinical trialists alike have rarely been confronted with a question of greater public health importance: Does HRT decrease or increase the risk of heart disease? At least every woman, every gynecologist and every primary care physician want to know the "correct" answer.

Randomized clinical trials (RCTs) are considered the "gold standard" in medical research. If epidemiologic studies and RCTs produce discrepant results, doubts are generally cast over the epidemiologic studies, and epidemiologists are put on the defensive and challenged to come up with an explanation for the differences.

Surely it is our responsibility as public health professionals to make sense of the data. The question is: Have the epidemiologic studies failed and thus misled clinicians and postmenopausal women, seducing them into using HRT when they would not have or should not have otherwise?

To respond to this we must first take a close look at the data. One of the fundamental issues is whether the data we are considering are indeed comparable. Are the epidemiologic studies and the RCTs comparing similar women, similar drugs and similar settings? We have to differentiate between primary and secondary prevention studies, and between unopposed estrogen and estrogen-progesterone combinations; we must also consider the characteristics of the study participants. The WHI is a primary prevention trial; only the estrogen-progestin arm of the trial has been stopped, which suggests that there is no substantial result from the unopposed estrogen arm. Interestingly, the results from observational studies and the WHI are consistent with respect to the risk of breast cancer: both types of studies indicate an increased incidence with the use of HRT, and this risk seems particularly high with the combination of estrogen and progesterone. 1,5 Similarly, an association between the use of opposed estrogen and an increase in stroke risk was apparent in both the epidemiologic studies and the WHI. 1,6

What seem to differ are the results for coronary heart disease (CHD). Although the Nurses' Health Study, for example, reported that the use of combined estrogen plus progesterone was associated with a reduction in risk of about 36% (95% confidence interval [CI] = 15%-51%), 6 the WHI found an increase of 29% (CI = 2%-63%) with this regimen. 1

Two possible explanations for this discrepancy come to mind. It has long been suspected that observational studies on HRT use are plagued by selection bias. In the United States, women who use HRT tend to be more health conscious, be leaner and more physically active, and smoke less and consult physicians more frequently than women who do not use HRT. It is possible that we cannot sufficiently account for all of the differences analytically, and so residual confounding remains in the observational data, threatening the internal validity of the epidemiologic studies. An alternative explanation is that the women included in the observational studies and the RCTs are not comparable. Indeed, women who participated in the WHI had an average body mass index of 28.5 kg/m2, which is more obese than women self-selecting themselves into the HRT-user group in observational studies. Furthermore, women recruited for randomization into the WHI had to be willing to start taking HRT at the flip of a coin, and hence their decision was not governed by menopausal symptoms. Moreover, women in the WHI started to take HRT at various ages during menopause. In fact, 66% of participants started taking the estrogen-progestin regimen at age 60 years or older and 21% began at 70 years or older. This does not reflect clinical practice (nor the observational setting) in which women are prescribed HRT as soon as they reach menopause; this situation exemplifies an external validity problem inherent in many RCTs. Clearly, the WHI eligibility criteria were established for practical reasons-to accrue as many participants as possible in a reasonable amount of time. This may have created a study population in which the association between HRT and CHD differed from that in women who initiated HRT use when entering menopause. It is biologically plausible that effect modification by age at HRT initiation exists. Women reaching menopause may have been protected from atherosclerosis by ovarian hormones and may be less susceptible to the prothrombotic effects of estrogen than older women who may have built up atherosclerotic plaques during times of estrogen depletion.

Given this constraint, it is unlikely that the WHI can answer the question of whether an estrogen-progesterone regimen initiated at the onset of menopause increases or decreases the risk of CHD. If the WHI cannot provide the answer, are there other studies we can look to? There are five other RCTs that have considered the effects of HRT on cardiovascular endpoints. 7-11 Three of these, however, are secondary prevention trials. 7-9 The other two primary prevention trials have short duration of follow-up and, therefore, no clinical endpoints are available. 10,11 The WHI is the only primary prevention trial currently available that provides clinical endpoints. Thus far no RCT has found a benefit of HRT use for the prevention of CHD.

What can we conclude from the available information? The data are quite convincing that the estrogen-progesterone combination initiated several years after menopause may raise the risk of CHD in obese women without severe menopausal symptoms. What we do not know is how substantial the residual confounding is that is likely to be present in the observational studies. And we do not have results from an RCT on the same question asked in epidemiologic studies: Does the use of estrogen plus progestin initiated at the onset of menopause among women with severe menopausal symptoms affect future CHD risk? In fact, we may never be able to address this question. At this point, ethical considerations do not permit randomizing postmenopausal women to an estrogen-progesterone combination. The Women's International Study of Long Duration Oestrogen after Menopause (WISDOM) in the United Kingdom, which had a design similar to that of WHI, was halted in October 2002 subsequent to the WHI termination. Although the steering committee of WISDOM wanted to continue the trial because important questions about the risk-benefit ratio of HRT remained unanswered post-WHI, an international committee charged with deciding WISDOM'S fate stopped it because a large reduction in the risk of CHD was no longer likely.

Have we checkmated ourselves?
The burden of proof seems to fall upon the epidemiologists. What can be done? There are a number of strategies we can pursue. First, we can reanalyze our epidemiologic data using different analytic approaches to minimize residual confounding. Such analyses would include controlling for confounders not previously adjusted for, such as lifestyle factors that might predict HRT use. Many such possible confounders may not have been measured in our studies. The problem of unmeasured confounders can be addressed by using the epidemiologic data to mimic as closely as possible the design of an RCT. Propensity scores, marginal structural models and structural nested models provide the statistical tools to estimate causal effects from observational data. 12,13 Furthermore, we can try to find women in our epidemiologic databases who initiated HRT later in their menopausal years, after age 60 years or even age 70 years, to explore whether the observational results really differ from the WHI findings. It is unlikely, however, that a sufficient number of such women exist. Conversely, additional analyses of the WHI data stratified by age at HRT initiation might provide interesting insights. Similarly, analyses stratified by body mass index and by duration of HRT use should be conducted, as these parameters differed between the WHI and the observational studies.

The clinical implications of the WHI results, however, may be less dramatic than they appear. The primary indication for HRT is the treatment of menopausal symptoms. HRT was never approved for the prevention of cardiovascular disease or any other chronic condition with the exception of osteoporosis. Therefore, women with their individual responses to menopause are well advised to follow their symptoms in their decision to initiate short-term HRT. Menopause is not a disease-general treatment for it is not required. Although HRT may help women to ease into menopause, most women are free of symptoms after several years even without hormonal substitution. Long-term HRT use has to be viewed in the context of an increase in cancer risk, which has been convincingly established.

Epidemiologists may use the HRT controversy as a reality check. There are research questions in medicine and public health that we may not be able to address adequately with observational studies because of inherent biases, such as uncontrollable confounding and uncorrectable measurement error. But we also need to keep in mind that epidemiologic studies remain the central element of public health and prevention research. Ethical or practical considerations prohibit RCTs for the majority of research questions in which we might be interested. Epidemiologic studies are essential for generating new hypotheses, and in many situations will provide the best available information. It is our responsibility to conduct them well, to be conscious of their pitfalls, and to be reasonably cautious in the interpretation of our observations.



References
1. Writing Group for the Women's Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. Principal results from the Women's Health Initiative randomized controlled trial. JAMA 2002; 288: 321-333. [Context Link]

2. National Institutes of Health, National Heart, Lung and Blood Institute. NHLBI stops trial of estrogen plus progestin due to increased breast cancer risk, lack of overall benefit. NIH News Release 9 July 2002. Available at: http://www.nhlbi.nih.gov/new/press/02-07-09.htm .

3. Stampfer MJ, Colditz GA. Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med 1991; 20: 47-63. ExternalResolverBasic Bibliographic Links [Context Link]

4. Grady D, Rubin SM, Petitti DB, et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann Intern Med 1992; 117: 1016-1037. 

5. Colditz GA, Hankinson SE, Hunter DJ, et al. The use of estrogens and progestins and the risk of breast cancer in postmenopausal women. N Engl J Med 1995; 332: 1589-1593. 

6. Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfer MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann Intern Med 2000; 133: 933-941. 

7. Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. The Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA 1998; 280: 605-613. 

8. Herrington DM, Reboussin DM, Brosnihan KB, et al. Effects of estrogen replacement on the progression of coronary-artery atherosclerosis. N Engl J Med 2000; 343: 522-529. 

9. Viscoli CM, Brass LM, Kernan WN, Sarrel PM, Suissa S, Horwitz RI. A clinical trial of estrogen-replacement therapy after ischemic stroke. N Engl J Med 2001; 345: 1243-1249. 

10. The Writing Group for the PEPI Trial. Effects of estrogen or estrogen/progestin regimens on heart disease risk factors in postmenopausal women. The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial. JAMA 1995; 273: 199-208. 

11. Hodis HN, Mack WJ, Lobo RA, et al. Estrogen in the prevention of atherosclerosis. The Estrogen in the Prevention of Atherosclerosis Trial Research Group. Ann Intern Med 2001; 135: 939-953. ExternalResolverBasic Bibliographic Links 

12. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55. 
13. Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology 2000; 11: 50-560. 



-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Olive Goddard
Sent: Wednesday, September 28, 2005 3:16 PM
To: [log in to unmask]
Subject: Re: Fw: 'outcomes research' (Olive Goddard is on holiday)


Dear Colleagues,

And here is a continuation to the previous email.

Regards,

Olive

>>> "Russell Blumer" <[log in to unmask]> 09/22/05 3:56 am >>>
To be more specific, here are a couple of statements which I think may be 
refutable:

"There is no evidence that I am aware of that demonstrates a meaningful 
difference between RCT and outcomes studies for the same of similar 
condition."


"In the modern era (within the last 15 years, from what I've read, there is 
NO difference in conclusions drawn between outcomes studies and RCT. (when 
studying similar conditions)."

"From what I've read...the only justification for the claim that RCT are in 
a meaningful way superior than outcomes studies extrapolated from studies 
done in the 1940's early 50s."

"from what I've read there is no convincing evidence that RCTs produce 
superior (in real terms) evidence than case series>"







Are the above statements correct?

----- Original Message ----- 
From: "Russell Blumer" <[log in to unmask]>
To: <[log in to unmask]>
Sent: Wednesday, September 21, 2005 10:18 PM
Subject: Fw: 'outcomes research' (Olive Goddard is on holiday)


> N.B.--- I use the term 'outcomes research as defined in the CEBM's
> Levels
> of Evidence hierarchy (level 2c):
>
> http://www.cebm.net/levels_of_evidence.asp#levels
> ----- Original Message -----
> From: "Olive Goddard" <[log in to unmask]>
> To: <[log in to unmask]>
> Sent: Wednesday, September 21, 2005 9:55 PM
> Subject: re: 'outcomes research' (Olive Goddard is on holiday)
>
>
> Olive Goddard is on holiday and will be returning on Wednesday 28th 
> September 2005 If your message is urgent please email 
> [log in to unmask] or telephone her on +44(0)1865 227191
>
>>>> russell.blumer 09/22/05 02:53 >>>
>
> To Whom it May Concern:
>
>           I'm involved in a debate in which someone is claiming that
> there
> is no meaningful objective evidence that the results of RCTs and 'outcomes 
> research' and other observational studies differ significantly when 
> evaluated in the context of a given intervention. This is to say that he 
> believes that outcomes research is essentially equally accurate and 
> meaningful to RCTs despite it's lower position in the 'Levels of Evidence' 
> hierarchy. He is basing this opinion upon only one study, though it was 
> published in the NEJM (comparing RCTs and observational studies):
>
> http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&d
> opt=Abstract&list_uids=10861324&query_hl=1
>
> I am well aware of the theoretical reasons for which RCTs are
> considered
> more reliable than outcomes research and occupy a higher level in the 
> hierarchy. What I'm looking for is something more objective, i.e., are 
> there any important studies which demonstrate the value of RCTs over any 
> and/or all other study designs in an objective, practical sense rather 
> than in theoretical construct?
>
> Thank you very much,
>
> Russell D. Blumer, MD
> Dept. of Radiology
> Humber River Regional Hospital
> Toronto, ON, Canada
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Anti-Virus.
> Version: 7.0.344 / Virus Database: 267.11.3/107 - Release Date: 
> 9/20/2005
>
>