I have neither time nor knowledge of the original literature you cite, so I
can't comment on the specifics you describe below. A few more general
comments:
"Evidence-based medicine" is not cost-effectiveness or cost reduction. It
is not a cabal, or cult. It is a set of principles applied to teaching and
practice that guide the selection of interventions for our patients. The
guiding principles is that physicians should know the level of evidence that
supports a particular decision. That doesn't mean you never do anything
unless it is supported by grade A/Level 1 evidence; far from it. That
would be unrealistic, given the state of the literature and body of studies
to support our practice. What it means is that when given a choice between
two or more therapies, you should have a preference for those that are
proven to be effective in well-designed studies. Evidence-based medicine is
not a research movement, except to the extent that education about
appropriate study design and common biases will hopefully lead to more,
better designed studies in the future (I think this is clearly happening
already).
At a policy level, the application of evidence-based principles is a
refreshing step forward. Of course, just as in clinical practice, we
shouldn't apply these principles thoughtlessly or carelessly. Also, EBM is
NOT cost-effectiveness, although it is often mistaken as such by
policy-makers and payors. The statement that "the level of evidence in
policy decisions is 0.50" betrays a poor understanding of statistics, if
nothing else. This suggests that we should pay for therapies when there is
a 50% probability that the "benefit" was due to chance.
I think it is reasonable that the government or insurers shouldn't be asked
to pay for treatments or tests that have not been shown to be beneficial in
well-designed studies. When there are no studies, we must rely on consensus
and expert opinion until studies are done. The technological imperative in
this country ("don't just stand there, do something") means that we have a
hard time waiting for evidence, though. Perhaps "don't just do something,
stand there" should be an acceptable alternative! The 50,000 Americans dead
because we prematurely adopted the anti-arrhythmics encanide and flecanide
would agree.
Mark Ebell, MD, MS
-----Original Message-----
From: [log in to unmask] [mailto:[log in to unmask]]
Sent: Tuesday, May 02, 2000 10:59 PM
To: EBHC
Subject: Some very naive questions
Dear List:
Please forgive the intrusion of a retired psychologist from the 'States, but
I urgently need suggestions from members of this list. I have learned
everything I know about "evidence-based-medicine" in the past six months,
and now barely know enough to ask, hopefully, the right questions.
THE BACKGROUND
As you may know, in the USA a federal agency called the "Health Care
Financing Administration" (HCFA) administers the national health insurance
program, "Medicare". Although this program pertains only to the elderly,
the majority of private health insurance programs look to Medicare for
leadership in evaluating technology and coverage issues, so their influence
extends far beyond their mandate.
Recently a new generation of staff members at HCFA decided to use an
"evidence-based-medicine" approach to reach coverage decisions. The new
approach was applied first to the use of "biofeedback" and "electrical
stimulation" to the treatment of urinary incontinence.
(Biofeedback is much more highly developed in the United States than it is
in Europe, where Electrical Stimulation has been widely used for many years.
But, as a reasonable estimate, 90% of the commercial support for biofeedback
comes from very small businesses (less than 20 employees), who are not in a
position to mount a serious challenge to government policy.)
Last fall HCFA invited professional organizations to submit written
testimony by early December in preparation for public hearings in January,
2000. At the last minute HCFA postponed the hearings and announced that
they would be held on April 12-13. Still, all professional organizations
were under the assumption that their prepared testimony had been submitted
to the "Medical - Surgical Panel" of the
"Medicare Coverage Advisory Committee" for consideration.
Less than a month before the hearings professional organizations were
informed that none of their prepared testimony had been distributed, but
instead, an "Evidence-based" evaluation of these technologies, prepared by
the "Technology Evaluation Center" of the "Blue Cross Blue Shield
Association", would be the sole evidence debated at the Panel meeting. The
professional organizations, which included the American Urological
Association, American College of Gynecologists, the American
UroGynecological Society, as well as the Association for Applied
Psychophysiology and Biofeedback, the Agency for Health Care Policy and
Research, as well as the national physical therapists' and nurses'
organizations -- all scrambled to revise their testimony in two short weeks
to accomodate the new rules for debate. [BTW, NO professional organization
desented on the value of biofeedback, and only one on electrical
stimulation.]
Two days before the public hearings, HCFA posted to its website a list of
"official" questions which would be addressed to the panel. And it was only
at the hearing itself that these questions were formulated into a formal
"flow chart", such that if the panel could not conclude that there was
sufficient "evidence-based" research PROVING the effectiveness of these
technologies, they were not allowed to even address the issue of whether or
not there was sufficient evidence, none the less, to justify coverage of
them in the absence of conclusive evidence.
THE VOTE ON SCIENCE
After reviewing the TEC report, the physician-members of the Panel voted 9
to 2 that the evidence was not conclusive for biofeedback. All of the males
voted against biofeedback, while the two females voted in favor of
biofeedback. Several male panel members remarked later that they agreed
with the principle voiced by one of them, that "the level of evidence in
science is 0.05, but the level in policy decisions is 0.50". But the panel
was NOT allowed to discuss policy.
As one might imagine, the process used by HCFA is now the subject of a
popular uprising that may or may not generate sufficient political pressure
to reverse the results of the HCFA hearings. That is another topic in its
own right. I have joined this list to pursue a different tact.
MY NAIVE QUESTIONS
1. Who decides which questions are addressed in "evidence-based-medicine?"
The TEC report investigated only those RCTs which compared "Biofeedback"
with "Pelvic Muscle Exercises Alone". In contrast, in the companion report,
they investigated RCTs that (1) compared Stim to placebo (sham), as well as
(2) Stim compared with alternative treatments.
In the first instance, the Burgio et al 1999 JAMA study (which was given the
highest methodological rating in Berghmans' recent review) was NOT
considered at all because Burgio compared biofeedback to drugs and placebo,
but NOT to PMEs alone. [If the same criteria that was used for stim was
used for biofeedback, Burgio would have been included, with very different
results.] Is this kind of manipulation common in
"evidence-based-medicine"?
2. By what standards are RCTs dismissed from consideration because they do
not present sufficient evidence of not guarding against selection bias,
etc.? The TEC report(s) never established that several studies WERE
defective in these areas, but merely intimated that they MIGHT be.
3. To what extent has "evidence-based-medicine" investigated behavioral
therapies such as biofeedback and pelvic muscle exercise? Clearly there are
many areas of medicine that lend themselves to clear and simple analysis.
In pharmacological studies, for example, it is fairly simple to ask how many
milligrams of the active ingredient were used? There is some controversy
in the incontinence field, however. In some of the studies that were
classified as "Pelvic Muscle Exercises alone", subjects actually received
repeated vaginal evaluations by a trained physiotherapist who coached
subjects on how to contract their pelvic muscles. These studies were then
compared with studies where the "feedback" was through a machine only. In
the two largest studies (Burns, 1993; Berghmans 1996) verbal feedback was
not statistically different, in outcome, from machine-feedback. (In the
largest study of all, Burgio 1999, machine feedback was dramatically
superior to drugs or placebo, !
bu!
t that comparision wasn't included.)
4. How do you experts account for the problems that may be created by uneven
or incompetant treatments being tested? In drug studies, for instance, its
seems warranted to assume that the subjects who were given "50 mg of drug X"
really GOT "50 mg of drug X", and those who got placebos got placebos. Drug
manufacturers are required to employ chemists who certify the integrity of
their products.
But in behavioral therapies, who is to decide if the subjects who got six
sessions of biofeedback training REALLY GOT six session of competent
biofeedback training? The therapist in Burns' study was never formally
trained in biofeedback techniques. At the time, her biofeedback subjects set
a new record LOW for treatment success (61% Sx reduction). The subjects in
Berghmans' study used an electrical stimulation electrode (circular
electrodes) instead of an EMG sensor (longitudinal electrodes); they then
set a new record for poor results from biofeedback (55% Sx reduction).
Would it be considered appropriate to compare surgeries performed by first
year medical students with surgeons who are board certified?
So my question is, who decides if the "treatment" meets minimal professional
standards? Or isn't that a consideration in "evidence-based-medicine"? If
so, isn't that a serious limitation of evidence-based-medicine? Or are
these factors normally considered?
Well, I have more quesitons, but that's enough for now. You can find the
relevant documents at the HCFA website, http://www.hcfa.gov/quality/8b.htm
and follow the links to biofeedback and Incontinence. The documents from
our professional societies are available on my website,
http://www.incontinet.com; there are detailed descriptions of available
testimony in the first three paragraphs in the text of the home page.
On behalf of tens of thousands of urologists, nurses, physicians, physical
therapists, and their many patients, I hope that some of the members of this
list will look into this subject and provide guidance as to the adaquacy of
the "evidence-based-medicine" being promoted by our government. It is even
a possibility that this recent activity will reflect badly upon the
"evidence-based" movement, at least in the United States.
Thank you in advance.
John D. Perry, PhD, MDiv, BCIA, FAACS
Committee on HCFA of the Association for
Applied Psychophysiology and Biofeedback
1192 Lakeville Circle
Petaluma, CA 94954 USA
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
