when I read a cross over trial, to help me assess if this is done well, I look for issues such as (and may not not be bang on but it helps me understand the issue a bit...yikes, I am a bit afraid to wade into this one as I am no expert in cross over):
1.) maybe a few lines on why the cross over was chosen over the parallel (rationale)??
2.) a clear indication of the rationale for the wash out period duration e.g. why was 4 weeks chosen and not 6 etc?? how did they arrive at the period needed for the effects to take hold and for the effects to wash out e.g. 3 weeks on intervention and 4 weeks on wash out/dissipation of effects?? where do they get that from? to me it has to be clearly stated/described.
3.) a clear treatment (discussion) on separating time/order effects/interaction??
4.) compliance measurement (discussion) ??
5.) did authors state a priori beliefs e.g. that that they did not suspect a treatment-time interaction nor that a negative correlation of treatment effect would occur?? did authors indicate clearly that precautions were taken to avoid any confounding of treatment effects and period effects??
6.) some treatment of why that particular outcome is suitable for a cross over e.g. I have read it stated as: e.g. a chronic disease with inter-subject variability with rapid (not delayed or protracted) ) response to treatment and return to baseline after stopping treatment??
7.) clear treatment of CONSORT-esque issues such as sequence generation, blinding, allocation concealment etc. ??
8.) clear mention of how threats to internal validity are handled??
9.) Maybe they need to explain how they tested for an interaction or show raw values for end of run-in and washout to convince us that was no significant difference b/w these (if just just stated no significant difference, maybe a problem here) ....maybe one should seek more details beyound "it was OK to combine these values" or "washout was complete"....these should be clearly delineated to be of higher quality...
10.) some say to pretest the data for evidence of carry over...was this considered??? note the wash out has to be long, long enough to rule it out.
11.) if the pre test for carry over effects show significance (carry over), then did the authors then analyze as a conventional 2 group parallel study???
12.) some say it is insufficient to conclude that no carry over is present because a statistical test says so...so need them to discuss this.
I also share this good url for you Brian for more reading....http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3345345/
my junior view...stay away from cross over trials unless absolutely needed. especially if you suspect carry over effects.
Best,
Paul E. Alexander
--------------------------------------------
On Thu, 7/4/13, Brian Alper MD <[log in to unmask]> wrote:
Subject: Re: Critical appraisal criteria for randomized crossover trials
To: [log in to unmask]
Received: Thursday, July 4, 2013, 3:03 PM
Perhaps we are using two different
meanings of the word “crossover”
In a randomized parallel trial
patients are randomly assigned to A vs. B. Over
time patients may crossover from one assignment to the
other, or perhaps after the randomized period is done all
patients in B
will be switched to A if there is a perception that A is
superior therapy. This “crossover”
is referring to changes in treatment assignment in ways
other than through the randomization process.
Crossover in this context may bias or complicate the
analysis.
In a randomized crossover trial the
initial design is give all patients a period of A and a
period of B. What is randomized is the order of which
period comes first. The intention is to compare
A vs. B within
the same patient. Crossover in this context is a key
factor of the primary research design.
We have analyzed many cancer trials
with outcomes of progression-free survival but they are
randomized parallel trials, often with complicated patterns
with multiple stages of care and potentially additional
randomizations (one randomization for induction
chemotherapy, another randomization for consolidation,
etc.). But this is not the randomized crossover trial
in which every patient gets both interventions being
compared.
One would generally not compare 2
different chemotherapy regimens by having the same patient
experience both regimens. Whereas one could
reasonably expect to compare two different analgesics in
this way for
a chronic pain.
The goal here is to define critical
appraisal criteria for trials designed as randomized
crossover trials with an expectation for paired
(within-patient) analyses. This is not about how
to manage crossover
that occurs in randomized parallel trials.
Brian S. Alper, MD, MSPH,
FAAFP
Editor-in-Chief, DynaMed
(dynamed.ebscohost.com)
From: Djulbegovic, Benjamin
[mailto:[log in to unmask]]
Sent: Thursday, July 04, 2013 9:52 AM
To: Brian Alper MD
Cc: [log in to unmask]
Subject: RE: Critical appraisal criteria for
randomized crossover trials
Unfortunately, these are rare
trials…vast majority of (modern) cancer trials are
based on clinical outcomes such as progression-free survival
(PFS) and employ a cross-over …standard practice
has always relied
on intention-to-treat analysis, with which clinicians are
never comfortable… my point in raising the issue of
cross-over in cancer trials is that it remains a
methodologically unresolved problem…most people (as
this exchange also demonstrates) dismiss them
as not true cross-over trials…but this will not make
a) cross-over disappear from cancer trials, b) how do you
assess the quality of these trials (indeed, how have you
assessed the quality of cancer trials in DynaMed where
primary outcome was PFS and the cross-over
was employed? hint: if you have appraised them as low
quality, in many cases you would not agree with the FDA,
which has granted approval for marketing for a number of
cancer drugs based on such a design)
ben
From: Brian Alper MD [mailto:[log in to unmask]]
Sent: Thursday, July 04, 2013 8:43 AM
To: Djulbegovic, Benjamin
Cc: [log in to unmask]
Subject: RE: Critical appraisal criteria for
randomized crossover trials
Not necessarily. The quality
assessment is specific to the outcome. So such a
trial could provide high-quality data on symptom control
(perhaps the reason for the trial) and if differences in
survival were
discovered between crossover periods one would have to
consider if this is potentially biased with period effects
(second period patients would be limited to those who
survived first period).
Brian S. Alper, MD, MSPH,
FAAFP
Editor-in-Chief, DynaMed
(dynamed.ebscohost.com)
From: Djulbegovic, Benjamin [mailto:[log in to unmask]]
Sent: Thursday, July 04, 2013 8:38 AM
To: Brian Alper MD
Cc: [log in to unmask]
Subject: Re: Critical appraisal criteria for
randomized crossover trials
Agree, but many oncology
trials do incorporate cross-over. So, they will all be
judged as low-quality trials?
Sent from my iPad
( please excuse typos
& brevity)
On Jul 4, 2013, at 8:21 AM, "Brian Alper MD"
<[log in to unmask]>
wrote:
Survival is not an outcome for which
a crossover trial would be an appropriate
design. There is no way to avoid a
carryover effect – washout period won’t reverse
death. A key part of critical appraisal of
crossover trials is to determine if the crossover design is
appropriate for the research being conducted.
Brian S. Alper, MD, MSPH,
FAAFP
Editor-in-Chief, DynaMed (dynamed.ebscohost.com)
From: Djulbegovic, Benjamin [mailto:[log in to unmask]]
Sent: Thursday, July 04, 2013 7:49 AM
To: Brian Alper MD
Cc: [log in to unmask]
Subject: Re: Critical appraisal criteria for
randomized crossover trials
Brian,
No oncology trial ( with
survival endpoint) would meet these criteria.
Best
Ben
Sent from my iPad
( please excuse typos
& brevity)
On Jul 4, 2013, at 5:58 AM, "Brian Alper MD"
<[log in to unmask]>
wrote:
EBH listserv members
Periodically we evaluate
the critical appraisal criteria we use for
DynaMed. This listserv has helped us in the past
(2008) when we added critical appraisal criteria for
randomized trials in the case of early trial
termination.
We have found critical
appraisal criteria for randomized parallel trials are not
optimal for analyzing crossover trials. The CONSORT
statement has numerous extensions but does not have an
extension for crossover trials. The Cochrane
Handbook was updated in 2011 (version 5.1.0) to include a
risk of bias assessment criteria for crossover trials.
In consideration of
critical appraisal criteria for randomized trials used today
and the risk of bias criteria for crossover trials from the
Cochrane Handbook we have developed the criteria listed
below as a planned revision to our level
of evidence criteria for DynaMed (found at
https://dynamed.ebscohost.com/content/LOE). We
welcome your feedback in consideration of refining these
criteria before we implement them.
DynaMed criteria
for level 1 (likely reliable) evidence for interventional
conclusions from crossover trials – must meet ALL
criteria to be considered level 1
evidence
1)
Conducted in
patients with condition not expected to change spontaneously
during course of trial
2)
Clinical outcome (also called
patient-oriented outcomes)
3)
Random allocation
method for order of assignment
(i.e. not assigned by date
of birth, day of presentation, “every
other”)
4)
Washout period
between interventions long enough to avoid carryover effects
between interventions
5)
Adequate duration
of intervention and assessment period to represent outcome
being measured
6)
Blinding of all
persons (patients, treating clinicians, outcome assessors)
if possible
7)
Follow-up
(endpoint assessment) of at least 80% of trial participants
AND adequate such that losses to follow-up could not
materially change results
8)
Accounting for
dropouts (even if not included in analysis)
9)
Analysis of
paired data
10)
Analysis not
suggesting period effects (i.e. effect resulting for
order of intervention), or period effects if present not
materially changing results
11)
Adequate
statistical power
12)
No
other factors contributing substantial bias, such as
subgroup or post hoc analyses
Thank you.
Brian
Brian S. Alper, MD, MSPH, FAAFP
Editor-in-Chief, DynaMed
Medical Director, EBSCO Information Services
[log in to unmask]
10 Estes Street
Ipswich, MA 01938-0231
Phone 800-356-6500 or 978-356-6500 ext. 2749
Cell 978-804-8719
Fax 978-356-7332
dynamed.ebscohost.com
"It only takes a pebble
to start an avalanche."
|