Michael Power wrote:
> ! might be appropriate for a study using multiple imputation;
> !! might be appropriate for a study using hierarchical Bayesian
> meta-analysis.
> !!! might be appropriate for a study using enriched enrolment.
>
In our journal club we just came across another example:
Eisenberg MJ, Filion KB, Yavin D, BĂ©lisle P, Mottillo S, Joseph L,
Gervais A, O'Loughlin J, Paradis G, Rinfret S, Pilote L.
Pharmacotherapies for smoking cessation: a meta-analysis of randomized
controlled trials. CMAJ. 2008 Jul 15;179(2):135-44
The article if freely accessible at
http://www.cmaj.ca/cgi/content/full/179/2/135
IMO the major problem here is not the use of the hierarchical Bayesian
meta-analysis. In fact we repeated some of the analyses using a
frequentist meta-analysis and obtained pretty much the same results.
The problem is with the forest plots that may mislead many readers: What
they plot as an OR of each study is not really what you may think an OR
is, but, in the words of the authors "Any differences between our
figures and simple unpooled analysis from each study is that, while we
too present results for each study alone, the estimates are taken from
our hierarchical model. This results in individual study estimates that
lead to smaller MSE than more simple analyses that ignore the random
effects."
Whatever that means, you may notice that it has some magic effect on the
data, so that in some cases (see figure 2 of the paper) a result of 6
quitters and 103 non-quitters with placebo vs. 2 and 103 with treatment,
which for most of us would give an OR of 2/6, or 0.32 (95% CI
0.06-1.62), is plotted as an OR of 1.88(0.91-2.57). So a
(non-significantly) negative study is plotted as if it was positive!
The overall result then is a smoothing of the results and an impression
of a much lower heterogeneity of results that is in practice (see the
graph of the same data in the letter we submitted to the journal at
http://www.cmaj.ca/cgi/eletters/179/2/135#19987 ).
Se I think that the readers should be warned that that is not the real
OR, that the reviewers should insist that it should be given a different
name, and that whatever model is used, authors should be requested to
give the plain ORs as well.
The paper also has an obvious big mistake in figure 7, which is of no
interest in this discussion except to remember that expert reviewers may
sometimes get distracted and cannot always be trusted ;-)
regards,
Piersante Sestini
|