Steve
Thanks very much for your clear explanation of hierarchical bayesian meta-
analysis, and for your practical and reassuring rules of thumb (ROT):
ROT1: the first rule in critical appraisal is to focus on how the data
was collected and not on how it was analyzed
ROT2: we have to trust the experts to some extent
I have no problems with your first ROT.
Your 2nd ROT is practical, but it needs to be balanced by a complementary
ROT about when NOT to trust the experts.
- ROT3: never trust an expert --- unless you have no other option; or
they have shown themselves to be trustworthy (you hinted at this by
suggesting that Brian check that a statistician had been involved)
I would also suggest another ROT for critical appraisal:
ROTn: if the raw data or simple stats do not suggest an effect, then the
effect found by sophisticated statistics is unlikely to be important in
practice.
I can think of examples in physics that support ROTn: neutrinos and
gravity waves need very clever machines to detect them, and they do not
affect our daily lives much.
In healthcare are there exceptions to ROTn's rule that important effects
are suggested by the raw data or simple stats?
In Brian's case does a graphic display of results of individual trials
suggest the effect measured by Bayesian hierachical meta-analysis (I have
not seen the paper)? If no, any effect is unlikely to be clinically
important. If yes, you could be less sceptical about the results.
Michael Power
Clinical Knowledge Summaries Service www.cks.library.nhs.uk
standard disclaimers (plagiarised from steve)
|