Martin/Michael
I challenge both of you to select any number of forest plots (have someone remove the pooled estimates) and then try to guess the statistical significance of the treatment effect (its size or or its direction). I bet that your visual impression will not be better than chance.
ben d
________________________________
From: Evidence based health (EBH) on behalf of Martin Dawes
Sent: Thu 7/3/2008 7:33 AM
To: [log in to unmask]
Subject: Re: quesetionable statistics in meta-analysis - ROT
Hi
I agree - particularly with the central premis that if it isn't obvious on the first eye-ball of data then it probably is not going/should not see the clinical light of day. Maybe there should be an additional questions similar to decision threshold for diagnostic tests that are stated/agreed before the study is undertaken. THey might include "Is this intervention feasible in practice" "What global reduction in adverse outcome is required to change practice".
At the same time maybe we should be doing a model of the cost effectiveness before the study is done. Maybe that is what should be required by ethics committees
Martin
On 03/07/2008 05:11, "Michael Power" <[log in to unmask]> wrote:
Hi Martin
I used the words "suggest an effect" deliberately because, ROTn-1: if
you plot the "raw" data for a meta-analysis (measure of central tendency
and confidence interval for each included study), you get a visual
impression that suggests the direction of effect. The meta-analysis
quantifies the visual impression of direction and provides a confidence
interval. If the meta-analysis is consistent with the visual impression,
it confirms its face validity. If the meta-analysis is NOT consistent
with your visual impression, you should check the numbers, graphing and
calculations!
Michael
-----Original Message-----
From: Martin Dawes, Dr. [mailto:[log in to unmask]]
Sent: 02 July 2008 18:07
Subject: Re: quesetionable statistics in meta-analysis - ROT
I agree with the consensus that is accumulating in this discussion on
ROT's but wonder if we have evidence for this following statement
ROTn: if the raw data or simple stats do not suggest an effect, then
the effect found by sophisticated statistics is unlikely to be important
in practice.
Pragmatically this sounds reasonable but isn't that what the Cochrane
logo shows us - advanced stats on small studies (plural) give a result
that is meaningful so maybe it should have a proviso - "in single
adequately powered studies if the raw data....."
Martin
Dr Martin Dawes
Chair Family Medicine
McGill University
515 Pine Avenue West, Montreal
Quebec, Canada H2W 1S4
Tel 514 398 7375 x0469
Fax 514 398 4202
Dr Martin Dawes
Chair Family Medicine
McGill University
515 Pine Avenue West, Montreal
Quebec, Canada
Tel 514 398 7375 x0469
Fax 514 398 4202
This transmission may be confidential or protected from disclosure and is only for review and use by the intended recipient. Access by anyone else is unauthorized. Any unauthorized reader is hereby notified that any review, use, dissemination, disclosure or copying of this information, or any act or omission taken in reliance on it, is prohibited and may be unlawful. If you received this transmission in error, please notify the sender immediately. Thank you.
|