Hi Ben (and colleagues)
I'm happy to be involved as an estimator. It would be very
interesting, indeed educational! I've been following the
"masterclass" on Bayesian hierarchical modelling initiated by Brian's
posting with great interest - been taking notes furiously.
Michael's first draft protocol looks well thought through. Are we
going to confine our Cochrane systematic reviews to general/internal
medicine?
Best wishes
Grant
Dr Grant M Dex
Clinical Lecturer & Clinical Sub-Dean
Division of Primary Care
Room 1307, Tower Building
The University of Nottingham
University Park
Nottingham, NG7 2RD UK
e-mail: [log in to unmask]
Telephone: 0115 823 0212
On 3 Jul 2008, at 22:21, Djulbegovic, Benjamin wrote:
> Fantastic, Paul
> Let's do it- happy to prepare the material. So, can I ask those
> interested to send me forest plots that we should include (it would be
> nice if the identifiers such as the name of interventions or trials id
> can be removed). Also, we can extend this invitation to other
> volunteers
> as well (so far, we have you and Martin)- so, I am asking those who'd
> like to participate to send me their contacts.
> This is going to be a fun- it is great to see that one off-hand remark
> can potentially proceed to an actionable project.
> Best
> Ben
> Ps Timetable: since I will be moving my office at the end of the
> month,
> it would be great to hear from everyone interested by then. I will
> then
> compile the material (including some of my own) sometimes in mid
> August
>
> -----Original Message-----
> From: Evidence based health (EBH)
> [mailto:[log in to unmask]] On Behalf Of Paul
> Glasziou
> Sent: Thursday, July 03, 2008 13:44
> To: [log in to unmask]
> Subject: Re: questionable statistics in meta-analysis - ROT
>
> Hi Martin & Ben
> You have me fascinated - I would have thought this was easy. So I'd
> love
>
> to be a subject.
> But I imagine some forest plots will be hard and some easy. They
> should
> have more than 4 trials and vary by
> a. number of studies b. degree of significance and c. heterogeneity.
>
> Ben - are you happy to do this? We could also ask for volunteers to
> send
>
> you forest plots (but they can't take part in the guessing
> competition),
>
> Cheers
> Paul
>
>
> Martin Dawes, Dr. wrote:
>>
>> OK - good challenge
>>
>> But let's do this properly
>>
>> Probably the first step is a pilot of 10 or so - 5 volunteers (I am
>> happy to be one of these) will look at the plots and then guess the
>> stat signif and its size and direction without cheating
>>
>> Someone will prepare the 10 plots by removing the pooled estimates
>> (maybe Ben as he proposed the challenge)
>>
>> Then another impartial observer will report back the anonymous pooled
>> and individual results from the experiment.
>>
>> Martin
>>
>> Dr Martin Dawes
>>
>> Chair Family Medicine
>>
>> McGill University
>>
>> 515 Pine Avenue West, Montreal
>>
>> Quebec, Canada H2W 1S4
>>
>> Tel 514 398 7375 x0469
>>
>> Fax 514 398 4202
>>
>> *** This communication is confidential. Please request explicit
>> permission before copying or forwarding. If you are not the intended
>> recipient, you are notified that any distribution or copying of this
>> email is strictly prohibited. If you have received this email in
>> error, please notify us immediately by return email and delete all
>> copies of the message. Thank you. ***
>>
>>
> ----------------------------------------------------------------------
> --
>>
>> *From:* Djulbegovic, Benjamin
> [mailto:[log in to unmask]]
>> *Sent:* 03 July 2008 10:00
>> *To:* Martin Dawes, Dr.; [log in to unmask]
>> *Subject:* RE: quesetionable statistics in meta-analysis - ROT
>>
>> Martin/Michael
>>
>> I challenge both of you to select any number of forest plots (have
>> someone remove the pooled estimates) and then try to guess the
>> statistical significance of the treatment effect (its size or or its
>> direction). I bet that your visual impression will not be better than
>> chance.
>>
>> ben d
>>
>>
> ----------------------------------------------------------------------
> --
>>
>> *From:* Evidence based health (EBH) on behalf of Martin Dawes
>> *Sent:* Thu 7/3/2008 7:33 AM
>> *To:* [log in to unmask]
>> *Subject:* Re: quesetionable statistics in meta-analysis - ROT
>>
>> Hi
>> I agree - particularly with the central premis that if it isn't
>> obvious on the first eye-ball of data then it probably is not
>> going/should not see the clinical light of day. Maybe there should be
>> an additional questions similar to decision threshold for diagnostic
>> tests that are stated/agreed before the study is undertaken. THey
>> might include "Is this intervention feasible in practice" "What
>> global
>
>> reduction in adverse outcome is required to change practice".
>> At the same time maybe we should be doing a model of the cost
>> effectiveness before the study is done. Maybe that is what should be
>> required by ethics committees
>>
>> Martin
>>
>>
>> On 03/07/2008 05:11, "Michael Power" <[log in to unmask]>
> wrote:
>>
>> Hi Martin
>>
>> I used the words "suggest an effect" deliberately because, ROTn-1: if
>> you plot the "raw" data for a meta-analysis (measure of central
> tendency
>> and confidence interval for each included study), you get a visual
>> impression that suggests the direction of effect. The meta-analysis
>> quantifies the visual impression of direction and provides a
> confidence
>> interval. If the meta-analysis is consistent with the visual
> impression,
>> it confirms its face validity. If the meta-analysis is NOT consistent
>> with your visual impression, you should check the numbers, graphing
> and
>> calculations!
>>
>> Michael
>>
>>
>> -----Original Message-----
>> From: Martin Dawes, Dr. [mailto:[log in to unmask]]
>> Sent: 02 July 2008 18:07
>> Subject: Re: quesetionable statistics in meta-analysis - ROT
>>
>> I agree with the consensus that is accumulating in this discussion on
>> ROT's but wonder if we have evidence for this following statement
>>
>> ROTn: if the raw data or simple stats do not suggest an effect, then
>> the effect found by sophisticated statistics is unlikely to be
> important
>> in practice.
>>
>> Pragmatically this sounds reasonable but isn't that what the Cochrane
>> logo shows us - advanced stats on small studies (plural) give a
>> result
>> that is meaningful so maybe it should have a proviso - "in single
>> adequately powered studies if the raw data....."
>>
>> Martin
>>
>>
>> Dr Martin Dawes
>> Chair Family Medicine
>> McGill University
>> 515 Pine Avenue West, Montreal
>> Quebec, Canada H2W 1S4
>> Tel 514 398 7375 x0469
>> Fax 514 398 4202
>>
>>
>>
>>
>> Dr Martin Dawes
>> Chair Family Medicine
>> McGill University
>> 515 Pine Avenue West, Montreal
>> Quebec, Canada
>> Tel 514 398 7375 x0469
>> Fax 514 398 4202
>>
>
> --
> Paul Glasziou
> Director, Centre for Evidence-Based Medicine,
> Department of Primary Health Care,
> University of Oxford www.cebm.net
> ph - +44-1865-289298 fax +44-1865-289287
>
>
> This transmission may be confidential or protected from disclosure
> and is only for review and use by the intended recipient. Access by
> anyone else is unauthorized. Any unauthorized reader is hereby
> notified that any review, use, dissemination, disclosure or copying
> of this information, or any act or omission taken in reliance on
> it, is prohibited and may be unlawful. If you received this
> transmission in error, please notify the sender immediately. Thank
> you.
|