Dear Jim,
You are right that technical definitions of deductive validity and proof
are, from a practical perspective, irrelevant. What we are interested in
is whether we have SUFFICIENT evidence to ACT. See attached figure I made
based on an idea that Paul developed. When the effect size outweighs the
combined effect of plausible confounders, we have sufficient evidence.
This sword, however, cuts in a way that needs to be addressed by GRADE:
when the ABSOLUTE effect size in a randomized trial is tiny (even if we
have used concealed allocation etc.), then it wouldn't take a large
confounder to 'tip the scale' and we need to question the conclusions of
the trial.
Best wishes,
Jeremy
On 31/01/2011 04:08, "Jim Walker" <[log in to unmask]> wrote:
>Perhaps thinking of the problem in engineering terms (rather than
>philosophical) can be useful:
>The goal in machining parts is not perfection, but achieving acceptable
>tolerances for the part (based on its intended use).
>From this perspective, the relevant question is something like "Is the
>likelihood that this observed relationship is due to chance low enough
>that it is appropriate to act as if it were a causal relationship?"
>
>
>Then we can avoid the trap of treating RCTs as if they were
>qualitatively different from (rather than quantitatively superior to)
>other study designs. That is, a well done non-RCT study may be entirely
>adequate to prompt a change in practice (e.g., Semmelweis).
>
>Jim
>
>James M. Walker, MD, FACP
>Chief Medical Information Officer
>Geisinger Health System
>
>The best way to predict the future is to invent it.
> - Alan Kay
>>>> Stephen Senn 01/30/11 6:48 PM >>>
>However, I don't think that it is necessary for RCTs to control for all
>confounders for valid inference to be based on them and it is precisley
>this point that I think many commentators are confused on.
>Stephen
>
>
>Stephen Senn
>
>Professor of Statistics
>School of Mathematics and Statistics
>Direct line: +44 (0)141 330 5141
>Fax: +44 (0)141 330 4814
>Private Webpage: http://www.senns.demon.co.uk/home.html
>
>University of Glasgow
>15 University Gardens
>Glasgow G12 8QW
>
>The University of Glasgow, charity number SC004401
>________________________________________
>From: Evidence based health (EBH) [[log in to unmask]]
>On Behalf Of Jeremy Howick [[log in to unmask]]
>Sent: 30 January 2011 23:20
>To: [log in to unmask]
>Subject: Can RCT help establish causation? A philosopher's reply
>
>Dear All,
>
>Stephen is correct that philosophers have been discussing the issue of
>how to establish scientific knowledge for centuries * if not longer *
>and it is fair to say that there is no widely accepted response to the
>problem Hume posed. (Hume famously argued that while we would be 'fools
>or madmen' to NOT accept scientific knowledge, inductive reasoning upon
>which scientific laws are base have no rational justification (for a
>simple explanation see: en.wikipedia.org/wiki/Problem_of_induction).)
>
>However, philosophers (and others) HAVE managed to provide "a scientific
>method that allows us to LOGICALLY distinguish coincidence from
>causation". See, for example, Cartwright (2007):
>
>journals.cambridge.org/action/displayAbstract?fromPage=online&aid=1002668
>
>A simple way to put Cartwright's claim is: "if RCTs control for all
>confounders, then their conclusions about the causal properties of the
>intervention are valid".
>
>The problem, of course, is that even the best randomized trial DO NOT
>control for ALL confounders. They are better than other methods at
>ruling out confounders, but are not perfect. Hence the EBM insistence
>that we keep UP TO DATE with the latest evidence that might overturn
>previous evidence. I try to explain this clearly in my forthcoming book
>(http://www.amazon.co.uk/Philosophy-Evidence-Based-Medicine-Jeremy-Howick/
>dp/140519667X).
>
>Best wishes,
>
>Jeremy
>
>From: "[log in to unmask]" >
>Reply-To: "[log in to unmask]" >
>Date: Fri, 28 Jan 2011 15:06:33 +0000
>To: "[log in to unmask]" >
>Subject: Can RCT help establish causation?
>
>Dear all
>I'd like to post this question to the group that I have been thinking
>about for some time... Is there a scientific method that allows us to
>LOGICALLY distinguish the cause-effect from the coincidence? David
>Hume, one of the most influential philosophers of all times, concluded
>that there is no such a method. This was before RCTs were "invented".
>Many people have made cogent arguments that (a well done) RCT is the
>ONLY method that can allow us to draw the inferences about causation.
>Because this is not possible in the observational studies, RCTs are
>considered (all other things being equal) to provide more credible
>evidence than non-RCTs. However, some philosophers have challenged this
>supposedly unique feature of RCT- they claim that RCTs cannot (on
>theoretical and logical ground) establish the relationship between the
>cause and effect any better than non-RCTs. I would appreciate some
>thoughts from the group:
>1. Can RCT distinguish between the cause and effect vs. coincidences?
>(under which -theoretical- conditions?)
>If the answer is "no", is there any other method that can help establish
>the cause and effect relationship?
>I believe the answer to this question is of profound relevance to EBM.
>
>Thanks
> Ben Djulbegovic
>
>
>
>
>IMPORTANT WARNING: The information in this message (and the documents
>attached to it, if any) is confidential and may be legally privileged. It
>is intended solely for the addressee. Access to this message by anyone
>else is unauthorized. If you are not the intended recipient, any
>disclosure, copying, distribution or any action taken, or omitted to be
>taken, in reliance on it is prohibited and may be unlawful. If you have
>received this message in error, please delete all electronic copies of
>this message (and the documents attached to it, if any), destroy any hard
>copies you may have created and notify me immediately by replying to this
>email. Thank you.
|