JiscMail Logo
Email discussion lists for the UK Education and Research communities

Help for RAMESES Archives


RAMESES Archives

RAMESES Archives


RAMESES@JISCMAIL.AC.UK


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

RAMESES Home

RAMESES Home

RAMESES  November 2014

RAMESES November 2014

Options

Subscribe or Unsubscribe

Subscribe or Unsubscribe

Log In

Log In

Get Password

Get Password

Subject:

Re: Example of causal inference in evaluation without a counterfactual

From:

Raymond Pawson <[log in to unmask]>

Reply-To:

Realist and Meta-narrative Evidence Synthesis: Evolving Standards" <[log in to unmask]>, Raymond Pawson <[log in to unmask]>

Date:

Wed, 12 Nov 2014 11:08:58 +0000

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (33 lines)

Hi Patricia (and all)

Now this is really important. Real causal inferences in science are made in the manner you suggest. Theories are built, contested and refined over time using a plethora of different empirical tests – none of them decisive in isolation. This is well described by the philosophers of science – Popper, Lakatos, Campbell (the real one) etc. Hill’s criteria capture the balance of many considerations that contribute to strong causal explanations and remain influential in the public health community. None of these sources translate simply into programme evaluation.
 
The great irony is that these ‘conjectures are refutations’ models describe perfectly the many different empirical tests that are applied in the lengthy process of developing clinical interventions. These start in basic science, pre-clinical work hypothesising the underlying disease pathology and potential mechanisms of action that might target the particular condition. Then there is a phase of therapeutic discovery in which compounds and techniques are tried, refined in an attempt to embody the conjectured mechanisms and then laboratory tested the see if the initial explanation holds promise. Then, still in pre-clinical phases, there are tests, for instance, about the absorption, distribution, metabolism and excretion of a drug. Only then we get to effectiveness with patients and the three phases of clinical trials, starting with proof of concept, dose-finding and safety studies, moving eventually to RCTs and regulatory proof. There is even a further stage about long-term follow-up and the detection of ‘rare events’. Failure is commonplace at any stage – in which case the hypotheses are refined and the hypotheses-testing cycle is resumed.

Here’s the problem. A) Cochranites have managed to convince themselves (and much of the medical establishment) that only the penultimate stage counts in assessing causality and effectiveness. B) Even those keenly attuned to the full cycle explanation often don’t like to use the language of ‘theory’ testing (they often prefer rather technical or mechanical accounts of each individual stage).

So you are right about the desperate need for good examples of non-counterfactual causal explanations. Finding them in medicine would be the ultimate coup de grace. Rather late in the day I’m trying to become an amateur medic in trying to piece together the full story as above. Any good sources graciously welcomed! 

Thanks by the way for the links to Julian and Jane’s excellent work. 

RAY
________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [[log in to unmask]] On Behalf Of Patricia Rogers [[log in to unmask]]
Sent: Tuesday, November 11, 2014 9:36 PM
To: [log in to unmask]
Subject: Example of causal inference in evaluation without a counterfactual

Dear RAMESES colleagues,

After such great suggestions of examples of realist evaluations, I'd like to make a related request.

Can you suggest any good examples of an impact evaluation that uses non-counterfactual causal inference - that is, doing lots of small tests that the data fit the theory of a causal relationship, and ruling out alternative explanations?

While there's good material that discusses these strategies (eg Bradford Hill's classic 1965 paper,<http://www.edwardtufte.com/tufte/hill> on Edward Tufte's site, Julian King's new e-book<http://www.julianking.co.nz/wp-content/uploads/2014/08/140826-BHC-web.pdf> on the Bradford Hill criteria. and Jane Davidson's webinar<http://betterevaluation.org/events/coffee_break_webinars_2013#webinarPart5> on causal inference) I struggle to find good examples that can be used in a workshop to clearly demonstrate the logic of non-counterfactual causal inference, can be readily understood and are clearly relevant.

The old epidemiological examples of John Snow's investigation of cholera in London and the link between lung cancer and smoking explain the logic but are too often dismissed as being not relevant because they are "the cause of an effect, not the effect of a cause"  - as in an impact evaluation of a known program.  (I think this is a specious argument but that's a hard position to change).

I'd really appreciate suggestions of good examples I can add to the BetterEvaluation site <http://betterevaluation.org/plan/understandcauses> and share in workshops and presentations.

Patricia Rogers

Top of Message | Previous Page | Permalink

JiscMail Tools


RSS Feeds and Sharing


Advanced Options


Archives

October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
August 2012
July 2012
June 2012
May 2012
April 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011


JiscMail is a Jisc service.

View our service policies at https://www.jiscmail.ac.uk/policyandsecurity/ and Jisc's privacy policy at https://www.jisc.ac.uk/website/privacy-notice

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager