From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Trish Greenhalgh
Sent: 30 January 2014 12:16
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.

Just to say, as someone observing from the sidelines -  
I've also found this discussion very useful.
Very valuable list.

Heather Trickey
DECIPHer
School of Social Sciences
Cardiff University
1-3 Museum Place
Cardiff CF10 3BD
Wales, UK

http://www.decipher.uk.net/    Twitter: @DECIPHercentre
http://cardiff.ac.uk/socsi/contactsandpeople/postgraduateresearchers/heather-trickey-phd-student-overview.html
Twitter: @HeatherTrickey
 
(mb) 07941 854883
+44 (0)29 20  879609
 


From:        "Hardwick, Rebecca" <[log in to unmask]>
To:        [log in to unmask],
Date:        30/01/2014 09:47
Subject:        Re: Advice on testing theories - and how programs work.
Sent by:        "Realist and Meta-narrative Evidence Synthesis: Evolving Standards" <[log in to unmask]>

.... me too – I tend to agree with Geoff on this one, my thinking was that mechanisms exist out there in the world independently of whether we have ‘activated’ them, found them, know about them – they are there, but are latent because the context isn’t ‘right’.  Thinking about it now, it is almost something I have taken on ‘faith’....  hmm.  
 
(... this was my understanding from reading Bhaksar.....  and on that note, it’s back to my data....  )
 
Rebecca
 
 
 
 
Rebecca Hardwick
Associate Research Fellow

01392 727408
email [log in to unmask]
 
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Geoff Wong
Sent: 30 January 2014 08:48
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.
 
Sorry, I can't resist jumping in here, even though I should be working on three other projects :-O
 
I suspect that all interventions / programmes are complex and that what we as researchers do is set the boundaries of the evaluation / review to make them seem 'more' or 'less' complex. To take the ideal case of a simple intervention, drugs/medication, it is naive to think these are simple. Before you can even get a drug licensed you have to (in almost all cases) explain how it works - i.e. what are its mechanisms of action at a cellular level, first in vitro, then in vivo (animal and then human). So in our stratified world, here we have to delve into the cellular stratum.
Once these have been established (as much as is possible) then you have the black box (the tablet/capsule/liquid) medication that can then be 'used' in drug trials.
Is the simplicity more about where we set the boundaries? I'd be interested in what others think.
 
On to mechanisms. Gill asked:
" The program works not by changing ‘the balance of existing mechanisms that fire’ but (arguably) by creating new mechanisms (new reasoning, new emotional states, new responses to existing contexts) within the client."
 
Forgive me for asking this, but can we create mechanisms? I had (perhaps erroneously) thought that they are pre-existing liabilities?
So we were always able to reason, respond in certain ways, but did not do so as the context did not trigger it?
 
This is a shocking simplistic example as I just want to illustrate a point.
Let us take therapy - e.g. cognitive behavioural therapy (CBT) - one common process therapist try to get clients to do is to reframe things (e.g. glass 1/2 full vs glass 1/2 empty thinking). One theory is that depression maybe perpetuated by continue negative reframing, so the therapist tries to get the client to reframe in a more positive way. So if we can see that the client starts to reframe more positively after some CBT sessions was the reframing mechanism created or activated?
It seems plausible that before a person becomes depressed, they were able to reframe in a more positive way (glass 1/2 full) - in other words they did not always think in a negative way about situations but perhaps were not aware that this was what they were doing? The therapist highlights this thought process and uses it for therapeutic gain.
Thus it might seem we created 'reframing' during CBT, but in fact it was already there????
 
I can think of more mechanisms that may seem to be created but already exist - fear, love, hate, over-optimism, greed etc. Purva gives us other real life examples. We all have the pre-existing response of wanting to 'fit in' or 'tow the party line' (good old reference group theory here), but we don't always do it.
Also in some cases we may wish to commit to something and in others not.
To use the old cliche, it all depends on the context, but does not mean we have created mechanisms?
 
Geoff
 
 
On 30 January 2014 00:26, Patricia Rogers <[log in to unmask]> wrote:
Not at all - not what I was trying to say, and thanks for checking.
 
  I think it's very helpful for them to think about policy and practices in terms of causal mechanisms, not just bundles of activities, to guide better translation of effective interventions to different situations (where the same mechanism might operate, but need different activities to trigger it, or different mechanisms).
 
But I think it's helpful to consider BOTH - what might we do within the existing context and what might we do to change the context?
 
On 30 January 2014 11:21, Gill Westhorp <[log in to unmask]> wrote:
Thanks Patricia.  
 
Is the implication that you don’t think it’s all that helpful to try to engage policy makers to think about mechanism??
 
Cheers
Gill
 
From: Patricia Rogers [mailto:[log in to unmask]]
Sent: Thursday, 30 January 2014 8:25 AM
To: Realist and Meta-narrative Evidence Synthesis: Evolving Standards; Gill Westhorp
Subject: Re: Advice on testing theories - and how programs work.
 
I think it's very helpful to think about ways in which program activities might trigger an Intended mechanism in an already favorable context by providing additional resources that are then able to be applied.

You could put this under the umbrella of context, since it's going from a low resource context to a higher resource context, but i don't find this a helpful way to frame it.  I think it's more useful for policymakers, practitioners and researchers to think hard about actions that might be taken to add resources, or change the context or both.

I'll be interested to hear others' ideas and experiences .

Patricia Rogers


> 2014-01-29 Gill Westhorp <[log in to unmask]>
>
> Hi all
>
> My turn to ask a question, in response to Geoff’s (as always, helpful) response below.  Geoff and I have had this conversation before but I’m curious as to what others think.  The question is: Do programs always and only work by changing the context and thereby changing the balance of mechanisms that fire?
>
>  
>
> It’s relatively straightforward in incentives-based programs to say ‘The new incentive changes the context in that it enables a proportion of the population who were already inclined to do x to now do x’ (or in the case of negative incentives such as increased costs or sanctions, ‘changes the weight of factors against the behaviour such that it increases the proportion of the population who decide against the behaviour’).
>
>  
>
> But what about therapeutic services?  One can of course argue that the context is different than it would be if no therapeutic service were available, but one can also argue that that is not what causes or enables the program to work.  The program works not by changing ‘the balance of existing mechanisms that fire’ but (arguably) by creating new mechanisms (new reasoning, new emotional states, new responses to existing contexts) within the client.
>
>  
>
> If one accepts the premise that such programs do not work by ‘changing the context and the balance of existing mechanisms that fire’, education programs are an even messier example, because they usually involve changing both the context and the ‘reasoning’ of the participants...

--
[log in to unmask]
Professor of Public Sector Evaluation
Centre for Applied Social Research
RMIT University, Australia
15.04.09 124 Latrobe Street Melbourne VIC 3000
ph (03) 9925 2854  m 04 09 386 499 (From overseas: m +614 09 386 499)
www.betterevaluation.org
 


 
--
[log in to unmask]
Professor of Public Sector Evaluation
Centre for Applied Social Research
RMIT University, Australia
15.04.09 124 Latrobe Street Melbourne VIC 3000
ph (03) 9925 2854  m 04 09 386 499 (From overseas: m +614 09 386 499)
www.betterevaluation.org