Hi all
I agree with Purva that whether it makes sense to do it depends on purpose.
But beware the notion that programs work through one or the other... Many
(most?) programs work through multiple mechanisms. So I'd ask two further
questions: 1. What's the relationship to program theory? 2. Is there a
relationship between category of mechanism and 'for whom' programs work?
Cheers
Gill
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Purva Abhyankar
Sent: Wednesday, 12 February 2014 1:47 AM
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.
Hi
In response to Reinhard's question "Does it make sense to separate between
cognitive, normative and emotive (belief system, ideology) mechanisms?"
In my opinion, it does make sense- and I remember reading somewhere that
mechanisms could be of cognitive, emotional or behavioural nature and at the
level of individuals, groups, or organisations.
I struggle to see how normative is different from cognitive - as a normative
mechanism still involves some change in people's cognition (or how/what
they think).
A broader question would be that what purpose would it serve to distinguish
between different types of mechanisms. One reason I can think of is that it
will be useful when drawing transferrable lessons from mechanisms observed
in one evaluation. E.g. If it is seen that a particular intervention or its
component leads to positive outcomes by changing the way people think/what
they think (cognitive), then 'change in people's thinking' becomes the
active ingredient which brings upon change. So interventions in other
contexts would focus more on developing something that changes how people
think, rather than replicating the component from a previous intervention.
In other words, the actual interventions/components may be different in
different settings, but the underlying mechanism could be transferred to
another setting. It might then be useful to know whether the mechanism is a
change in cognitions, affect/feeling or directly behaviours.
Apologies if I have just stated the obvious!
Purva
Dr. Purva Abhyankar
NMAHP Research Unit
University of Stirling
Unit 13 Scion House
Stirling University Innovation Park
Stirling
FK9 4NF
Phone: +44 (0)1786 466 102
Email: [log in to unmask]
Latest papers:
Abhyankar P, Volk R, Blumenthal-Barby J, Bravo P, Buchholz A, Ozanne E, et
al. Balancing the presentation of information and options in patient
decision aids: an updated review. BMC Medical Informatics and Decision
Making 2013;13(Suppl 2):S6.
Fagerlin A, Pignone M, Abhyankar P, Col N, Feldman-Stewart D, Gavaruzzi T,
et al. Clarifying values: an updated review. BMC Medical Informatics and
Decision Making 2013;13(Suppl 2):S8.
Laterst Methodological papers
Abhyankar P, Cheyne H, Maxwell M, Harris F and McCourt C. (2013) A realist
evaluation of a normal birth programme. Evidence Based Midwifery, 11(4):
112-119.
Latest NMAHP RU Newsletter:
NMAHP Research Unit Newsletter - Winter 2013
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Reinhard Huss
Sent: 07 February 2014 18:02
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.
Hallo
As an observer and newcomer to this discussion, I would like to ask the
following question:
Does it make sense to separate between cognitive, normative and emotive
(belief system, ideology) mechanisms?
Purva's last example seems to describe a normative mechanism where the
health board has made a commitment to certain norms for childbirth in
Scotland.
Best wishes
Reinhard
Senior Teaching Fellow
University of Leeds
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Purva Abhyankar
Sent: 30 January 2014 08:12
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.
Hi
In response to Gill's question: whether programmes always and only work by
changing the balance of existing mechanisms that fire OR do they also work
by creating new mechanisms (new reasoning, new emotional states, new
responses to existing contexts) within the client - I would say programmes
do both as we found in our evaluation of a normal birth programme in
Scotland.
Briefly, the programme introduced three components (can be seen as
opportunities or resources to make change happen), two of which were
implementation of normal birth pathways and appointment of a consultant
midwife to facilitate the implementation of pathways and other normality
promoting policies.
Consistent with Gill's first explanation, the pathways did change the
balance of existing mechanisms that fire - in that the pathways enabled
those midwives who were already inclined towards normality to actually
implement the normality practice. It might be fit to say here that the
pathways changed the context by making 'normality' a policy and a norm
practitioners were expected to follow. This was of course facilitated by the
consultant midwives whose job was to ensure that pathways are implemented
smoothly. However, the pathways alone didn't go as far as introducing new
mechanisms among those who weren't already so inclined towards normality.
Respondents in our evaluation reported that the programme didn't convert the
unconverted.
But we also found that another component of the programme, the appointment
of consultant midwife at the level of the health board, did also introduce
new mechanisms in addition to changing the balance of existing mechanims.
This post (a senior midwife appointed to promote normality in childbirth)
created the 'commitment' mechanism- i.e. it signalled the commitment from
the health boards to implement the normal birth programme and highlighted
the importance of doing so. So before the programme, existing normality
policies were inconsistently implemented due to lack of a strong approach -
normality was kind of optional. But with the programme's new resource,
normality became officially endorsed. This endorsement at the higher level
filtered through to the practice level in that all practices that deviated
from normality now had to be justified.
This is not to say that it was rather easy for us to classify x as context,
mechanism or outcome- I feel, in common with other evaluations, this was
slightly muddled. But that apart....
Purva
Purva Abhyankar
Research Fellow
NMAHP Research Unit
University of Stirling
Iris Murdoch Building
Stirling
FK9 4LA
Tel: 01786 466 102
Email: [log in to unmask]
________________________________________
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[[log in to unmask]] On Behalf Of Gill Westhorp
[[log in to unmask]]
Sent: 29 January 2014 20:35
To: [log in to unmask]
Subject: Re: Advice on testing theories - and how programs work.
Hi all
My turn to ask a question, in response to Geoff's (as always, helpful)
response below. Geoff and I have had this conversation before but I'm
curious as to what others think. The question is: Do programs always and
only work by changing the context and thereby changing the balance of
mechanisms that fire?
It's relatively straightforward in incentives-based programs to say 'The new
incentive changes the context in that it enables a proportion of the
population who were already inclined to do x to now do x' (or in the case of
negative incentives such as increased costs or sanctions, 'changes the
weight of factors against the behaviour such that it increases the
proportion of the population who decide against the behaviour').
But what about therapeutic services? One can of course argue that the
context is different than it would be if no therapeutic service were
available, but one can also argue that that is not what causes or enables
the program to work. The program works not by changing 'the balance of
existing mechanisms that fire' but (arguably) by creating new mechanisms
(new reasoning, new emotional states, new responses to existing contexts)
within the client.
If one accepts the premise that such programs do not work by 'changing the
context and the balance of existing mechanisms that fire', education
programs are an even messier example, because they usually involve changing
both the context and the 'reasoning' of the participants...
This of course plays back into some broader questions (for example, when and
how to classify 'x' as context, mechanism or outcome) and I wonder how
others out there think about it and deal with it?
Cheers
Gill
From: Geoff Wong [mailto:[log in to unmask]]
Sent: Wednesday, 29 January 2014 7:30 PM
To: Realist and Meta-narrative Evidence Synthesis: Evolving Standards; Gill
Westhorp
Subject: Re: Advice on testing theories - Realist Review
Dear Mairi,
I hope I have not mis-interpreted you intentions and level of understanding
of realist review, if I have, I apologies in advance. Especially as you
asked about theory testing and I am going to be mention something else.
I note that your question is:
Which mechanisms within child health interventions, positively influence
child health promoting care giver behaviour?
This is not a 100 miles from a related question:
Which child health interventions, positively influence child health
promoting care giver behaviour?
My point of making this comparison between questions echos the detailed
explanations above... that an intervention is not a mechanism.
To put it very crudely, interventions / programmes try to change context
such that the relevant mechanisms are triggered to generate the desired
outcomes.
I am sure you are aware of this and if you are, then please do ignore this
email.
If not you may wish to (if you have not already) access the RAMESES training
materials for realist synthesis:
http://www.ramesesproject.org/media/Realist_reviews_training_materials.pdf
A last bit of (perhaps again tangential advice) is that you may also wish to
spend some time to ensure that you focus your review appropriately. As you
mention the realist review you plan is part of your PhD. Time is always
short and realist reviews can grow into unmanagable beasts. In addition you
are 'learning' a new method. You might find section 3 on Focussing reviews
of the abovementioned resource of some help.
Good luck
Geoff
On 28 January 2014 23:52, Gill Westhorp
<[log in to unmask]<mailto:[log in to unmask]>> wrote:
Hi Mairi
To add to Justin's beautiful answer:
So what does it mean when we say we will 'test' a realist theory? Firstly,
it means our theory will take a certain structure - it will identify
hypothesised (or previously researched, if we're lucky) mechanisms, the
outcomes that they generate, and the contexts in which they do and don't
work.
It will then consider 'what might constitute evidence for these elements of
the theory?' Because mechanisms operate a) as a function of 'the powers and
liabilities' of the system, and at a different level of the system (either
higher or lower) than the outcomes of interest, this means investigating
different things than the 'outcomes level'. Most reviews will not be able
to investigate all aspects of the theory and will select a particular
aspect.
To return to your example: I agree with Justin that goal-setting is a
program strategy or activity rather than a mechanism. (Simple test: can
you see it happening? Is it something that the program (the people in the
program) actually 'do'? - if yes, it's an activity/strategy, not a
mechanism). If you were investigating goal setting, you'd be asking
questions like: So how and why does goal setting change the outcomes that
programs generate? What does it provide for whom along the program
implementation pathway? How does it change the 'reasoning' (catch all term
for whatever happens inside the heads of decision-makers along that route)
of whom? What different decisions are generated as a result of that
different reasoning, and how do those different decisions generate different
outcomes? When does and doesn't that happen? Having hypothesised those
elements - yes, you would then toddle off to investigate the literature
about those elements, then bring it back and apply it to your particular
topic.
The other way in which realist theory testing differs from other forms of
theory testing is in the notion of 'cases as nuggets of evidence along the
theory chain'. I.E. - having laid out the THEORY (not just the program
theory per se, but the realist theory as well), the 'nuggets' of evidence
are aligned against the specific aspect of the theory to which they relate.
This helps to overcome a common problem for new realist analysts, who often
want to argue that evidence of outcomes equals evidence of the hypothesised
mechanism. It doesn't - because several different mechanisms could have
generated the same outcome.
I've answered here primarily in terms of mechanism but of course the same
sorts of processes also apply to outcomes (NB outcomes at different levels
at different points in time in different levels of the system are all
'outcomes' in realist terms - even though they can also become new contexts
or operate as 'resources' for new mechanisms in the next stage of the
process) and to context. The trick for context is to work out which
ELEMENTS of context relate to / affect which mechanisms (and/or the ability
of decision-makers to put their decisions into practice).
Cheers
Gill
-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]<mailto:[log in to unmask]>] On Behalf Of
Mairi Anne Young
Sent: Wednesday, 29 January 2014 4:13 AM
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Advice on testing theories - Realist Review Hi,
I'm a 2nd year Dental Public Health PhD student working on a protocol for a
Realist Review. My research question is: Which mechanisms within child
health interventions, positively influence child health promoting care giver
behaviour?
The review will form part of my PhD which is an evaluation of a Lay Health
Worker role within the Childsmile dental programme.
While researching Realist Review methodology and developing my protocol the
only stumbling block I'm coming up against is the concept of 'theory
testing'. I understand the searching is an iterative process, one which
involves identifying and testing theories. However I'm failing to understand
what exactly is involved within this testing phase.
For example, if it arose that Goal Setting was a theory, would I then be
re-entering the literature with the intention of locating evidence related
to Goal Setting? Would this evidence have to confirm why Goal Setting is an
effective tool?
I would be grateful for any advice or information anyone could provide on
this matter.
Kind regards,
Mairi
Mairi Young BA (hons) MSc MBPsS
PhD Student
The University of Glasgow, Glasgow Dental Hospital & School Level 8 COH
Office 378 Sauchiehall Street Glasgow G2 3JZ
Email: [log in to unmask]<mailto:[log in to unmask]>
Tel: 0794 7744 517<tel:0794%207744%20517>
--
The University of Stirling has been ranked in the top 12 of UK universities
for graduate employment*.
94% of our 2012 graduates were in work and/or further study within six
months of graduation.
*The Telegraph
The University of Stirling is a charity registered in Scotland, number SC
011159.
--
The University of Stirling has been ranked in the top 12 of UK universities
for graduate employment*.
94% of our 2012 graduates were in work and/or further study within six
months of graduation.
*The Telegraph
The University of Stirling is a charity registered in Scotland, number SC
011159.
|
