Hi again
Yes, we could get lost in analysis unless we consider the purpose before we choose the approach to categorising mechanisms. Which leads me to think about framing and setting boundaries for the review. 
If I were the Chief Executive of Nike, and I wanted to commission a review exploring what adverts work best to convince people in, let's say Southeast Asia, to buy bike helmets - which categorisation system would we use
> rational, normative, emotive
>  individual / group; natural science / social; level of system (worker, unit, organisation, system).

Let's say there is a lot of published research and many sources of info across stakeholders. But I'm not prepared to pay someone to use all of these categorisations.
So what process do we go through to decide?

Best wishes
Janet

> On 14 Feb 2014, at 17:04, Gill Westhorp <[log in to unmask]> wrote:
> 
> Hi all
> Reinhard's bike helmet example is a nice examples of different mechanisms in
> different contexts for the same person.  The HIV example is, as framed, an
> example of the same mechanism firing (or not) in different cultural contexts
> (which is not to say that a normative intervention couldn't work in Zimbabwe
> - it would be a matter of finding the right norm for that context.) 
> 
> Janet's example is a nice example of multiple mechanisms for the same person
> at the same time - which of course may reinforce or undermine each other.
> 
> And then of course you've got all the examples where there are multiple
> mechanisms firing for multiple participants in the same program even though
> they're apparently in the 'same' context (because, of course, they aren't.) 
> 
> But I'm not sure any of this gets to the issue of 'purpose' for the proposed
> categorisation system (rational, normative, emotive).  There are lots of
> ways one can categorise mechanisms - e.g. individual / group; natural
> science / social; level of system (worker, unit, organisation, system).
> Whether it makes sense to use a particular categorisation will IMHO depend
> on purpose.  If it helps to clarify program theory in a particular
> situation, I'm all for it.  If it's being proposed as a general schema, I'm
> yet to be convinced. 
> 
> Cheers
> Gill 
> 
> 
> 
> -----Original Message-----
> From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
> [mailto:[log in to unmask]] On Behalf Of Reinhard Huss
> Sent: Friday, 14 February 2014 4:20 AM
> To: [log in to unmask]
> Subject: Re: Advice on testing theories - and how programs work.
> 
> Hallo Purva, Gill and Janet,
> I was raising the question about different types of mechanism, because these
> mechanisms may interact in a synergistic or antagonistic way. It may also be
> important to consider the sequence of interventions which trigger certain
> types of mechanisms.
> 
> I think the term rational is probably more appropriate than cognitive
> mechanism, because I refer to the rational understanding of an issue.
> 
> To be more concrete I present a historical self-experiment:
> 
> 1) I am a keen cyclist and for a long time I had many discussions with many
> people about the protective function of bicycle helmets. In my assessment
> the "rational" evidence on the protection of bicycle helmets was
> inconclusive or weak and I also "believed" that I was a very experienced
> cyclist who did not need such a helmet.
> 
> 2) Next I was pushed and convinced by some relatives and colleagues to use a
> helmet in order to set a "normative" example to my son who was learning to
> ride a bicycle.
> 
> 3) During this period (When I set the example to my son) I crashed with
> another cyclist due to my mistake. My helmet split into two parts, but my
> head remained intact. Since this event I am convinced due to this powerful
> "emotional" experience that wearing a helmet is good for me and I do it all
> the time. Sometimes I forget to put on the helmet, but even when I am in a
> rush I go back and collect my helmet.
> 
> In my work as a health professional I made similar observations regarding
> different mechanisms to protect against HIV infections, e.g. a normative
> intervention to use condoms in casual sexual relationships may work better
> (trigger certain mechanism) in the social and cultural context of Thailand
> than in the context of Zimbabwe.
> 
> Best wishes reinhard
> 
> -----Original Message-----
> From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
> [mailto:[log in to unmask]] On Behalf Of Janet Harris
> Sent: 13 February 2014 21:31
> To: [log in to unmask]
> Subject: Re: Advice on testing theories - and how programs work.
> 
> Hi Purva, Gill, Reinhard
> This thought builds on Gill's comment: cognitive, normative and emotive
> mechanisms often get triggered at the same time e.g we understand something,
> the process of cognition means we are busy fitting it into to our belief
> system ( or not - we may be experiencing cognitive dissonance).  And
> depending on how much we need to belong, we are either rejecting or
> accepting based on social norms.
> So multiple mechanisms, perhaps occurring in a relatively short period of
> time...
> Best wishes
> Janet
> 
> Sent from my iPad
> 
>>> On 12 Feb 2014, at 02:10, Gill Westhorp <[log in to unmask]>
>> wrote:
>> 
>> Hi all
>> I agree with Purva that whether it makes sense to do it depends on
> purpose.
>> But beware the notion that programs work through one or the other...
>> Many
>> (most?) programs work through multiple mechanisms.  So I'd ask two 
>> further
>> questions:  1. What's the relationship to program theory? 2. Is there 
>> a relationship between category of mechanism and 'for whom' programs work?
>> 
>> Cheers
>> Gill
>> 
>> -----Original Message-----
>> From: Realist and Meta-narrative Evidence Synthesis: Evolving 
>> Standards [mailto:[log in to unmask]] On Behalf Of Purva Abhyankar
>> Sent: Wednesday, 12 February 2014 1:47 AM
>> To: [log in to unmask]
>> Subject: Re: Advice on testing theories - and how programs work.
>> 
>> Hi
>> 
>> In response to Reinhard's question "Does it make sense to separate 
>> between cognitive, normative and emotive (belief system, ideology)
> mechanisms?"
>> In my opinion, it does make sense- and I remember reading somewhere 
>> that mechanisms could be of cognitive, emotional or behavioural nature 
>> and at theR level of individuals, groups, or organisations.
>> I struggle to see how normative is different from cognitive - as a 
>> normative mechanism still involves  some change in people's cognition 
>> (or how/what they think).
>> 
>> A broader question would be that what purpose would it serve to 
>> distinguish between different types of mechanisms. One reason I can 
>> think of is that it will be useful when drawing transferrable lessons 
>> from mechanisms observed in one evaluation. E.g. If it is seen that a 
>> particular intervention or its component leads to positive outcomes by 
>> changing the way people think/what they think (cognitive), then 
>> 'change in people's thinking' becomes the active ingredient which 
>> brings upon change. So interventions in other contexts would focus 
>> more on developing something that changes how people think, rather than
> replicating the component from a previous intervention.
>> In other words, the actual interventions/components may be different 
>> in different settings, but the underlying mechanism could be 
>> transferred to another setting. It might then be useful to know 
>> whether the mechanism is a change in cognitions, affect/feeling or
> directly behaviours.
>> Apologies if I have just stated the obvious!
>> 
>> Purva
>> 
>> Dr. Purva Abhyankar
>> NMAHP Research Unit
>> University of Stirling
>> Unit 13 Scion House
>> Stirling University Innovation Park
>> Stirling
>> FK9 4NF
>> 
>> Phone: +44 (0)1786 466 102
>> Email: [log in to unmask]
>> 
>> 
>> Latest papers:
>> Abhyankar P, Volk R, Blumenthal-Barby J, Bravo P, Buchholz A, Ozanne 
>> E, et al. Balancing the presentation of information and options in 
>> patient decision aids: an updated review. BMC Medical Informatics and 
>> Decision Making 2013;13(Suppl 2):S6.
>> 
>> Fagerlin A, Pignone M, Abhyankar P, Col N, Feldman-Stewart D, 
>> Gavaruzzi T, et al. Clarifying values: an updated review. BMC Medical 
>> Informatics and Decision Making 2013;13(Suppl 2):S8.
>> 
>> Laterst Methodological papers
>> Abhyankar P, Cheyne H, Maxwell M, Harris F and McCourt C. (2013) A 
>> realist evaluation of a normal birth programme. Evidence Based Midwifery,
> 11(4):
>> 112-119.
>> 
>> Latest NMAHP RU Newsletter:
>> NMAHP Research Unit Newsletter - Winter 2013
>> 
>> 
>> -----Original Message-----
>> From: Realist and Meta-narrative Evidence Synthesis: Evolving 
>> Standards [mailto:[log in to unmask]] On Behalf Of Reinhard Huss
>> Sent: 07 February 2014 18:02
>> To: [log in to unmask]
>> Subject: Re: Advice on testing theories - and how programs work.
>> 
>> Hallo
>> As an observer and newcomer to this discussion, I would like to ask 
>> the following question:
>> 
>> Does it make sense to separate between cognitive, normative and 
>> emotive (belief system, ideology) mechanisms?
>> 
>> Purva's last example seems to describe a normative mechanism where the 
>> health board has made a commitment to certain norms for childbirth in 
>> Scotland.
>> 
>> Best wishes
>> Reinhard
>> Senior Teaching Fellow
>> University of Leeds
>> 
>> 
>> 
>> 
>> 
>> 
>> -----Original Message-----
>> From: Realist and Meta-narrative Evidence Synthesis: Evolving 
>> Standards [mailto:[log in to unmask]] On Behalf Of Purva Abhyankar
>> Sent: 30 January 2014 08:12
>> To: [log in to unmask]
>> Subject: Re: Advice on testing theories - and how programs work.
>> 
>> Hi
>> 
>> In response to Gill's question: whether programmes always and only 
>> work by changing the balance of existing mechanisms that fire OR do 
>> they also work by creating new mechanisms (new reasoning, new 
>> emotional states, new responses to existing contexts) within the 
>> client - I would say programmes do both as we found in our evaluation 
>> of a normal birth programme in Scotland.
>> Briefly, the programme introduced three components (can be seen as 
>> opportunities or resources to make change happen), two of which were 
>> implementation of normal birth pathways and appointment of a 
>> consultant midwife to facilitate the implementation of pathways and 
>> other normality promoting policies.
>> 
>> Consistent with Gill's first explanation, the pathways did change the 
>> balance of existing mechanisms that fire - in that the pathways 
>> enabled those midwives who were already inclined towards normality to 
>> actually implement the normality practice. It might be fit to say here 
>> that the pathways changed the context by making 'normality' a policy 
>> and a norm practitioners were expected to follow. This was of course 
>> facilitated by the consultant midwives whose job was to ensure that 
>> pathways are implemented smoothly. However, the pathways alone didn't 
>> go as far as introducing new mechanisms among those who weren't already so
> inclined towards normality.
>> Respondents in our evaluation reported that the programme didn't 
>> convert the unconverted.
>> 
>> But we also found that another component of the programme, the 
>> appointment of consultant midwife at the level of the health board, 
>> did also introduce new mechanisms in addition to changing the balance of
> existing mechanims.
>> This post (a senior midwife appointed to promote normality in
>> childbirth) created the 'commitment' mechanism- i.e. it signalled the 
>> commitment from the health boards to implement the normal birth 
>> programme and highlighted the importance of doing so. So before the 
>> programme, existing normality policies were inconsistently implemented 
>> due to lack of a strong approach - normality was kind of optional. But 
>> with the programme's new resource, normality became officially 
>> endorsed. This endorsement at the higher level filtered through to the 
>> practice level in that all practices that deviated from normality now had
> to be justified.
>> 
>> This is not to say that it was rather easy for us to classify x as 
>> context, mechanism or outcome- I feel, in common with other 
>> evaluations, this was slightly muddled. But that apart....
>> 
>> Purva
>> 
>> 
>> 
>> 
>> Purva Abhyankar
>> Research Fellow
>> NMAHP Research Unit
>> University of Stirling
>> Iris Murdoch Building
>> Stirling
>> FK9 4LA
>> 
>> Tel: 01786 466 102
>> Email: [log in to unmask]
>> ________________________________________
>> From: Realist and Meta-narrative Evidence Synthesis: Evolving 
>> Standards [[log in to unmask]] On Behalf Of Gill Westhorp 
>> [[log in to unmask]]
>> Sent: 29 January 2014 20:35
>> To: [log in to unmask]
>> Subject: Re: Advice on testing theories - and how programs work.
>> 
>> Hi all
>> My turn to ask a question, in response to Geoff's (as always, helpful) 
>> response below.  Geoff and I have had this conversation before but I'm 
>> curious as to what others think.  The question is: Do programs always 
>> and only work by changing the context and thereby changing the balance 
>> of mechanisms that fire?
>> 
>> It's relatively straightforward in incentives-based programs to say 
>> 'The new incentive changes the context in that it enables a proportion 
>> of the population who were already inclined to do x to now do x' (or 
>> in the case of negative incentives such as increased costs or 
>> sanctions, 'changes the weight of factors against the behaviour such 
>> that it increases the proportion of the population who decide against the
> behaviour').
>> 
>> But what about therapeutic services?  One can of course argue that the 
>> context is different than it would be if no therapeutic service were 
>> available, but one can also argue that that is not what causes or 
>> enables the program to work.  The program works not by changing 'the 
>> balance of existing mechanisms that fire' but (arguably) by creating 
>> new mechanisms (new reasoning, new emotional states, new responses to 
>> existing contexts) within the client.
>> 
>> If one accepts the premise that such programs do not work by 'changing 
>> the context and the balance of existing mechanisms that fire', 
>> education programs are an even messier example, because they usually 
>> involve changing both the context and the 'reasoning' of the
> participants...
>> 
>> This of course plays back into some broader questions (for example, 
>> when and how to classify 'x' as context, mechanism or outcome) and I 
>> wonder how others out there think about it and deal with it?
>> 
>> Cheers
>> Gill
>> 
>> From: Geoff Wong [mailto:[log in to unmask]]
>> Sent: Wednesday, 29 January 2014 7:30 PM
>> To: Realist and Meta-narrative Evidence Synthesis: Evolving Standards; 
>> Gill Westhorp
>> Subject: Re: Advice on testing theories - Realist Review
>> 
>> Dear Mairi,
>> I hope I have not mis-interpreted you intentions and level of 
>> understanding of realist review, if I have, I apologies in advance.
>> Especially as you asked about theory testing and I am going to be mention
> something else.
>> I note that your question is:
>> Which mechanisms within child health interventions, positively 
>> influence child health promoting care giver behaviour?
>> This is not a 100 miles from a related question:
>> Which child health interventions, positively influence child health 
>> promoting care giver behaviour?
>> 
>> My point of making this comparison between questions echos the 
>> detailed explanations above... that an intervention is not a mechanism.
>> To put it very crudely, interventions / programmes try to change 
>> context such that the relevant mechanisms are triggered to generate 
>> the desired outcomes.
>> I am sure you are aware of this and if you are, then please do ignore 
>> this email.
>> If not you may wish to (if you have not already) access the RAMESES 
>> training materials for realist synthesis:
>> http://www.ramesesproject.org/media/Realist_reviews_training_materials
>> .pdf A last bit of (perhaps again tangential advice) is that you may 
>> also wish to spend some time to ensure that you focus your review 
>> appropriately. As you mention the realist review you plan is part of 
>> your PhD. Time is always short and realist reviews can grow into 
>> unmanagable beasts. In addition you are 'learning' a new method. You 
>> might find section 3 on Focussing reviews of the abovementioned 
>> resource of some help.
>> Good luck
>> 
>> Geoff
>> 
>> On 28 January 2014 23:52, Gill Westhorp 
>> <[log in to unmask]<mailto:[log in to unmask]>> wrote:
>> Hi Mairi
>> To add to Justin's beautiful answer:
>> 
>> So what does it mean when we say we will 'test' a realist theory?
>> Firstly, it means our theory will take a certain structure - it will 
>> identify hypothesised (or previously researched, if we're lucky) 
>> mechanisms, the outcomes that they generate, and the contexts in which 
>> they do and don't work.
>> 
>> It will then consider 'what might constitute evidence for these 
>> elements of the theory?'  Because mechanisms operate a) as a function 
>> of 'the powers and liabilities' of the system, and at a different 
>> level of the system (either higher or lower) than the outcomes of 
>> interest, this means investigating different things than the 'outcomes 
>> level'.  Most reviews will not be able to investigate all aspects of 
>> the theory and will select a particular aspect.
>> 
>> To return to your example: I agree with Justin that goal-setting is a 
>> program strategy or activity rather than a mechanism.  (Simple test:
>> can you see it happening?  Is it something that the program (the 
>> people in the
>> program) actually 'do'? - if yes, it's an activity/strategy, not a 
>> mechanism).  If you were investigating goal setting, you'd be asking 
>> questions like:  So how and why does goal setting change the outcomes 
>> that programs generate? What does it provide for whom along the 
>> program implementation pathway?  How does it change the 'reasoning'
>> (catch all term for whatever happens inside the heads of 
>> decision-makers along that route) of whom?  What different decisions 
>> are generated as a result of that different reasoning, and how do 
>> those different decisions generate different outcomes? When does and 
>> doesn't that happen? Having hypothesised those elements - yes, you 
>> would then toddle off to investigate the literature about those 
>> elements, then bring it back and apply it to your particular topic.
>> 
>> The other way in which realist theory testing differs from other forms 
>> of theory testing is in the notion of 'cases as nuggets of evidence 
>> along the theory chain'.  I.E. - having laid out the THEORY (not just 
>> the program theory per se, but the realist theory as well), the 
>> 'nuggets' of evidence are aligned against the specific aspect of the
> theory to which they relate.
>> This helps to overcome a common problem for new realist analysts, who 
>> often want to argue that evidence of outcomes equals evidence of the 
>> hypothesised mechanism.  It doesn't - because several different 
>> mechanisms could have generated the same outcome.
>> 
>> I've answered here primarily in terms of mechanism but of course the 
>> same sorts of processes also apply to outcomes (NB outcomes at 
>> different levels at different points in time in different levels of 
>> the system are all 'outcomes' in realist terms - even though they can 
>> also become new contexts or operate as 'resources' for new mechanisms 
>> in the next stage of the
>> process) and to context.  The trick for context is to work out which 
>> ELEMENTS of context relate to / affect which mechanisms (and/or the 
>> ability of decision-makers to put their decisions into practice).
>> 
>> 
>> Cheers
>> Gill
>> 
>> -----Original Message-----
>> From: Realist and Meta-narrative Evidence Synthesis: Evolving 
>> Standards 
>> [mailto:[log in to unmask]<mailto:[log in to unmask]>] On 
>> Behalf Of Mairi Anne Young
>> Sent: Wednesday, 29 January 2014 4:13 AM
>> To: [log in to unmask]<mailto:[log in to unmask]>
>> Subject: Advice on testing theories - Realist Review Hi,
>> 
>> I'm a 2nd year Dental Public Health PhD student working on a protocol 
>> for a Realist Review. My research question is: Which mechanisms within 
>> child health interventions, positively influence child health 
>> promoting care giver behaviour?
>> 
>> The review will form part of my PhD which is an evaluation of a Lay 
>> Health Worker role within the Childsmile dental programme.
>> 
>> While researching Realist Review methodology and developing my 
>> protocol the only stumbling block I'm coming up against is the concept 
>> of 'theory testing'. I understand the searching is an iterative 
>> process, one which involves identifying and testing theories. However 
>> I'm failing to understand what exactly is involved within this testing
> phase.
>> 
>> For example, if it arose that Goal Setting was a theory, would I then 
>> be re-entering the literature with the intention of locating evidence 
>> related to Goal Setting? Would this evidence have to confirm why Goal 
>> Setting is an effective tool?
>> 
>> I would be grateful for any advice or information anyone could provide 
>> on this matter.
>> 
>> Kind regards,
>> Mairi
>> 
>> 
>> 
>> 
>> Mairi Young BA (hons) MSc MBPsS
>> PhD Student
>> The University of Glasgow, Glasgow Dental Hospital & School Level 8 
>> COH Office 378 Sauchiehall Street Glasgow G2 3JZ
>> 
>> Email: [log in to unmask]<mailto:[log in to unmask]>
>> Tel:    0794 7744 517<tel:0794%207744%20517>
>> 
>> --
>> The University of Stirling has been ranked in the top 12 of UK 
>> universities for graduate employment*.
>> 94% of our 2012 graduates were in work and/or further study within six 
>> months of graduation.
>> *The Telegraph
>> The University of Stirling is a charity registered in Scotland, number 
>> SC 011159.
>> 
>> 
>> 
>> --
>> The University of Stirling has been ranked in the top 12 of UK 
>> universities for graduate employment*.
>> 94% of our 2012 graduates were in work and/or further study within six 
>> months of graduation.
>> *The Telegraph
>> The University of Stirling is a charity registered in Scotland, number 
>> SC 011159.