Hi All,

I haven't chimed in yet because I am up to my eyeballs in both a
meta-narrative review (the impacts of mining on indigenous communities),
a Cochrane systematic review (later high school start times) and a
realist review of the same. I am in a bit of a comparative/busy mode at
the moment. As Barend and Marjolein noted, it is kind of a luxury to be
able to think more about the methods from an in-depth methodological
perspective at present.

I do however think that Trisha has touched on something interesting. As
a systematic reviewer and ethnographer I agree there is indeed
usefulness in letting our interpretive selves reign (if not entirely
free than at least) supreme in the conduct of metanarrative and realist
reviews. It is probably folly for us to sit and wait claiming "not
enough information" hoping that research and intervention accounts will
improve. Perhaps we can help encourage better conduct and reporting of
research and interventions (and intervention evaluations) but in the
meantime what to do with what we have?

I wonder what "systematic ethnographic techniques" or "systematic
reflection and deliberation" would look like in a practical sense? How
do reviewers "get into the minds" of the researchers or evaluators to
begin to understand the theories they might use or understand (or those
they don't)? If we are indeed feeling wary about "assumptions" ...how
can we get more comfortable with this idea, concept or approach? Is it
in systematic or standard processes, building precedent, arguing for its
value within our different communities, demonstrating rigor and
validity, and/or something else? As reviewers I think we need to be able
to say (in light of some, but not all, missing information) - this is
what I think is going on, and this is the way I came to that decision-
and I am confident in the result and the process (by some measure).

Colleen






On 7/12/2011 3:44 AM, Trisha Greenhalgh wrote:
> Barend and Marjolin said
>
> " Concerning the MRT: we only made the MRTs retrospectively as from the
> literature we could not define MRTs (or program logic) for most of the
> programs as too little information was given in the articles. Staying close
> to the literature was exactly what we wanted, not using our assumptions of
> what the authors had as their MRT as theory building. "
>
> This is an interesting issue which I've encountered in realist evaluation
> too.  In my experience people who are "doing stuff" don't have a theory
> which they can articulate (that's 'theory' in both senses - they don't have
> a mechanism of action and they certainly don't a middle-range theory).  But
> I'm not sure that it's necessarily good science to keep faithful to this
> lack of espoused theory when interpreting other authors' work.  It MIGHT be
> OK (indeed, it might be good practice) to use systematic reflection and
> deliberation to impugn the theory/ies that are implicit in authors' actions.
> There's a parallel here in ethnography - human actors are often not very
> good at articulating what they're up to but they're very good at (say) doing
> their jobs. By using systematic ethnographic techniques one can often build
> up a robust picture of social action EVEN THOUGH the actors themselves
> haven't expressed their actions in those terms. One aspect of validity here
> is whether the actors recognise and engage with the explanations...
>
> Comments Gill?  (and others...)
>
>
> Trisha Greenhalgh
> Professor of Primary Health Care and Director, Healthcare Innovation and
> Policy Unit
> Centre for Primary Care and Public Health
> Blizard Institute
> Barts and The London School of Medicine and Dentistry
> Yvonne Carter Building
> Turner Street
> London E1 2AT
> t : 020 7882 7325 (PA) or 7326 (dir line)
> f : 020 7882 2552
> e: [log in to unmask]
>
>
> http://www.icms.qmul.ac.uk/chs/staff/trishagreenhalgh.html
>
>
> -----Original Message-----
> From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
> [mailto:[log in to unmask]] On Behalf Of Marjolein Dieleman
> Sent: 08 July 2011 13:29
> To: [log in to unmask]
> Subject: Reaction to messges about realis review retention studies
>
> We greatly thank Mark and Gill for their reactions to our realist review and
> for taking the time to read it and write such elaborative reactions. We took
> quite some time to react, as it is for us at the moment a luxury to take
> time and think- and of course it is not easy to respond either.
>
> In particular the way we have operationalised "mechanisms" and the way that
> Mark and Gill responded to our operationalisation gave us food for thought.
> We called mechanisms reactions and understood it in the way Gill explained:
> "changed reasoning in response to resources or opportunities provided by the
> program, leading to changed behaviours, in turn causing changed outcomes".
> However, we believe that a program can trigger several mechanisms and their
> sum can have a positive, negative or a net-zero outcome. Reactions were for
> us a result of the trigger, and hence are mechanisms that make a program or
> intervention work or not work.
>
> Gill described the different ways of conceptualising mechanisms- and it
> would be interesting to find more examples to describe these levels, as for
> us it was not very clear how these were different. The example from the text
> that Gill provided triggered another thought on our side: in Ecuador apathy
> and resentment caused two different types of outcomes:- poor quality but
> also less likely to be retained- so in fact in this case the sum of the
> mechanisms caused two different outcomes.
>
> We really would like to be able to explain this to outsiders in an
> understandable way without compromising the concept of mechanism- any
> suggestions?
>
> Concerning the MRT: we only made the MRTs retrospectively as from the
> literature we could not define MRTs (or program logic) for most of the
> programs as too little information was given in the articles. Staying close
> to the literature was exactly what we wanted, not using our assumptions of
> what the authors had as their MRT as theory building. For testing the
> retrospectively created theories, 'our' MRTs, was not enough time and
> resources available but would have been of course the most ideal.
>
> Looking forward to your reactions!
> Barend and Marjolein