 |
 |
How very reassuring, Gill. I agree totally, BUT this is an anathema to the traditional [stereotypical] systematic reviewer, who has REPLACED reflection and theory-building with a technical lexicon of coding, categorising, checking, double-checking and above all COUNTING. The link between theory-building systematic review and reflective practice is key, and one that has not previously been made explicit in many how-to-do-it guides. Some of this is just the good old principles of qualitative research, where validity (and scholarship more generally) are defined in terms of reflexivity, making assumptions explicit, immersion in the data, critical awareness of the inevitability of error, etc etc rather than in terms of the 'view from nowhere' and some universal, context-free truth. However, I think we're getting somewhere beyond this. The role of the systematic [theory-building] reviewer is not merely to reflect (and thereby build theory) but also to prompt others to reflect. Dare I introduce a Bakhtinian perspective: there is no story without an audience, and the way the story is told is shaped and constrained by the relationship between the narrator and the audience. When we produce a review, especially when we do so for particular policymakers in particular contexts - and especially when we are trying to pull out theories of change - we form a triangle: the reviewer + the review + the audience, all of which exist in a dynamic tension. This tension is perhaps most pressing when we're doing a 'rapid review' to a deadline for a painfully tiny budget! But always the audience is present, even if only in a distant and assumed way. It's also true that the [realist] evaluator exists in a three-way tension: the evaluator, the policymakers/practitioners and the 'case'. The evaluator seeks to pull out the theories-in-use - and in order to do so, s/he reflects and asks the practitioners to reflect. A good evaluator will tease out theories of action and facilitate the process by which the practitioners engage with and add to emerging interpretations. So realist evaluation is inextricably bound up with reflection/facilitation. As you say Gill, the realist reviewer both loses something (direct access to the reflections of the practitioners and the authors of the original studies) and gains something (a degree of critical distance). Can anyone take this thread forward or challenge it. My day job too calls now! (does anyone else find it ironic that we're doing our thinking outside our day jobs?) trish Trisha Greenhalgh Professor of Primary Health Care and Director, Healthcare Innovation and Policy Unit Centre for Primary Care and Public Health Blizard Institute Barts and The London School of Medicine and Dentistry Yvonne Carter Building Turner Street London E1 2AT t : 020 7882 7325 (PA) or 7326 (dir line) f : 020 7882 2552 e: [log in to unmask] http://www.icms.qmul.ac.uk/chs/staff/trishagreenhalgh.html -----Original Message----- From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards [mailto:[log in to unmask]] On Behalf Of Gill Westhorp Sent: 13 July 2011 00:33 To: [log in to unmask] Subject: Re: How much should we impugn when authors don't make things explicit? Hi all Ah yes, I so agree. I consider theory building to be a necessary part of science and therefore of good science. I consider good theory building practice to include being as well informed as one can be, within all the practical constraints. And I see realist (and probably meta-narrative but I haven't tried one of those) approaches as being purpose designed for the task... it's one of the reasons why I like it. To expound: I believe that there is no such thing as 'interpretation free' analysis - there are after all levels of interpretation and decision-making reflected in every step of every piece of prior research or evaluation that we synthesise, as well as every step of our review process itself. I don't see a problem with taking that one step further and making 'next step interpretations' (ie theory building) - so long as we are explicit about the fact that that's what we're doing. In fact, I almost consider it a duty. Who else is in a better position to do it than those who have immersed themselves in the research and evidence? So while I have sympathy for the desire to stay 'close to the literature', I see that as 'being as well informed as one can be' before proposing a theory that accounts for the findings. As for practitioners not having theory - here I in fact disagree. I think they do have theories - NB plural - albeit at the naieve level sometimes! It's possible in realist evaluation to unpack those. I tell stories in my training about experiences in doing so - it's often when practitioners find out that they're operating on different theories than their colleagues, even if they work together all the time. This provides a wonderful opportunity to deepen reflective practice, and to assist them to access relevant MRT and evidence related to same. The difficulty in realist synthesis is that we don't have the same direct access to the practitioners to find out what their theories are, and those who did the primary research/evaluation didn't always either find out or didn't record same. That's both a strength and a weakness for the synthesis analysts - a weakness because it reduces the clues about where to look for theory and because it means we can't check whether interventions were in fact built on 'the same' theory (now there's an issue for reflective thought: what would constitute adequate evidence that the theories were in fact 'the same'?). But it can be turned into a strength because our job as synthesists is not necessarily to test THEIR theories but to build and test theory ACROSS (insert here whichever version you're doing) ... across manifestations of an intervention; across interventions using similar theories; across interventions using different theories but similar mechanisms; or in MNR, across similar topics but from widely disparate theoretical bases...) Gotta go. Day job calls. Cheers Gill