Paul and Marcello have identified a major issue which definitely needs
sharpening: what exactly IS this "story" or "narrative" that is so central
to meta-narrative review?  Another question follows from this: how do we
identify that meta-narrative?  (I guess these two questions are the
'ontology' and 'epistemology' of meta-narrative review respectively).  

I think it might help if I told the story of how this technique emerged.

Back in 2003, we had gained a grant from the SDO programme to study
"diffusion, spread and sustainability of service innovations", and we had an
explicit brief to explore beyond health services research to get ideas from
other disciplines.  

The first task was to familiarise ourselves with the research in this
general area. We began of course by looking at Everett Rogers' classic work
from the 1940s in rural sociology (the famous Iowa corn farmers, who varied
in the speed with which they adopted new faming methods developed in
universities). Rogers spent 40 years building on concepts such as the
S-shaped adoption curve and the phenomenon of social influence.  Ev Rogers
was still alive in 2003. I briefly corresponded with him and his
co-researcher Tom Valente, and got a great account from them of how
'diffusion of innovation' came and went in rural sociology (i.e. it became
fashionable for people to do their PhD on an aspect of this topic, and for
research calls to invite further work on DoI). But then (said Rogers and
Valente), the rural sociologists moved on to study other things. They showed
this very systematically by counting the number of publications per year in
the main journals - I can find the reference if anyone's interested.

My team discovered that whilst rural sociologists had long since dropped DoI
(because they felt all the key questions had been answered and had moved on
to what they saw as the 'next stage' questions), other groups of scientists
- some of whom knew about Rogers' work and sought to build on it, but some
of whom ploughed their own furrow - were still researching diffusion of
innovations, but they were doing so in a completely different way. They
didn't even mean the same thing when they used the words.  What WAS clear
was that if you were a rural sociologist you researched diffusion of
innovation in THIS way (or you didn't pass your PhD), but if you were (say)
a health services researcher, you researched it in THAT way (ditto). 

So we decided that our first task was to map the multiplicity of different
ways in which the topic of DoI was being (and had been) researched. It was
clear from our data (600+ full text papers on diffusion of innovation) that
the natural unit of analysis was the broad research tradition in which
people were working. In order to unravel these traditions, we drew on Thomas
Kuhn's book 'The structure of scientific revolutions'. Anyone who thinks
they might want to do a meta-narrative review should read Kuhn's book.  He
describes four key aspects of what he calls a paradigm (which underpins a
research tradition): [1] concepts (how the 'thing' is defined and
conceptualised), [2] theories (how the 'thing' is linked to other things in
explanatory models), [3] methods (preferred study designs and approaches)
and [4] instruments (preferred techniques and metrics). Kuhn's take was that
science is a social construction: the undergraduate and postgraduate
training needed to produce (say) a 'rural sociologist' or a 'health services
researcher' was not merely to teach core facts but to convey a shared set of
assumptions and ways of viewing the world without which advanced scientific
work in that discipline cannot progress. As Newton said "I stand on the
shoulders of giants".  

Using this Kuhnian view of science, it follows that the logical way of
unpacking a complex literature is to see whose shoulders particular
scientists were standing on when they did their work - and what were the
(evolving and sometimes internally contested) shared assumptions which
shaped and constrained their work. The key question is, how do we do this in
a rigorous way?  The way we did it for our DoI review was to physically put
the 600 papers into piles by asking ourselves questions like "which of these
guys attend the same conferences as each other, which of them would get sent
whose paper or grant application to review, which ones published in the same
journals as each other, etc?".  (Before someone chips in with the suggestion
that this was effectively a poor man's social network analysis, it sure was
- my colleague Henry Potts has suggested we could and should do this more
systematically using bibliographic software, so if anyone wants to have a go
at this we'd love to see if it works better than sorting papers into piles
on a big table). 

When Henry and I (and others) took on a more challenging topic - the
implementation of electronic patient records - we discovered that quite a
few papers seemed to defy classification. Some of these were just 'pants'
(quasi-scientists who mess around with lots of different paradigmatic
approaches and think they've done something clever by cherry-picking bits
from each of them) but some are brilliant pieces of interdisciplinary work
from which some of the best insights in the field are emerging.
Distinguishing the former from the latter is difficult and I haven't worked
out a systematic way of doing that yet.

Another issue I'd like this list to consider (perhaps AFTER we've had a go
at the basic stuff) is the question of ideology / political economy.  As
David Nicolini says, Kuhn was very insightful in developing his idea of
paradigms but he was naïve in assuming that these different ways of doing
science are ideologically or politically neutral. As we all know, this isn't
the case. Particular conceptualisations, theorisations and methodological
approaches in science are more or less aligned with prevailing ideologies of
the day (e.g. the experimental paradigm in health services research with its
focus on RCTs and limited interest in theory aligns closely with the new
public management "find out what works and implement it").  I've already
been in correspondence with a researcher who asked me what the difference is
between a scientific meta-narrative and what Foucault called a 'discourse'.
I suspect that for some review questions (though I suspect only a small
minority of them), it will be productive to take an explicitly Foucauldian
perspective on the meta-narrative. 

That's probably enough for one posting - hope this helps get some good
discussion going!  

Trish   


-----Original Message-----
From: Realist and Meta-narrative Evidence Synthesis: Evolving Standards
[mailto:[log in to unmask]] On Behalf Of Paul Ward
Sent: 01 July 2011 00:34
To: [log in to unmask]
Subject: Re: meta-narrative syst rev

Hi Marcello

I guess one of the initial issues is whether you are trying to uncover
differing 'conceptualizations' or 'theories' of community within and across
different academic disciplines.  For me, the term 'storyline' is rather
vague and is particularly difficult to operationalise.  I guess you could
potentially examine 'storyline' as 'narrative', and in this way examine the
genealogy of 'community' in different disciplines, but this wouldn't
necessarily help with MRT

I agree that there has already been a lot of work on uncovering the
different meanings of community, but I'm not aware of such a structured and
systematic analysis as the one you are engaged in.  In the 'discipline' of
public health, the term seems to have lost meaning, in that it is variably
used as a proxy for neighborhood, place, locality, groups of people, social
networks etc etc - so I really look forward to seeing the outcomes of your
work

Kind regards

Paul

******************************
Professor Paul Ward
Discipline of Public Health
Flinders University

On 01/07/2011, at 2:27 AM, "Marcello Bertotti" <[log in to unmask]> wrote:

> at the Institute for Health and Human Development (IHHD) based at UeL we
are conducting a meta-narrative systematic review on the meanings of
community from different research traditions (sociology, psychology,
anthropology etc) funded by the Arts and Humanities Research Council. The
idea is to build 'storylines' from each research tradition and compare
research across these. 
> 
> At the moment, we are in the process of coding about 250 initial
references based upon a database search on the concept/meanings of
community, alongside other sources such as our own searches and outcomes
from an academic and policy advisory groups. We hope to identify key authors
in each research tradition and build from that a range of draft
meta-narratives (based on Trisha Greenhalgh's work). 
> 
> What i would like to ask the group is whether they have been engaged in
similar work and whether they can provide any useful tip or lessons they
learnt, even from the 'realist' type of work they have been engaged in. A
specific problem that is tormenting me is the theoretical level of the
'storylines'. We don't want to repeat longstanding work on community but
provide some sort of middle range theories (I don't really know if I am
using the right terminology here). I would like to interact with anybody who
has faced such problems in the past.
> 
> i am obviously happy to provide further details if needed.
> 
> Marcello 
>