Dear Justin,
Thank you very much for the very interesting and timely e-mail.
I am junior researcher in international development and very interested in methodological discussions in realist approach.
My initial interest and desire to use realist approach in a systematic review I am leading exposed me to a lot of challenges including methodological ones. My initial intention was to follow up existing standards such as RAMESES in the international development field and I approached the methodological issues quite pragmatically. But I realized that journals that are open to publishing realist review in international development are very limited and the ones that can publish a realist protocol in international development hardly exist. So I started to work on introducing some more standard Campbell review elements that can increase the chances of publication. In this process, I really enjoyed learning more about methodological issues and drafted a methodological paper and draft guidelines that I want to share with the realist review community. Actually I joined the group recently to identify senior researchers whom I can ask for advises and possibly to identify collaborators for the systematic review. So your e-mail is quite exciting and timely!
Here is a brief text I wanted to use as a base for discussion. I omitted elements that are conceptually intense.
https://www.researchgate.net/publication/301788078_A_Guide_for_Conducting_Systematic_Reviews_in_International_Development_and_Social_Sciences_using_the_case_of_effectiveness_of_Agricultural_Research_for_Development_AR4D_Interventions_with_Multi-stakeh
If you think that there is room for collaboration, I will be very happy to discuss my work, collaboration possibilities and share the method paper I drafted for your comments. If it is relevant, I would love to look for synergies between the draft I am developing and the proposal you mentioned or alternatively publishing the draft together.
Kind Regards
Murat
Murat Sartas
International Institute of Tropical Agriculture (IITA)
Innovation Systems Scientist/Integrated Systems for Humidtropics
Wageningen University
Researcher/Knowledge, Technology and Innovation
[log in to unmask]<mailto:[log in to unmask]>
[log in to unmask]<mailto:[log in to unmask]>
http://www.sartas.org<http://t.signauxdix.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJW7t5XZs3LyCLnMfDkBWd0HJ8W1p7ZwM56dWfBf5SwWDl02?t=http%3a%2f%2fwww.sartas.org%2f&si=5644807320895488&pi=1fad5a984c8c43109a9981adec962a1e>
On Jun 11, 2016, at 6:32 PM, Jagosh, Justin <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Dear RAMESES forum members:
I am hoping to stimulate a discussion on the forum about people’s experience using realist methodology and suggestions for how advancements in the methodological field should be directed. This work will feed into an ESRC grant proposal I’ll be submitting later this year.
For the grant, I am seeking to assemble an interdisciplinary, international cohort of collaborators – people who are interested in methodological questions and would like to devote some of their time and headspace to advancing complexity-sensitive approaches to methodology. So if you would like to share your thoughts about methodology on the forum – this will be read with interest. If you are interested in dedicating your time more substantively to methodology – specifically to be affiliated with the grant in some capacity, please write to me personally. I’m not sure as of yet what the configuration of collaborators will be, and I can’t guarantee your involvement, but please do introduce yourself, your interests in terms of methodology, yours ideas, experience, expertise, and expectations. The more detail you provide the better but it doesn’t have to polished. Firsthand experience with methodological successes and failures will be read with interest. I will respond but it may take some time. If you are going to be in touch, please do so by the end of June- and send your email to me directly at [log in to unmask]<mailto:[log in to unmask]> as opposed to answering on the forum.
I would like the grant structure and design to develop through participatory dialogue with interested parties, and there may be various options for involvement in terms of co-investigator, institutional partner, advisory board member, Delphi panel member and so on. I am keen to involve junior researchers and Ph.D students in the grant in some capacity – fresh minds have a good chance at contributing to emerging complexity science and structuring their mental faculties for the demands of pragmatic complexity thinking.
We know that the uptake of realist methodologies is on the rise, in tandem with the development of other kinds of complexity-sensitive approaches. As realist evaluation becomes mainstreamed it is rubbing up against other paradigms and now we are seeing questions arise about whether the boundaries of realist evaluation should be kept clear and protected – so that we can distinguish between genuine (standard) and non-genuine (sub-standard) forms, or whether the logic underpinning realist methodology should be applied to inspire complexity-sensitivity across stages of research including open-system experiments that rely on counter-factual comparisons. It’s a worthy debate.
From my purview, I think that a trans-disciplinary forum on understanding causal logic in research is lacking and needed. Focussed study of causal logic is needed because causal assumptions are at the core of most research methodologies and/or policy, whether it is made explicit or not. Even when a research field or discipline does not engage with causation, the corresponding policy equivalent will. For example, a field such as descriptive epidemiology in which rates and states of morbidity are tracked, measured and published - no causal claims are made – yet causation still exists at a policy level in the subject area because groups of people will be set to the task of designing efforts to change status quo, supported by the findings from descriptive statistics. That change effort will necessarily involve causation – because you can’t design a programme without an implicit or explicit assumption that X will change Y, via Z. So causation is a fundamental element, whether it be it in the research process itself, or in the translation of research to practice.
Studying causation can help to address the differences (and differences of opinion) with regard to the acceptability of methodology for different areas of inquiry. It is likely that there are more than a few people on this forum who would say that RCTs are suitable for drug trials – in which effect size calculations are needed to inform drug policies, but not so suitable for complex social interventions in which social contingency is the linchpin. Currently the MRC guidelines for complex interventions suggests that whenever possible, a randomization is desirable. What is missing in my opinion is a fit-for-purpose analysis in which we collectively agree that certain causal logics are suitable for certain types of research questions and research areas. This is not to create blanket statements, but one cannot disregard difference in nature between, for example, an experiment to test the pharmacological efficacy of a drug, and an intervention to prevent schoolyard bullying.
And between these two hypothetical ends, there is a spectrum of interventions that have, one could argue, degrees of durability of their core objects. What is under-developed is the promotion and adoption of clearly established taxonomies for categorizing interventions that will enable a fit-for-purpose framework leading to the customization of methodologies to areas of inquiry. If people disagree with this I would be keen to hear opposing viewpoints.
Central to advancing the methodological field is dialogue, debates and disputation amongst a trans-disciplinary community of researchers and evaluators and this forum is a good starting point. We are facing incredibly challenging problems that do not simply require more research – multi-morbidity, chronic diseases, health inequalities, climate change adaptation etc. More suitable research, in which the foundations are realistic, pragmatic and complexity-sensitive – and in which the researchers and end-users develop their analytic capacities to deal with such complex problems – this is key I think. So with that I’m opening up to discussion –either online on this forum, or offline, via personal email correspondence. What would you propose for research to advance the methodological field?
Many thanks,
And looking forward to your responses.
Justin
Justin Jagosh, Ph.D
Senior Research Fellow
Director, Centre for Advancement in Realist Evaluation and Synthesis (CARES)
University of Liverpool, UK
www.liv.ac.uk/cares<http://www.liv.ac.uk/cares>
|