Dear all 

Many thanks for pitching in. As ever, some interesting stuff.

If you remember, there were two aspects to my question. One about
reviews containing only one study, and another about reviews within
which there may be a number of included studies but  which include one
or indeed many individual comparisons using one study. Most of you
addressed the first but we also have a few comments on the second from
Cochrane groups. 

- Cause of problem

Single-study reviews can come about through overexclusion of relevant
but possibly poorer quality studies.

Multiple single-study comparisons within one review can come about by
allowing too broad a focus to the review initially.


- Pros

1 At least the lack of evidence is presented in a clear and unequivocal
way that may inform some of the less discerning consumers of health
information

2 You are more confident that it IS the best evidence of  that question
- otherwise you wouldn't know if the single study represents the best
available evidence vs. what happens to be recognized by someone without
awareness of other studies. Even empty reviews save people the hassle of
replicating lit searches that have been conducted by others.  


3 The review will document a useful and effective search strategy for
future researchers. They will have a clear idea of the outcomes of
greatest interest/importance and also have a clear guide for the
research methods most likely to be useful and improve the design e.g.
increase follow up, suitably powered/sample size etc, sub group analysis
powered.
 
Can provide a good visual summary and one Cochrane group allows
single-study comparisons provided no summary measure is displayed.

- Cons

1 MAY cause some to think there is stronger evidence since is  a SR
2 May NOT encourage the next SR
3 Limits any firm conclusions.

- What does it tell you?

1 the paucity of research in the area and thus the opportunities...makes
you dig deeper into why studies are lacking...what is the problem? 
 
2 the paucity of controlled studies in the area
 
3 There may be heterogeneity in comparison studies/study groups that
limit pooling for a summary estimate measure (studies too different)

- Other comments

Depends on the type of study you refer to as ' one study'

What is more important: (1) the number of included trials, (2) the total
sample size of the included trials or (3) the quality of the included
trials.

Consequences for a single-study/low no of study review - may be
publication bias as less likely to be published

Consequences of too many single-study comparisons: takes a long time to
finish the review!


-  Suggestions for managing the problem  in reviews

?maybe present a further analysis including poorer quality RCTs or
non-RCT studies. 

Arguably, large reviews may be more informative to particular areas of
practice, but I think smaller and empty reviews perform important
functions in Cochrane overall.

? how should we handle excluded studies?

Can use RevMan to determine the effect size of each outcome of each
comparison - have to enter all the data - but that does not mean that it
all has to appear in the final review

Judge everything on merits - no fixed policy

Try to advise authors to group comparisons into broader headings rather
than atomise everything into individual comparisons - choosing primary
outcomes for comparison which make clinical sense and just summarise the
secondary outcomes to keep review manageable. 

-  Lastly...

One respondent asked how many Cochrane reviews are 'blank'? I don't have
data exactly on this but a few years ago we analysed a random selection
of  110 reviews looking to see how many offered an 'inconclusive' bottom
line. We made it about 41, of which 10 were due to lack of evidence
rather than poor quality. So my guess would be 10% but of course if
anyone knows different.............


All the best from Lancaster

Andrew Smith

Andrew Smith
Consultant Anaesthetist 
Director, Lancaster Patient Safety Research Unit 
Royal Lancaster Infirmary
Ashton Road
Lancaster
UK
 
tel +44 1524 583517
fax +44 1524 583519
email   [log in to unmask]
http://www.lpsru.org.uk/ 
 
DISCLAIMER: This e-mail is confidential and privileged. If you are not the intended recipient please accept our apologies; please do not disclose, copy or distribute information in this e-mail or take any action in reliance on its contents: to do so is strictly prohibited and may be unlawful. Please inform [log in to unmask] that this message has gone astray before deleting it.  Comments or opinions expressed in this email are those of their respective contributors only. The views expressed do not represent the views of the Trust, its management or employees.
University Hospitals of Morecambe Bay NHS Trust is not responsible and disclaims any and all liability for the content of comments written within. Thank you for your co-operation