Dear James,
The issue of generalisability (also called applicability, external
validity, and several other things) is a difficult and thorny one, with
little in the way of empirical evidence to help us.
There is a Cochrane Methods Group on Applicability and Recommendations
who's main interest has been this question. A extensive review of the
literature done a couple of years ago (initiate by David Henry and Di
O'Connell in Newcastle, Australia) revealed surprisingly little work on
this crucial issue. I have attached a Word document with a two page summary
that the Cochrane Methods group uses at Colloquia workshops on the issue,
and is based on this review. It recommends 5 steps in the process, and
gives a brief relevant bibliography. In part, the structure of the Clinical
Evidence summaries is based on this work (e.g., the need for quantifiable
prognostic risk such as Rod Jackson's CDV-risk chart, and separate sections
on Benefits and Harms).
Let me know if you need more details*
Best wishes,
Paul Glasziou (for the Cochrane Methods Group on Applicability and
Recommendations)
* Note: I am way for the next 7 days
At 01:17 AM 14/12/99 , James Woodcock wrote:
>A question that we have come across and cannot find a satisfactory answer
>to is about generalising evidence. All evidence requires some level of
>generalisation to apply to anyone outside the original trial population.
>Inclusion/ exclusion criteria by themselves are not satisfactory; because
>a positive effect in a trial does not tell us who in the trial benefited
>or what differences are important. External validity depends to a large
>part on local conditions:see Dave Sacket's contribution to this list on
>levels of evidence. But what is wanted is to advise those in the field how
>to take their local conditions in to account or the whole evidence base
>goes through so many hoops that the end practice is evidence descended
>rather than based. Extrapolation based on risk factors can be useful but
>when is it likely to be valid? Has it been tested? Is there any way of
>expressing our level of confidence when applying the evidence as
>confidence intervals from the trials are clearly over optimistic.
>James Woodcock
>epidemiological editor Clinical Evidence BMJ Publishing group
>
>______________________________________________________
>Get Your Private, Free Email at http://www.hotmail.com
Paul Glasziou
Harvard Centre for Risk Analysis
718 Huntington Avenue, E-221
Boston, MA 02115
Fax: 1-617-432-0190 ph: 1-617-4320095
|