Over a year a ago I raised the issue on ALLSTAT of Guidelines for reporting
an observational study. The volume of response was disappointing.(The
previous summary is appended below). Some while ago I came across those
produced by the American Association for Public Opinion Research
http://www.aapor.org

Specifically at
http://www.aapor.org/default.asp?page=survey_methods/standards_and_best_prac
tices/best_practices_for_survey_and_public_opinion_research

I send this as it maybe helpful to some and may help to improve practice.
I feel the issue is an important one.
It is a pity that there does not seem to be such a set of guidelines
available which is based this side of the Atlantic.

Paul Marchant

Centre for Research and Graduate Studies
Leeds Metropolitan University
Calverley St., Leeds.
LS1 3HE

Tel 0113 2833074
Fax 0113 2836764





> -----Original Message-----
> From: Marchant, Paul           [RDO]
> Sent: 20 November 2001 18:15
> To:   [log in to unmask]
> Subject:      SUMMARY Guidelines for reporting an observational study
> (e.g. a Su rvey)
>
>
> Some time ago I sent a message which asked if there was a set of
> authoritative and useful guidelines for observational studies as there is
> for RCTs, in the form of the CONSORT statement. There was a flurry of
> activity on the value of and possible improvement to the CONSORT statement
> generated by Allan Reese. However after 3 weeks my total substantive reply
> is one. That was from Robert Newcombe and is appended and contains my
> original posting.   Thanks Robert! It is a good point you make that we
> ought not to put people off RCTs by making it more onerous to write up a
> RCT than to do another scientifically weaker approach, i.e. observational
> study. It does indeed seem to me to be another very good reason for having
> quite tight guidelines for observational studies and expecting them to be
> adhered to.
>
> For RCTs which have intrinsic scientific strengths, power calculations are
> routinely done , protocols are produced... the consort guidelines exists
> which encourage transparency. Whereas, with surveys it can be the case
> that much is not specified in advance and much is not specified in the
> write up, e.g. response rates, adjustments made and what happens with
> sensitivity analysis ... if any was done. There is therefore a great lack
> of transparency, often with such work.  ( I know there are survey
> specialists who operate to high standards but many are not survey
> specialists.)
>
> I would have thought that somewhere there ought to be something
> authoritative written which says what we expect to be able to see, so that
> all may judge.  As it is, anyone can knock out a questionnaire, get a
> minimal response and report the results as the pure truth about the
> 'World'. It seems often the case that statistically significant results
> are reported that could well have occurred by chance, because of the many
> outcomes that were examined.
>
> I feel that some published standards ought to be expected of such work, as
> it is a very tricky area any way  (i.e. observational study, problems of
> meaning and measure.... non-response.....)
>
> Something for some group to develop and promote perhaps?? It might do
> something to help the reputation of statistics.
>
>
>
> Paul Marchant
>
> Centre for Research and Graduate Studies
> Leeds Metropolitan University
> Calverley Street
> Leeds
> LS1 3HE
> England, UK
>
> [log in to unmask]
>
> Tel:  +44  (0) 113 2833074
> Fax:  +44  (0) 113 2836764
>
>
> --------------------------------------------------------------------------
> -----------------------------------
> From: Robert Newcombe [SMTP:[log in to unmask]]
> On 29 Oct 2001, at 15:44, Marchant, Paul [RDO] wrote:
>
> "The Consort statement www.consort-statement.org  is extremely useful
> in
> guiding anyone on  how to  write  up a randomised control trial.
> Indeed
> even how to think about an RCT. The check-list is extremely valuable.
>
> I wonder if there is anything similar for reporting a non-randomised
> study.. not just in health but for say a social survey. On the
> Consort
> web-site there are links to Quorum and Moose, meta-analysis of RCTs
> and
> Observational studies respectively. However what I would like is
> something
> comprehensive and authoritative, along the lines of Consort, for
> reporting
> a single observational study.
>
> Can anyone help?"
>
> No, but it's all a very interesting and important issue.  You
> perceptively notice that the CONSORT guidelines have the hidden
> agenda of guiding people on how to think about an RCT.
>
> I've been reponsible for persuading/catalysing two clinical journals
> (Gut and J Orthodontics) to introduce the CONSORT guidelines, and Int
> J Endodontics, with which I am now associated, is likely to move in
> the same direction soon. In each case, I have strongly argued 3
> points:
>
> These guidelines should influence how research is conducted as well
> as reported.
> They only specifically relate to RCTs, but very much the same
> principles should apply to other research.
> We should avoid introducing the perverse objective of making it
> harder to perform and publish an RCT (the most informative type of
> study re therapeutic potential) than other types of studies.
>
> There are quite a lot of less specific guidelines about, both
> relating to points that authors should heed in preparaing a paper for
> publication, and also checklists for referees to use in assessing
> manuscripts - among others, the BMJ has done quite a lot in this
> area.  But I'd be interested to see what else you come up with.
>
> Hope this helps.  Best wishes.
>
> Robert Newcombe.
>
>
> ..........................................
> Robert G. Newcombe, PhD, CStat, Hon MFPHM
> Senior Lecturer in Medical Statistics
> University of Wales College of Medicine
> Heath Park
> Cardiff CF14 4XN, UK.
> Phone 029 2074 2329 or 2311
> Fax 029 2074 3664
> Email [log in to unmask]