Thanks Dr. Conroy:
Also, Ms. Benson:
With all due respect, not every study is statistically flawed and I hope
you are not suggesting that. All studies have limitations, some of which are
statistical and some of which are not.
Some studies attempt to address findings that were never a part of the
original intent or the original design. Even statistics won't save the day
there.
Power deserves discussion in all outcomes: Positive, negative,
non-significant.
You note that your students know how to check p-values and confidence
intervals but do they understand that these two measures are based upon
different assumptions about the underlying data - - frequencies vs
probabilities?
Interpreting the medical literature is not easy. Analysis of assumptions,
appropriateness of design to the intended question, application of the
protocol, follow-up loss, identification of appropriate outcome measures,
all have a role. Focusing only on the statistical aspects does not IMHO do
service to students.
Bill
>>> Ronan Conroy <[log in to unmask]> 10/14/2008 2:40 PM >>>
On 14 Oct 2008, at 18:44, Benson, Teresa wrote:
> From those of you who review/appraise articles regularly, I would
> like to hear what kinds or errors you find most often in the
> statistical methodology. For example, do you find researchers:
> -using parametric tests with skewed data distributions,
Skewed distributions can be analysed using parametric tests.
Generalised linear models, for example, allow this, as do negative
binomial regression and quantile regression. There are plenty of
parametric approaches to such data. They are usually more powerful
than rank-based procedures, and allow the calculation of useful
measures of effect size. Even tests that we think of as nonparametric
calculate parameters, of which the most useful is the Wilcoxon rank
sum test. The parameter behind the test is directly equivalent to the
area under the ROC curve.
>
> -using one-tailed tests when they should have done two-tailed,
I have seen virtually no one-tailed tests ever.
>
> -using multiple t-tests instead of an ANOVA, or
ANOVA does not test any useful hypothesis. You will find that most
people in the area of EBH favour regression models where each term has
an interpretation. We don't want to know that 'there is a
relationship' - we want to know what that relationship is. Shotgun t-
tests have almost died out, at least in my corner of the woods.
>
> -assigning effects as fixed when they should be random effects; and/or
I cannot think of a single erroneous decision in medical practice that
has been based on such a confusion.
Statistical errors are usually not of this sort. They are usually
undetectable in publication, and consist in the selection of which
data to present, or which models to build. Type III errors, in a word.
Ronan Conroy
=================================
[log in to unmask]
Royal College of Surgeons in Ireland
Epidemiology Department,
Beaux Lane House, Dublin 2, Ireland
+353 (0)1 402 2431
+353 (0)87 799 97 95
+353 (0)1 402 2764 (Fax - remember them?)
http://www.flickr.com/photos/ronanconroy/sets/72157601895416740/
P Before printing, think about the environment
|