Summary of my query results, I have not included names, thanks to all who
replied:-
I don't think you have a problem here. What you are proposing to do is not a
multiplicity of tests, but a check that the underlying assumptions of the
t-test are true for your dataset. Provided you do it before you actually do
the analysis, then you are not doing more than one test on the same dataset,
and therefore multiplicity is not an issue. Only if you did BOTH the t-test
AND the Wilcoxon test would there be questions raised. It may be worth
bearing in mind that, if you have more than 50 subjects per group, the
normality assumptions will hold in he absence of major distributional
problems.
**
Normality is overstated and mis-understood. Let me start by saying that the
paired t-test is a special case of ANOVA. Providing your tests have been
randomised the tests for ANOVA are valid under the weaker assumption that
the residuals (y minus Anova mean) have constant variance. A plot of the
residuals against means will tell you whether the the magnitude of the
residuals is related to the mean and whether there are any obvious outliers.
The second point is that statistical analyses have deterministic and
stochastic components. Prior to analysis it is unlikely the data will have a
normal distribution. Post analysis the RESIDUALS are likely to have a
normal-ish distribution. If the residuals have a highly skewed distribution
or exhibit other systematic patterns you may need to respecify the
deterministic component of the analysis e.g. analyse log response. Overall
tests for normality are overated and oversensitive. In short look at your
data and residuals but don't trust a test of normality. Finally only use the
Wilcoxon test if you have a small number of responses and the residuals are
skewed or your client insists on a nonparametric test..
**
I would say the approach that you are describing is appropriate. However, I
would go about it slightly differently. In an analysis plan/protocol,
written prior to performing the analysis (and prior to breaking the
treatment blind) I would state that the primary analysis is the paired
t-test. I would then also say that assumptions underlying the use of this
test (normality of differences) would be investigated, and if the
assumptions seem not to hold then in addition to performing the parametric
analysis the non-parametric Wilcoxon signed rank test would also be
performed. What you then call the primary analysis is a 'sticky' issue, but
I would be pushed in the direction of stating the non-parametric be deemed
the primary analysis if analysis assumptions seem to fail. Therefore, you
are only ever stating one analysis to be the primary analysis (which cannot
be influenced by your own preference), and hence multiplicity issues do not
come into play. I would also suggest, that even if analysis assumptions do
not hold, the regulatory authorities would probably still want to see the
parametric analysis as well as the non-parametric.
**
Excellent point. By testing for normality, we do not prove it, merely fail
to
find a
contradiction with it. I think the underlying philosophy is that if you are
not
sure
whether the normality assumptions are appropriate you should not be in the
business of
clinical trials. Assess the nature of the variable from past trials in
which it
was
used. If it was normal then, it is not going to be categorical today, and
probably
neither highly skewed. In any case, the paired t-test is quite robust with
respect to
departures from normality, so testing for normality before t-testing does
have a
touch
of paranoia to it. The properties of the t-test for non-normal variables
(e.g.,
with
five ordinal categories) can be established (or insight gained) by
simulations.
It does
not take long to do them (in SAS), and the learning is worth it.
`Who cares whether the analysis is correct? We only want it to look good.
We
only want
the client/regulator to see it as faultless.'
Guidance is not an order ...
The multiplicity point is valid, but not as screamingly as in model
selection
(e.g., when
testing for carryover).
>From: katherine tilson <[log in to unmask]>
>Reply-To: katherine tilson <[log in to unmask]>
>To: [log in to unmask]
>Subject: QUERY: Testing for normality and CPMP guidance
>Date: Fri, 14 Sep 2001 08:47:56 +0000
>
>Dear Allstat,
>
>Is it appropriate to test for normality before the analysis of a primary
>variable? I am performing a paired t-test on my primary variable and was
>first going to see if it was normally distributed, if not I was going to
>perform the Wilcoxon test. I am concerned that the new EMEA CPMP guidance
>on
>multiplicity seems to dissuade from this pre-testing process.
>
>Thanks
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
|