Hello everybody,
two questions concerning significance tests:
1. Does there exist a non-parametric significance test which tests the null
hypothesis that two samples stem from the same distribution, not considering
only single features such as the mean value (like the Wilcoxon-test), but
*synchronously* mean value, width of the distribution, higher moments for
non-gaussian-shaped distributions, and the number of maxima for multimodal
distributions?
2. A very basic question: Does it make sense to use different significance
tests for different features, or does one have to use a test which considers
all features of the distribution which are of interest? Example:
There be three samples A, B, and C. One like to know if two samples differ
with respect to the mean value and/or with respect to the width of the
distribution. One perform the (mean-value-sensitive) Wilcoxon-test as well
as the (width-sensitive) F-test to the sample pairs A-B, A-C, and B-C. The
Wilcoxon-test yield a high significance for the pairs A-B and A-C, but a low
one for the pair B-C (e.g. values 0.01, 0.01, and 0.7), and the F-test yield
a low significance for A-B, but high significances for the pairs A-C and B-C
(e.g. values 0.5, 0.0, 0.0).
Which one of the following two statements is right:
I: A and B differ significantly in mean value, but not in width, and B and C
differ significantly in width, but not in mean value.
II: It is suspicious and unscientific to argue that way, since you pick the
test which fits your hypothesis, applying a bias on your results and
enforcing significance where there is none in reality. To free your results
from any bias you have to perform a test which synchronously considers all
features of the distribution you are interested in.
Kind regards,
Volker Knecht
|