Rosie McEachan wrote:
> Dear all,
>
> I was wondering if anyone had some advice on the following statistical
> matters:
>
> I have a dataset containing 319 variables. Without going into too much
> detail, the variables are a combination of 29 health behaviours being
> rated on 11 characteristics (e.g.so data set is in following format:
> Behaviour1characteristic1, behaviour1characteristic2). To make things
> just a bit more complicated, I should point out that respondents only
> responded to 110 variables each. This was because respondents were only
> asked to answer questions on 10 randomly selected behaviours (out of the
> possible 29). This means that there are an awful lot of missing values!
> Scores were on 1-7 likert type scale
>
>
>
> Basically, I want to demonstrate that there are no variations in
> responses to these variables according to gender, and social class (this
> is in a very simple dichotomised form).
You can't do this, because statistical tests don't answer that question.
You can say that you have not got evidence that they differ by gender
or social class.
A problem you are going to have is that if you test 319 variables, and
there were no population differences, you are going to get p value less
than 0.05 16 times, and less than 0.01 3 times (on average).
So, you could Bonferroni correct. But the problem with Bonferroni
correction is that it reduces your power, so you have much less chance
of finding differences. You can say "Aha! As I said, I found no
differences." And someone else can say "But you had your eyes shut, so
you wouldn't have."
> Can anyone suggest how I might
> do this? Parametric statistics are not appropriate because the variables
> are not normally distributed.
Parametric statistics do not rely on a normal distribution in the
sample, they rely on a normal sampling distribution, which is dependent
on the distribution in the sample AND the sample size. With larger
samples, you are almost guaranteed a normal sampling distribution,
whatever your data look like.
> I also am a wary of carrying out
> parametric or non-parametric tests (e.g. mann whitney u) because it
> would mean there would be a huge amount of comparisons (increasingly
> type 1 error etc). Am I going to just have to eyeball the descriptive
> data / histograms to judge their similarity and if so, how could I
> justify this approach as being scientific?
>
>
You could look at effect sizes.
With your missing data, you are going to have to do something
appropriate. Have a look at a paper in Psych Methods, from about 2002
called "Missing data: our view of the state of the art" by Schafer and
Graham. You have a massive advantage that you know your data are
missing completely at random. There was also a paper by Brendan Bunting
in Structural Equation Modelling, which examined ways of dealing with
data which have a planned missingness structure. If you can't get hold
of that, email Brendan, he's usually nice - [log in to unmask]
>
> Secondly, I also want to demonstrate that there is minimal within group
> variation. In other words, that everybody answered a particular
> question with a similar response. At present I have just been looking at
> standard deviations (hoping that they would be small) and other
> descriptive data. Can anyone think of a more formal way of doing this?
> And if not, how could you justify what is a ‘small’ and what is a
> ‘large’ standard deviation in relation to a seven point scale.
>
If I were you I would work out the standard deviation for different sets
of probabilities - e.g. if all response are equally likely, what's the
SD. If half the people say "4" and the others form a symmetrical
normal(ish) distribution, what's the standard deviation, and then
compare your results with those.
>
>
> If anyone has any help or advice on either of these two issues I would
> be very grateful!
>
>
>
You might want to try emailing psych-methods, another jiscmail list.
Jeremy
--
Jeremy Miles
mailto:[log in to unmask] http://www-users.york.ac.uk/~jnvm1/
Dept of Health Sciences (Area 4), University of York, York, YO10 5DD
Phone: 01904 321375 Mobile: 07941 228018 Fax 01904 321320
"No fair, you changed the outcome by measuring it!"
- Professor Hubert J. Farnsworth
|