Here's the address of the documentation for staff here at Leeds on
using MCQ papers in our VLE.
http://www.leeds.ac.uk/sociology/nbmanual/nbb4v2.pdf
Section 9 deals with the statistical
analysis of class results. Correlation is used on questions that give
a sliding scale of performance, e.g. multiple response questions and
AnoVa is used where it is a right/wrong score.
For the former marks on a questions are correlated against marks on
the exam as a whole or against some other performance indicator
provided to the software. The correlation coefficient can be
interpreted in a similar way as a discrimination index - it is a number
between -1 and +1. A value of +1 indicates a perfect linear
relationship between performance on the question and performance on the
"reliable" measure of performance, -1 also indicates a perfect linear
relationship but in reverse. A score of 0 indicates that there is no
relationship between performance on the question and performance on the
test as a whole. The correlation is accompanied by a probability that
the coefficient deviates from 0 due only to random variation in the
data.
The statistic from AnoVa is the difference in performance for two groups
of students - those who got this question wrong and those who got it
right. This is expressed as the gap in the student's mean mark on some
reliable performance indicator. Like a discrimination index a positive
value shows that the students who got this question right were the
generally more able students and a negative value shows that they were
the generally less able students.
If you want to program these stats into your tools the maths can be
found
in any reasonable stats text book. If you are willing to do it "by
hand"
you can use the analysis toolpack standard add-in for MS Excel.
Jon
Carole McLennay wrote:
>
> Many thanks to all who kindly provide such helpful advice. I can see how
> useful the significance level would be and I, too, would be grateful for
> details of both the AnoVa and the correlation usage
>
> Carole.
> >
> > My personal preference is to use AnoVa or correlation (depending on the
> > scoring scheme for the question) to analyse class performance. A
> > correlation coefficient has the same usefulness as for example a
> > disrimination index but it is accompanied by a significance level so
> > you know when to take it with a pinch of salt.
> >
> > I can give some details of this if people think it will be of interest
> > and I would welcome criticism.
> >
> > Jon Maber
> >
> >
|