Carole,

The TAL system does exactly what you suggest. You can
collect all the student responses in a database and then
analyse them to find how difficult each question is.

The problem that one has is that if you change the order of
the responses you change the difficulty of the question. If
you are dealing with large numbers of students then the
average facilty you get will be across the set of
randomised orders of responses. It is arguable that if you
set a reasonable number of questions to a student and each
of the questions has a randomised order of responses then
they are likely to balance out.

Some questions are more liable to the effect of changed
order of response. Particularly those that you can go
through the answers in order until you find an answer that
is correct!

The current discussion is about generating 'different'
questions by permuting the set of possible answers that
each student gets. I dont really regard these questions as
different but their difficulty will certainly vary.

Jon Sims Williams
On Thu, 22 Mar 2001 11:24:18 -0000 Carole McLennay
<[log in to unmask]> wrote:

> Please excuse me for interjecting with regard to Jon's comment on different
> difficulties.  I am ignorant of XML but have found the exchange of ideas
> stimulating because I am interested in the subject, particularly vis a vis
> the data tracking of student individual responses so that these responses
> can be imported into a spreadsheet for analysis of, for example, facility
> value (whether the questions are too easy/difficult) and discrimination
> index (a measure which sorts out the 'good' students from the 'poor'
> students.
>
> Am I correct in assuming that these indivdual responses could all be
> transferred into a database on the host server? If so, surely the quiz
> question could include a code (for level of difficulty) which could be
> transferred into the database (for subsequent amendment/updating)?
>
> Thanks for this exchange of ideas.
>
> Carole
> ----- Original Message -----
> From: jon sims williams <[log in to unmask]>
> To: <[log in to unmask]>
> Sent: Thursday, March 22, 2001 2:55 PM
> Subject: Re: XML, resource discovery and MCQ sharing
>
>
> > Dear Martin and David,
> >
> > Thank you for clarifying the issue for me. Yes you can
> > generate all these marvellous number of questions from a
> > set of possible answers. The problem will be that all the
> > generated questions will have different difficulties. This
> > does not matter if you are just providing students with
> > practice but if you want to use the results for formal
> > assessment then getting questions of random difficulty is a
> > problem.
> >
> > Jon Sims Williams

----------------------
Jon Sims Williams
Dept. Engineering Maths,
University of Bristol
Bristol BS8 1TR
Email: [log in to unmask]
Tel; 0117-928 7757, Fax: 0117-925 1154