I am suggesting that for summative assessments it is best
to have the variety of different MCQs stored as separate
questions, since the cost of storage is so low now and then
you can collect data, as we do, on the difficulty of each
question without having to rely on a panel of wise men -
who frankly wont know.
On Thu, 22 Mar 2001 10:20:53 +0000 David Davies
<[log in to unmask]> wrote:
> Hi Jon
> >Yes you can generate all these marvellous number of questions from a
> >set of possible answers. The problem will be that all the generated questions
> >will have different difficulties.
> Such as?
> > but if you want to use the results for formal
> > assessment then getting questions of random difficulty is a
> > problem.
> Not random difficulty. The difficulty should be consistent otherwise you're
> disadvantaging students. Perhaps I misunderstood and you mean random
> allocation of consistent difficulty questions?
> If so, then it's more difficult, sure, but lots of courses do it.
> Maths is a classic (and easy) example.
> I'm not suggesting I can tell anyone how to set up an MCQ group, far from
> it, but a model that seems to work is you get 2-3 lead teachers in a module
> to agree a set of equivalent difficulty stem/branch interactions. You let
> the computer worry about allocating and delivering delivering them. If
> you're really worried about quality, which in summative assessments we
> should be, then your external examiner ratifies that your question bank is
> of consistent difficulty. Not forgetting of course that it's equally valid
> to have mixed difficulty questions in a summative exam. This is perhaps
> desirable if you want to differentiate between the different levels or
> competency of students. This is the notion of the 'golden' question(s).
> I think however that the issue in this thread is that if the computer
> assigns random branches to a stem to a) generate a large number of
> stem/branch questions and b) to give different students different question
> on the same topic, then your local QA measures must ensure that the
> alternate branches are of equivalent difficulty.
Jon Sims Williams
Dept. Engineering Maths,
University of Bristol
Bristol BS8 1TR
Email: [log in to unmask]
Tel; 0117-928 7757, Fax: 0117-925 1154