Not wishing to appear pedantic, but the probability of obtaining a mark of 4
or more by randomly choosing  from 10 multiple choice questions with one
correct answer and three distractors is approximately 0.2.  This would mean
that from 50 students guessing approximately 10 would achieve a pass mark of
40%.  The reason for this is, of course, too few questions. (reduce the
number of questions to two, with a pass mark of 50%, and 22 out of 50
students guessing would pass)
My concern in using any correction formulae is for the borderline student
who knows just enough to pass the examination, but by incorrectly guessing
other questions has a pass mark reduced to fail.

alan heslington

 ----------
From: Farthing D W (Comp)
To: [log in to unmask]
Subject: RE: Negative marks
Date: 17 February 1999 17:07

>Generally, we encourage our staff to apply negative marking in such a
>way:
>
>(here follows a little arithmetic)
>let n = total number of choices (including the correct answer)
>offered to the student.
>Marks for the correct answer = n-1
>Marks deducted for each incorrect answer = 1
<snip>
>e.g.   if I have a 5-choice question, I get
>4 for the correct answer
>-1 for the incorrect answer
>
>For someone who knows nothing, the balance is 4 vs -4.

Please bear in mind that guessing compensation techniques (negative marks)
work only for the average case. In fact some candidates will be lucky and
guess more right. The key questions are how many, and by how much?

Suppose EVERYONE sitting a test with ten multiple-choice questions guessed
the answers at random. The average mark should be about zero after guessing
compensation is applied. However, one in 50 of them might get a mark of 40%
or more; in most UK institutions this is enough to pass. (The calculation is
derived from the statistical distribution of random guesses, assuming a
4-choice question.)

The likelihood of gaining 40% through guessing reduces if the test has more
questions. 5-choice questions are also less susceptible to random guessing,
but studies have shown that in practice more choices don't help because it's
hard to devise convincing distracters.

This brings me to a greater problem: poor distracters. Candidates can
increase their chance of guessing correctly by eliminating the obvious
distracters. I always try to use distracters that are true statements, but
true about something else. For example, if I asked "What is Mailbase?", I'd
have distracters describing of other Internet technologies, not make up
something about manned stations on the moon etc.

So the message is, use guessing compensation by all means, but don't rely on
it to combat random guessing.

Dave W Farthing
School of Computing
University of Glamorgan, UK
[log in to unmask]
"To be sure of hitting the target, shoot first, and call whatever you hit
the target."


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%