Email discussion lists for the UK Education and Research communities

## WEB-ASSISTED-ASSESSMENT@JISCMAIL.AC.UK

#### View:

 Message: [ First | Previous | Next | Last ] By Topic: [ First | Previous | Next | Last ] By Author: [ First | Previous | Next | Last ] Font: Proportional Font

#### Options

Subject:

Re: facility and discrimination

From:

This list, hosted by the CASTLE project, is for those interested in conduct" <[log in to unmask]>

Date:

Fri, 23 Mar 2001 13:58:11 +0000

Content-Type:

text/plain

Parts/Attachments:

 text/plain (44 lines)
 ```I'm not sure exactly when AnoVa was invented but its use has only become common (I mean in general not just for MCQs) in recent decades so I'd probably go along with the spirit of the opinion given below but point out the AnoVa is generally the accepted way to compare two or more means with each other. As mentioned before we have to be careful about terminology. The traditional maths that goes with the term "discrimination index" at least in the UK is to place the students in two groups according to performance on the test, ignoring the mediocre third of the students as a first step. Then for each question you simply calculate the mean score of the "top" and "bottom" students on a scale 0.0 to 1.0 and find the difference. There are a number of problems with this. It treats performance on the test as a whole as a logical selection into two groups when clearly it is a score on a sliding scale with a roughly normal distribution. It makes the opposite mistake on the score for the question when it is a true MCQ. It also gives no guidance on how significant the result is. A "craft" advice is often given that if the number of students falls below "n" you should bump up the numbers by not discarding the middle third. There are no real statistical justifications for the validity of any of this and my experience is that you have the double problem with academics that either a) they don't understand the purpose or value of the analysis or b) they understand the statistics well enough to argue that it shouldn't be done. The upshot is they get away with never examining the performance of the questions. If you do the stats right you still need to convince half your audience but at least it can't be shot down on the basis of unsound methodology. Apart from any of that the statistical analysis is a complete waste of time if noone intends to take action as a result and for that to happen they have to understand how to interpret the result. A common problem is that a simplistic explanation can become current - e.g. if you get a negative discrimination index on your question you are a cr*p teacher and if all your questions have a positive discrimination index you are a brilliant teacher. The document I referenced before attempts to deal with this issue by presenting various scenarios. Jon Maber ```