Colleagues,
Thanks to all who contributed to my call!
A selection of the responses, with some annotation is set out below:
Designing & Managing MCQ's
http://www.le.ac.uk/castle/resources/mcqman/mcqcont.html
This handbook is designed to familiarise UCT staff with the benefits and limitations of using Multiple Choice Questions (MCQs); explain how the Technology Assisted Testing (TAT) programme at UCT can help individual lecturers and departments; provide lecturers and departments with basic advice and guidelines about designing, administering and marking MCQs.
CAA Centre Blue Papers:
http://www.caacentre.ac.uk/resources/bluepapers/index.shtml
Bluepaper 1: Principles of Assessment by Mhairi McAlpine
This paper reviews the context and tensions within educational assessment and refreshes memory of the purposes of assessment.
Bluepaper 2: A Summary of Methods of Item Analysis by Mhairi McAlpine
Item analysis is a method of gauging the quality of an examination by looking at its constituent parts (items). The primary purpose of most examinations in higher education is that of a measurement tool, for assessing the achievements of the examination candidates, and thus how future learning can be supported and directed. This paper details the methods of item analysis for this purpose, and also considers how they might be used for the wider functions given above.
Bluepaper 3: Design Requirements of a Databank by Mhairi McAlpine
This Bluepaper gives an insight into some of the concerns and considerations that must be taken into account when designing an item banking system.
What are Objective Test Questions?
http://www.caacentre.ac.uk/resources/objective_tests/index.shtml
Objective tests require a user to choose or provide a response to a question whose correct answer is predetermined.
Such a question might require a student to select a solution from a set of choices (MCQ, true-false, matching); identify an object or position (graphical hotspot); or
Supply brief numeric or text responses (text input).
Because the correct answers to objective test questions are pre-determined, they are well suited to the many forms of CAA that involve automated marking. The electronic marking of the responses is completely non-subjective as no judgement has to be made on the correctness or otherwise of an answer at the time of marking. However, it is worth noting that in terms of in-built bias, an objective test is only as objective as the test's designer makes it. The tutorial below offers an introduction to a selection of question types in popular use with advice on construction and best practice. As the question type most commonly associated with CAA is multiple choice, particular emphasis has been given to this and should be viewed first. The principles detailed within that section should however, be of use when considering other types.
For a related Bibliography, please see:
http://www.caacentre.ac.uk/resources/bibliography/index.shtml
Constructing Written Test Questions For the Basic and Clinical Sciences" [NBME 2001]
This is a pretty comprehensive manual (115 A4 pages) "written to help faculty members improve the quality of the MCQs used in their examinations" - the manual is available from: http://www.nbme.org/PDF/2001iwg.pdf
Writing Objectives and relating to MCQs
http://web.uct.ac.za/projects/cbe/mcqman/mcqappc.html
This site introduces Bloom's Taxonomy and relates this classic (and possibly somewhat dated) approach to the design of MCQs. A good start for those who are unfamiliar with this approach or need updating.
Perceived drawbacks with using CAA
http://www.ltss.bris.ac.uk/caa_4.htm
A useful summary
Tripartite Interactive Assessment Delivery System [TRIADS]
http://www.derby.ac.uk/assess/newdemo/mainmenu.html
TRIADS is a highly flexible interactive assessment system capable of delivering a wide variety of question styles in a wide variety of modes to facilitate the testing of higher order learning skills. There are in-built facilities to support randomized selection and sequence in the delivery of questions, groups of questions and question content that make it suitable for assessments undertaken in an open environment. It is this potential for 'in-depth' and flexible assessment that separates TRIADS from the proprietary assessment systems currently available.
|