 |
 |
Everyone is welcome to attend David Draper's talk TOMMORROW: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Friday 12th May 2000, 2:15pm Room SM1.3, School of Mathematics, University of Bristol Statistical analysis of performance indicators in UK higher education David Draper (University of Bath) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Abstract: ~~~~~~~~~~ In recent years the Higher Education Funding Council for England (HEFCE) and the corresponding bodies in Scotland, Wales, and Northern Ireland have become increasingly interested in monitoring the quality with which universities carry out their public mandate. Last December HEFCE published results on a number of performance indicators for 1996-97, including drop-out rates after the first year of undergraduate study, comparing observed (O) and expected (E) outcomes for all 165 government-funded higher education institutions in the UK (this publicly available report makes interesting reading). Interpreting this effort in the language of causal inference, with a binary outcome such as dropout at the student level, (a) the process of students choosing and attending universities gives rise to an observational study (rather than a controlled experiment), meaning that measuring and controlling for potential confounding factors (PCFs) is crucial; (b) the supposedly causal factor at the university level is the underlying - and *unobserved* -- quality of the university; and (c) there are many student-level PCFs (including age, A-level qualifications, and subject of study). HEFCE's method of computing expected outcomes given the PCFs turns out to be a form of indirect standardization, which is almost equivalent to a version of hierarchical model-based inference in which the university effects are treated as fixed. A PhD student, Mark Gittoes, and I have shown that when the difference D = ( O - E ) is correctly calibrated the numbers of unusually under- and over-achieving universities in 1996-97, as far as dropout rates were concerned, were both surprisingly high. In this talk I will (1) describe a method for calibrating D; (2) relate this method to regression reformulations of the problem, both hierarchical and non-hierarchical; and (3) present league-table-style results for all UK universities and discuss their policy implications. Time permitting, I will also (4) compare fixed-effects and random-effects formulations of the model-based version of the HEFCE approach on their ability to correctly identify "good" and "bad" universities, and (5) explore the effects of unobserved student-level PCFs on the results. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For further details, see our Web site http://www.stats.bris.ac.uk/~guy/Avon/ or contact Nicky Welton ([log in to unmask], Tel: (0117) 965 6261 ext. 3227). Location maps for Bristol University can be obtained from http://www.bris.ac.uk/directions.html ---------------------------------------- Dr. Nicky Welton, Office 2P10, CSM, University of the West of England, Bristol. BS16 1QY. Email: [log in to unmask] Tel: (0117) 965 6261 ext 3227 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%