Everyone is welcome to attend David Draper's talk TOMMORROW:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Friday 12th May 2000, 2:15pm
Room SM1.3, School of Mathematics, University of Bristol
Statistical analysis of performance indicators in UK higher
education
David Draper (University of Bath)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Abstract:
~~~~~~~~~~
In recent years the Higher Education Funding Council for
England (HEFCE) and the corresponding bodies in Scotland,
Wales, and Northern Ireland have become increasingly
interested in monitoring the quality with which universities
carry out their public mandate. Last December HEFCE
published results on a number of performance indicators for
1996-97, including drop-out rates after the first year of
undergraduate study, comparing observed (O) and expected (E)
outcomes for all 165 government-funded higher education
institutions in the UK (this publicly available report makes
interesting reading).
Interpreting this effort in the language of causal
inference, with a binary outcome such as dropout at the
student level, (a) the process of students choosing and
attending universities gives rise to an observational study
(rather than a controlled experiment), meaning that
measuring and controlling for potential confounding factors
(PCFs) is crucial; (b) the supposedly causal factor at the
university level is the underlying - and *unobserved* --
quality of the university; and (c) there are many
student-level PCFs (including age, A-level qualifications,
and subject of study). HEFCE's method of computing expected
outcomes given the PCFs turns out to be a form of indirect
standardization, which is almost equivalent to a version of
hierarchical model-based inference in which the university
effects are treated as fixed.
A PhD student, Mark Gittoes, and I have shown that when the
difference D = ( O - E ) is correctly calibrated the
numbers of unusually under- and over-achieving universities
in 1996-97, as far as dropout rates were concerned, were
both surprisingly high. In this talk I will (1) describe a
method for calibrating D; (2) relate this method to
regression reformulations of the problem, both hierarchical
and non-hierarchical; and (3) present league-table-style
results for all UK universities and discuss their policy
implications. Time permitting, I will also (4) compare
fixed-effects and random-effects formulations of the
model-based version of the HEFCE approach on their ability
to correctly identify "good" and "bad" universities, and
(5) explore the effects of unobserved student-level PCFs on
the results.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For further details, see our Web site
http://www.stats.bris.ac.uk/~guy/Avon/
or contact Nicky Welton ([log in to unmask], Tel:
(0117) 965 6261 ext. 3227).
Location maps for Bristol University can be
obtained from http://www.bris.ac.uk/directions.html
----------------------------------------
Dr. Nicky Welton,
Office 2P10, CSM,
University of the West of England,
Bristol. BS16 1QY.
Email: [log in to unmask]
Tel: (0117) 965 6261 ext 3227
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
