Just replying to say that I'd be really interested if anyone has done calculations like this, and has found a way to make them statistically responsible.
I teach a second-year class for which I cannot have a prerequisite, but where there is a first-year class that would have given people some background. I'm therefore naturally interested to see whether there is or isn't a big difference between the final second-year results of those who took the first-year class, and those who didn't.
I have done the calculations using a t-test before now, but it strikes me that there's actually another question to be resolved before we can really do tests like this: which test of difference to use? T-tests are only really suitable for continuous data that has a real zero; grade data is continuous, but I don't think it has a real zero value. Where I am, as is common in UK universities, we give grades as a percentage, but arguably it isn't a real percentage because it's vanishingly rare to get under 30% or over 80%. Most grades are concentrated between 50 and 70, and getting over 70 (a 'First') is what good students are aiming at. So the difference between 20 and 30 is tiny (both fails, one more abysmal than the other), and the difference between 75 and 80 is pretty small (both over 70, so who cares?); but the difference between 59 and 60 is huge, and 69 to 70 can change your life if repeated. The 9-0 boundaries are class boundaries, so a degree with an average score of 60 is substantially and substantively better than one with an average of 59, and someone with a degree average of 70 is much more likely to get hired than someone with a 69 average.
So, given all that, how do we do these tests anyway? Some sort of transformation to make the line straight, and then inferential stats? Anyway, doing it statistically responsibly is much more complicated than it appears, even if you could do a quick and dirty t-test for a first eyeball of the data. All suggestions and discussion welcome!
(Probably killed it now.)
Damien
--
Damien Hall
Newcastle University (UK)
|