Dear Kal

In response to your question " is there any empirical evidence that anonymous marking actually changes the discrepancy in marks awarded to different groups?"
I found the following from Edinburgh University Students' Association (EUSA) https://www.eusa.ed.ac.uk/eusapolicy/academic/anonymousmarking/ - NB I've not checked this and references are not given - but it could be useful to follow up:

"Research by the Association of University Teachers found that before anonymous marking was introduced 42% of male students achieved a 2:1 or a first in the Arts Faculty of the University of Wales, compared to just 34% of female students.

The year immediately following the introduction of anonymous marking saw 42% of male students still achieving a 2:1 or a first, but the number of women achieving the same results increasing to 47%.

Research has also found links between discriminatory marking and race, age and other personal characteristics.

At the University of East London black graduates achieved grades on average 4.2% lower than their white counterparts, while Asian students at the University of Glasgow Dental School made up 8% of those who failed but 20% of the student body as a whole.

NUS has found that while 44% of students' unions believed there was bias and discrimination at their institutions, two thirds of the institutions that received favourable ratings had fully anonymous marking in place."

Elsewhere the NUS http://www.nusconnect.org.uk/campaigns/highereducation/archived/learning-and-teaching-hub/anonymous-marking/
list the following 'further reading' (though some of this is rather old)

Belsey, C. (1988) Marking by Numbers, AUT Woman, 15
Bradley, C. (1984) Sex bias in the evaluation of students, British Journal of Social Psychology, 23,
Bradley, C. (1993) Sex bias in student assessment overlooked?, Assessment and Evaluation in Higher Education
Dennis, I. and Newstead, S.E. (1994) The strange case of the disappearing sex bias, Assessment and Evaluation in Higher Education
Francis, B., Robson, J. and Read, B. (2001) An analysis of undergraduate writing styles in the context of gender and achievement, Studies in Higher Education
Newstead, S.E. and Dennis, I. (1990) Blind marking and sex bias in student assessment, Assessment and Evaluation in Higher Education
Steinberg, J. (1982) On examining blind, New Society, 16 September


I'd be keen to hear other views on the evidence. On a more general note though, there is plenty of evidence from society at large ( e.g. http://www.tuc.org.uk/equality-issues/equality-audit ) that structural factors and unconscious bias in the forms of racism, ageism sexism etc, are endemic and result in considerable inequality in terms of outcomes - including the jobs people get and, we can assume, in the grades and qualifications they are awarded. This seems to me to make a compelling case for undertaking, where practicable, anonymous marking of summative work at university where it is to count towards a degree or other award.

Cheers

John


-----Original Message-----
From: Kal Winston [mailto:[log in to unmask]]
Sent: 31 January 2015 11:07
To: John Hilsdon
Cc: [log in to unmask]
Subject: Re: Marking (anonymised)

Is there any empirical evidence that anonymous marking actually changes the discrepancy in marks awarded to different groups? Or indeed, that non anonymous marking results in consistent and measurable differences? I've searched the literature, and have found a lot of opinion, but no data. Can anyone share specific references?
If this is not evidence-based, we risk losing all the essential benefits of assessment for learning that knowing the students offers, as stated so clearly earlier in this thread, for no actual gain at all.

On 31/01/2015, John Hilsdon <[log in to unmask]> wrote:
> Dear all
>
> Thanks for an interesting question, Peter and thanks all who have
> contributed for raising important issues about the various kinds and
> purposes of marking and giving feedback to students on their work. As
> I'm sure many of you already know, the NUS takes the view that
>
> "Anonymous marking for exams and coursework is crucial to ensure
> equality and fairness for students. Research tells us that black
> students receive lower marks than their white counterparts. Lesbian,
> gay and trans students report that their coursework has been marked
> unfairly simply because of their sexuality or gender expression."
>
> http://www.nus.org.uk/en/news/mark-my-words-not-my-name/
>
> As others have pointed out, however, it is not a simple binary choice
> between anonymous marking and having students names displayed on their work.
> Summative work by individuals, submitted under exam or test conditions
> should certainly, in my view, be anonymised; but for work at formative
> stages and for responses to group projects, the learning development
> purpose of these activities requires a conversation or dialogue in
> order for them to be effective. I don't think this is usually possible where the participants'
> identities are not known to the marker (or to each other). In
> formative activities there is also the issue of learning as a function
> of, and concomitant with, the development of community ('community of
> practice', if you like). In the latter a goal of mutual trust and
> respect between participants is vital. Additionally, knowledge of each
> other by participants (as opposed to anonymity), allows for a
> developing understanding and appreciation of context (e.g. socio
> cultural factors such as home background, ethnicity and linguistic
> heritage) so that the relative merits of participants' contributions
> (e.g. their writing, oral presentation or practical work) can be seen
> in a way that accounts for the diversity of our backgrounds. I agree
> with Neil that marking and feedback is about making judgements - for
> me the discussion is really interesting when we ask who can judge
> students' work and why. I think raising this question helps us ask not
> only about the role of self and peer feedback and assessment in
> learning, but also the role students could play in setting the
> groundrules and benchmarks for assessment. Of course, there will
> continue to be a need for summative forms of assessment and for
> academic judgements where anonymity will help us achieve greater equity in determining how students' work is graded for the purposes of achieving qualifications and awards.
>
> Happy weekends, folks
>
> John
>
> -----Original Message-----
> From: learning development in higher education network
> [mailto:[log in to unmask]] On Behalf Of LENT Neil
> Sent: 30 January 2015 16:58
> To: [log in to unmask]
> Subject: Re: Marking (anonymised)
>
> Hello folks,
>
> I'm just in the midst of some anonymous marking so this discussion is
> quite timely.
>
> I was going to suggest what Diane has just suggested in terms of
> marking anonymously but then feeding back more personally. I suspect
> this could be a useful compromise. It seems to me though that the
> anonymous marking is perceived as more objective and therefore
> accurate, whereas feedback should perhaps have a more motivational
> effect and therefore be more effective when it is more personalised.
> I'm not convinced that any marking system we use is objective. For
> example, we  can still react to anonymous work personally despite not
> knowing whose work we are reacting to. I think in many ways marking
> (whether anonymous or not) is dressed up as a process of measurement
> when, in fact, it is more a process of judgement. Anonymity doesn't
> help challenge this but then again we do tend to have biases which can negatively impact on how we mark that anonymity seems to reduce.
>
> I'm not sure how well anonymous marking  fits with the growing concern
> with working in partnership. I guess I worry that a focus on grading
> and marks (being accurate etc) promotes a product orientated version
> of higher education rather than one that focuses on process and interaction.
>
> I hope this is useful and not too much of an end of the week rant...
>
> Cheers,
>
> Neil
>
> Dr Neil Lent
> Lecturer in University Learning and Teaching Institute for Academic
> Development University of Edinburgh
> 7 Bristo Square
> Edinburgh EH8 9AL
> Tel: 0131 651 7199
> Email: [log in to unmask]
>
>
> _________________________________________
>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
> -----Original Message-----
> From: learning development in higher education network
> [mailto:[log in to unmask]] On Behalf Of Nutt, Diane
> Sent: 30 January 2015 16:10
> To: [log in to unmask]
> Subject: Re: Marking (anonymised)
>
> A quick thought or two
> (and sorry if I am  reeating ideas from other people, I think I have
> looked at all the emails on this so far!)
>
> This is a debate HE has been having for a long time. I remember heavy
> discussions with an external examiner when i was a new lecturer!
>
> I've tended to favour knowing the student whose work you are marking,
> HOWEVER I also know how influenced we can be from knowing our students.
>
> Hazel's thoughts are good ones and make sense, but I wonder too if
> there is also some value in separating marking from feedback. ie
> marking work when you dont know whose it is, but then finding out who
> it is before you provide feedback. Im sure there are lots of
> logistical problems with this approach, but it might allow for a personalised learning experience fairly framed?
> Diane
>
> ________________________________________
> From: learning development in higher education network
> [[log in to unmask]] on behalf of Hazel Corradi [[log in to unmask]]
> Sent: 30 January 2015 15:51
> To: [log in to unmask]
> Subject: Re: Marking (anonymised)
>
> Hi,
> I can see people are not keen on anonymised marking, but there is
> evidence to suggest that people can favour certain students without
> even realising it or wanting to.
> We recently looked at this paper in our teaching and learning journal club:
> John M. Malouff, Ashley J. Emmerton, and Nicola S. Schutte The Risk of
> a Halo Bias as a Reason to Keep Students Anonymous During Grading
> Teaching of Psychology July 2013 40: 233-237, first published on May
> 8, 2013
> doi:10.1177/0098628313487425
>
> Obviously, there are be reasons for not having anonymous marking and I
> think it would probably depend on the type of marking. Where the work
> is formative and low stakes, the benefits of knowing the student will
> be high. e.g. where tutors are marking tutorial work, or if there is
> an opportunity for guided feedback over a period. However, at least on
> the course I teach, this is not the majority of the marking, nor for the majority of the credits awarded.
> For large scale, high stakes, summative assessment, anonymisation will
> be fairer. This is why all of our exams are anonymous. This is
> probably also one of the reasons why our exams have a broader spectrum
> of marks than coursework only units. I think it can be hard to give
> students low marks 'to their face'.
>
> The main problems with anonymisation tend to be logistical (e.g.
> students want to add their name to their work), but even Moodle (our
> e-learning system), now allows anonymous marking and uploading of
> feedback in a relatively simple way.
>
> Having read the paper, I am now going to mark my final year essay
> coursework (worth many credits!) anonymously.
> Hazel Corradi
>
> Senior Teaching Fellow
> Biology and Biochemistry
> University of Bath
>
> PS Hope I haven't sent this twice, I don't think it worked the first
> time ________________________________
> [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymout
> h.ac.uk/worldclass>
>
> This email and any files with it are confidential and intended solely
> for the use of the recipient to whom it is addressed. If you are not
> the intended recipient then copying, distribution or other use of the
> information contained is strictly prohibited and you should not rely on it.
> If you have received this email in error please let the sender know
> immediately and delete it from your system(s). Internet emails are not
> necessarily secure. While we take every care, Plymouth University
> accepts no responsibility for viruses and it is your responsibility to
> scan emails and their attachments. Plymouth University does not accept
> responsibility for any changes made after it was sent. Nothing in this
> email or its attachments constitutes an order for goods or services
> unless accompanied by an official order form.
>


--
Kal
________________________________
[http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass>

This email and any files with it are confidential and intended solely for the use of the recipient to whom it is addressed. If you are not the intended recipient then copying, distribution or other use of the information contained is strictly prohibited and you should not rely on it. If you have received this email in error please let the sender know immediately and delete it from your system(s). Internet emails are not necessarily secure. While we take every care, Plymouth University accepts no responsibility for viruses and it is your responsibility to scan emails and their attachments. Plymouth University does not accept responsibility for any changes made after it was sent. Nothing in this email or its attachments constitutes an order for goods or services unless accompanied by an official order form.