 |
 |
The algorithm used in WebPA is similar to the one published by Li in 2001. When I introduce students to peer assessment I make the process clear and make an example marking available (attached) so they can see how the algorithm cuts out this potential bias - because it can work both ways - with a generous student ending up with a lower mark for themselves. As Peter says, it isn't perfect and judgement comes in somewhere. For me, I get students to document their activities so that I can refer to the evidence in case of dispute or 'strange' marking habits. Works fine for me and only one contested mark in five years which was subsequently withdrawn once I read that they never attended meetings or did any work... I think there is also a logged request for a range of algorithms to be available, since there is more than one way to divide marks, it just depends on your preference. Regards Paul Li, L. K. Y. (2001). Some Refinements on Peer Assessment of Group Projects. Assessment & Evaluation in Higher Education, Carfax Publishing Company. 26: 5-18. -----Original Message----- From: WebPA Project [mailto:[log in to unmask]] On Behalf Of Peter Willmot Sent: 08 December 2008 12:10 To: [log in to unmask] Subject: Re: Selfishness? Hi This is the dilemma of self and peer assessment - should you give them control (when control inevitably leaves it open to this kind of abuse. Having had experience of WebPA for over 10 years, here is my take on it in bullet form. 1) Using Web-PA you only allow students partial control - because you can check (and query if necessary all results that you think are erroneous after the marking is submitted). Normally very little, if any editing is required. 2) When I introduce Web-PA I always make it clear that marking is anonymous except that the only other person who will see the detailed breakdown is ME! I am careful to advise them that their marks are likely to be monitored. 3) It is true that weak students generally overplay their contribution but it is also just as true that other members of a team with a weak student tend to be over-critical of him/her and mark them perhaps lower than is justified. The scenario you paint is rare in my experience and could probably be picked up prior to final publication. 4) There is an academic case that students undergoing teamwork have a responsibility for the cohesion of the team. t Hence a mechanism for moderating the overoptimistic or downright inaccurate scores of a weak student and for the potentially vindictive scores of peer members is a good thing and Web-Pa effectively provides that through self and peer assessment. 5) The main drivers for this imperfect, yet sound solution are - a) to give the students responsibility and make them think about what is good and bad work b) I have not yet found a better solution to group marking and c) there is much evidence that the students who use this system and generally more content than with other systems. Have a look at my paper on this at http://www.engsc.ac.uk/journal/index.php/ee/issue/view/21 At the end of the day, the decision is yours. Regards Peter -----Original Message----- From: WebPA Project [mailto:[log in to unmask]] On Behalf Of Nicola Wilkinson Sent: 08 December 2008 11:01 To: [log in to unmask] Subject: Selfishness? I have received an email from Mike asking about selfishness within WebPA. Can anyone help to answer his question (see the email text below) Thanks Nic -------------------------------------------- Nicola Wilkinson eLearning Systems Developer WebPA Project, engCETL Web: http://webpaproject.lboro.ac.uk/ JISCmail: http://www.jiscmail.ac.uk/lists/webpa.html Project Blog: http://webpa-tec.blogspot.com/ -------------------------------------------- Hi there. I'm very interested in the WebPA system but I have been looking over the documentation and I can't see any way of preventing students from selfishly manipulating marks to boost there own scores. Is there a mechanism that you haven't explained in the demonstrations, or is the only solution to prevent students from assessing themselves (something I don't want to do)? What I mean is, say a report for a 4-person group got 80%, and I had decided to use a 40% PA weighting. Therefore, each student effectively controls a 10% stake of their 80% mark. What is to prevent one of them from marking themselves 5/5 on every criterion and their peers 1/5? Assuming the other scores were completely even, this would give them a webPA score of 1.375. 1.375 x 40% = 55%, giving a total score for the selfish student of 95% and scores for the others of 0.875 x 40% + 40% = 75%. Essentially, the selfish student has taken 5% from each of the others. Is there a flag to alert the assessor of this? In this example it would be fairly obvious, but I imagine that when the other numbers vary it would be easy to miss that. Thanks, Mike