I'd echo Lynette's comments and we use a very similar approach on undergrad 2nd and final year modules and at postgraduate level also. In particular we explain to students that their textual 'feedback' (we use the term 'justification' of marks as the comments are confidential so not technically feedback) will be used to 'validate' the peer assessment process so if there is a distribution of marks that is not supported by comments this will be looked at. Furthermore as part of the group assessment we ask the teams to submit minutes of their meetings (either in paper or via course Wiki) to evidence their group process. Again if students are criticising one another on certain issues that do not appear to have been addressed via the group minutes and discussions we reserve the right to moderate the marks. Both internal examination board, external examiners and accrediting professional body have been more than happy with the approach.



Having said all that in 4 years of using WebPA I'm yet to have to moderate a mark. Prevention appears to be better than cure :)



Alastair



---------------------------------------

Alastair Tomlinson

Chartered EHP, Senior Lecturer

Programme Director, MSc Applied Public Health

029 2020 1528

[log in to unmask]





-----Original Message-----

From: WebPA [mailto:[log in to unmask]] On Behalf Of Lynette Johns-Boast

Sent: 25 April 2013 09:17

To: [log in to unmask]

Subject: Re: Using WebPA in degree classification modules



Hi Linsey



For the past six years I have been using peer assessment (the last three years using a branched version of WebPA) to generate 75% of marks for an industry-based capstone software engineering design course where 3rd and 4th year students are required to work for two semesters in small teams of 5-7 students developing a software solution to a real-world problem for their client. At four points during the course team progress is assessed using gateway reviews which involve the team, the client, the team mentor and course academics. Peer assessment is used to calculate a moderator which is used to generate individual marks from the team mark.

Students like using peer assessment in this way as it empowers them. It enables students who want to work hard and get a good mark while not rewarding those students who have less drive and seems to overcome many (if not all) of the problems associated with social loafing and free-riding.



The assessment scheme makes it clear, and we are at pains to explain to students how we use peer assessment. We are also explicit that we reserve the right to amend the marks generated by peer assessment if we feel they are not a true reflection of contribution and that the marks they receive therefore may not be exactly as generated by WebPA. We have rarely had to intervene and amend the peer assessed marks. We do, however, exclude self-assessment when calculating the peer assessed rating as otherwise students can affect their own results in a way that is not necessarily reflective of contribution. We collect written feedback as well as the numeric ratings (using a Likert scale of 1-5).

The written feedback provides valuable insight into team contribution and validates the numeric ratings. The peer assessment data we collect usually validates the performance we see in the gateway review process and what mentors observe while supporting teams.  Like Neil we use SVN repositories and collaborative sites so the academics involved have access to all artefacts and team communication which enables us to verify claims if necessary.



Since the introduction of peer assessment, the complaints about marks have dropped to almost zero. After all who better to know what is going on in a team than the members of the team itself. Now with minimal effort by the academics we can award defensible individual marks for group work. The peer assessment data collected can also be used to mentor students to improve their performance - we have some great evidence of this working extremely well.



The course forms part of a 4-year accredited software engineering degree and our accrediting body have been very accepting of the benefits the students draw through the use of peer assessment to generate individual marks from group artefacts.



Cheers

Lynette





On 25/04/2013 01:13, Linsey Duncan-Pitt wrote:

> Thank you Neil, Keith and Paul

>

> This is really helpful information.  I saw a presentation on this a little while ago but gleaned from that and the WebPA resources that there were checks and balances.  However the real experience of people using it at the degree classification module level is what I was really interested in.  I like the safety net options you mentioned Neil.

>

> Best Regards

> Linsey



--

====================================================

Lynette Johns-Boast

Lecturer

Research School of Computer Science

The ANU College of Engineering and Computer Science CS&IT Building 108 The Australian National University Canberra ACT 0200 Australia



T: +61 2 6125 4526

F: +61 2 6125 0010

M: +61 405 611 859

W: http://cecs.anu.edu.au



Available Tuesday, Wednesday & Thursday

________________________________



From 1st November 2011 UWIC changed its title to Cardiff Metropolitan University. From the 6th December 2011, as part of this change, all email addresses which included @uwic.ac.uk have changed to @cardiffmet.ac.uk. All emails sent from Cardiff Metropolitan University will now be sent from the new @cardiffmet.ac.uk address. Please could you ensure that all of your contact records and databases are updated to reflect this change. Further information can be found on the website here.<http://www3.uwic.ac.uk/English/News/Pages/UWIC-Name-Change.aspx>



Ar Dachwedd y 1af 2011 newidiodd UWIC ei henw i Brifysgol Fetropolitan Caerdydd. O Ragfyr 6ed, fel rhan o'r newid yma, bydd pob cyfeiriad e-bost sy'n cynnwys @uwic.ac.uk yn newid i @cardiffmet.ac.uk. Bydd yr holl ebyst a ddanfonir o Brifysgol Fetropolitan Caerdydd yn cael eu danfon o‘r cyfeiriad @cardiffmet.ac.uk newydd. Gwnewch yn siwr eich bod yn diweddaru eich cofnodion cyswllt a'ch cronfeydd data i adlewyrchu hyn. Gellir cael rhagor o wybodaeth ar y wefan yma.<http://www3.uwic.ac.uk/English/News/Pages/UWIC-Name-Change.aspx>