Hi Tim,
Originally, WebPA didn’t have a peer-only marking mode, so the example on the website is based around a simple self-and-peer submission.
For peer-only grading, non-submitting students are assumed to have given everyone 5 marks - the actual number doesn’t matter, so long as it’s the same for everyone and greater than zero, it all normalises out to them giving everyone the same fractional score. The algorithm then runs as normal with these faked-compensatory scores.
As I said before, the closest I could get to your grades was to run the algorithm without the ‘compensate for non-submitting’ bit. It wasn’t a perfect match for what you see, but it was close. There should not be a way to accidentally do this through the software, I had to hack the database directly after the scores were entered. If I remember correctly, it's changing the assessment_type field, from '0' to '1' (self-and-peer to peer-only).
--
Paul Newman
------------------------------------------------------
Senior PHP Developer
Learning Technology & Digital Innovation Group, IT Services
Loughborough University
------------------------------------------------------
From: WebPA [mailto:[log in to unmask]] On Behalf Of [log in to unmask]
Subject: Re: Problems with WebPA results
PS I have studied the worked example of the algorithm on the website to see if I can calculate the scores manually, but it does not show how to compensate for non-responders.
On 21 Nov 2016, at 10:03 PM, Timothy Allen <[log in to unmask]> wrote:
Hi Paul,
I really appreciate your detailed and clear reply. I have just received another similar example from another instructor, which is consistent with your suggestion as to the likely problem. In this group:
• 4 out of 6 students did not submit ratings
• self assessment was not enabled
• as a result students only received 1 or 2 separate ratings for each question
• even though most ratings were 4 or 5, the WebPA scores were very low. One student got two 5/5s, but just got a rating of 0.66
• non-responders got higher ratings than responders.
I have passed on these details to our WebPA admin and will ask him what version we are running.
Many thanks,
Tim.
|