My research involves analysing errors in a learner corpus ( a collection of texts written by English as a foreign language learners). I am sorting the errors into 56 error categories (mainly 8 major error categories): Form, Grammar, lexico grammar, lexis, Word redundant or missing, Style, Infelicities.To ensure reliability, a sample of twenty papers will be analyzed by a second coder. i.e. He will read the text, identify the errors and give them an appropriate tag/code. The second analyst will be given an error tagging manual to guide him in giving the codes.
The researcher (the first analyst) has used the same manual.The research aims to find the most common and the least frequent errors in these written texts. One of the statistics I need to have using SPSS 16 is:
I will need to - count how many phrases/words, etc. were considered as errors by both judges (because sometimes one judge doesn't consider something an error, whereas the other does). Once I have a list of these agreed upon 'error tagging targets', Iwill need to do a statistic to assess how much the judges agreed in the error tags they used to code these errors. The test to do is the Kappa test.
I do not know how to prepare my data for the Kappa test. How many variables I will need? What are these variables?. What would be the count?Could you please provide me with a detailed explanation for my research?
|