 |
 |
Hi David, You quoted some ICOGRADA survey data in the recent DRN. I've been collecting ICOGRADA survey data for several years to do a behavioural/attitudinal profile of ICOGRADA active designers. It looked like a nice paper! I found ICOGRADA survey data is deeply suspect to the point of being false. A few months ago, I looked over the ICOGRADA data and was surprised at a few of them so I started looking more closely at validity issues. I found the data is worthless - or rather totally unreliable - because it is effortless for any interested/unscrupulous individual to modify it by voting multiple times. The numbers of participants is low and I found I could modify the survey ratios with little effort. If the numbers of participants were higher a simple script could be used to automate the revoting process. This is a serious data collection weakness that totally compromises the validity of the surveys. The usual method of addressing this problem is through identification of the participants and only allowing a single vote. There are many straightforward ways of doing this. ICOGRADA's web managers should be aware of them. ICOGRADA's surveys also are compromised by the usual data collections problems associated with online surveys relating to bias of results from, e.g. unbalances self-selection in participation, leading questions, and poor category choice. Advice to PhD students is not to use the ICOGRADA data and when designing online surveys to do them properly! Best wishes, Terry