> On 22 Jun 2018, at 3:53 pm, Ali Ilhan <[log in to unmask]> wrote:

>  Are you
> suggesting that we should abandon survey research completely? I am asking
> because there are some areas in which simply we do not have any other
> viable alternatives (at least at the moment)

Quick answers. No, that is not viable, as you point out. What I’m suggesting is a much more critical approach to the use of surveys and to including the results of surveys as valid evidence in literature reviews.

You can see a routine use of this approach in one of our literature reviews for our Members. https://communication.org.au/designing-usable-medicine-information/ <https://communication.org.au/designing-usable-medicine-information/> 

David
-- 

60 Park Street • Fitzroy North • Melbourne • Australia • 3068

> 
> Dear David,
> 
> As you have mentioned there are well known problems in survey research, I
> am pasting an article below for those who might not know the technical
> issues that we are talking about:
> 
> http://www.nber.org/papers/w21399
> 
> *Abstract*
> *"Household surveys, one of the main innovations in social science research
> of the last century, are threatened by declining accuracy due to reduced
> cooperation of respondents. While many indicators of survey quality have
> steadily declined in recent decades, the literature has largely emphasized
> rising nonresponse rates rather than other potentially more important
> dimensions to the problem. We divide the problem into rising rates of
> nonresponse, imputation, and measurement error, documenting the rise in
> each of these threats to survey quality over the past three decades. A
> fundamental problem in assessing biases due to these problems in surveys is
> the lack of a benchmark or measure of truth, leading us to focus on the
> accuracy of the reporting of government transfers. We provide evidence from
> aggregate measures of transfer reporting as well as linked microdata. We
> discuss the relative importance of misreporting of program receipt and
> conditional amounts of benefits received, as well as some of the
> conjectured reasons for declining cooperation and survey errors. We end by
> discussing ways to reduce the impact of the problem including the increased
> use of administrative data and the possibilities for combining
> administrative and survey data."*
> 
> But that said, I want to reiterate my point,  these problems are there in
> any research that claim to "measure" people's opinions. (by the way I like
> the word ecumenical, perhaps I can found the church of puritanical
> methodologists. Maybe I can attract a few followers who knows? :) The
> problem is not the instrument but the problem of getting into people's
> heads (unless you are a psychic of some sort). Is ethnography reliable? In
> so far as your interpretation skills, and the willingness of your
> informants in being truthful.
> 
> I am writing again (I promise I will not pontificate more and I will not
> write under this topic again), to understand your position better. Are you
> suggesting that we should abandon survey research completely? I am asking
> because there are some areas in which simply we do not have any other
> viable alternatives (at least at the moment)
> 
> Two examples
> 1) the US government uses a survey called the current population survey
> (CPS) for details see https://www.bls.gov/cps/cps_htgm.htm to measure
> unemployment in the US. Without this survey, FED does not have any other
> reliable figure about unemployment in the US (and they triangulate it other
> admin data).
> 
> 2) Crime rates: Official crime statistics (which themselves are a form of
> survey research) are notoriously not reliable. The reason is the fact that
> many kinds of crimes is underreported (people do not always go to the
> police or to the courts). So criminologists use something called
> "victimization surveys" to be able to understand the prevalence of crime in
> a geographic area and triangulate this information with official crime
> statistics.
> 
> Warm wishes,
> 
> ali
> 
> 
> -----------------------------------------------------------------
> PhD-Design mailing list  <[log in to unmask]>
> Discussion of PhD studies and related research in Design
> Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
> -----------------------------------------------------------------



-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------