Dear David,
We don't want to go to the extremes. There is no universal method. And many methods that are rebuked are pretty good if people know in which situations to use them and how to use them. Even the survey and the experiment. Pollsters use them pretty well. If there are "errors" within 15 percentage points, they are actually ordered and paid by the clients.
The real surveys in 2016 US Presidential Elections were quite accurate. But the media were airing the data that the clients wanted to trumpet. No one came forward to talk about the realistic data. Then, people were surprised of the election outcome. (Sociologists foresaw the eventual election outcomes as early as late September.) One simple trick in an election campaign is to convince your voters that you will win. Then voters become more enthusiastic and go to the ballot boxes. Very often the biggest problem of the politicians is to make voters go to the ballot boxes. That is why the politicians are trying to convince their voters that they will win with a 20% margin.
In polling, the agencies make studies how the sequence of questions and their wording will affect the results.
Surveys work well if people know how to do them. And if the phenomenon that is researched is well understood and there are good theoretical models to guide the development of the survey questions. Also, if the sampling strategy and implementation are appropriate. Ali already talked about the methodology of survey design.
Surveys can be manipulated with ease. The data collected depend on the wording of questions, the sequencing of questioners, etc. If you want to raise your party approval values we can do this with ease, up to 10-15 percentage points. This the data can be easily "defended" even when the professed theoretical model and the sampling strategy are scrutinized and questioned.
And there is nothing worse than a poor survey. May be a poor experiment? Poor surveys are less reliable even than qualitative studies. With the poor surveys we have "garbage in, garbage out." No use.
Many researchers think that developing the survey questionnaire is like making questions from thin air. These researchers fool themselves that they use a very powerful method, while in fact they have no control over data quality because of incompetence.
Every time we make a survey, we make social science research. We need to be experts in the social sciences. At least a Ph.D. in a social science discipline that has a good track record of using the survey research design. Like sociology, political science, marketing research, etc. All the rest is perfunctory, and most often leads to "garbage in, garbage out." The advent of Survey Monkey (and many others) made every third-grader a pollster. And everyone claims they make research. I am sorry... We should do better and teach the kids better.
The current discussion on survey methods highlights the nebulous boundaries of design research. What is design research? Is there design research? When do we make design research? What should be the education and training of a design researcher? And so on, and on, and on...Or, we are making social science research without even realizing this? There are so many examples.
There is a simple rule: every time we study the relationship between any kind of object/phenomenon and society/groups/individuals, we make social science. Every time we talk with/interview/observe humans, we make social science. Now let's see what remains for design research. So much about design research:) Let's change the name of the discussion list:)
Best wishes,
Lubomir
Lubomir Popov, Ph.D., FDRS, IDEC, CSP
Professor, School of Family and Consumer Sciences
American Culture Studies Affiliated Faculty
Bowling Green State University, OH, U.S.A.
-----Original Message-----
From: PhD-Design - This list is for discussion of PhD studies and related research in Design <[log in to unmask]> On Behalf Of [log in to unmask]
Sent: Wednesday, June 20, 2018 8:32 PM
To: [log in to unmask]
Subject: Re: - Call for participation in International survey on Design expertise
> On 20 Jun 2018, at 3:13 pm, Ali Ilhan <[log in to unmask]> wrote:
>
> Surveys might be problematic in your context, but they have their
> uses. For some types of research questions, they are the best tool we
> have. For some others, they are an absolute pain in the neck. You can
> say the same thing easily for experiments, interviews, ethnography etc.
Without extending this discussion much further than I have already, I will just say that I think your ecumenical approach underestimates the extent of the validity crisis in survey research. It’s not a case of horses for courses. Some horses are born lame and are best euthanised.
David
--
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|