I assure you I am not chanting from the sideline. I've been involved directly in numerous museum evaluations in the UK and internationally (including in developing world contexts where things are even more challenging). I am not trying to be annoying: I just
hate to see wasted resources on audience data that are inaccurate, and therefore not helpful for anyone. I am very sympathetic and personally experienced in the practical dilemmas involved.
I agree, sometimes paper surveys are appropriate to include as an option. I just see them used in far more cases than they actually need to be (with associated costs to the museum that I noted below).
|On 7 Sep 2018, at 18:57, Bob Clark <[log in to unmask]>
Re: Collecting visitor data
Your points about the unknowable bias inherent in most visitor surveys are beyond challenge, and have been made often before.
If we had the very substantial resources that would be required to plan and run museum user surveys to the standards of a social sciences research project, I have no doubt that is what all of us would do. But for the great majority of museums, we don't have
and will never get that sort of cash for these sorts of activities. At the same time, there is a constant, impossible-to-ignore, demand from strategic bodies, funders and governing bodies for information on who our visitors are, why they come (or don't),
what they like (or don't), and so on, and no such bodies would accept us saying that because we cannot provide rigorously-collected, scientifically-reliable data, we are not going to provide any data at all. If we tried that, we'd just be sent away and told
to do the best we can with what we've got. Which, most of the time, is what we do, and it is demoralising when the social scientists chant from the sidelines that it is all rubbish. What would be really useful is for the social scientists to undertake a
study of common limited-resource survey methodologies, and try to place quantum on the extent and nature of the inherent bias.
I do not believe there is the slightest suggestion here - most certainly there is not on my part - that paper-based surveys generate more scientifically-accurate results than tablet-based surveys. The issue is simply that at least in some locations, paper-based
surveys provide more data.
The Auchindrain Trust
mailto:[log in to unmask]
Friday, September 7, 2018, 12:12:47 PM, you wrote:
|Yes, it is all about getting the small details right in survey research and evaluation. I would not advise leaving the tablet at a reception
desk without a systematic sampling plan (and detailed instructions and training for the reception staff) in place.
My default approach for visitor surveys is to intercept people on their way in to the site (30 seconds - 2 minutes), following a systematic sampling procedure. People complete a relatively short survey on the way in with a data collector either holding the
tablet or handing it over temporarily. Then at the end of the day, they automatically receive a post-visit survey request (with automatic reminders). Observable data are collected on non-respondents by the data collector (called a ‘refusal log’) in order to
calculate response rate and identify any systematic sampling bias that may be visible.
In order to do this kind of fundamental quality assurance, a person is needed to administer the initial survey (or you need advance access to people’s details). Without this kind of systematic approach, quantitative claims about visitors cannot legitimately
be made (e.g. % of people satisfied with their experience) as the level of sampling bias is unknowable.
Of course, this is challenging when you do not ever have a data collector available, so creative solutions have to be identified. I don’t see how paper survey forms per se make these issues better though (and someone then needs to put the time into typing up
the data, there is risk of data entry errors, you cannot use skip logic elegantly, etc.).
Dr Eric Jensen, Fellow Higher Education Academy
Associate Professor (Senior Lecturer), Department of Sociology, University of Warwick
Latest book: -
Doing Real Research (SAGE): https://us.sagepub.com/en-us/nam/doing-real-research/book241193
Check out a sample chapter here -
Sociology at the University of Warwick ranked:
The Guardian, Complete University Guide and The Times Good University Guide – 3rd
QS World University Ranking - 23rd
|On 7 Sep 2018, at 16:20, Elin Bornemann <[log in to unmask]>
It has been our experience that many visitors are reluctant to interact with tablets as well. We tried doing visitor surveys via a tablet for a while, but it was not a success.
We never hit on the right app to use either, they all had problems. It might have worked better if we had used the tablet in conjunction with a person, i.e. have a person approach visitors with tablet in hand. The way we tried it, however, was with the tablet
set up on the reception desk, and we didn’t get much data that way. We have now gone back to paper.
Abingdon County Hall Museum
From: Museums Computer Group [mailto:[log in to unmask]]
On Behalf Of Jensen, Eric
Sent: 07 September 2018 03:36
To: [log in to unmask]
Subject: Re: Collecting visitor data
Just a quick note that I have sometimes found visitors are reluctant to have their data collected via smartphone. (Perhaps because these are so associated with being personal devices)
So I would recommend a low cost android tablet as the on-site data collection device. You can get these got as little as £45 each. (Though if you can afford the £100-120 level, the added processor speed does make the data collection interaction smoother)