Print

Print


Hi Bob,

No, that is not the case. I have supported a lot of evaluations that were far from properly resourced (ranging from £0 budgets).

Regarding the need to make good evaluation practice as logistically and financially feasible for museum professionals as possible: I am working on it!

Best,
Eric

On 7 Sep 2018, at 22:05, Bob Clark <[log in to unmask]<mailto:[log in to unmask]>> wrote:

Eric,

I'm very aware that you are far from just a theoretician when it comes to museum evaluation, but suspect you are in the fortunate position of only ever being called upon to do such work when it is properly resourced.  You thus get good, scientifically-reliable, results.

There is, however, a fundamental difference of viewpoint between where you stand and what represents reality for most of us.  You consider resources to be "wasted" when they produce data you feel to be "inaccurate and therefore not helpful for anyone".  May I, however, offer you an alternative perspective?  However flawed the data generated by vernacular surveys may be, time and again it satisfies the needs of governing bodies and funders, and museums like the one I run get grants.  By my definition that is not money wasted, and regardless of how inaccurate it may be, it has certainly been helpful to a lot of people.

I suspect there isn't a single person on this forum who does not wholly accept what you say about the inaccuracy of what the great majority of us are doing.  In my view however, until and unless the social sciences world can offer us methodologies that are logistically and financially realistic, we have no option but to meet internal and external demands for information with "inaccurate" results that satisfy the needs of those who require them.


Bob
-----

Friday, September 7, 2018, 2:59:16 PM, you wrote:


        Hi Bob,

I assure you I am not chanting from the sideline. I've been involved directly in numerous museum evaluations in the UK and internationally (including in developing world contexts where things are even more challenging). I am not trying to be annoying: I just hate to see wasted resources on audience data that are inaccurate, and therefore not helpful for anyone. I am very sympathetic and personally experienced in the practical dilemmas involved.

I agree, sometimes paper surveys are appropriate to include as an option. I just see them used in far more cases than they actually need to be (with associated costs to the museum that I noted below).

Best,
Eric



        On 7 Sep 2018, at 18:57, Bob Clark <[log in to unmask]<mailto:[log in to unmask]>> wrote:

Re: Collecting visitor data
Dr Jensen

Your points about the unknowable bias inherent in most visitor surveys are beyond challenge, and have been made often before.

If we had the very substantial resources that would be required to plan and run museum user surveys to the standards of a social sciences research project, I have no doubt that is what all of us would do.  But for the great majority of museums, we don't have and will never get that sort of cash for these sorts of activities.  At the same time, there is a constant, impossible-to-ignore, demand from strategic bodies, funders and governing bodies for information on who our visitors are, why they come (or don't), what they like (or don't), and so on, and no such bodies would accept us saying that because we cannot provide rigorously-collected, scientifically-reliable data, we are not going to provide any data at all.  If we tried that, we'd just be sent away and told to do the best we can with what we've got.  Which, most of the time, is what we do, and it is demoralising when the social scientists chant from the sidelines that it is all rubbish.  What would be really useful is for the social scientists to undertake a study of common limited-resource survey methodologies, and try to place quantum on the extent and nature of the inherent bias.

I do not believe there is the slightest suggestion here - most certainly there is not on my part - that paper-based surveys generate more scientifically-accurate results than tablet-based surveys.  The issue is simply that at least in some locations, paper-based surveys provide more data.



Bob Clark
Director
The Auchindrain Trust
mailto:[log in to unmask]

-----

Friday, September 7, 2018, 12:12:47 PM, you wrote:


        Yes, it is all about getting the small details right in survey research and evaluation. I would not advise leaving the tablet at a reception desk without a systematic sampling plan (and detailed instructions and training for the reception staff) in place.

My default approach for visitor surveys is to intercept people on their way in to the site (30 seconds - 2 minutes), following a systematic sampling procedure. People complete a relatively short survey on the way in with a data collector either holding the tablet or handing it over temporarily. Then at the end of the day, they automatically receive a post-visit survey request (with automatic reminders). Observable data are collected on non-respondents by the data collector (called a ‘refusal log’) in order to calculate response rate and identify any systematic sampling bias that may be visible.

In order to do this kind of fundamental quality assurance, a person is needed to administer the initial survey (or you need advance access to people’s details). Without this kind of systematic approach, quantitative claims about visitors cannot legitimately be made (e.g. % of people satisfied with their experience) as the level of sampling bias is unknowable.

Of course, this is challenging when you do not ever have a data collector available, so creative solutions have to be identified. I don’t see how paper survey forms per se make these issues better though (and someone then needs to put the time into typing up the data, there is risk of data entry errors, you cannot use skip logic elegantly, etc.).

Best,
Eric

---------------
Dr Eric Jensen, Fellow Higher Education Academy
Associate Professor (Senior Lecturer), Department of Sociology, University of Warwick
http://warwick.academia.edu/EricJensen
Latest book: - Doing Real Research (SAGE): https://us.sagepub.com/en-us/nam/doing-real-research/book241193
Check out a sample chapter here - https://us.sagepub.com/sites/default/files/upm-binaries/73894_Jensen_Chapter_6.pdf

Sociology at the University of Warwick ranked:
The Guardian, Complete University Guide and The Times Good University Guide – 3rd
QS World University Ranking - 23rd



        On 7 Sep 2018, at 16:20, Elin Bornemann <[log in to unmask]<mailto:[log in to unmask]>> wrote:

It has been our experience that many visitors are reluctant to interact with tablets as well. We tried doing visitor surveys via a tablet for a while, but it was not a success. We never hit on the right app to use either, they all had problems. It might have worked better if we had used the tablet in conjunction with a person, i.e. have a person approach visitors with tablet in hand. The way we tried it, however, was with the tablet set up on the reception desk, and we didn’t get much data that way. We have now gone back to paper.


Elin Bornemann
Collections Officer
Abingdon County Hall Museum
Market Place
Abingdon
OX14 3HG
01235 523703



From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of Jensen, Eric
Sent: 07 September 2018 03:36
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: Collecting visitor data

Just a quick note that I have sometimes found visitors are reluctant to have their data collected via smartphone. (Perhaps because these are so associated with being personal devices)

So I would recommend a low cost android tablet as the on-site data collection device. You can get these got as little as £45 each. (Though if you can afford the £100-120 level, the added processor speed does make the data collection interaction smoother)

Eric


________________________________
To unsubscribe from the MCG list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=MCG&A=1






________________________________
To unsubscribe from the MCG list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=MCG&A=1


________________________________
To unsubscribe from the MCG list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=MCG&A=1






________________________________

To unsubscribe from the MCG list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=MCG&A=1


****************************************************************
       website:  http://museumscomputergroup.org.uk/
       Twitter:  http://www.twitter.com/ukmcg
      Facebook:  http://www.facebook.com/museumscomputergroup
 [un]subscribe:  http://museumscomputergroup.org.uk/email-list/
****************************************************************