Hello everyone,
We've been looking at stats quite a bit lately, and this culminated in
Monday's dissemination event here at LTM that Bilkis and Richard
mentioned. We wanted to explore some of the practical issues around
different organisations taking a more strategic approach to collecting
stats, and see how they could be used more effectively by the London Hub
partners.
It's easy to dismiss stats as being pointless. There are issues around
their accuracy, and they can provide a misleading picture of use. I
suspect that Nielsen's remarks about bounce rates are probably more
suited to e-commerce type environments, where a single page view is
unlikely to result in a purchase. So from that point of view they
probably are a problem. However, as Mia points out a single page view on
a museum web site may result in the user achieving their particular goal
perfectly well. This is the problem of course, they don't tell you
anything about the user's reasons for being there and whether they got
what they wanted. But then the same can be said of museum visitor
numbers, which we've been happily using for years now...
However, I do think that the "stats are worthless so we shouldn't
bother" argument is a bit too convenient, and rather misses the point in
many ways. I think stats can work and do have a role to play, but it's
important to recognise their limitations and see them as part of a
bigger picture. The main conclusion we came to as a result of our work
here in London was that the type of data you collect really depends on
what you're trying to do.
In terms of stats, we found that there are some simple figures that can
(and are) be used as 'headline' data, for the purposes of internal and
external advocacy and reporting. This can be collated and used pretty
consistently by different organisations like the Hub partners, and it's
relatively easy for non-technical people to get their heads round. So
for example, here at LTM we've recently become a charitable trust. As
part of this process we now use a Balanced Scorecard type approach to
report different aspects of museum performance to staff, Board and other
stakeholders. This includes headline figures like:
- Visit to the museum, broken down by ticket type
- Late openings visits
- Gift Aid donations
- Web Visits
- Admissions Income
- Retail Shop Income
- Catering Income
- Guide Books sold
I'm sure other museums take a similar approach. In this context, I think
it makes perfect sense for web visits to be included. They give an
overall indication of the level of use that the our online offering is
experiencing, just as the museum visitor numbers do for our physical
offering. Apart from anything else though, it makes it clear that the
web is an important delivery mechanism for the museum and we're willing
to be assessed on that.
I think this sends an important message. As IT professionals, we can't
on the one hand endorse the web as an essential tool for museums, but
then hold our hands up in horror when we're asked to actually be
measured on it.
Once you get into the more detailed stuff - as Jeremy says - it's more
difficult to do this sort of thing because the data is much more closely
tied to site purpose, audience, architecture and so on. So broad
comparisons are much less meaningful, but we found that the stats can
still tell us a lot about how the site is used and inform developments.
This kind of data is more generally suited to internal organisational
use rather than the external advocacy and reporting.
Evaluating the 'impact' and 'value' of web offerings does seem to be a
hot topic at the moment, and its important to see stats as part of this.
The work we've been doing has been complemented by more qualitative
audience research as well. As with the stats work, we wanted to look at
practical ways in which the London Hub could approach market research in
a more joined-up way. So we used online questionnaires, in-gallery
visitor surveys and focus groups to try and get a decent picture of who
our online users are, what they want, and try and work out whether they
are getting it or not.
I think it's important to see the statistical data in the context of
this more qualitative market research and as part of a broader package
of measures and methodologies - both web related and otherwise - that
help to establish a museum's overall performance.
We found that using different techniques to actively find out more about
our online audiences can give us an insight into users' motivations and
expectations and fill in many of the gaps left by web stats. The
research was pretty broad-brush and did seem to raise more questions for
us than it answered. However, here are some of the main findings from
the market research at the London Hub, pasted in from a summary
document:
- By far and away the main reason for visiting our museum web sites is
to plan a visit to the physical museum. This appears to be the case
across the Hub museums, and was confirmed through feedback during the
focus groups.
- Educational uses represent the exception to this rule, with teachers
and families using the sites to look for teaching materials and ideas,
online games, and material to help with homework.
- In general terms, the demographics of the Hub's online visitors
broadly reflect those of the people who visit the museums.
- Overall, the research revealed that people would like more information
about what they can expect to see and do when they visit the museum.
Information on the museum's objects, and details of events and
exhibitions also ranked highly. These all relate to visitor information,
and reinforce the view that people see the primary role of the museum's
web presence as enabling and supporting the museum visit.
- There's very little appetite for user-generated content on our web
sites. In fact it consistently scored lowest in surveys and this was
confirmed in focus groups. (We need to do more work on this, but it
seems to support Linda's recently published findings about apparent lack
of demand for 'Web 2.0' type services from HE and FE audiences).
I'm sure that others will disagree strongly with a lot of this and will
have different findings, and we've realised that we need to do more
research to understand why people are telling us this. But this work so
far has certainly raised questions in my mind about a number of areas.
Specifically, how people want us to present collections online, and
whether a wholesale approach to 'Web 2.0' type applications like UGC is
really justified without a clear purpose, audience and justification in
mind.
Hope this helps,
Dylan
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Mike Ellis
Sent: 03 July 2008 09:20
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
It's a really familiar refrain: stats don't work. What I'm not seeing is
anyone actually doing anything about it.... :-)
Mike Ellis
Professional Services Group
Eduserv
[log in to unmask]
tel: 01225 470522
mob: 07017 031522
fax: 01225 474301
www.eduserv.org.uk
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Ottevanger, Jeremy
Sent: 02 July 2008 16:00
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
How many pages does the average Facebook user see on an average visit?
Their own, perhaps? They can see what their mates are up to right there.
Lots of Gmaps activity also takes place in one page view. For some
sites, as Mia says, a higher number of page views probably indicates
failure rather than success, in that the user should have been able to
do what they wanted in fewer clicks. It all depends how the site works.
As ever, there isn't a statistic that is meaningful in all situations
(or indeed most).
Perhaps Nielsen has a point, and some bounce rate measure would be an
appropriate KPIs to report to DCMS or whoever. But to my
mind most stats are best suited to use by site owners/developers for
their own purposes of evaluating and improving the effectiveness of
their services, knowing the specific peculiarities of their offering. I
can't think of a single (automated)indicator that won't mean opposite
things for different sites. But I think Dylan is itching to say
something....;-)
Cheers, Jeremy
Jeremy Ottevanger
Web Developer, Museum Systems Team
Museum of London Group
46 Eagle Wharf Road
London. N1 7ED
Tel: 020 7410 2207
Fax: 020 7600 1058
Email: [log in to unmask]
www.museumoflondon.org.uk
Museum of London is changing. Visit www.museumoflondon.org.uk to find
out more.
Explore how the Great Fire shaped the city
www.museumoflondon.org.uk/londonsburning
Jack the Ripper and the East End a major new exhibition at Museum in
Docklands, opens 15 May
Before printing, please think about the environment
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Ridge, Mia
Sent: 02 July 2008 15:46
To: [log in to unmask]
Subject: Re: [MCG] Jakob Nielsen says '"Unique Visitors" Must Die'
Bilkis wrote:
> I don't know if you have come across this, but given the discussions
> that took place at the MCG in Leicester in June and the Museums Hub
> workshop on Monday, this makes some interesting reading:
>
> Reduce Bounce Rates: Fight for the Second Click:
> http://www.useit.com/alertbox/bounce-rates.html
Thanks Bilkis!
Nielsen says, "bouncers should be considered a negative statistic: the
site failed to engage them enough to entice even a second pageview."
Maybe. But could it also be the case that some users have very simple
needs (for example, is the museum open tomorrow, and until what time if
so?) and that a good 'visit this venue' page answers that need entirely?
In that case, is a 'bouncer' a positive statistic?
While it might be nice to entice these visitors to spend more time on
our site, they simply may not be looking for an online experience at
that time. He does at least say "it's important to realize that there's
no such thing as a single bounce rate" but then he focuses on referrer
source rather than user requirements.
Nielsen's always good fodder for debate. Am I the only one who can
never quite agree with him?
cheers, Mia
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
Visit Last Stop, an art exhibition of Ralf Obergfell's beautiful photographs that celebrate the Routemaster bus. The exhibition runs until 27 July and in London Transport Museum's CBS Outdoor Gallery www.ltmuseum.co.uk
_______________________________________________________________________
This email and any attachments are confidential and intended solely for the use of the addressee. If you have received this in error please notify [log in to unmask] and delete immediately. Any use, disclosure, forwarding, printing or copying of this email is strictly prohibited. London Transport Museum (LTM) hereby excludes any warranty and any liability as to the quality or accuracy of the contents of this email and any attached transmitted files.
LTM may monitor all email communications through its internal and external networks.
London Transport Museum Limited is a charitable company registered in England and Wales (company number 6495761 registered charity number 1123122) whose registered office is at Windsor House, 42-50 Victoria Street, London SW1H 0TL. London Transport Museum Limited is a subsidiary of Transport for London and as such is a company controlled by a local authority for the purposes of Part V Local Government and Housing Act 1989. VAT No. 756 2770 08
Copyright in this email and attachments belongs to TFL.
This email and attachments have been swept for the presence of computer viruses and LTM accepts no responsibility for any virus imported with this document.
**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************
|
