Maybe I'm just being obstreperous here, but...
There's an awful lot said about UGC and web 2.0 which seems to indicate
that very often, focus group respondents aren't entirely sure what UGC
looks like, and what web 2.0 is capable of. All these things, as with
every other aspect of the web, can be executed exceptionally well, and
until the audience as a whole has an experience of this kind of
excellence, then *of course* the audience will resist it, mired as it is
in a reputation of pointless babble and ill-informed (and occasionally
downright rude) comment.
The same could easily be said for collections information, but the
success of the Flickr Commons project would seem to contradict both
those findings in a fairly comprehensive way: A blend of lots of
collections information, UGC and web 2.0 - it might not be the be-all
and end-all to the possibilities for Museums making use of newer kinds
of interaction and content online, but it's certainly a pretty
impressive start.
Of course, then you get to trying to measure engagement, and to the
issue of Digg, StumbleUpon et al: Sites like 24 Hour Museum get a lot of
traffic from StumbleUpon (not so much from Digg), and people tend to
write nice things about it (but not everyone does, and that's important
too). If we want to have a conversation with our users, we could do
worse than start there - there are all the requisite tools for a full
and comprehensive conversation with our audience about what it is that
we do, and what does and doesn't work: Every site will have a different
place where its users congregate (and leaving 'share this' stamps
through the site will encourage more congregation); surely it makes
sense to go there too, and talk to your audience...
Similarly, the point was made earlier in this thread about measuring
whether your site is fit for purpose: Each page will have its own
function within the site, and a user's optimum interaction with that
site will vary hugely from page to page: If someone spends ages poring
over your 'Contact Us' page, there's probably something wrong.
Similarly, someone might spend ages reading a long article, then return
to it when writing their blog and look at it very quickly while
fact-checking. Both of these interactions are valid, engaged
interactions.
But user loyalty also has its problems, particularly if you're dealing
with a tourist audience, who may be using your site to find out more
before their one and only visit to your institution while they're in the
town/country...
Equally, I keep coming across sites which seem (from Google) to have the
content I want, but usually don't - so I'm possibly logged as a 'loyal'
user of those sites (I read them often, then try something else, then go
away again, frustrated) - but I'm anything but (BBC springs to mind on
this score...)!
Also, measuring engagement brings with it the thorny issue of having to
actually engage with the audience: Opening up UGC doesn't mean that the
content is generated and left; if it's not part of a wider discussion,
it stagnates - the conversation must be two-way and it must have a
purpose at its centre - moreover, that purpose needs to be appropriate
to the forum; there's not much point in trying to discover the ins and
outs of what users think of your site via Flickr (unless like Jason
Calacanis for Mahalo, you're specifically looking to crowd source
expertise about the design), and you'll struggle to have a meaningful
conversation about object-level online collections via StumbleUpon
unless you're happy to trawl through pages and pages of reviews.
It seems to me that the framework for understanding our statistics, and
creating any kind of standard, needs to begin with the premise that
optimum performance is going to vary wildly from one site to the next,
but that there are some good trends to aspire to: Can we start by
examining the types of site and the types of content we are publishing,
and work towards patterns and trends which indicate success, and others
which indicate the need for improvement? (e.g. proportion of time spent
on homepage vs contact (or any other) page, specific comments related to
those pages on third party sites, most and least successful content,
length of 'long tail', uptake of 'calls to action', subscriptions to
e-news, RSS... The list goes on...
Perhaps we need to look at keeping a studied eye on new types of metrics
available, and the new types of interaction - which will involve far
more qualitative research than quantative?
Ruth Harper
Administrator, Culture24
www.culture24.org.uk
Direct Line: 01273 623269
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Dylan Edgar
Sent: 03 July 2008 14:30
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
Hi James,
Yes very good point, I should have clarified that. We explored the
reasons behind the responses in more detail during the focus groups. We
found that in general, where people did want information about objects
it was in context of the experience of visiting (or potentially
visiting) the museum. So, people generally wanted to know what they
could expect to see at the museum.
There were some other interesting points to note from those who did want
collections info online. I think it's safe to say that they seemed to be
more interested in quality of information rather than quantity of
records. Furthermore, contextual interpretive information surrounding
the object data was seen as being very important. So perhaps there's an
argument for fewer, higher-quality records with good contextual info
rather than just striving to get as much online as possible?
Obviously there were individual exceptions to these rules (mainly in
educational use), but I think it's fair to say that this was the overall
general message we were getting. We ran the focus groups according to
different audience types. As an illustration, here are some summaries
pasted in from the reports:
************************************************
Adults 25+
----------
Most participants were not particularly interested in finding out more
information about the museum collections via the website/browsing online
collections. Some were not aware that this kind of information existed
on museum websites.
Of those with some interest they felt that it should be clear that what
you see is not on display at the museum. Others felt this resource would
only be useful when studying e.g. History of Art.
Families
--------
Most families were interested in finding out more information about the
museum collections via the website/browsing online collections if they
related specifically "to what the children are doing at school".
Families generally wanted layered information about online collections.
The information should be presented in a "tidy" and "appealing" way,
perhaps starting with "a picture of something that everybody would
associate with that era".
18-24
-----
Some participants were not interested in looking at museum collections
on line and would "prefer to go to the gallery myself", "you can't beat
going to the real live, hands-on experience", "the atmosphere in
museums, you couldn't create that online".
Some participants wanted the option of a good depth of information about
objects as well as information which placed the object in context: "not
just what the object was and where it was found, but what that object
might mean".
Most participants preferred being able to access information on
collection "highlights" and then being able to drill down to more
information if needed.
************************************************
I hope this is a representative summary of the responses. The focus
groups were obviously only a small sample, but the findings did seem to
make sense when compared to the broader responses from online
questionnaires and in-gallery surveys which generated a much larger
sample. Again though, it would be nice to do more research in this area
specifically, seeing as so many museums are putting collection records
online.
Hope this helps - I'm in the process of working out a place to put all
of these findings at the moment, so if anyone's interested I can try to
circulate more widely...
Cheers,
Dylan
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
James Watson
Sent: 03 July 2008 13:00
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
Hi Dylan.
You wrote:
> - Overall, the research revealed that people would like more
information
> about what they can expect to see and do when they visit the museum.
> Information on the museum's objects, and details of events and
> exhibitions also ranked highly. These all relate to visitor
information,
> and reinforce the view that people see the primary role of the
museum's
> web presence as enabling and supporting the museum visit.
I'm probably misunderstanding, but does 'information on the museum's
objects' always 'relate to visitor information'?
All the best
James
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
James Watson
Digital Collections Manager
Digital Media
National Maritime Museum
Greenwich
London
SE10 9NF
[log in to unmask]
020 8312 8506
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Dylan Edgar
Sent: 03 July 2008 12:08
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
Hello everyone,
We've been looking at stats quite a bit lately, and this culminated in
Monday's dissemination event here at LTM that Bilkis and Richard
mentioned. We wanted to explore some of the practical issues around
different organisations taking a more strategic approach to collecting
stats, and see how they could be used more effectively by the London Hub
partners.
It's easy to dismiss stats as being pointless. There are issues around
their accuracy, and they can provide a misleading picture of use. I
suspect that Nielsen's remarks about bounce rates are probably more
suited to e-commerce type environments, where a single page view is
unlikely to result in a purchase. So from that point of view they
probably are a problem. However, as Mia points out a single page view on
a museum web site may result in the user achieving their particular goal
perfectly well. This is the problem of course, they don't tell you
anything about the user's reasons for being there and whether they got
what they wanted. But then the same can be said of museum visitor
numbers, which we've been happily using for years now...
However, I do think that the "stats are worthless so we shouldn't
bother" argument is a bit too convenient, and rather misses the point in
many ways. I think stats can work and do have a role to play, but it's
important to recognise their limitations and see them as part of a
bigger picture. The main conclusion we came to as a result of our work
here in London was that the type of data you collect really depends on
what you're trying to do.
In terms of stats, we found that there are some simple figures that can
(and are) be used as 'headline' data, for the purposes of internal and
external advocacy and reporting. This can be collated and used pretty
consistently by different organisations like the Hub partners, and it's
relatively easy for non-technical people to get their heads round. So
for example, here at LTM we've recently become a charitable trust. As
part of this process we now use a Balanced Scorecard type approach to
report different aspects of museum performance to staff, Board and other
stakeholders. This includes headline figures like:
- Visit to the museum, broken down by ticket type
- Late openings visits
- Gift Aid donations
- Web Visits
- Admissions Income
- Retail Shop Income
- Catering Income
- Guide Books sold
I'm sure other museums take a similar approach. In this context, I think
it makes perfect sense for web visits to be included. They give an
overall indication of the level of use that the our online offering is
experiencing, just as the museum visitor numbers do for our physical
offering. Apart from anything else though, it makes it clear that the
web is an important delivery mechanism for the museum and we're willing
to be assessed on that.
I think this sends an important message. As IT professionals, we can't
on the one hand endorse the web as an essential tool for museums, but
then hold our hands up in horror when we're asked to actually be
measured on it.
Once you get into the more detailed stuff - as Jeremy says - it's more
difficult to do this sort of thing because the data is much more closely
tied to site purpose, audience, architecture and so on. So broad
comparisons are much less meaningful, but we found that the stats can
still tell us a lot about how the site is used and inform developments.
This kind of data is more generally suited to internal organisational
use rather than the external advocacy and reporting.
Evaluating the 'impact' and 'value' of web offerings does seem to be a
hot topic at the moment, and its important to see stats as part of this.
The work we've been doing has been complemented by more qualitative
audience research as well. As with the stats work, we wanted to look at
practical ways in which the London Hub could approach market research in
a more joined-up way. So we used online questionnaires, in-gallery
visitor surveys and focus groups to try and get a decent picture of who
our online users are, what they want, and try and work out whether they
are getting it or not.
I think it's important to see the statistical data in the context of
this more qualitative market research and as part of a broader package
of measures and methodologies - both web related and otherwise - that
help to establish a museum's overall performance.
We found that using different techniques to actively find out more about
our online audiences can give us an insight into users' motivations and
expectations and fill in many of the gaps left by web stats. The
research was pretty broad-brush and did seem to raise more questions for
us than it answered. However, here are some of the main findings from
the market research at the London Hub, pasted in from a summary
document:
- By far and away the main reason for visiting our museum web sites is
to plan a visit to the physical museum. This appears to be the case
across the Hub museums, and was confirmed through feedback during the
focus groups.
- Educational uses represent the exception to this rule, with teachers
and families using the sites to look for teaching materials and ideas,
online games, and material to help with homework.
- In general terms, the demographics of the Hub's online visitors
broadly reflect those of the people who visit the museums.
- Overall, the research revealed that people would like more information
about what they can expect to see and do when they visit the museum.
Information on the museum's objects, and details of events and
exhibitions also ranked highly. These all relate to visitor information,
and reinforce the view that people see the primary role of the museum's
web presence as enabling and supporting the museum visit.
- There's very little appetite for user-generated content on our web
sites. In fact it consistently scored lowest in surveys and this was
confirmed in focus groups. (We need to do more work on this, but it
seems to support Linda's recently published findings about apparent lack
of demand for 'Web 2.0' type services from HE and FE audiences).
I'm sure that others will disagree strongly with a lot of this and will
have different findings, and we've realised that we need to do more
research to understand why people are telling us this. But this work so
far has certainly raised questions in my mind about a number of areas.
Specifically, how people want us to present collections online, and
whether a wholesale approach to 'Web 2.0' type applications like UGC is
really justified without a clear purpose, audience and justification in
mind.
Hope this helps,
Dylan
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Mike Ellis
Sent: 03 July 2008 09:20
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
It's a really familiar refrain: stats don't work. What I'm not seeing is
anyone actually doing anything about it.... :-)
Mike Ellis
Professional Services Group
Eduserv
[log in to unmask]
tel: 01225 470522
mob: 07017 031522
fax: 01225 474301
www.eduserv.org.uk
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Ottevanger, Jeremy
Sent: 02 July 2008 16:00
To: [log in to unmask]
Subject: Re: Jakob Nielsen says '"Unique Visitors" Must Die'
How many pages does the average Facebook user see on an average visit?
Their own, perhaps? They can see what their mates are up to right there.
Lots of Gmaps activity also takes place in one page view. For some
sites, as Mia says, a higher number of page views probably indicates
failure rather than success, in that the user should have been able to
do what they wanted in fewer clicks. It all depends how the site works.
As ever, there isn't a statistic that is meaningful in all situations
(or indeed most).
Perhaps Nielsen has a point, and some bounce rate measure would be an
appropriate KPIs to report to DCMS or whoever. But to my mind most stats
are best suited to use by site owners/developers for their own purposes
of evaluating and improving the effectiveness of their services, knowing
the specific peculiarities of their offering. I can't think of a single
(automated)indicator that won't mean opposite things for different
sites. But I think Dylan is itching to say
something....;-)
Cheers, Jeremy
Jeremy Ottevanger
Web Developer, Museum Systems Team
Museum of London Group
46 Eagle Wharf Road
London. N1 7ED
Tel: 020 7410 2207
Fax: 020 7600 1058
Email: [log in to unmask] www.museumoflondon.org.uk
Museum of London is changing. Visit www.museumoflondon.org.uk to find
out more.
Explore how the Great Fire shaped the city
www.museumoflondon.org.uk/londonsburning
Jack the Ripper and the East End a major new exhibition at Museum in
Docklands, opens 15 May Before printing, please think about the
environment
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
Ridge, Mia
Sent: 02 July 2008 15:46
To: [log in to unmask]
Subject: Re: [MCG] Jakob Nielsen says '"Unique Visitors" Must Die'
Bilkis wrote:
> I don't know if you have come across this, but given the discussions
> that took place at the MCG in Leicester in June and the Museums Hub
> workshop on Monday, this makes some interesting reading:
>
> Reduce Bounce Rates: Fight for the Second Click:
> http://www.useit.com/alertbox/bounce-rates.html
Thanks Bilkis!
Nielsen says, "bouncers should be considered a negative statistic: the
site failed to engage them enough to entice even a second pageview."
Maybe. But could it also be the case that some users have very simple
needs (for example, is the museum open tomorrow, and until what time if
so?) and that a good 'visit this venue' page answers that need entirely?
In that case, is a 'bouncer' a positive statistic?
While it might be nice to entice these visitors to spend more time on
our site, they simply may not be looking for an online experience at
that time. He does at least say "it's important to realize that there's
no such thing as a single bounce rate" but then he focuses on referrer
source rather than user requirements.
Nielsen's always good fodder for debate. Am I the only one who can
never quite agree with him?
cheers, Mia
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
Visit Last Stop, an art exhibition of Ralf Obergfell's beautiful
photographs that celebrate the Routemaster bus. The exhibition runs
until 27 July and in London Transport Museum's CBS Outdoor Gallery
www.ltmuseum.co.uk
_______________________________________________________________________
This email and any attachments are confidential and intended solely for
the use of the addressee. If you have received this in error please
notify [log in to unmask] and delete immediately. Any use,
disclosure, forwarding, printing or copying of this email is strictly
prohibited. London Transport Museum (LTM) hereby excludes any warranty
and any liability as to the quality or accuracy of the contents of this
email and any attached transmitted files.
LTM may monitor all email communications through its internal and
external networks.
London Transport Museum Limited is a charitable company registered in
England and Wales (company number 6495761 registered charity number
1123122) whose registered office is at Windsor House, 42-50 Victoria
Street, London SW1H 0TL. London Transport Museum Limited is a subsidiary
of Transport for London and as such is a company controlled by a local
authority for the purposes of Part V Local Government and Housing Act
1989. VAT No. 756 2770 08
Copyright in this email and attachments belongs to TFL.
This email and attachments have been swept for the presence of computer
viruses and LTM accepts no responsibility for any virus imported with
this document.
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
This message has been scanned for malware by WebSense Mailcontrol for
the National Maritime Museum, Greenwich
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The service is
powered by MessageLabs. For more information on a proactive anti-virus
service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________
Visit Last Stop, an art exhibition of Ralf Obergfell's beautiful
photographs that celebrate the Routemaster bus. The exhibition runs
until 27 July and in London Transport Museum's CBS Outdoor Gallery
www.ltmuseum.co.uk
_______________________________________________________________________
This email and any attachments are confidential and intended solely for
the use of the addressee. If you have received this in error please
notify [log in to unmask] and delete immediately. Any use,
disclosure, forwarding, printing or copying of this email is strictly
prohibited. London Transport Museum (LTM) hereby excludes any warranty
and any liability as to the quality or accuracy of the contents of this
email and any attached transmitted files.
LTM may monitor all email communications through its internal and
external networks.
London Transport Museum Limited is a charitable company registered in
England and Wales (company number 6495761 registered charity number
1123122) whose registered office is at Windsor House, 42-50 Victoria
Street, London SW1H 0TL. London Transport Museum Limited is a subsidiary
of Transport for London and as such is a company controlled by a local
authority for the purposes of Part V Local Government and Housing Act
1989. VAT No. 756 2770 08
Copyright in this email and attachments belongs to TFL.
This email and attachments have been swept for the presence of computer
viruses and LTM accepts no responsibility for any virus imported with
this document.
**************************************************
For mcg information and to manage your subscription to the list, visit
the website at http://www.museumscomputergroup.org.uk
**************************************************
**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************
|
