RE: evaluating museum websites
Sounds interesting.
As with all evaluations, the main questions are: What do you want to find
out? Why do you want to find it out? What will you do with the information
you gather? I (and probably others on this list) would be very interested
to know your responses to those 3 questions and the objectives of the
evaluation.
Our early evaluation of the Science Museum website is at
http://www.nmsi.ac.uk/eval/index.htm, as you probably know. The information
is a bit dated now. For example, we are currently running at an annualised
rate approaching 1 million online 'visits' per annum (as opposed to page
hits or even file hits, as some institutions report, which are of course
far higher) and about a tenth of that to the specifically 'education'
pages. The whole website, including how we construe the 'education' area is
being redesigned based on this and subsequent evaluation.
More specifically, we are currently engaged in a wide-ranging evaluation
with the Exploratorium (San Francisco) of 3 online resources produced
specifically to support science learning (two produced by the Exploratorium
and one by us in partnership with the Franklin Institute in Philadelphia).
Indeed, we are doing this not just to generate evaluative data but also to
develop evaluation methodologies per se on the web. The approach is
three-fold:
1. Log analysis, using software real-time and offline to analyse log files.
This gives us information on URLs accessed, ISPs acessing, referral paths
etc etc. An example of some data that can be obtained can be seen at
http://www.nmsi.ac.uk/usage/monthly/november99.html for the whole Science
Museum site. The Exploratorium in particular is exploring the depth of
information we can extract from this sort of data.
2. Online questionnaires. You can see an example if you click (before 17
December!) on http://www.nmsi.ac.uk/flights/. This allows quantitative data
to be collected quite quickly (between 200 and 500 responses on each of the
3 sites in a period of 1 month, which is probably a response rate of about
5%) although with problems of validity and sampling. Interestingly, the
system allows the contributor to see his/her answers in the context of
everyone else's (after submitting them). We are getting e-mails from
volunteers to follow up more qualitative/judgemental aspects (how well the
site met their needs etc etc). The present Cornucopia evaluation is not
dissimilar (perhaps since we recommended the evaluator to the MGC!)
3. Direct observation and personal discussion. Classroom observation and
personal follow-up by e-mail/phone.
Each of these methods gives different data, from the very quantitative to
the very qualitative, and we believe that such a combined methodology is
probably needed to give a balanced picture of what is actually happening.
An interesting published evaluation (not of a museum website) is the
evaluation of the 'Why Files', in Public Understanding of Science, vol 7,
285-311 (1998). I believe that the Marco Polo research project
(MCIWorldCom) may be worth contacting and that John Falk at the Institute
for Learning Innovation http://www.ilinet.org/ is embarking on an
evaluation of US science museum websites.
Roland
At 17:53 09/12/99 -0000, you wrote:
>Posted on behalf of my colleague
>PLEASE EXCUSE CROSS POSTINGS
>
>In the new year, mda will be conducting an evaluation of museum
>websites on the UK National Grid for Learning. Two aspects of the
>study include:
>
>**the identification of existing evaluations of museum websites in
>the UK and abroad
>
>** the identification of best practice exemplars in educational
>websites outside the museum sector.
>
>Could anyone recommend existing evaluations or some best
>practice exemplars? Many thanks for any assistance.
>
>Katherine Futers
>
>*--------------------------------------------*
>Katherine Futers - mda Standards Support Officer
>Tel: +44 1223 315760 Fax: +44 1223 362521
>[log in to unmask] www.mda.org.uk
>
>
>Alan Bentley
>mda Outreach Manager
>01943 464997
>Advice Point 01223 366097
>www.mda.org.uk
>
>
>
Dr Roland Jackson, Head of Education and Programmes
Science Museum, London SW7 2DD
http://www.nmsi.ac.uk/education/
Phone: 44 (0)207 942 4710
Fax: 44 (0)207 942 4712
If you cannot access these from outside the UK,
dial: 44 171 942 4710 (fax 4712)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
