 |
 |
Hi Bridget, I have been mulling over your thoughts during Christmas (by way of pleasant relief from family gatherings and rubbish TV ;-). I agree that it's harder to measure the success of your strategies if the learning outcomes are loosely expressed. But it seems to me that a goal such as encouraging open-ended discovery need not necessarily result in loosely expressed outcomes. So, taking your example "develop skills in comparison of objects", as designers we might usefully think about comparison skills: what are they? How are they developed? What are the common barriers or pitfalls to developing such skills? Then maybe we could begin to develop a learning activity which encourages people to make comparisons and which helps them to develop comparative skills. This would be an open ended activity in the sense that the learners would be free to choose what they compared but the learning outcome is quite specific and quite easy to measure therefore. So I am not sure that the opposite of measurable learning activities is open ended discovery. Putting that another way, I think it is possible to define specific learning outcomes for open ended discovery activities providing one accepts a shift in focus from learning content to the learning process. For example, we might devise a learning activity in which learning comparison skills in the context of medieval surgical implements is more important than learning about medieval surgical implements themselves. Such an activity would be quite exploratory and open ended in one sense and yet focussed and precise in another. A consequence of loosely expressed learning outcomes is that one has to accept that the actual learning emerges from the interaction between learner and learning experience without being able to predict it very precisely in advance. But from a design perspective this approach has a major flaw. Under this regime a good learning activity design will be one that engenders a lot of positive responses. But until the design has undergone extensive testing, with large numbers of participants, its overall impact on users will not be apparent. There will not be enough interactions for the overall pattern to emerge. Given the tight budgetary and time constraints that surround most web development projects, lengthy field trials are usually out of the question during the development phase. This effectively leaves the designer without a tool for testing the effectiveness of the design until most of the investment has been made. So while "emergent" learning outcomes are useful for retrospective assessment of overall impact I think they are of little use for supporting a developmental user-centred design approach. We need "predictive" measures to assess performance iteratively against clearly specified intended learning outcomes during the design development process. I hope the above exmple explins why I think that clearly specified learning outcomes need not be didactic.. I think a couple of good real life examples are: "AccessArt" (http://www.accessart.org.uk/index.php), an online learning resource for "pupils, students, and lifelong learners as well as teachers, gallery educators and artists" is a good example of how online resources can be used to support clear learning objectives without prescribing the precise outcomes. The online drawing workshop for 16+ learners encourages and supports exploration of understanding of seeing and drawing. Another good example is the set of learning "challenges" in the British Museum's Ancient Egypt Interactive learning web site (see for example the temple challenge at http://www.ancientegypt.co.uk/temples/home.html. In both cases the activities provided carefully "scaffold" the learning experience. Id be interested in responses to these ideas and suggestions of further examples. Stephen -----Original Message----- From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of Bridget McKenzie Sent: 01 December 2006 11:16 To: [log in to unmask] Subject: Re: Learning design Hello This is an interjection, I hope a little bit relevant, just to raise a thought about instructional learning design in a context that favours constructivist learning. Museum learning is about open-ended discovery, so learning outcomes will often be loosely expressed e.g. 'develop skills in comparison of objects', 'develop tolerance of other cultures'. Therefore it's harder to measure the success of your strategies. You may be able to discern changes in learners if you evaluate a sustained project, including museum visits, experimental activities and use of the linked web resources. Or formative evaluation can work if the resources are seen being used in a discovery learning context. If you haven't got time/money to do this you can only guess that open-ended tools would be successful in reaching the desired outcomes if people seem to like them and are using them. It could be tempting to design more and more instructional or didactic web tools, simply because you know you're supposed to evaluate them and it's easier to assess whether users 'got it' or not. Much better to support an expansion of e-learning as dialogue, so that discovery happens online in interactions between people with questions, people with insights and cultural artefacts. The proof of that kind of pudding is definitely in the eating, and not only divined in an expensive evaluation. Bridget ----- Original Message ----- From: "Joe Cutting" <[log in to unmask]> To: <[log in to unmask]> Sent: Friday, December 01, 2006 10:34 AM Subject: Learning design > Stephen, > >> > Hi, does anyone have experience of using user-centred design to create > either exhibitions/displays and/or web sites please? I would be really > interested to find out how useful you found this approach. > >> > Dear colleagues, have you built learning activities into your Museum > web site? Was it important for you to be able to demonstrate the > effectiveness of your designs? Did you test the designs during > development, and/or after they were completed? > >> > > This is pretty much established best practice and becoming more > widespread > as > funders are demanding summative evaluation on exhibitions and projects > after completion. > They're also demanding that institutions specify objectives and audiences > upfront with their > funding applications - although I would agree that these can easily get > lost along the way. > It tends to be much easier to do formative evaluation on exhibits and > micro-sites than > whole institution sites due to the difficulty of specifying audiences and > objectives for a whole institution site. > The amount of evaluation done tends to depend on the attitudes and > resources of the institution but if you're looking into this > your main issues are going to be > Institutions don't tend to release the results of evaluation unless > they're completely positive. This is for a variety of political > reasons connected with the press and funders. > There's also very little movement by institutions to publish anything > about their development methods and > experiences - in general its just not seen as a priority. There are > honourable exceptions like the Tate's Multimedia tour page > http://www.tate.org.uk/modern/multimediatour/re_keyfindings.htm. There's > also bits and pieces by Ben Gammon scattered around the web > like this one http://ukupa.org.uk/events/presentations/science_museum.pdf > (google "Ben Gammon" for more). > > Given that there's a lot of work going on, your problem is unlikely to > be > finding examples - its much more likely to be > getting people to tell you about them in any detail - particularly if user > testing wasn't done or produced "bad" results. > My recommendation would be either to get hold of a list of recent projects > from a major funder like HLF or Wellcome and then > go and interview a selection of the project managers or pick one project > which did use a lot of user testing and study it in detail. > To give you an idea, one project I worked on last year did around 20 > formative evaluation studies in 4 months of development so there's a > lot to get your teeth into. > > All the best with your project > > Joe > > > > > Joe Cutting > Computer exhibits and installations > www.joecutting.com > The Fishergate Centre, 4 Fishergate, York, YO10 4FB > 01904 624681 > > As of 30th October 2006 I have a new office so > please note my new address and phone number > ************************************************** > For mcg information and to manage your subscription to the list, visit > the > website at http://www.museumscomputergroup.org.uk > ************************************************** > > ************************************************** For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk ************************************************** ************************************************** For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk **************************************************