Hi Gunter, I read your considered and balanced blog post when it popped up via my trusty API-monitor (Google Reader).
You make a very good point below. I'm mainly about the mouth and not the trousers, since we don't (as yet) offer our collections through an API here (though we do offer some other stuff under the radar). But I'm one of those very keen on the idea of APIs; however this very concern of scalability is one of the reasons why I'm also enthusiastic about pushing from both the bottom up (museums making their stuff available) and the top down (projects to start clumping content together and reduce the number of APIs that need to be built or pointed at). There seem to me to be different aspects to the question, though.
Firstly, there's the simple idea of offering an API for developers to work with directly, which will be fine for developers happy to develop stuff using the collections of single museums.
Secondly there's the federated search thing, which I guess will work for two to a few-ish institutions before getting unwieldy and getting swamped by problems with ranking, unwieldy datasets and so on. I think you're right that this is what NMOLP has done via OpenSearch (I'm sure I'll be set straight if this is wrong). I guess it wouldn't be too hard to build an API on top of a set of OpenSearch end-points so that 3rd party developers could do something with them all together. Without it then they're still required to query one at a time with however many syntaxes there are (but hopefully at least one response format).
Then there's the idea of searching all the collections that are out there, which might be just about possible at the moment via the small number of collections APIs of which I, at least, am aware. But as you point out, it surely won't scale far for live, distributed search. If we're dreaming of Total World Collection Search (or London, UK, or US) a plethora of APIs could still be part of a version of the picture, though, because of their potential role in aggregation for centralised search. The aggregation could come by various means, of course (one of the big search engines having a crack at our structured data, or perhaps a mega-harvester run by press-ganged conference delegates at MW 2010) but I'm sure standardisation of some sort would help!
It seems to me, following this, that the obvious approach has to be to both to encourage APIs at all museums, giving us (hopefully) some easyish payoffs, whilst at the same time working towards (possibly several kinds of) centralised search. APIs have other rewards too, including for an institution's own developers or for hooking up with software companies one-to-one (perhaps Martyn Farrows would like to give an example here, I'm too shy!)
Hopefully with a few more APIs in effect out there we'll see some sort of path beaten towards standardisation based on the standards that are already out there (thanks to you, for one). My htought for the day, though, is to wonder whether there might be some mileage in talking to a company like GnipCentral, who do for social software companies what we might find that we need in the cultural heritage sector by brokering requests to many and various APIs, standardising the request syntax and the response format and performing the search across multiple providers simultaneously. Though I may have misunderstood the model... Perhaps it won't be necessary, but it could be one way to both make it possible to query multiple APIs with one request, and to standardise the formats each way.
What I'm dying to see is something cool done with the APIs that exist. I know that it's possible with the contents of just a single museum - we all do it all the time with our own stuff - so I'm certain there must be developers out there who'll work with the Powerhouse/National Maritime/Brooklyn/AN Other Museums' APIs, I just want to see it happen. I'm sure they can't all be waiting for Europeana or some such to offer an API of loads of collections, so perhaps they'll show up soon, but it's a little disappointing there aren't any examples yet (that I know of). Not that it stops me wanting to get our own API ASAP!
Cheers, Jeremy
Jeremy Ottevanger
Web Developer, Museum Systems Team
Museum of London
46 Eagle Wharf Road
London. N1 7ED
Tel: 020 7410 2207
Fax: 020 7600 1058
Email: [log in to unmask]
www.museumoflondon.org.uk
Museum of London is changing. Visit www.museumoflondon.org.uk to find out more.
Explore how the Great Fire shaped the city www.museumoflondon.org.uk/londonsburning
Before printing, please think about the environment
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of Waibel,Guenter
Sent: 06 March 2009 17:16
To: [log in to unmask]
Subject: [MCG] 'Creative Spaces' & APIs
"To get live, we need APIs; they are, of course, the way forward as Richard Light, Mia and Mike Ellis all say. API's need standards, and the Collections Trust work with DACS and towards new BSI data standards is excellent."
I've noted the emphasis on APIs on this list not just in discussions around Creative Spaces, but almost every time the topic of sharing data comes up on this list.
To my mind, APIs are a wonderful thing because they allow for the kinds of mash-ups which create something utterly new out of two (or more) streams of data. They are great for plugging one institution to another (or a couple of other) data sources. Moreover, they send a powerful signal about an institutions commitment to openness and sharing. For that reason alone I would want all 2,500 museums in the UK, and all the 17,500 museums in the US for that matter, to have one, and I rejoice the news that the Brooklyn Museum just launched theirs.
On the other hand, if your goal is to create a collective collection of a significant subset of those 2,500 or 17,500 or whatever other figure you'd like to toss around, how useful are those APIs? (This is a genuine question, not a rhetorical one - I aim to expand my horizons!)
This may be where the conversation veers into the distributed vs centralized data store arena. As I understand it, APIs allow you to execute a distributed search, which may scale to the 9 national museums in Creative Spaces (I assume they use an Open Search API), but they usually don't allow you to take possession of the data. Does this approach scale to even all of the London museums (241 as very empirically counted on http://www.londonnet.co.uk/ln/guide/about/museums.html)?
What's the mechanism you envision for a collective collection beyond the 9 national museums to flow together? I'd be curious to hear how folks on the list think the API approach will scale...
Günter
P.S.: I wrote a blog about Creative Spaces as well, in case anybody cares to hear more opinion on the matter: http://hangingtogether.org/?p=636.
-----Original Message-----
From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of Jon Pratty
Sent: Thursday, March 05, 2009 5:04 PM
To: [log in to unmask]
Subject: Re 'Creative Spaces' - reposted
[An MCG person asked me to repost this in full, to keep things on the list, rather than off list on blogs...fair enough.]
I think this has been one of the best ideas threads we've had for a long time. Yes, it's been passionate, and that does indeed get people thinking, and firing up laptops in reply.
I think voices who advocated tact in the exchange (Nick Poole, myself via Twitter, and others) did so because we're already engaged in working with museum people all over the regions, not always in the most glamorous places; we're all working for peanuts, doing about ten million things at once, including managing that puzzling interface between museum directors and the onward march of digital technology...and so, not surprisingly, people are sensitive about projects they put lots of time into.
To me, that's one of the reasons there needs to be some tact in the way we review each other's projects; if you'd been behind the scenes of projects like NMOLP you'll have seen the sort of passion it arouses. I saw people (like Terry and Carolyn, and the teams of writers like Rachel and Rowena L) working like absolute stink to get the project done, and ploughing through all sorts of effluent to manage relationships across and through the project. Those who stuck the course deserve medals.
I think the emotionality was also caused by the big fees funding the project - big ticket jobs like this cause a certain amount of envy, and that too, leads to comment that doesn't always please. One gets a picture sometimes of vast (National museum) battleships manoevring around a smallish patch of sea, each one guarding it's own flanks, carefully manning the bulwarks, in case a stray shell cuts the rigging, or someone jumps ship.
Best things coming out of the Creative Spaces debate for me?
A) The emerging discussion about 'the plumbing' (nice metaphor from Paul
Walk) being the first job to tackle when working on these complex cross-collection projects. Yep. Of course the data scheme underneath is critical. The website (if there needs to be one!) should be sat on top of the database well down the line of projects like this. Sorting out how you get the data, on what (copyright) terms it's given, and how the data is related and relational is the first key task.
B) Another plus has been the thread (from Frankie, Mike Ellis, Kate Fernie and others) about how social nets work in reality, and why you might want, or not want, to play for a while, culturally. This stuff needs to be explored more. Already one or two culture orgs have made abortive attempts at trying to get things going, and they mostly failed 'cause they didn't spot that sites get massive visits when they get the bigger publishing picture about mass audiences, massive budgets and massive human resources and tech support. That insight mainly comes from expertise that's mostly, at the moment, outside the museum sector.
C) We're starting to get the idea too, that the cool culture venture we dream about here might not be a big project, but smaller-scale, evolutionary, more experimental, more informal. There aren't any more big pots of money (like ISB) now for this kind of work. We've got to be coming up with sustainable and scaleable ideas, so some wisdom about the scope and depth of project concepts needs to be found when ideas are still at the back of an envelope stage.
My interests in this?
I've long evangelised (and written about, in 2005) 'the inside out web museum.' At my former workplace, my enthusiasm for a more 'datacentric'
publishing offer drove quite a bit of our re-design thinking, though the final realisation of those ideas is still in the pipeline. But look outwards at recent tech trends and think about how near we are to some sort of breakthrough. We're wrong to expect a 'killer app,' but continuous development and playful experimentation like the (Mike Ellis) Mashed Museum sessions at UKMW08 will get us nearer to some sort of nirvana.
Where to go now, post-Creative Spaces? We ALL need and deserve (as a sector, everywhere) access to data channels that come to us, and do the necessary spidering and data mining to make the most of all the content we might choose to expose and share. And, importantly, let it be live data exchange, not a day old, or a week old, or some such OAI-harvested old hat.
The next culture web must be live; after all we have come to expect that through our day-to-day fun with Twitter and FB.
To get live, we need APIs; they are, of course, the way forward as Richard Light, Mia and Mike Ellis all say. API's need standards, and the Collections Trust work with DACS and towards new BSI data standards is excellent.
Sharing freely and offering culture content to others for their own use opens doors to commerce and business models, so some movement there gets us towards a more commercially-geared culture web.
And finally? The success of #hashtags on Twitter (check #fakeanimalfacts) proves people can come up with vocabs and impromptu syntax that bind humour, culture, conferences and news together using simple XML. My research interest now is to see how we can map some simple #-like tagging and vocab structures (and maybe the National Curriculum) so we can have cultural fun without needing to build big and expensive portalised web projects...
JP
**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************
**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************
**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************
|
