Hi Jeremy,
I'm intrigued by this potential use of OAI-PMH as a gateway to  
identified resources. The architect in me is screaming "NO!"..... but  
the pragmatist sees the value in what you say about the fact that this  
is already deployed in a significant number of cases.

I think OpenSearch is a very limited solution - but it seems to  
perform its limited function pretty well.

Picking up on your point about the aggregators developing better  
search interfaces *and* exposing the underlying resources with stable  
URLs: I'm working on a project in the HE sector looking at something  
very similar:

  - harvesting metadata records using OAI-PMH into a simple  
aggregation 'container'
  - providing a search interface (machine and UI) via a fantastic  
RESTful Lucene implementation called SOLR (check it out if you don't  
know it - it's really good open source software)
  - making the harvested source records accessible via RESTful URLs

I realise this does not move us towards the ideal of the linked open  
data, or web of resources. I mention this only in the sense that I  
think your call to the aggregators to step up in this way might be  
something the community can pursue. On a related note, I set out to  
convince myself that all of the above could be achieved with mostly  
'off-the-shelf' components, with a little development to glue them  
together. I think I've been fairly successful in this respect -  
certainly I'm using completely open source and freely available code,  
libraries etc.

Happy to share this - perhaps offline, as I have already helped to  
digress this thread already!!

Cheers,

Paul


On 4 Jun 2008, at 11:13, Ottevanger, Jeremy wrote:
> Hi Paul
>
> Thanks for your thoughts, you're basically bang on target as usual,  
> but
> let me try to clarify. It was of course me that injected OAI-PMH into
> the discussion, which betrays both my ignorance of the technology  
> and my
> desperation to find some sort of (public) machine-facing data  
> sources on
> museum objects out there on the web. As you say, it's not a technology
> designed to act as a general search interface (more's the pity), but  
> the
> reason I was interested in it in relation to this discussion is that  
> it
> does offer the facility for those museums that employ it (quite a  
> few, I
> think) to offer an object-specific URL to a view of the data (rather
> than a human UI on top of the data). It's not be what OAI-PMH is
> intended for, but it seems an obvious enough extension of its role,  
> and
> one that could be productively combined with PURLs. And of course it
> does use HTTP, XML and DC... There are doubtless better  
> implementations
> of these technologies/standards either available or possible for
> expressing museum data, but it's the existence of OAI services out  
> there
> right now that made me look at it as a no-cost solution for those
> already using such gateways.
>
> There are of course better alternatives to OAI-PMH and co-opting OAI
> gateways for what I'd really like to see. After all, whilst you can
> point at individual records with OAI, you can't search the gateway
> usefully (except for aggregation purposes). But the only alternative
> I've seen used my museums in the wild has its drawbacks, too:  
> OpenSearch
> is a fine way to federate search, but it's not AFAIK intended (or  
> used)
> to return specific objects. So whilst you could load up your search
> results with lots of semantic goodness, as Jim has done, it can't
> operate as a "home URL" for any given item (someone please correct  
> me if
> I'm wrong).
>
> So there are better things that could be done than what's out there  
> now,
> but between them OAI-PMH and OpenSearch take us two different steps in
> the right direction. Perhaps together they could be really useful, or
> perhaps those aggregators that have harvested from OAI gateways and
> built the search facilities on top of them that make them useful to  
> the
> public could expose both the search results and the item end-points in
> machine-friendly ways that would kill both of those birds together.  
> And
> of course the data format then could be whatever you wanted and OAI- 
> PMH
> could go back into the shadows...
>
> Cheers, Jeremy
>
>
>
>
> Jeremy Ottevanger
> Web Developer, Museum Systems Team
> Museum of London Group
> 46 Eagle Wharf Road
> London. N1 7ED
> Tel: 020 7410 2207
> Fax: 020 7600 1058
> Email: [log in to unmask]
> www.museumoflondon.org.uk
> Museum of London is changing. Visit www.museumoflondon.org.uk to  
> find out more.
> Explore how the Great Fire shaped the city www.museumoflondon.org.uk/londonsburning
> Jack the Ripper and the East End a major new exhibition at Museum in  
> Docklands, opens 15 May
> Before printing, please think about the environment
>
>
>
> -----Original Message-----
> From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
> Paul Walk
> Sent: 04 June 2008 10:26
> To: [log in to unmask]
> Subject: Re: [MCG] Linking Open Data
>
> Hi,
> This has been an interesting thread so far - I'd just like to chip in
> with some quick observations. First though, a question:
>
> Do we have a problem with broken links to open data? I don't see how  
> if,
> as Jeremy says, very little data has been made openly linkable to
> date....
>
> The community *should* care about persistent identifiers - they are
> closely coupled with issues of preservation after all. But, based on
> experience in other sectors, this can also take ages to be implemented
> and adopted by the community. In fact, it hasn't always been adopted
> where one might have expected it to. It would be a shame to delay good
> efforts to link open data while we waited for this to become a  
> reality.
>
> Regarding OAI-PMH:
> I'm a little puzzled by this sub-thread. OAI-PMH is for harvesting.
> It's not designed for the sort of 'interoperation' we might aspire to
> now - and which a few brave pioneers have shown to be possible.  
> Where a
> central service wants to provide services based on aggregations of  
> data
> from individual, distributed collections, then OAI-PMH is a reasonable
> candidate mechanism. Where individual collections want to  
> participate in
> the resource-oriented web at large, we need different mechanisms.
> Fortunately some good protocols and standards already
> exist: HTTP, XML, RSS, RDF, DC.....
>
> Cheers,
>
> Paul
>
>
> On 3 Jun 2008, at 14:46, Ottevanger, Jeremy wrote:
>> Hi Nick,
>>
>> You're right, I shouldn't be too down-hearted. I was a bit off-topic
>> anyway, on account of having spent the previous day looking for any
>> machine-friendly object records (with mashup day in mind). Mia also
>> pointed out that I was mixing up two issues a bit - PURLs and APIs.
>> There may be lots more out there than I found anyway, but if it's not
>> findable it's not much use!
>>
>> You're right that there's a challenge in making the case, but really
>> that's not even really necessary - surely there's not much to do to
>> make your OAI interface available to more than just harvesters, you
>> just have to tell people how to find it and use it. Off-list contacts
>> suggest this is the case. It's probably easier to do this even than  
>> to
>
>> build the HTML version of each record.
>>
>> Still, there's a case to be made and hopefully we're as a group
>> starting to build it. I'm very much looking forward to UKMW to see
>> where we go with this, and to mashup day before that.
>>
>> Very good to hear that your plans are progressing re URIs etc. You've
>> got my support, for what it's worth!
>>
>> Cheers, Jeremy
>>
>>
>>
>>
>> Jeremy Ottevanger
>> Web Developer, Museum Systems Team
>> Museum of London Group
>> 46 Eagle Wharf Road
>> London. N1 7ED
>> Tel: 020 7410 2207
>> Fax: 020 7600 1058
>> Email: [log in to unmask] www.museumoflondon.org.uk
>> Museum of London is changing. Visit www.museumoflondon.org.uk to find
>> out more.
>> Explore how the Great Fire shaped the city
>> www.museumoflondon.org.uk/londonsburning
>> Jack the Ripper and the East End a major new exhibition at Museum in
>> Docklands, opens 15 May Before printing, please think about the
>> environment
>>
>>
>>
>> -----Original Message-----
>> From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
>> Nick Poole
>> Sent: 02 June 2008 21:34
>> To: [log in to unmask]
>> Subject: Re: [MCG] Linking Open Data
>>
>> Hi Jeremy,
>>
>> Thanks for that. I think it's important not to be downhearted about
>> the number of museum datasets not openly available to the network. I
>> know the principles have been axiomatic in Techworld for a while, but
>> they're very, very far from it in Humanworld (much less in  
>> ManagerLand
>
>> but I'm getting carried away...)
>>
>> I think there are a combination of factors at play here:
>>
>> It's all a bit new
>> It's not yet plug'n'play
>> It's a bit scary
>> It's untested in the cultural world
>> I don't have to (it's not in Accreditation...)
>>
>> I maintain that we're on a growth curve and we're still rounding off
>> the 'digitise everything' phase in anticipation of the 'share
>> everything'
>> phase (in a sensible, managed way, I hope). I'm sure others will get
>> back to the list with examples of opened-up datasets, and I think  
>> they
>
>> (and Seb and Jim) deserve a round of applause for being early-
>> adopters.
>> It's going to take, in reality, 2-3 years before this kind of
>> behaviour becomes a norm across an industry which spans, lest we
>> forget, some very very small players. It'll also probably take some
>> enlightened funder to invest not in digitising new stuff, but in
>> opening up the old stuff.
>>
>> I don't think it's a lack of imagination - look at it the other way
>> round: Why should they? There's no impetus to do it, other than the
>> tech community promising some pretty loosely-defined long-term
>> benefits. To drag out the old maxim 'what gets measured gets managed'
>> - nobody's looking, so why should the majority of museums bother
>> (particularly when there are so many other things which *are*  
>> measured
>
>> these days). I'll bet that the majority who set up OAI targets did so
>> in response to some bit of project funding or other and that their
>> managers may not even know that they *are* exposing the data.
>>
>> Now, if we could get Digital delivery into Accreditation, or, heaven
>> forbid, Local Authority improvement indicators, we'd be laughing...
>>
>> On the subject of persistent identifiers/authoritative URI's -
>> absolutely. The Collections Trust is actually leading on a Work
>> Package under the EDL/Europeana to scope and implement this. We  
>> have a
>
>> number of persistent identifier schemes for institutions (including
>> libraries and archives), but it is still our fundamental aim in life
>> to enable the formation of an expression as:
>>
>> http://www.InstitutionURI/CollectionURI/ObjectURI (don't click on it,
>> it goes nowhere) - where 'ObjectURI' is a persistent identification
>> scheme that is applicable (and applied) to physical and digital
>> objects.
>>
>> This is by no means a simple challenge - not least because you're not
>> starting from a clean slate - but we've got a bit of cash and the
>> impetus to do it (everything else, in the sense of non-semantic,
>> current-gen interoperability, seems to depend on it to some degree).
>> Later in the year, we'll be announcing it as a bit of work and
>> enlisting some help from the Group and the International community to
>> look at options. It'd be even better if we got some grassroots  
>> support
>
>> for doing it (hint hint).
>>
>> Cheers,
>>
>> Nick
>>
>>
>>
>>
>>
>> Nick Poole
>> Chief Executive
>> Collections Trust
>>
>> www.collectionstrust.org.uk
>> www.collectionslink.org.uk
>> www.cuturalpropertyadvice.gov.uk
>>
>>
>> Tel:  01223 316028
>> Fax:  01223 364658
>>
>> Company Registration No: 1300565
>> Reg. Office: 22 Hills Road, Cambridge, CB2 1JP.
>>
>> The Collections Trust believes that everybody, everywhere should have
>> the right to access and benefit from cultural collections. Our aim is
>> to develop programmes and standards which help connect people and
>> culture.
>>
>> The Collections Trust was launched from its predecessor body, the  
>> MDA,
>
>> in March 2008.
>>
>> -----Original Message-----
>> From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
>> Ottevanger, Jeremy
>> Sent: 02 June 2008 17:49
>> To: [log in to unmask]
>> Subject: Re: Linking Open Data
>>
>> Hi Richard,
>>
>> "It seems to me that museums should get in there and publish
>> authoritative URIs for their own collection objects."
>>
>> I'd agree, but I'm nursing some disappointment with reality at the
>> moment (yet again - it's the curse of the optimist). Having never
>> really checked all that thoroughly I was sure that there must be
>> plenty of museums out there that do actually publish their data in
>> some machine processable way. After all, I reasoned, we know that
>> plenty of them have used OAI-PMH for data aggregation projects, and
>> institutions like the Getty must (you'd think) have their stuff
>> accessible in CDWA [Lite]. A corrolary of this would hopefully be  
>> nice
>
>> stable URLs for each object.
>>
>> My trawl yesterday was disappointing. I should make honourable  
>> mention
>
>> of Seb at Powerhouse and Jim at NMM, both of whom have made available
>> an OpenSearch interface [1]. Besides that, I could find no
>> institutions actually publicising/promoting programmatic access to
>> their collection data (i.e. in some way other than as web pages). It
>> may be that there are feeds out there that just aren't promoted on  
>> the
>
>> websites, so my Googling doesn't turn them up (this is the case with
>> Powerhouse and NMM, too, at present, but I knew of their existence).
>> If so perhaps I'm missing scores of other museums that have also
>> opened up their data. I have to say I was rather surprised. I did  
>> find
>
>> examples of OAI-PMH use by museums stretching back to 2002, but it's
>> always been about aggregation and no-one has taken the step of
>> exposing the resources to the wider web. I guess it's not the same as
>> saying that there aren't still some relatively stable URLs out there
>> (any PURLs, anyone?) but it's worse than I thought. I'm dearly  
>> hoping,
>
>> though, that someone will pipe up with a few examples that I'm  
>> unaware
>
>> of.
>>
>> So, not knowing an awful lot about OAI gateways, I'm wondering, are
>> there technical reasons why so few (if any) of the institutions that
>> have these gateways have made the interface public, or is it a matter
>> of politics, or perhaps just not making the small mental leap from
>> using the gateway for aggregation to using it for direct
>> dissemination?
>>
>> On the question of authoritative URIs, is there a job for Collections
>> Trust here? I would really like to see work on a mechanism for
>> standardising/hosting stable URLs, and I can imagine them organising
>> it.
>> Certainly I'd have thought it needs some sort of centralised effort  
>> at
>
>> least to kick start things, if not control them in the longer term.
>>
>> Cheers, Jeremy
>>
>>
>> [1] for convenience, here are links to searches on each of these,
>> cribbed from Seb and Jim's earlier MCG posts. Note that the objects
>> themselves don't have an end-point URI other than their web page, but
>> at least they're nice metadata rich pages.
>>
>> http://www.powerhousemuseum.com/collection/database/opensearch/ 
>> search.
>> ph
>> p?s=compass&start=1&show=50
>>
>> http://www.nmm.ac.uk/collections/requestHandlers/doQuickSearch.cfm? 
>> sea
>> rc
>> hterm=midshipman&startrow=&format=rss
>>
>>
>>
>>
>>
>> Jeremy Ottevanger
>> Web Developer, Museum Systems Team
>> Museum of London Group
>> 46 Eagle Wharf Road
>> London. N1 7ED
>> Tel: 020 7410 2207
>> Fax: 020 7600 1058
>> Email: [log in to unmask] www.museumoflondon.org.uk
>> Museum of London is changing. Visit www.museumoflondon.org.uk to find
>> out more.
>> Explore how the Great Fire shaped the city
>> www.museumoflondon.org.uk/londonsburning
>> Jack the Ripper and the East End a major new exhibition at Museum in
>> Docklands, opens 15 May Before printing, please think about the
>> environment
>>
>>
>>
>> -----Original Message-----
>> From: Museums Computer Group [mailto:[log in to unmask]] On Behalf Of
>> Richard Light
>> Sent: 02 June 2008 10:17
>> To: [log in to unmask]
>> Subject: [MCG] Linking Open Data
>>
>> Hi,
>>
>> The Linking Open Data initiative:
>>
>> http://en.wikipedia.org/wiki/Linked_Data
>> http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/ 
>> LinkingOpe
>> nD
>> a
>> ta
>>
>> looks interesting.  Having a quick play with dbPedia:
>>
>> http://dbpedia.org/About
>>
>> shows the possibilities: real "Semantic Web" query support at last!
>> ("find all musicians born in Berlin before 1900", that sort of thing)
>>
>> Has anyone taken the Triplification Challenge?  It seems to me that
>> museums should get in there and publish authoritative URIs for their
>> own collection objects.
>>
>> Richard
>> --
>> Richard Light
>> XML/XSLT and Museum Information Consultancy [log in to unmask]
>>
>> **************************************************
>> For mcg information and to manage your subscription to the list,  
>> visit
>
>> the website at http://www.museumscomputergroup.org.uk
>> **************************************************
>>
>> **************************************************
>> For mcg information and to manage your subscription to the list,  
>> visit
>
>> the website at http://www.museumscomputergroup.org.uk
>> **************************************************
>>
>> **************************************************
>> For mcg information and to manage your subscription to the list,  
>> visit
>
>> the website at http://www.museumscomputergroup.org.uk
>> **************************************************
>>
>> **************************************************
>> For mcg information and to manage your subscription to the list,  
>> visit
>
>> the website at http://www.museumscomputergroup.org.uk
>> **************************************************
>
>
> --------------------------------------------
> Paul Walk
> Technical Manager
> UKOLN (University of Bath)
> http://www.ukoln.ac.uk/
> [log in to unmask]
> +44(0)1225383933
> --------------------------------------------
>
> **************************************************
> For mcg information and to manage your subscription to the list, visit
> the website at http://www.museumscomputergroup.org.uk
> **************************************************
>
> **************************************************
> For mcg information and to manage your subscription to the list,  
> visit the website at http://www.museumscomputergroup.org.uk
> **************************************************


--------------------------------------------
Paul Walk
Technical Manager
UKOLN (University of Bath)
http://www.ukoln.ac.uk/
[log in to unmask]
+44(0)1225383933
--------------------------------------------

**************************************************
For mcg information and to manage your subscription to the list, visit the website at http://www.museumscomputergroup.org.uk
**************************************************