Dear John,
I by no means intended to suggest that such a method would provide a good method of assessing research for RAE/REF purposes, since as an Arts researcher I am aware that citation counts are an absolutely misleading way of determining the importance of a paper in most disciplines, even including many science disciplines. They are a cheap way to do what was previously done with the RAE, which, although it may have needed reform because it allocated funding on the basis of research rather than on a balanced assessment of research and teaching as a whole, was certainly more appropriate for the purpose than citation counts appear to be in the case of many disciplines, notably in the Arts.
How in fact do you numerically describe the subjective impact and importance of a particular journal or other resource? I would say that you logically cannot, since it is a subjective judgement. The matter of citing more sources for certain subjects than others will be fine: after all, one normally only compares papers with their like, i.e. papers on the same subject when assessing their relative importance (unless one is purely engaged in a funding exercise, on which see my earlier comments). One could make similar comments on the other issues you describe. In very tiny discliplines such as my own, one might hope that their are so few papers that a competent academic might be familiar with them anyway. All this points to the fallacy of using citations for the purposes of research funding allocation. I deliberately avoid the term research assessment, since I am not clear that citation methods constitute any genuine assessment at all.
However, wearing another hat I am a librarian and I am therefore concerned with resource discovery, an area in which such a simplistic method as I describe would indeed be of significant benefit. Leaving aside research assessment, therefore, how do we simply work out where the papers are for each subject? I am assured that OAI provides a means to query across repositories for the data that would be needed for the sort of resource discovery (not assessment) tools that I describe.
Simply, I think you have misunderstood the purpose of my comments. Leaving aside my distrust of citation analysis as an entirely secondary issue, does anybody have any comments relating to my earlier proposal purely as a resource discovery tool? It would work on the most granular level, i.e. on the basis of very narrow subject-based searches, so it wouldn't matter in the slightest, for instance, that published research about Shelley gave far higher counts than, say, counts for research about Andrew Motion. You'd only compare like with like in order to find what you wanted. This is what social bookmarking enables, without any attempt to force an assessment of worth on the researcher.
Best wishes,
Talat
-----Original Message-----
From: Repositories discussion list [mailto:[log in to unmask]] On Behalf Of John Smith
Sent: 06 May 2008 12:05
To: [log in to unmask]
Subject: Re: Google, OAI and the IRs
Talat,
Currently there are various ways to indicate quality or importance in a publication.
1. The journal it is published in (Nature/Science, etc, confers more kudos than The Butterfly Magazine). This is prepublication accreditation (or certification).
2. The citations it attracts. Like journals not all citations are equal and simply counting them is not enough. As a first pass we could take the Google approach and recursively count the citations of the citing papers, this has some merit but how deep should we go, should we just consider the citations of the citing paper, or include the citations of the papers that cite it, etc. We must also consider the citing patterns of the subject; some subjects cite many papers as the norm some only a few. Another variable is the size of the subject, that is, how many active researchers are there in this subject. In a very specialised area no one can get a large number of citations since there are few citers. And so on :-) .
The above is just a list of requirements or problems a Social Networking publication model would need to provide/solve. I can see it may be possible but I don't think it will be easy.
Regards,
John Smith.
> -----Original Message-----
> From: Repositories discussion list [mailto:JISC-
> [log in to unmask]] On Behalf Of Talat Chaudhri [tac]
> Sent: 06 May 2008 10:34
> To: [log in to unmask]
> Subject: Re: Google, OAI and the IRs
>
> Hi Andy,
>
> As I've said before, I don't think we need to adopt an either/or
> approach to the question of mandates versus social networks, and I
> fully agree with you that the latter is sadly missing from our
> current repository solutions. (I'm still of the view that what
> institutions need to achieve is also valuable, so let's leave that
> aside here.)
>
> Citing someone's work is a "social" act in itself, almost
> identical conceptually to tagging (but perhaps not gravitas as
> things currently stand in academia) but just operating in a rather
> more specific fashion by attaching very specific connections
> between certain areas of commentary and specific parts of a
> resource, rather than simply saying "here's a cool paper I read".
> Both of these acts, however, are potentially very valuable indeed.
> In a published paper, you would only expect conventional citation,
> but in a blog, say, you could expect either strict citation or the
> more general link to someone else's resource. Let's say we can
> develop a protocol more flexible than either, that can be used
> purely to link to a whole resource but that could also point to a
> specific point within it. It needs to do several things in order
> to achieve more than a simple hyperlink does:
>
> * provide an automatic indication of how popular that resource is,
> both where cited and where found (as well as perhaps in some sort
> of tag cloud in the repository housing the item or in a subject
> based harvester or search tool). This needs to update whenever a
> new link is added, of course, wherever it may be on the web,
> requiring some feedback mechanism form the citation end. It must
> also have a means of doing this in a way that is not open to abuse
> or forgery of statistics.
> * provide a method of linking not just to the whole item, but also
> to specific referencers such as page numbers or paragraphs in an
> online version. If you aren't making a strict citation, you must
> be able to leave this out.
> * provide a method of distinguishing between page/paragraph
> citations and total resource citations, so that statistics about
> those citations will be able to clearly show the difference
> between the "cool paper!" and the "Prof Bernstein makes this
> specific remark about epistemology on page such and such"
> citations.
>
> Is it useful to distinguish between something like "strict" and
> "general" citations in this way?
>
> I don't think this is technically difficult, but I can't see that
> either conventional online citations or tags can perform all of
> these functions. Will OAI-ORE do any of this? Are these useful
> speculations? Thanks,
>
>
> Talat
>
> -----Original Message-----
> From: Repositories discussion list [mailto:JISC-
> [log in to unmask]] On Behalf Of Andy Powell
> Sent: 02 May 2008 17:42
> To: [log in to unmask]
> Subject: Re: Google, OAI and the IRs
>
> Sarah,
> Lorcan sent me a private email a while back to say that my use of
> "social network" in the context of repositories was likely to lead
> to
> confusion and he is probably right.
>
> To try and clarify, probably unsuccessfully... I am *not*
> suggesting
> that we need to turn repositories into Facebook! I am suggesting
> that
> if one looks at almost any successful repository-like service,
> whether
> it is Flickr, or Slideshare, or YouTube, or arXiv, or RePEc -
> there is a
> significant but sometimes subtle 'social' aspect to the success of
> the
> service.
>
> These services work because they focus on *sharing* content - both
> between narrow groupings of "friends" and with a wider "global" or
> semi-global audience - funnily enough, not disimmilar to the way
> that
> research takes place.
>
> Scholarly repositories as they currently stand, do not tend to do
> this
> successfully - partly because of the technology focus on the OAI-
> PMH,
> partly because of their 'institutional' rather than 'discipline'
> or
> 'global' focus (which doesn't match the social networks adopted by
> researchers) and partly because of functional creep into things
> like
> preservation and institutional RAE returns. Because repositories
> are
> not seen as successful places to *share* content, they are not
> seen as
> compelling - and they are not used. Our only solution -
> mandates :-(
>
> Dismissing the importance of the "social" aspects of respositories
> on
> the basis that some people don't have time to look at "social
> networks"
> (like Facebook or whatever) therefore rather misses the point.
>
> So, apologies for introducing some confusion... I probably need to
> start
> using a different phrase than "social network" in the future and
> will
> try to come up with something.
>
> Andy
> --
> Head of Development, Eduserv Foundation
> http://www.eduserv.org.uk/foundation/
> http://efoundations.typepad.com/
> [log in to unmask]
> +44 (0)1225 474319
>
> > -----Original Message-----
> > From: Repositories discussion list
> > [mailto:[log in to unmask]] On Behalf Of Sarah
> Currier
> > Sent: 02 May 2008 16:03
> > To: [log in to unmask]
> > Subject: Re: Google, OAI and the IRs
> >
> > OK, I'm going to do a classic "Friday afternoon" posting here:
> >
> >
> >
> > But if not for information on repositories and similar
> > services, what would people on social networks talk about ?
> >
> >
> >
> > Thanks for saying what I always think at this point in the
> > conversation Les.
> >
> > And something additional occurs to me. For many, many
> > students and researchers, online "social networks" may not be
> > a time-efficient way of finding what they need. I think the
> > concept of "social networks" online may become a new version
> > of "old boys' network", or perhaps a new version of the
> > "social capital" (bleachh, sorry for that painful term)
> > enjoyed by the middle classes, enabling them to navigate the
> > waters of education with ease. Or maybe it already is.
> >
> > For those accessing higher education whose existing social
> > networks consist of (a) non-academics, (b) families with no
> > interest in the internet, (c) their own children and the
> > support network around child-rearing, and for those who have
> > to spend a good bit of their time simply earning money to
> > live while they research or learn, or paying off the debts
> > they've accrued so far, or supporting families, there may not
> > be much time for tutuing about online. I'm thinking of one
> > person I know who is doing a PhD as a 40-something poor
> > working class non-digital-native. She wants to get onto a
> > research system, search in as few clicks as possible, delving
> > into whatever resource she has access to via the open web or
> > the university library's services, and get what she needs for
> > the work she's doing. She doesn't have broadband at home and
> > maybe never will. She wants to get an accurate return,
> > representing the best possible match to what she's searching for.
> >
> > I'm sure she will eventually become part of more and more
> > social networks related to her subject area and its related
> > professional area. Some of these may be online. But it
> > seems to me that the social networking stuff is still a huge
> > luxury to most of the world, with regard to the accessibility
> > (in its broadest sense) of knowledge and learning. Systems
> > which at their basic level allow accurate retrieval of
> > appropriate resources (e.g. repository and metadata
> > management) grow out of a library tradition of making
> > knowledge available. There is still work to be done on this.
> > (Don't worry, I'm very aware of how social mediation also
> > takes place within information management- how systems such
> > as cataloguing and classification can embed human biases).
> >
> > I always have this suspicion that when esteemed, accomplished
> > and highly intelligent colleagues such as Andy say things
> > like "[...] we are focusing our attention in the wrong
> > place", they are bored with dealing with metadata and
> > information management, and they perceive that issues of
> > "social networking" are either more interesting to them
> > professionally now, or that it is a more lucrative and
> > high-status area of research and development to move into.
> > (Andy, I'm not implying this mercenary last option is what
> > you're thinking at all!).
> >
> > And that's fine: it may well be that folk like my friend may
> > reap hitherto un-dreamt of benefits from all this online
> > social networking stuff in a few years' time- but don't
> > conflate it with needing to take attention and funding away
> > from the basic information infrastructure, without which, as
> > Les says, we wouldn't have much to talk about in our networks.
> >
> > Having just read Jewel's response as well, which I am in
> > agreement with: where is the funding going? Why isn't it
> > going into making sure there are robust systems for open
> > access to research data and learning resources,
> > well-catalogued with good-quality metadata? It's just such a
> > basic need in education.
> >
> > Please insert disclaimer about none of this being remotely
> > related to Intrallect's position on these matters: purely
> > some personal thoughts...
> > S.
> >
> >
> > --
> >
> >
> > Sarah Currier
> > Product Manager, Intrallect Ltd.
> > http://www.intrallect.com
> >
> > 2nd Floor, Regent House
> > Blackness Road
> > Linlithgow
> > EH49 7HU
> > United Kingdom
> >
> > Tel: +44 870 234 3933
> > Mob: +44 (0)7980855801
> > E-mail: [log in to unmask]
> > --
> >
> > P Please consider the environment before printing this email.
> >
>
>
> -----
> Dr Talat Chaudhri, Ymgynghorydd Cadwrfa / Repository Advisor
> Tîm Cynorthwywyr Pwnc ac E-Lyfrgell / Subject Support and E-
> Library Team
> Gwasanaethau Gwybodaeth / Information Services
> Prifysgol Aberystwyth / Aberystwyth University
> Llyfrgell Hugh Owen Library, Penglais, Aberystwyth, Ceredigion.
> SY23 3DZ
> E-bost / E-mail: [log in to unmask]
> Ffôn / Tel (Hugh Owen): (62)2396
> Ffôn / Tel (Llandinam): (62)8724
> Ffacs / Fax: (01970) (62)2404
>
> CADAIR: http://cadair.aber.ac.uk
> Cadwrfa ymchwil ar-lein Prifysgol Aberystwyth
> Aberystwyth University's online research repository
> Ymholiadau / Enquiries: [log in to unmask]
|
