Comments here have come mainly from library staff. 

We feel the relevance algorithm still needs improvement and are concerned that subject headings in particular are not weighted before the third relevance group. 

There is also the question of the order of entries within each relevance group.  Our current default is to sort the entries by title but colleagues have been asking for a date sort *within* the relevance groups to bring the latest entries to the top, which is particulatly useful where there many editions.  We could change the default sort to date but they want the option of sorting within the relevance groups by title or date as appropriate to the search - known item, partially known item or purely subject search.  I would be interested to know what others think about this.

    Anne 

-------------------------------------------------------
Anne Turner, LITS Information Services Systems Manager, Adsetts Centre, Sheffield Hallam University, City Campus, Pond Street, Sheffield, S1 1WB tel. 0114 225 2118,  fax. 0114 225 3859 international +44 114 225 2118 [log in to unmask]
-------------------------------------------------------


-----Original Message-----
From: This list is for current and potential users of the Innopac system [mailto:[log in to unmask]] On Behalf Of Mieko Yamaguchi
Sent: 31 July 2007 10:50
To: [log in to unmask]
Subject: Re: Any relevance ranking feedback?

We only went Pro a week ago and since the library is very quiet we haven't had any feedback from users or library staff for that matter! 
We are still in the process of redesigning our WebPac and haven't made a big deal of going Pro at this stage either.

Now that CSDirect FAQ has detailed descriptions of how relevance ranking works (only a year or so ago Innovative declined to publish this information claiming that it was proprietary) we have created a very brief explanation of what each group means.  Click on "What's this?" 
next to any relevancy icon.

http://library.bangor.ac.uk/search/a?searchtype=Y&searcharg=web+design&SORT=D&searchscope=1

I suspect that library staff who understand how relevance ranking of keyword searches works are probably more concerned than other groups. 
After all, users who are missing something (e.g. titles in group 3 because the search phrase exactly matches a subject heading or group 5 because the search words almost match a subject heading) may not realise what they are missing.

About 50% of keyword searches carried out by users appear to be single word searches.  Even if there is a single word title which exactly matches the searched word (quite common in journal titles), the user may not notice it whatever sort order you offer within a result set. 
However, I gather there may be an enhancement to treat single word title hits differently.

Mieko

Karen Stevenson wrote:
> Hi Graeme,
> We introduced WebPAC Pro and relevance ranking in early Feb. We didn't make a big deal of it; just an info page with the changes. Negative reaction came from Library & academic staff ("I know better than the system what is relevant for me").  We introduced the Pro enhancement which allows the user to resort the entire list by Title or by Date when it became available. And those who had commented were happy with this.
> 
> I'm not aware that we had any feedback from other user groups (e.g. students) and would suspect that either they haven't noticed or are not bothered one way or the other.
> Karen
> 
> ---------------------------------------
> Karen Stevenson
> Information Systems,
> Glasgow University Library
> Hillhead Street, Glasgow G12 8QE
> 
> Tel: 0141 330 6724
> Email: [log in to unmask]
> ----------------------------------------
> 
>> -----Original Message-----
>> From: This list is for current and potential users of the Innopac 
>> system [mailto:[log in to unmask]]On Behalf Of Graeme 
>> Leng-Ward
>> Sent: 28 July 2007 11:35
>> To: [log in to unmask]
>> Subject: Any relevance ranking feedback?
>>
>>
>> Dear All,
>>
>> Has anyone got any early feedback from patrons on the relevance 
>> ranking display for keyword searching in WebPacPRO catalogues?
>>
>> Have the criticisms of the III's relevance ranking algorithm been 
>> born out by any negative reaction so far?
>>
>> Or have patrons just taken to it easily? or maybe not even noticed 
>> the change?
>>
>> Cheers,
>> Graeme

-- 
Mieko Yamaguchi                          [log in to unmask]
Head of Technical Services               Phone: +44 (0)1248 382970
University of Wales Bangor               Fax:   +44 (0)1248 382979


--
Gall y neges e-bost hon, ac unrhyw atodiadau a anfonwyd gyda hi, gynnwys deunydd cyfrinachol ac wedi eu bwriadu i'w defnyddio'n unig gan y sawl y cawsant eu cyfeirio ato (atynt). Os ydych wedi derbyn y neges e-bost hon trwy gamgymeriad, rhowch wybod i'r anfonwr ar unwaith a dilëwch y neges. Os na fwriadwyd anfon y neges atoch chi, rhaid i chi beidio â defnyddio, cadw neu ddatgelu unrhyw wybodaeth a gynhwysir ynddi. Mae unrhyw farn neu safbwynt yn eiddo i'r sawl a'i hanfonodd yn unig  ac nid yw o anghenraid yn cynrychioli barn Prifysgol Cymru, Bangor. Nid yw Prifysgol Cymru, Bangor yn gwarantu bod y neges e-bost hon neu unrhyw atodiadau yn rhydd rhag firysau neu 100% yn ddiogel. Oni bai fod hyn wedi ei ddatgan yn uniongyrchol yn nhestun yr e-bost, nid bwriad y neges e-bost hon yw ffurfio contract rhwymol - mae rhestr o lofnodwyr awdurdodedig ar gael o Swyddfa Cyllid Prifysgol Cymru, Bangor.  www.bangor.ac.uk

This email and any attachments may contain confidential material and is solely for the use of the intended recipient(s).  If you have received this email in error, please notify the sender immediately and delete this email.  If you are not the intended recipient(s), you must not use, retain or disclose any information contained in this email.  Any views or opinions are solely those of the sender and do not necessarily represent those of the University of Wales, Bangor.
The University of Wales, Bangor does not guarantee that this email or any attachments are free from viruses or 100% secure.  Unless expressly stated in the body of the text of the email, this email is not intended to form a binding contract - a list of authorised signatories is available from the University of Wales, Bangor Finance Office.  www.bangor.ac.uk