JiscMail Logo
Email discussion lists for the UK Education and Research communities

Help for JISC-REPOSITORIES Archives


JISC-REPOSITORIES Archives

JISC-REPOSITORIES Archives


JISC-REPOSITORIES@JISCMAIL.AC.UK


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

JISC-REPOSITORIES Home

JISC-REPOSITORIES Home

JISC-REPOSITORIES  July 2010

JISC-REPOSITORIES July 2010

Options

Subscribe or Unsubscribe

Subscribe or Unsubscribe

Log In

Log In

Get Password

Get Password

Subject:

Re: OA policies and their "weight"

From:

Stevan Harnad <[log in to unmask]>

Reply-To:

Stevan Harnad <[log in to unmask]>

Date:

Mon, 19 Jul 2010 09:56:09 -0400

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (98 lines)

Dear Reme, if I may also make an intervention in your exchange with Steve Hitchcock about the MELIBEA OA policy evaluator:
http://www.accesoabierto.net/politicas/ 

The MELIBEA service is extremely timely and promising, and could be potentially useful and even influential in shaping OA mandates -- but that makes it all the more important to get it right, rather than releasing MELIBEA prematurely, when it still risks increasing confusion rather than providing clarity and direction.

You are right to point out that -- unlike the CSIC's University Ranking and the Repository Ranking -- the policy evaluator is not really a ranking. But you have set up the composite algorithm and the graphics to make it a ranking just the same. 

You are also point out, correctly, that the policy criteria for institutions and funders are not (and should not be) the same. Yet, with the MELIBEA coding as well as the algorithm, they are treated the same way. 

You also point out, rightly, that gold OA publishing policy is not central to institutional OA policy making, yet there it is, as part of the MELIBEA algorithm.

You also point out that the color code has nothing to do with the "green" OA coding -- yet there it is, competing with the widespread use of green to designate self-archiving, and thereby causing confusion, both overt and covert.

I would be more than happy to give you feedback on every aspect of MELIBEA -- it could be a useful and natural complement to the ROARMAP registry of OA policies. 

But as it is designed now, I can only agree with Steve Hitchcock's points and conclude that consulting MELIBEA today would be likely to induce confusion and would not help in bringing the all-important focus and direction to OA policy-making that I am sure CSIC, too, seeks, and seeks to help bring about.

Here are just a few prima facie points:

(1) Since MELIBEA is not, and should not be construed as a ranking of OA policies -- especially because it includes both institutional and funder policies -- it is important NOT to plug it into an algorithm until and unless the algorithm has first been carefully tested, with consultation, to make sure it weights policy criteria in a way that optimizes OA progress and guides policy-makers in the right direction.

(2) For this reason, it is more important to allow users to generate separate flat lists of institutions or funders on the various policy criteria, considered and compared independently, rather than on the basis of a prematurely and arbitrarily weighted joint algorithm.

(3) This is all the more important since the data are based on less then 200 institutions, whereas the CSIC University Rankings are based on thousands. Since the population is still so small, MELIBEA risks having a disproportionate effect on initial conditions and hence direction-setting; all the more reason NOT to amplify noise and indirection by assigning untested initial weights without carefully thinking through and weighing the consequences.

(4) A potential internal cross-validator of some of the criteria would be a reliable measure of outcome -- but that requires much more attention to estimating the annual size and growth-rate of each repository (in terms of OA's target contents, which are full-text articles), normalized for institution size and annual total target output. Policy criteria (such as request/require or immediate/delayed) should be cross-validated against these outcome measures (such as percentage and growth rate of annual target output).

(5) The MELIBEA color coding needs to be revised, and revised quickly, if there is to be an algorithm at all. All those arbitrary colors in the display of single repositories as ranked by the algorithm are both unnecessary and confusing. The objective should be to order and focus clearly and intuitively. Whatever is correlated with more green OA output (such as a higher level or faster growth rate in OA's target content) should be coded as darker or bigger shades of green. The same should be true for the policy criteria, separately and jointly: in each case, request/require, delayed/immediate, etc., the greenward polarity is obvious and intuitive. This should be reflected in the graphics as well as in any comparative rankings.

(6) If you include repositories with no OA policy at all (i.e., just a repository and an open invitation to deposit) then all you are doing is duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA, is to highlight, weigh and compare specific policy differences among (the very few) repositories that DO have policies.

(7) The sign-up data -- http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are also rather confusing; the criteria are not always consistent, relevant or applicable. The sign-up seems to be designed to make a funder mandate the generic option, whereas this is quite the opposite of reality. There are far more institutions and institutional repositories and policies than funders. There should be separate criterial lists for institutional policies and for funder policies; they are not the same. There is also far too much focus on gold OA policy and payment. If included at all, this should only be at the end, as an addendum, not the focus at the beginning, and on a par with green OA policy. 

(8) There is also potential confusion on the matter of "waivers": There are two aspects of a mandate. One concerns whether or not deposit is required (and if so, whether that requirement can be waived) and the other concerns whether or not rights-reservation is required (and if so, whether that requirement can be waived). These two distinct and independent requirements/waivers are completely conflated in the current version of MELIBEA.

I hope there will be substantive consultation and conscientious redesign of these and other aspects of MELIBEA before it is can recommended for serious consideration and use.

Stevan Harnad


On 2010-07-19, at 5:18 AM, Remedios Melero wrote:

> Dear Steve,
> 
> I apologize for the delay in my response, but I will try to give some explanations to make clear some issues you raised in your message (my comments are in capital letters, to distinguish them from yours)
> 
> 
> 
> El 15/07/2010 11:22, Steve Hitchcock escribió:
>> Reme,    Thank you for bringing this new service to our attention. OA policies are vitally important to the development of institutional repositories, and services that can highlight and bring attention to this development can be valuable.
>> 
>> There are a few aspects of the validation aspects of the new MELIBEA service that confuse, and possibly trouble, me. The first is the main indicator, %OAval, which is the most visible result for a policy. What do you expect this will tell people about a given policy? I randomly selected a couple of policies, one of which was for my own school, to find they each scored about 50%. I would expect these to be among the leaders in terms of OA policies, so this seems a surprisingly unhelpful score.
>> 
>> So what's the explanation? Note that the objects being evaluated are institutional OA policies; they are effectively being presented in relation to institutional repositories when the policy specifies where to archive is an IR with a URL. It seems that the scores include ratings for OA publication policy, libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score if Yes), etc., some of which an institution might specify but which might not apply to an IR 
>> http://www.accesoabierto.net/politicas/politicas_estructura.php
>> . However you weight these factors they are still contributors to the overall score, so a policy that is specific to an IR is immediately handicapped, or appears to be unless there is more context to understand the scores.
>> 
>>   
>> 
> AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  BUT TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  AND WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS POSSIBLE" (this could be a month after publication or years after publication, depending on one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH ("Gold  (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED BY  SAME ECONOMIC MODEL.
> 
> 
>> Which leads me to another question on the visualisation of the validator, and its use of green, gold (and red) in the meter. Do the green and gold refer the the classic OA colours? This would be quite convenient, since it would appear that the green repository policies I mentioned above are achieving almost full scores in the green zone of the meter. However, I suspect this cannot be the case, because it would assume that institutions must have a green AND gold policy, but not simply gold (whatever argument could be put for that). 
>>   
>> 
> COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE A SPECTRA. 
> 
>> It is important that new services should help reveal and promote OA policies, as you seek to do, but at the same time not to prejudice the development of such policies by mixing and not fairly separating the contributing factors, especially where these relate to different types of OA.
>>   
>> 
> I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
> IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH COULD DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but to remember we should  assume  reponsible attitudes).
> 
> However we will revise the model to see if we can make any improvement to make it clear, we are working also in a graph interface to show some data in graphical form.
> Best wishes
> Reme
> 
>> 
>>> R. Melero 
>>> IATA, CSIC 
>>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
>>> TEl +34 96 390 00 22. Fax 96 363 63 01 
>>> E-mail 
>>> [log in to unmask]
>>>  
>>> 
>>> http://www.accesoabierto.net
>>> 
>>> 
>>> --      
>>>     
>>> 
> 
> 
> --      
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f

Top of Message | Previous Page | Permalink

JiscMail Tools


RSS Feeds and Sharing


Advanced Options


Archives

March 2024
February 2024
January 2024
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
November 2005
October 2005


JiscMail is a Jisc service.

View our service policies at https://www.jiscmail.ac.uk/policyandsecurity/ and Jisc's privacy policy at https://www.jisc.ac.uk/website/privacy-notice

For help and support help@jisc.ac.uk

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager