United Nations High Commissioner for Refugees
Evaluation Policy and Analysis Unit
**Terms of Reference**
Evaluation of the Department of International Protection’s Protection
Information System (PIS)
1. Background
With the disbanding of the Centre for Documentation and Research (CDR) in
2000, and despite the fact that several of its functions were redeployed
within the organization, concerns were raised regarding functions relating to
UNHCR’s mandated responsibilities, namely, legal database development and the
production of country of origin information and background papers. In
response, the Department of International Protection was designated to create
a Protection Information Section (PIS). The purpose of PIS was to backstop
UNHCR’s protection role by undertaking the provision of timely and reliable
country of origin analysis and legal reference information/materials to assist
policy formulation and decisions relating to refugee status determination. The
primary tool and focus for the dissemination of such information was to be
Refworld, the CD-Rom containing information on UNHCR as an organization,
legal, national legislation, and countries of origin, Refworld, created in
the mid-ninties in the former CDR, had been discontinued in 2000. The first
challenge facing the new PIS was to update the content and re-issue it. Though
a user survey of Refworld has been undertaken in 2003, a comprehensive
evaluation has not. Moreover, the last evaluation of UNHCR’s protection
information function was undertaken in respect of the Centre for Documentation
and Refugees in 1993.
In the light of this background, and concerns regarding the continuing
usefulness of the Refworld CD-Rom, there is a clearly indicated need at this
time for an evaluation of PIS, and its major information product Refworld on
website and CD-Rom.
2. Purpose of the Evaluation
The purpose of the evaluation will be to review PIS outputs, products and
management, with special emphasis on its main product Refworld, with a view to
assessing Refworld’s relevance, efficiency, effectiveness, value-added, impact
and sustainability.
2.1 Major Evaluation Question
What are the key components of an effective protection information system?
3. Areas of Assessment
The evaluation will review all PIS outputs, products and services as they have
developed over 3 years. Emphasis will be given to PIS’ current and major
products and organization. As the perception of clients is key to assessing
the effectiveness of PIS’ performance as a whole, and on production of its
main information product, Refworld in particular, the evaluation will seek to
obtain a broad spectrum of user opinions and comments, including UNHCR staff
and senior management, donors, other UN agencies, governments, NGOs, judges
and academics. At the same time, while the evaluation might not so easily
measure the impact of PIS, the impact as perceived by its clients should be
gathered and factored into the evaluation report.
3.1 Purpose and Objectives
Has PIS been able to fulfill its purpose and objectives. Is the purpose still
relevant and, if not, what should it be and how achieved?
3.2 Institutional role
What is the institutional role of PIS? What is PIS’ contribution to UNHCR? How
far is PIS mainstreamed in other UNHCR-wide programmes? Does PIS bring value-
added to the mandate and mission of UNHCR? How is PIS used or not by other
departments and sections within UNHCR? How effective has PS been at
institutional networking with other partners in the UN system (WFP, UNIEF,
UNDP, UNOCHA) and NGOs, judiciary and academics? How are PIS’ relations with
other HQs units to which certain former CDR functions were deployed, namely,
the library, archives, EPU and ESS? What should be the institutional role of
PIS?
3.3 Staffing and Management
Is PIS adequately staffed, funded and managed? With its current staffing
arrangements, is PIS capable of meeting its planning and delivery objectives?
Is it capable of sustainable, future development (refer to sustainability at
para 8).
3.4 PIS users
Who are the main users of PIS products, with what frequency and for what
reasons do they access PIS products and services? Who should be the target
audience for UNHCR protection-related information? Where do the main users of
PIS products place Refworld in the list of their most used information
products? In what ways is Refworld distinct or different than other sources of
information? Where would one go for similar information in the absence of
Refworld? What are the weaknesses of Refworld?
3.5 PIS products
How well does Refworld meet user needs, including communities of practice, in
terms of its content, means of delivery, frequency, language coverage and
cost? What has been the impact of PIS’ Refworld on its clients? Where does
Refworld rank against other related information sources? What is the value-
added of current products? Does Refworld continue to fill a relevant niche in
the information field? To what extent does it overlap with other information
providers and products?
3.6. Technology platform
How relevant is Refworld in the light of developments in communications
technology and the internet? Are there other viable and sustainable options
for dissemination of Refworld content? Should production be managed separate
from content?
3.7 Cost effectiveness
Is Refworld providing value for money? Are the costs of running PIS to produce
Refworld in line with the outputs and impact? Should users pay, or should
Refworld be free? Are current marketing, distribution, payments and accounting
systems supporting optimal income generation?
3.8 Sustainability
Is PIS able to secure income sufficient to meet its needs and to support
needed growth? What is the potential for PIS to reduce reliance on donor
funding? What other options for funding could be drawn on?
3.9 Future
Is there scope for PIS to be further expanded, or alternatively merged with
another information service? What are the options for PIS outside of DIP, and
even UNHCR?
4. Method
The evaluator will assess the above key areas through:
- Interviews with current and former CDR and Refworld staff, and those from
relevant inter-institutional units and departments.
- Interviews with UNHCR (HQs and field) staff
- Interviews with external stakeholders and clients other UN, NGOs,
governments, academics and media
- Review all internal systems and documentation
- Assess internal organization and management
5. Outputs
An initial feedback meeting, 10 March 2005, will be held by the Evaluator with
an Evaluation Committee led by EPAU before the Evaluator embarks on field
visits. The full findings of the evaluation will be presented in a written
report to include:
Full findings on all areas of assessment above and, in addition, special
emphasis on
- PIS’s accomplishments in relation to its planning.
- Findings on PIS’s client satisfaction, impact and influence.
Recommendations on how to increase impact.
- Findings and recommendations on internal organization and management
structures including staffing and resources.
- Findings and recommendations on institutional role.
- Findings and recommendations on content and platform for dissemination
- Suggestions on ways to overcome constraints faced by PIS: recruitment,
resources, content, and sustainability of its major product, Refworld.
6. Evaluator Profile
The successful candidate will possess demonstrated knowledge and experience in
two areas:
6.1 Experience in information management systems at the international level
with sound grounding in methods, information dissemination, library systems,
COI, legal databases and their evaluation, and
6.2 Experience in evaluating organizational and management structures.
7. Schedule
The evaluation will commence in March 2005, with a first draft due mid-April
2005. Findings of the first draft will be shared for comments. The final draft
is due 31 May 2005. The schedule* appears below.
(*as if the project were to start on 1 March)
ACTIVITY DATES DAYS
Phase I - Preparations
Literature Review 1-4 March 3
Develop Interview Questionnaire 4-8 March 3
Setup interview schedule 9-12 March 3
Management and structure analysis 13-23 March 10
Interviews 23-29 March 7
Evaluator presents initial findings 31 March 1
Phase II - Field Missions
Dakar 1-3 April 2
Nairobi 4-7 April 4
Washington 9-10 April 2
Ottawa 11-12 April 2
Phase III - Drafting Evaluation
Initial draft for comments 14 April 11
Final Report 31 May 8
Total 55
8. Reporting Requirements
A succinct report that includes a short executive summary of up to 2,000 words
and a main text of no more than 15,000 words. Annexes should include a list of
all persons interviewed, a bibliography, a description of the method used and
purpose of the evaluation, and a summary of survey results.
The 1st draft report is due mid-April 2005.
The final report is due 31 May 2005.
9. Payments
The Evaluator will receive 20% of the amount of the contract upon signing, 40%
upon completion of the first draft, and the last 40% upon successful
completion of the final report.
EPAU, HQEP00
UNHCR HQs
Geneva - Switzerland
**Request for Proposals:**
Evaluation of the Department of International Protection’s Protection
Information System (PIS)
1. Introduction
The United Nations High Commissioner for Refugees is a programme of the United
Nations General Assembly. Its Chief Executive, the High Commissioner, is
elected by the General Assembly on the nomination of the Secretary-General and
reports to the General Assembly through the Economic and Social Council.
Within UNHCR, the Department of International Protection provides governance
and doctrine in respect of UNHCR’s mandate which is to ‘provide international
protection’ and ‘to seek permanent solutions to the problems of refugees ‘
through repatriation, local integration or resettlement. Concisely, UNHCR’s
primary purpose is to safeguard the rights and well-being of refugees.
As part of its role to oversee the Office’s primary purpose which is to
safeguard the rights and well-being of refugees, the Department of
International Protection created the Protection Information Section (PIS).
Initiated in 2001, the purpose of PIS was to backstop UNHCR’s protection role
by undertaking the provision of timely and reliable country of origin analysis
and legal reference information/materials to assist policy formulation and
decisions relating to refugee status determination. The primary tool and focus
for the dissemination of such information was to be Refworld, the CDRom
containing information on UNHCR as an organization, legal materials, national
legislation, and countries of origin originating from governments,
intergovernmental organizations, NGOs academics and courts. Refworld, created
in the mid-ninties in the former CDR, had been discontinued in 2000.
The first challenge facing the new PIS was to update the content and re-issue
it. Though a user survey of Refworld has been undertaken in 2003, a
comprehensive evaluation has not. Moreover, the last evaluation of UNHCR’s
protection information function was undertaken in respect of the Centre for
Documentation and Refugees in 1993. An evaluation of PIS and Refworld is
needed. Within UNHCR, the Evaluation and Policy Analysis Unit (EPAU) is
responsible for undertaking evaluations of operations, programmes and themes
on behalf of the organization. On the basis of the above introduction and
organizational concerns regarding the continuing usefulness of the Refworld
CDRom, the EPAU will undertake an evaluation.
EPAU is seeking proposals for an evaluation to be carried out by a single
evaluator. Proposals should include:
- CV of the evaluator.
- A clear methodology and schedule of how the evaluation will be structured
over the entire period, from 1 March to 31 May 2005, with travel to Dakar,
Nairobi, Washington and Ottawa included.
- A description of past performance in implementing external evaluations of
global information systems.
- References
- An indication of how the final report will be structured.
- A budget outline of total costs and expenses, including all travel.
2. Purpose of the Evaluation
The purpose of the evaluation will be to review PIS outputs, products,
technology and management, with special emphasis on its main product Refworld,
with a view to assessing Refworld’s relevance, efficiency, effectiveness,
value-added, impact and sustainability.
3. Timeframe
The evaluation is proposed to take place over a 4 month period, commencing
March 2005 with a first draft report for comments due mid-April 2005.
4. Call for Proposals
All those interested in submitting proposals for the evaluation of the
Protection Information Section (PIS)’s Refworld, please contact
[log in to unmask] with a proposal and budget. The deadline for submissions is 28
February 2005.
EPAU
February 2005
++++++++++++++++++++++++++++++++++++++++++++++++++++
Note: The material contained in this communication comes to you from the
Forced Migration Discussion List which is moderated by the Refugee Studies
Centre (RSC), University of Oxford. It does not necessarily reflect the
views of the RSC or the University. If you re-print, copy, archive or
re-post this message please retain this disclaimer. Quotations or extracts
should include attribution to the original sources.
List archives are available at: http://www.jiscmail.ac.uk/lists/forced-migration.html
|
