UNICEF ECA Regional Office: “Short-Term National / International Institutional Consultancy to conduct a developmental evaluation of the of UNICEF ECD response to COVID-19”
UNICEF ECA Regional Office: “Short-Term National / International Institutional Consultancy to conduct a developmental evaluation of the of UNICEF ECD response to COVID-19”
Reference: RFP-ECA-2020-01 (LRPS-2020-9160960)
Beneficiary country(ies): Multiple destinations (see ‘Countries’ tab below)
Registration level: Basic
Published on: 28-Aug-2020
Deadline on: 07-Sep-2020 19:00 (GMT 3.00) Baghdad, Riyadh, Moscow, St. Petersburg
Background
As the first part of the year evolved, the coronavirus increasingly spread across the ECA region and ushered in a new wave of global and domestic shocks. COVID-19 is a global pandemic which threatens young children and their rights in countries across the ECA region and exposes them to massive disruption to their healthcare, education, access to basic needs and services like food, protection and social interaction with family members, teachers, peers and communities. The imposed preventative measures by the local governments such as quarantines, school and day care closures, travel restrictions and border closures and others, have a deeper impact on the wellbeing of children and their parents.
There are major health and development risks to children and their families that arise from the pressure on healthcare and education systems resulting in reduced access to routine health, childcare and early education services. Vulnerable children in need of specialized services are at high risk of compromised development during the crisis and through the recovery as these services are not easily accessible.
Children and parents from the most vulnerable groups (poor children, ethnic minorities such as Roma, migrant and refugee children, children with disabilities) are likely to be even more affected with the COVID- 19 and its secondary impact. Due to the lower access to digital means of communication, reduction in outreach services, segregation and linguistic and financial barriers they have poorer access to timely information, support to parental guidance, or distance learning opportunities, increasing equity gaps and jeopardizing longterm social cohesion.
Too often, early childhood development (ECD) falls through the cracks in emergency responses – but there are key measures UNICEF and partners are taking to ensure that early childhood development is supported within the COVID-19 responses and that opportunities to strengthen policy goals with respect to young children are leveraged.
Purpose
As the crisis unfolds, adaptation of ECD-related services provided to families is taking place rapidly in response to the current, evolving context. Services may include remote pre-natal care, home visiting, parental counselling, early childhood education, or early childhood intervention for children with disabilities. Some of these services are halted, some have already been adapted to reach children and families through digital means, others are looking into expanding reach and effectiveness further.
Many countries on the region are already implementing or planning to implement various data collection mechanisms at the household level, mainly to understand the economic impact of COVID-19 in families and children. However, there is not much evidence generation and data collection taking place with respect to services that affect young children and the providers i.e. workforce who are in direct contact with families. Combining household level data with service-level data in an iterative manner can help to shape and direct decisions towards increased relevance and effectiveness of core services and support to families and children.
UNICEF ECARO seeks to guarantee business continuity of these services for young children and ensure relevance and effectiveness during the period of crisis, as well as integrating lessons learned into future planning and rebuilding of more resilient systems and service provision modalities. For this purpose, ECARO plans to undertake an evaluation of the adaptation of services for young children and their families in response to COVID-19 to response to the changing needs of the service users.
The primary purpose of this evaluation is to critically assess UNICEF’s efforts in ECD programming that are being adapted to meet the changing needs of young children and families and do so with the view of going forward to be better prepared to the similar situation in the future. By collecting “good enough” evidence recurrently and dedicating resources for ongoing analysis, UNICEF and partners can strengthen the process of adaptation of these services and their effectiveness in an agile manner. The knowledge generated from the evaluation, and the specific lessons learned that it identifies, will inform evidence-based decision-making, evidence-based advocacy, and resource-mobilization, while contributing to learning within UNICEF.
The secondary purpose is to document the specific substance and progress of UNICEF’s work in this area to date to contribute to the final evaluation which will be conducted in 2021.
The objectives of the evaluation are to:
- assess the extent to which the ECD activities are being implemented in the selected countries, how they are meeting the needs of young children and families especially when their needs change as the COVID-19 outbreak evolves.
- assess the effectiveness of the ECD activities in improved programming and systems strengthening support to governments in the selected countries
Object and Scope of the Evaluation
The evaluation will focus on services for young children and their families, traditionally supported by health and education delivery platforms and frontline workers but which are now being altered to fit in to the COVID-19 restrictions. Relevant services are determined at country level and may include remote pre-natal care, home visiting, parental counselling, early childhood education, early childhood intervention for children with disabilities or any other type of services to families with young children.
However, the selected project activities must be delivered through ‘a workforce’ (frontline workers) and fall into one of the following categories (these excluding criteria):
- • moving an existing service to a digital mode of engagement or delivery or complementing the existing service with new modalities of distance support (increase effectiveness);
- • introducing new services to address new needs for current service users (increase relevance);
- • adapting existing or incorporating new services or service provision modalities to meet the needs of new groups of service users (increase relevance) The evaluation will not focus on the project activities which do not fit either of the three stages of adaptation or those which do not provide service delivery through frontline workers.
The inclusion criterion for the countries is determined based on interest of the Country Office (CO) to participate, existence of respective services supporting young children prior to COVID-19 and after the onset of the crisis, relevant adaptations introduced to core services, regional representation, and ability of countries to engage directly with frontline workers, in partnership with relevant government counterparts.
The key OECD-DAC evaluation criteria which are most important to the evaluation are relevance, effectiveness and sustainability.
The time period covered is from March 2020 when the COVID-19 outbreak started and up to February 2021.
Intended user(s) and use(s)
The primary intended users of the evaluation are UNICEF COs selected as the ‘in depth’ study countries, UNICEF ECA Regional Office and national governments and partners. The findings will also be useful to those UNICEF COs who are starting their new programme cycles in 2021 to reflect on the lessons learned that are applicable to their contexts.
The findings of the evaluation will inform the ECD programming in the near future in relation to the situations such as that of COVID-19 to ensure the continuity of ECD-related services delivered to children and families regardless of the pandemic.
Evaluation questions
The evaluation proposes to investigate the following main evaluation questions:
- Q1. What adaptations in ECD services made in response to COVID-19 crisis are more (or less) effective in terms of delivering on the expected service outcomes and the needs of families in the selected countries? For which population group and under which circumstances do the adapted ECD service delivery work best and in what ways? (relevance, effectiveness)
- Q2. What are key requirements in terms of staff capacity, technology, as well as enabling environment, for the introduction and continuous delivery of the effective adaptations of services? (effectiveness)
- Q3. To what extent adaptations introduced in response to COVID can improve resilience of services and contribute to long-term effectiveness and efficiency in service provision? (sustainability)
There will be sub-questions developed for each CO to ensure contextualisation of the evaluation to their specific context and programme characteristics. Those questions will be developed and fined tuned during the inception stage.
Methodology
This proposal presents a developmental evaluation approach to contribute to increasing the relevance and effectiveness of the available services for young children and address current and emerging needs of the ECD workforce so they can better support families. Given the very dynamic context that is rapidly changing, the exercise will be both retrospective and prospective and will try to evaluate suitability and relevance of the adaptations to the respective context in the past, present and future times. It will look back and evaluate adaptations that were introduced at the onset of COVID-19 pandemic, even if these had been discontinued. It will equally try to assess ongoing changes and the extent to which they support current and future needs and modalities of service provision.
While the evaluation methodological approach will be finalized at the inception phase, initial advice on the comprehensiveness of the evaluation approach is expected at the proposal stage. Methodological rigor will be given significant consideration in the assessment of proposals. Hence bidders are invited to interrogate the approach and methodology suggested in the ToR and improve on it or propose an approach they deem more appropriate. Bidders are encouraged to also demonstrate methodological expertise in.
Three-layered data collection and analysis
The initial step in this exercise will be to map adaptations introduced in ECD-related services across participating countries. This will be the basis for the identification of the respective service providers to participate in data collection and the key respondents for indepth interviews. Then the following 3 main components will be conducted:
1. Survey design and data collection in the in-depth study countries. This includes:
- i. Frontline workers survey (telephone/SMS/text): sampling of workforce, questionnaire formulation; implementation of the questionnaire three times in a period of 6-9 months- through digital means or other forms (e.g. telephone). Questions should be tailored to the workforce needs in terms of access to families and children, operational challenges (access to materials, supplies, information, transport etc.), skills gaps and capacity needs to respond to the current context (e.g. availability of training and guidance, connection with the national authority, network of professionals, peers etc. ), mentoring and supervision in the current context, etc.
- ii. Key in-depth interviews with selected government officials in the depth study countries
- iii. Key-in-depth interviews (telephone) to a small number of front-line workers in the depth study countries
- iv. Key in-depth interviews (telephone) to selected service end-users in the depth study countries at specific times will complement the data collection process.
- v. Other existing data available in-depth study countries, such as social media data, HH surveys, situation analysis, etc.
2. Data analysis and formulation of programmatic recommendations through rapid cycle testing/analysis in the depth study countries, ideally in combination with household level data collected by UNICEF or partners, as well as other existing information available. Data analysis will aim to provide rapid inputs to UNICEF staff and national counterparts, and enable the formulation of evidence-based recommendations, advocacy and communication strategies, and programmatic decisions. The recurrent analysis (done each time after the surveys are produced)’ objectives will be to:
- i. assess if ECD workforce have relevant capacity and skills to adapt to and deliver new modalities (what is missing- trainings, technology, supportive policy, supportive supervision- which is already in the note);
- ii. assess the support available and required by front line workers and formulate better program response;
- iii. assess the effectiveness of the adapted tele-and remote services they are providing, how can they be improved, are there new opportunities to consider post COVID-19; and ultimately
- iv. inform future policy formulation and adaptations that reflect lessons learned from this context
The participating in-depth study country offices will have three briefs.
3. Regional synthesis and country level reports. A regional synthesis of lessons learned, findings, and recommendations will be developed. Further, based on the iterative data collection at country level and the overall emerging findings. There will be a synthesis report at the end of this exercise.
The proposed period of this exercise is August – March 2021. Surveys and interviews will ideally be conducted in August, September, December. The iterative process can continue in the future.
Sampling Procedures
Timeframe As an ongoing evaluation, the evaluation will look primarily at recent developments and assess the direction of work in this area. Data collected will cover the eight-month period.
Geographical scope: The evaluation will have ‘desk review’ and ‘in-depth’ study countries.
The former will include all the ECA RO 21 countries and territories that have adapted ECD related services in response to COVID-19, while the latter will focus on Georgia, Croatia, Ukraine, and Moldova (see Annex 1 for project details by country). Selection of specific municipalities, cities, districts within each in-depth study country will be detailed at the inception stage. Selection of specific respondents will also be identified separately for each country.
Programming: As stated above, the programme focus is on ECD project activities which are falling under one or more of the three stages of adaptation as discussed above. Reporting requirements
It is expected that the evaluation will have a joint Inception Report, analytical briefs based on rapid cycle data collection, Evaluation Report for each in-depth country based on the briefs, and a regional synthesis report of lessons learned, findings, and recommendations based on the evaluation reports for countries.
Outlines and descriptions of each evaluation products are meant to be indicatives, and include:
Inception report: The inception report (in English) will be key in confirming a common understanding of what is to be evaluated, including additional insights into executing the evaluation. At this stage evaluators will refine and confirm evaluation questions, confirm the scope of the evaluation, further improve on the methodology proposed in this ToR and their own evaluation proposal to improve its rigor, as well as develop and validate evaluation instruments. The report will include, among other elements: i) evaluation purpose and scope, confirmation of objectives and the main themes of the evaluation; ii) evaluation criteria and questions, final set of evaluation questions, and evaluation criteria for assessing performance; iii) evaluation methodology (i.e., sampling criteria), a description of data collection methods and data sources (incl. a rationale for their selection), draft data collection instruments (with a data collection toolkit as an annex), an evaluation matrix that identifies descriptive and normative questions and criteria for evaluating evidence, a data analysis plan, a discussion on how to enhance the reliability and validity of evaluation conclusions, a description of the quality review process and a discussion on the limitations of the methodology; iv) proposed structure of the final report; v) evaluation work plan and timeline; vi) resources requirements (i.e., detailed budget allocations, tied to evaluation activities, work plan) deliverables; v) annexes (i.e., organizing matrix for evaluation questions, data collection toolkit, data analysis framework). The inception report will be 2025 pages in length (excluding annexes), and will be presented at a formal meeting of the reference group.
Analytical briefs based on the iterative and rapid data collection in each country will be produced three times during the course of the evaluation on the analysis of the ECD project adaptations and provide overall emerging findings from the survey and interviews. There will be three briefs soon after each cycle of data collection.
Country Evaluation report for each in-depth country will answer the sub-questions relevant to each in-depth country. The report (in English) will not exceed 40 pages, excluding the executive summary and annexes. A complete draft report will include: i) Title page and opening pages; ii) executive summary; iii) brief background, purposes, scope and objectives iv) an analysis of key issues against the evaluation questions v) the lessons emanating from those experiences; and vi) annexes. These reports will be delivered in March 2021.
A Synthesis report will answer the main evaluation questions. The report (in English) will not exceed 40 pages excluding the executive summary and annexes. A complete draft report will include: i) Title page and opening pages; ii) executive summary; iii) brief background, purposes, scope and objectives iv) an analysis of key issues against the evaluation questions v) the lessons emanating from those experiences; and vi) annexes.
This report will be delivered in March 2021.
All reports should be electronically delivered to UNICEF. Completed data sets should be returned (filled out questionnaires, surveys, interview notes and tapes, etc.) to UNICEF.
Competitive scope
Evaluator qualifications
The evaluation will be conducted by engaging an institution. The proposed team will consist of two international consultants and one national technical expert in each country (Georgia, Croatia, Ukraine, Moldova).
1. The senior-level Team Leader must have the following competences:
- • Having extensive evaluation experience (at least 10 years) with an excellent understanding of evaluation principles and methodologies, including capacity in an array of qualitative and quantitative evaluation methods, and UNEG norms and standards. It will be a requirement that the team leader can prove extensive expertise in developmental evaluations.
- • Having experience on ECD sector reforms – planning, implementing, managing or monitoring such programmes.
- • Holding an advanced university degree (Masters or higher) in international development, public policy or similar, including sound knowledge of policy and systemic aspects; familiarity with ECD and education programmes.
- • Bringing a strong commitment to delivering timely and high-quality results, i.e., credible evaluations that are used for improving strategic decisions.
- • Having in-depth knowledge of the UN’s human rights, gender equality and equity agendas.
- • Having a strong team leadership and management track record, as well as excellent interpersonal and communication skills to help ensure that the evaluation is understood and used.
- • Specific evaluation experience in the ECD and education sector is strongly desired, but is secondary to a strong mixed-method evaluation background, so long as the ECD and education expertise of the other team members (see below) is harnessed to ensure the team’s collective understanding of issues relating to systems strengthening from a UN or NGO perspective.
- • Previous experience of working in Europe and Central Asia context is a must.
- • S/he must have the ability to concisely and clearly express ideas and concepts in written and oral form as well as the ability to communicate with various stakeholders in English.
The Team Leader will be responsible for undertaking the evaluation from start to finish, for managing the evaluation team, for the bulk of data collection, analysis and consultations, as well as for report writing in English and communication of the evaluation results.
2. The international Junior Analyst must have the following competencies:
- Minimum 5 years’ experience in quantitative and qualitative data design, collection and analysis for large scale surveys, research projects or evaluations.
- Holding advanced university degrees (Masters-level) in Statistics, public policy or similar.
- Familiarity in UN’s human rights, gender equality and equity agendas; knowledge of ECD programming a plus.
- S/he must have the ability to concisely and clearly express ideas and concepts in written and oral form as well as the ability to communicate with various stakeholders in English.
The Junior Analyst will work under the stewardship of the Team Leader in support data collection and analysis.
3. One national consultant (one per participating country – 4 countries) must have the following competencies:
- • Holding advanced university degrees (Masters-level) in Statistics, public policy or similar.
- • Hands-on experience in collecting and analysing quantitative and qualitative data.
- • Strong expertise in equity, gender, equality and human rights-based approaches to evaluation and expertise in data presentation and visualisation.
- •Having good communication, advocacy and people skills and the ability to communicate with various stakeholders and to express concisely and clearly ideas and concepts in written and oral form.
- • Excellent English communication and report writing skills; knowledge of local languages is obligatory.
- • Having excellent understanding of the ECD and education system in ECA region through previous experience or research work.
The national consultant will play a major role in data collection, analysis and presentation, and preparation of the debriefings and will make significant contributions to the writing of the main evaluation report.
It is vital that the same individuals that develop the methodology for the proposal will be involved in conducting the evaluation. In the review of the proposal’s, while adequate consideration will be given to the technical methodology, significant weighting will be given to the quality, experience (CV’s and written samples of previous evaluations) and relevance of individuals who will be involved in the evaluation.
Submission Guidelines
- The Proposals must be received at the below address by latest 07 September 2020, Monday, at or before 19:00 hrs (Ankara time). Proposals received after the stipulated date and time will be invalidated. Due to the nature of this RFP, there will be no public opening of proposals. *Full proposals must be submitted in ENGLISH and must be received not later than the closure date and time as duly signed, stamped and dated.
**Due to the COVID-19 pandemic situation, exceptionally for the subject tender, electronic submission of the proposals by email will be accepted.
- Accordingly, bidders are requested to send electronic version of their proposals by email as duly signed &stamped pdf documents to ([email protected]) in two (2) separate emails with the tender reference in their email subjects. Proposals should not be uploaded to any type of cloud/sharing platforms. The submission documents should be sent as email attachments.
- Email Subject:
- Confidential: RFP-TURA-2020-11 (LRPS-2020-9159638)
- one email / emails for technical proposals (each should have a number such as 1/4, 2/4, etc.)
- one email for financial proposals (password protected)
- To help us track our procurement effort, please indicate in your email where (vacanciesinturkey.com) you saw this tender/procurement notice.
Financial proposals shall be secured by a password, the password will be requested only from the technically qualified bidders and therefore eligible for opening of the financial proposals. All other provisions of the tender are still valid and proposals received in any other manner will be invalidated.
**Please note that this tender is open both for national and international institutional consultancy companies which are legally registered.
- All requests for formal clarification or queries on this RFP must be submitted in writing to Supply Unit via email (till 03 September 2020, 18:00 at the latest) to: [email protected]