Consultancy for End of Project Evaluation in Sudan (ToR)

Terms of References

I. Description of Project Being Evaluated

Project duration: 09/2018 – 12/2021 (40 months)

Sudan has experienced ongoing conflicts and displacement for more than a decade, leading to food insecurity, livelihoods, health and nutrition. In the Darfur region, the protracted conflict in connection with natural disasters and underinvestment in social services has contributed significantly to the current humanitarian situation. The cumulative effects predispose individual households and communities to malnutrition, high morbidity and mortality from communicable and easily preventable diseases.

World Vision (WV), with funding from the German Federal Ministry for Economic Cooperation and Development (BMZ) through its instrument for transitional development assistance (TDA), has been implementing the multi-sectoral project “Building Community Resilience through improved agricultural production capacity, improved nutritional status and better access to health services and facilities for vulnerable returnee and host community households in Mershing and Alwehda Region” since September 2018 in South Darfur. This 40-month intervention is addressing root causes of vulnerability by building community resilience in the areas of food and nutrition security and rehabilitation of basic social and productive infrastructure as well as contributing to the (re)integration of refugees and IDPs. The project targets a total of 194,074 beneficiaries, including 55,382 returnees, 117,662 IPDs, and 21,030 particularly vulnerable members of the host communities whose livelihoods are largely based on agricultural activities, the intended project outcomes are:

(1) improved food security and livelihoods situation of vulnerable households through building productive capacities.

(2) improved local capacity of communities and State Ministry of Health to prevent and manage undernutrition.

(3) improved quality, availability and accessibility of health and nutrition services. The project has a strong focus on SDG 1 (No Poverty), 2 (No Hunger) and 3 (Health and Welfare). Focusing on women and strengthening their capacities, the project also contributes to SDG 5 (gender equality).

Key intervention packages aim at strengthening productive capacities and provide households with seeds and agricultural tools, training households on sustainable, climate-adapted farming practices, establishing seed bank systems, supporting women's savings and credit groups, and installing irrigation schemes. Through these interventions, households are empowered and supported to engage in productive activities, enabling them to satisfy their nutritional needs through adequate and healthy food. The creation of adequate food and monetary reserves, made possible by more efficient agriculture, makes a strong contribution to strengthening the resilience of the beneficiaries. The resulting improvement in food safety status complements the reduction of malnutrition, which is anticipated as an impact of the nutrition and health activities implemented in partnership with the local NGO Jabal Mara Charity Organisation for Rural Development (JMCO), including the establishment of community-based nutrition groups, use of vegetable gardens, community events, and screenings to detect malnutrition. The dissemination of nutritional and health-related knowledge among community members, by community members themselves, initiates a process of behavioural change that will contribute to the widespread adoption of good nutritional practices. Combined with the ability to identify, diagnose, and treat cases of malnutrition early on, the prevalence of malnutrition is anticipated to decrease. In addition to community capacity, public structures are encouraged to provide quality and accessible health services. A total of three primary health care centres are being rehabilitated and/or expanded, equipped with medical equipment and furniture, trained by health ministry staff and supported in providing basic health services. The focus is on the health of pregnant and breastfeeding women as well as infants and toddlers. The support of health centres and the development of technical capacities will support the overburdened public health structures of the target municipalities to fulfil their mandate more effectively. The impact of this improvement is expected to be reflected in increased health service utilization and improved health status, especially among women and infants. Based on preliminary successes, the project was expanded with additional funds in March 2021 to cover additional beneficiaries and the project completion date was extended until 31/12/2021.

Logframe (Excerpt)

Outcome 1: Improved food security and livelihoods situation of vulnerable households through building productive capacities

1.1: 40% of households which consume two or more meals per day

1.2: Reduction of average number of cereal deficit months for a household to 3 months

1.3: 50% of households with acceptable food consumption score

1.4: 50% increase of average annual household income

Outcome 2: Improved local capacity of communities and State Ministry of Health to prevent and manage under-nutrition

2.1: 50% reduction in the prevalence of undernutrition among children under 5 years

2.2: 60% of households have been educated in diversified and balanced diet

2.3: 60% of Lead Mothers are able to recognize symptoms of malnutrition and issue corresponding recommendations

2.4: 80% of CHPs know at least 6 IYCF practices

Outcome 3: Improved quality, availability and accessibility of health and nutrition services within the host communities

3.1: 3 health facilities rehabilitated/expanded

3.2: 72,155 people receive curative and preventive consultation at the health facility

3.3: 60% of health facilities reporting no stock out of 10 essential drugs

Output 1.1: Increased agricultural productivity through availability of improved seed varieties and utilization of sustainable agricultural production techniques

1.1: 60% households that attain target yields for the staple crops (sorghum - 360 Kg/Fed, millet 200 Kg/Fed and groundnuts - 270 Kg/Fed).

1.1.2: 4,500 HHs receive seed for staple crops

1.1.3: 60% of targeted farmers adopt and practice at least 6 improved techniques of agricultural production

Output 1.2: Established community managed seed banking system

1.2.1: 4 seed banks established and used

1.2.2: 60% of households' access and use improved seeds

1.2.3: 4 community committees established and put functional to manage the seed banks

Output 1.3: Increased income and promotion of economic reintegration of returnees within the host communities through women saving groups

1.3.1: 60% of Women Saving Group (WSG) members reporting an increase of annual income

1.3.2: 28 WSG using installed small-scale irrigation system (SSIS)

1.3.3: 30 local saving groups for women established

Output 2.1: Increased knowledge and practice of appropriate maternal and child nutrition, hygiene, sanitation and family health practices within the targeted communities

2.1.1: 60% increase of knowledge on child and maternal health within the target communities

2.1.2: 810 peer education sessions (led by Lead Mothers) conducted

2.1.3 4,400 Mothers with children under 5 years attended care group sessions

Output 2.2: Increased access to micronutrients to reduce micro-nutrition deficiency among children under 5 years and PLW in communities

2.2.1: 50% of HHs reflect a reduction of micronutrient deficiency

2.2.2: 370 Kitchen gardens established

2.2.3: 370 Lead mothers (LMs) (360) and Community Health Promoters (CHPs) (10) provided with kitchen garden seeds and tools

Output 2.3: Enhanced knowledge of improved IYCF practices for children under-2 years within targeted communities

2.3.1: 4800 caregivers receiving infant and young child feeding (IYCF) counselling through LMs and CHPs

2.3.2: 60% exclusive breast feeding to children less than 6 months

Output 3.1: Primary Health centres rehabilitated/expanded and capacity of health staff developed

3.1.1: 3 health facility providing full range of PHC services according to BPHS

3.1.2: 3 primary health care centers rehabilitated/expanded

3.1.3: 36 PHCC health staff trained

Output 3.2: Increased utilization of health care services

3.2.1: 1700 children U2 received routine vaccinations (BCG, DPT, OPV, Measles)

3.2.2: 1300 Women of reproductive Age (WRA) received vaccinations against Tetanus and other routine vaccinations

3.2.3 4,100 pregnant women received ANC+2 services

II. Evaluation Target Audiences

Donor: BMZ expects findings about resilience and about all Log Frame indicators and an assessment of the intervention based on the DAC-criteria for development interventions.

Co-financing Org: World Vision Germany and Implementer World Vision Sudan need to render accountability towards the donor BMZ and expects findings about resilience and about all LF indicators and an assessment of the intervention based on the DAC-criteria for development interventions. Any findings and/or recommendations for internal learning are of added value, such as good practices and/or specific challenges.

Local Partner Organisation (Jabal Mara Charity Organisation for Rural Development) and beneficiaries: should be actively involved in the evaluation process and be informed about the results and recommendations of the evaluations.

III. Evaluation Purpose and Objectives

The purpose of the evaluation is to assess and document the impact and effectiveness of project interventions to render accountability to the donor and target communities. Any findings and/or recommendations for internal learning are an added value, such as good practices and/or specific challenges. They will contribute to a learning process which enables World Vision to draw lessons from its experience in order to improve future programming.

The evaluation is expected to use existing (monitoring) data and where necessary collect new data on all of the Logframe indicators as well as on specific objectives about resilience. The approach on strengthening resilience and its objectives are outlined in the strategy for transitional development assistance (TDA) from BMZ (https://www.bmz.de/en/news/publications/publikationen-reihen/30738-30738) and should be considered in the evaluation.

The evaluation shall provide a detailed analysis about the contribution of World Vision and the local partner JMCO to the changes and effects of the project as opposed to other actors and factors.

The evaluation will collect and analyze Gender Equality, Disability and Social Inclusion (GEDSI) data to identify any gaps for women, men, persons with disabilities and other vulnerable groups in access, participation, decision making, systems and well-being. It will determine the main causes of these gaps and identify both the positive and negative effects of the program on different social groups.

There is indication that positive changes in the project area motivated neighbouring communities that were not targeted by the project to adopt certain techniques that were introduced by the project. A rough estimation of the reach and favoring factors of the diffusion should be investigated.

The intervention has to be assessed against the DAC criteria for development interventions, which are relevance, coherence, effectiveness, efficiency, impact and sustainability (https://www.oecd.org/dac/evaluation/revised-evaluation-criteria-dec-2019.pdf).

Key Objectives with Key QuestionsRelevance** The extent to which the intervention objectives and design respond to beneficiaries’, global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change.

• To what extent are the objectives of the project aligned with global, regional and country-specific policies and priorities, especially policies and priorities of BMZ, World Vision and local partners?
• To what extent do the project’s objectives consider the needs and priorities of the target groups, including women and men, people with disabilities and other vulnerable groups, as well as their capacities?
• To what extent did the project reach most vulnerable population groups, especially most vulnerable children and their families? -> For the livelihoods components it may also be interesting to see if the project targeted to right individuals in the household (e.g. training the persons that are actually responsible within the household for the activities the training focuses on; if seeds were distributed to the person mainly responsible to grow the respective crop(s) etc.).
• To what extent were the interventions adapted to changing circumstances (risks and potentials) during implementation, especially Covid-19?
• Does the program logic allow to achieve the project’s objectives (Quality of ToC)?
• Are the activities and outputs of the project consistent with the intended impacts and effects?
• Were the targets set realistic?
• To which extent have the community and stakeholders participated in planning and implementation of projects interventions?

Coherence The compatibility of the intervention with other interventions in a country, sector or institution.

Internal coherence

• To what extent does the project contribute to synergies and/or interlinkages with other interventions of World Vision in the same region?

External coherence

• To what extent is the project complementary with other interventions in the same region?**

-> For livelihood components, this should include the complementarity of promoted techniques, provided trainings and distributed inputs and equipment. Are these in line or conflicting with agricultural policies and support provided by other interventions (e.g. if the government and/or other interventions promote conservation agriculture with minimum tillage, the hand-out of ploughs would indicate a slightly different incentive/impulse)?**

• To what extent did the project build on existing local structures and systems, especially local farming systems, and to what extent have they been used?**

Nexus implementation

• How does the project tie in with humanitarian assistance and peacebuilding interventions and contribute to “Collective Outcomes” for humanitarian assistance, peacebuilding and development cooperation?

Effectiveness To assess the progress made towards achieving the project’s goal and objectives based on the logframe, design and monitoring data; the extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups.

• To what extent did the programme achieve its outputs, outcomes and goals?
• To what extent did the project contribute to changes and effects as opposed to other actors and factors?
• To what extent did the project follow technical standards and/or project models from World Vision?
• What were the major internal (technical, financial, organizational) and external factors influencing the achievement or non-achievement of the objectives?
• Did the pandemic Covid-19 affect the project implementation and/or the achievement of objectives?
• How many people have benefitted and how is the distribution of benefits among different social groups (e.g. men, women, children, most vulnerable, etc.)?
• What are the effects of the project activities on women, men, persons with disabilities and other vulnerable groups?
• Did the program use the right activities for women, men, persons with disabilities and other vulnerable groups?
• Have critical components for strengthening resilience in the 3 dimensions (stabilization capacity, absorption capacity, transformation capacity; based on TDA-strategy of BMZ) on the different levels of individual, household, community been implemented and do they show the intended results?
• To what extent did the project contribute to strengthen resilience in the 3 dimensions on the different levels?

Efficiency The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.

• Have the outputs been delivered in an economic and timely way?

Impact To assess the potential impact of the program on the targeted communities

• The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.
• It does so by examining the holistic and enduring changes in systems or norms, and potential effects on people’s well-being, human rights, gender equality, and the environment.
• Did the project contribute to any positive and negative, intended, and unintended higher-level effects in the targeted communities? -> Especially in Child Well-being according to the child well-being objectives of World Vision? -> Did the project contribute to gender equality? -> Did the project contribute to spread environmental-friendly agricultural techniques?
• What real difference has the intervention made to the beneficiaries?
• Are the beneficiaries and local structures more resilient than before as a result of project interventions?
• To what extent has a diffusion of new practices taken place in neighbouring communities of the project area (rough estimation)?
• What were the determining factors for the persons to adopt new approaches of neighbouring communities?

Sustainability The extent to which the net benefits of the intervention are likely to continue.

• How likely is it that the positive changes will be sustained after the project has ended?
• Had the project planned and implemented an exit strategy agreed with local stakeholders to increase the likelihood of sustainability?
• To what extent does exist local ownership? -> To what extent are key responsibilities transferred to local stakeholders?
• Do the local stakeholders have the capacities to sustain positive changes after the project has ended? Did the project contribute to strengthen these capacities?

IV. Evaluation Methodology

The consultant will be responsible to design the evaluation, including its detailed methodology: refinement of evaluation questions, sampling, data collection instruments, data management, methods for data analysis and interpretation, etc.

The baseline study does not capture most of the logframe indicators, so an alternative approach to establish the counterfactual needs to be designed by the consultant. Data exists for some logframe indicators from a KAP Survey from October 2020 which might serve as midline. We expect the consultants to check the available previous studies and monitoring data to assess to what extent and how this data might be used for the evaluation. Furthermore, we expect that the consultant will check the logframe indicators and advise to what extent some indicators might measure the same or very similar aspects and could be skipped, if the donor accepts

For the logframe indicators measured at baseline, the methodology from baseline (and if applicable other studies like post harvest assessment) for measuring the logframe indicators needs to be replicated (including sampling, tools, etc.) to enable statistically sound comparison of baseline and endline results, where feasible, and should be tailored to serve the purpose of the evaluation. Any deficits in baseline methodology and therefore deviances in the evaluation methodology needs to be discussed and outlined in the inception report and noted in the evaluation report.

The evaluation should follow a collaborative and participatory mixed methods approach that draws on both existing data (project design, monitoring data and former data collection) and collect new quantitative and qualitative data to answer the evaluation questions. The evaluation should combine evaluation tools based on international standards and guidelines like the DAC Quality Standards for Development Evaluation. The choice of method must consider the needs and capacities of the different target groups and stakeholders (e.g. women, men, persons with disabilities, children and adolescents, beneficiaries, staff of World Vision and local partners).

It is expected that the consultant will

• check the quality of the project’s impact logic and if necessary, to develop a realistic impact logic based upon on the conducted interventions

• will make a detailed analysis about the contribution of World Vision, the local partner JMCO and other factors and/or actors to the changes, effects and impacts.

• Disaggregation of data according to gender and other relevant social categories. The evaluation applies a Gender Equality, Disability and Social Inclusion (GESI) lens in approaches, methods, and processes, assessing the relevance, performance, efficiency, and impact (both intended and unintended) of the project on women, men, persons with disabilities and other vulnerable groups. As a minimum, the CGIAR Standards for the collection of sex-disaggregated information should be applied.

Data Management and analysis will fall as well under the responsibility of the consultant/consultancy firm and will include data entry, cleaning and analysis.

Ethical considerations

The evaluation will respect the essential WV ethical guidelines concerning conducting studies with children (anyone under 18) and community members (Adults). It is crucial to ensure that the risks of potential harm to participants resulting from the data collection process are minimized and are outweighed by the potential benefits of the outcomes of the study. This also requires the planning and design of a favorable and non-harmful research setup (e.g. location, time, social environment). The consultant is required to abide by the same ethical principles and guidelines developed by WV during this mission, including child safeguarding.

V. Documents

The consultant will be provided with relevant documentation to support the desk review of secondary information once the contract is signed. The consultant will be encouraged to identify any other sources or appropriate additional information that may be required to supplement what is provided by the project*.* Documents available include but are not limited to:

• Baseline report
• Post-harvest assessment
• Project design documents
• Project progress reports
• Monitoring data**
• If available, former data collections on specific components, i.e. S3M nutrition surveys
• TDA strategy of BMZ

VI. Limitations

• The security situation in South Darfur remains precarious especially in the last few months due to tribal clashes and general insecurity. Precautions will be taken in close coordination with local authorities to ensure a smooth flow of activities during data collection. Where the situation will be extremely challenging, proper advice and remedial action will be sought.
• The visa process for Sudan might take up to a month, which has to be considered in the scheduling of data collection timelines in case international travel is planned.
• As of July 2021, according COVID19 prevention procedures, new arrivals in the country have to complete a one-week quarantine, followed by a COVID-19 test.

VII. Products/ deliverables

• Working language: English
• Participation in a virtual kick-off discussion for clarifications of the task
• Templates for the inception report and the evaluation report will be provided

Inception Report: Deadline: Three weeks before the data collection starts. The inception report needs to be approved by World Vision Sudan and Germany.

The consultant will prepare and submit an inception report in English detailing how the evaluation will be carried out from his/her point of view. The report will outline the evaluation design, sampling methods to be used and questions to be answered and detailed work plan for the entire exercise. Draft questionnaires, interview guides and other data collection tools will be submitted to WV for review and approval before data collection starts. As part of the inception report, the consultant must provide a data analysis plan showing the questions and analysis for each of the project indicators to be investigated. Further mandatory elements:

Preliminary Evaluation Report: 3.12.2021

The consultant will submit a draft evaluation report in English to WV Sudan and Germany. The draft report will be reviewed, and comments provided on the report within two weeks of submission.

Final evaluation Report, including 2-page summary fact sheet

The consultant will submit the detailed final report in English outlining the evaluation methodology, findings, lessons learned and recommendations. The report shall incorporate specific simple and achievable recommendations, including the most appropriate strategies that can be undertaken and/or incorporated by WV Sudan and partners to attempt to address the issues identified. The final report should address the issues and questions raised in this ToR and correspond to the evaluation objectives set out above.

VIII. Expected Qualifications of Consultant

• In-depth knowledge of South Darfur State and the respective localities including government and community level service delivery structures
• Masters/PhD degree in Agriculture, Agricultural Economics, Health and Nutrition, Development studies, Peace and Security, Monitoring and Evaluation or other relevant field of study
• 5+ years of international evaluation experience
• Experience with evaluations of BMZ-funded projects, especially financed with the instrument Transitional development assistance.
• Proven strong expertise in quantitative and qualitative evaluation methods including demonstrated skills with statistics analysis software (EPI-Info, SPSS, SAS, STATA or others) and
• Experience in designing or leading Gender Equality Disability and Social Inclusion (GESI)-responsive evaluations or programs, and conducting GESI analysis
• Experiences conducting evaluations of nutrition or food security programs
• Expertise in resilience analysis
• Professional proficiency in English, Arabic an asset

IX. Additional expectations for the consultants

• A team of international and local consultants would be preferred but is not mandatory.
• In our understanding a mixed team would help to combine the understanding of the needs for the German donor and for the Sudanese context and as well as might help to absorb challenges in data collection if circumstances do not allow international travelling or travelling at all.
• Generally, a willingness to travel to the project site in Sudan for data collection is expected if circumstances like security conditions and COVID-19 regulations allow travel. Therefore, our preferences for the evaluation are the following:

1^st priority: Data collection in the project site can be done personally by the evaluation team, including lead consultants.

2^nd priority: Data collection can be done semi-remote, e.g. lead consultants cannot travel but will instruct local partners to do the data collection in the project site

3^rd priority: Data collection will be done remotely only and as far as possible, because no travel allowed.

X. Authority and Responsibility

· WV Sudan will

• be responsible for the selection and contracting of the consultants
• support in provision of reference documents like project proposal, M&E Plan, previous reports.
• Assist in communication to localities and community for the assessment
• Review the assessment plan and tools before use.
• Translate the evaluation tools into Arabic language.
• Hire enumerators.
• will closely observe during training of enumerators.
• will support and facilitate the data collection and help in coordination with the different stakeholders.
• The M&E team conduct internal supervision and data verification during data collection and review the draft report and provide feedback and comments to the consultant
• The interim project manager and project coordinator will be responsible for the field level coordination of the exercise and working closely with the consultant(s) and other evaluation staff. They will also coordinate logistics arrangement with the relevant department as and when needed.
• Will be responsible to organize in-country accommodation and travelling for the consultants, costs will be part of the evaluation budget.

· The consultants will be responsible:

• for their own international travelling to Khartoum.
• for the training of the enumerators
• evaluation design, data collection, data analysis, report writing

· WV Germany will:

• support the selection of the consultancy and agrees on the final candidate
• support the quality assurance of the evaluation,
• review and approve inception report and (draft) evaluation report

XI. Expected timeframe

• Call for applications 19.07.-27.08.2021
• Selection and contracting of evaluation team 30.08.-24.9.21
• Kick-off 27.09.2021
• Inception report 08.10.2021
• Review inception report by WV Sudan & Germany Until 15.10.2021
• Data collection 01.11.-15.11.21
• Debriefing workshop 19 .11.2021
• Preliminary Evaluation Report 03.12.21
• Review of draft evaluation report by WV Until 17.12.2021
• Submission of final evaluation report 10.01.2022

How to apply:

The following information needs to be submitted for application:

• An application letter of 1-2 pages outlining the consultants understanding of the Terms of Reference and general methodological suggestions, especially how the counterfactual could be established without baseline for this project.
• a budget with amount of working days and daily rates based on the details in the TOR.
• Work plan with rough timeline
• Organizational capacity statement, past experience and activities related to Project Evaluations.
• Curriculum Vitae for all the consultants and proposed capacity of any complementary staff (CVs submitted should be of staff that will actually be on this assignment, including contact details and IDs).
• 3 work samples of relevant work undertaken in the last 3 years
• Names, addresses, telephone numbers of three organizations that will act as professional referees.
• Full names, post office box, telephone numbers, e-mail address, physical address and contact person of the consultancy
• Copy of Covid-19 vaccination certificate for those traveling to the field

Deadline for CVs, references and work samples: 27th August 2021 17:00 CEST

Place of submission: Ibrahim_Bakhit@wvi.org,Ibrahim Bakhit, Supply chain manager of WV Sudan