Request for Proposal (RFP) for the Final Evaluation of the Skills to Succeed Global Grant 7 (GG7)

1. Background

Save the Children Federation, Inc. (hereafter “SCUS”) is the world’s leading independent organization for children. We work in 120 countries, including the US and helped more than 166 million children, including more than 55 million children directly. We save children’s lives, we fight for their rights, and we help them fulfill their potential. We work together, with our donors and partners, to inspire breakthroughs in the way the world treats children and to achieve immediate and lasting change in their lives.

Save the Children’s Skills to Succeed (S2S) program supports adolescents and youth who experience the most inequality, including youth with disabilities, migrants and refugees, to gain the skills and access the opportunities they need to continue their education, get decent jobs or start businesses, and break the intergenerational cycle of poverty. During Global Grant 7, the program works with very young adolescents (ages 8-14) to build their social-emotional and digital skills so they can succeed in school and be prepared for the future of work. Considering the 2025 goal to reach 2 million Next Gen (ages 8-14), through this grant Save the Children will reach 263,040 Next Gen, helping them acquire the mindsets and skills they need to thrive in education and in the future of work. Additionally, the program aims to make the changes sustainable and effective; thus, the program seeks to also work, and benefit educators, parents and other adults who are critical spheres of influence in children’s educational and employment choices. In this sense, Save the Children will reach 60,824 adults to ensure a supportive enabling environment. Of the adults reached, Save will build the capacity of 3,774 educators.

As part of the S2S research, Save the children will conduct final evaluation of the Global Grant 7 in three countries (Bangladesh, China, Italy). The main objective of the evaluation is to understand the impact of the S2S program on very young adolescents and gather perceptions and lessons learnt from implementing the program. Save the Children is looking to hire an Evaluation Consultant to conduct the GG7 final evaluation in consultation with the S2S teams.

Please see full RFP below or a formatted version here.

2. Scope of the study

1. Evaluation Objectives

This study is being conducted in the last year of a 3-year grant for the S2S program (Global Grant 7 (GG7)).

The study main objectives are:

1. Understand the impact of the S2S program on very young adolescents’ outcomes such as Socio-Emotional Skills, Growth Mindset, Digital Skills, Renewed Commitment to Education, and Gender Equality and Social Inclusion perceptions.
2. Gather perceptions and lessons learnt from implementing the S2S trainings and activities to very young adolescents, teachers/trainers, and parents.
3. Obtain information of potential strategies, models or activities that partners would find sustainable to carry out after the program ends.

The study team will undertake consultation with the SC team at the commencement of the project in order to further refine the study questions (note that you can find this information in a table format here.)

2. Key Evaluation Questions

Research Question

Methodology

Very Young Adolescents (VYA):

1) How effective was the program in building SEL, growth mindset, and digital skills for very young adolescents (ages 8-14)?

Qualitative

• FDGs/KIIs withVYA
• FDGs/KIIs with Parents
• FDGs/KIIs with Teachers/Trainers

Quantitative

• Data analysis using inferential statistics with countries’ tools (e.g., ISELA, Cosmonauta*)

*Save the Children to provide data

2a) How effective was the program in re-invigorating commitment to education and career readiness? (e.g., do children have more ideas about their future careers, knowledge of labor market opportunities)

2d) How effective was the program in changing attitudes related to gender equality and social justice for VYAs?

Qualitative

• FDGs with VYA

Quantitative

• Data analysis using inferential statistics with countries’ tools (e.g., ISELA, Cosmonauta* and Endline Surverys)

*Save the Children to provide data

Enabling environments:

3a) How effective was the approach to training teachers/facilitators?

3b) How effective was the program in changing attitudes related to gender equality and social justice for parents, teachers, and other adults?

3c) How relevant are the trainings to the VYAs to successfully make their way through education trajectories and future employment?

Qualitative

• FDGs with teachers
• FGDs with parents
• FDGs with VYA
• KII with Adult Volunteers

Systemic approaches and sustainability:

4a) What models did countries use to ensure sustainability of program interventions? What were some challenges, lessons learned, and best practices?

4b) What is the opinion of the schools and implementing partners in terms of relevant and effectiveness of the programs? Which, if any, parts of the program will they continue even after the program ends?

Qualitative

• KII with STC Staff
• KII with Schools’ Authorities
• KII with Key Other Partners

Operational Learnings:

5) What were the lessons learned for implementation? What worked well? What should be done differently next time?

Qualitative

• FDGs/KIIs with STC Staff
• FDGs/KIIs with VYA
• FDGs/KIIs with Teachers/Facilitators

3. Study methodology

1. Study Data Collection Plan

The following study data collection plan is proposed, which includes a combination of existing and primary data collection.

The consultant will develop the research protocol and tools for all qualitative data collection. However, data collection with VYA/youth, teachers/facilitators, parents and local partners will be conducted by local enumerators hired and managed by SC country office teams. Raw translated data will be provided to the global consultant for analysis.

All primary data collected during the study must facilitate disaggregation by gender and age. The study data collection plan includes estimates for the number of focus group discussions (FDGs) and Key Informant Interviews (KIIs) to be conducted. The study data collection plan will be further revised at the commencement of the project.

The study team is required to adhere to the Save the Children Child Safeguarding; Protection from Sexual Exploitation and Abuse; Anti-Harassment, Intimidation and Bullying; and Data Protection and Privacy policies throughout all project activities.

All secondary data, such as reports, past studies and quantitative data will be provided by Save the Children.

The reach of the study will contain three main countries that worked with Very Young Adolescents (8 – 14 years old):

• Bangladesh
• China
• Italy

2. Ethical Considerations

It is expected that this study will be:

• Child participatory. Where appropriate and safe, children should be supported to participate in the evaluation process beyond simply being respondents. Opportunities for collaborative participation could include involving children in determining success criteria against which the project could be evaluated, supporting children to collect some of the data required for the evaluation themselves, or involving children in the validation of findings. Any child participation, whether consultative, collaborative or child-led, must abide by the 9 Basic Requirements for meaningful and ethical child participation.
• Inclusive. Ensure that children from different ethnic, social and religious backgrounds have the chance to participate, as well as children with disabilities and children who may be excluded or discriminated against in their community.
• Ethical: The study must be guided by the following ethical considerations:
  • Safeguarding – demonstrating the highest standards of behavior towards children and adults.
  • Sensitive – to child rights, gender, inclusion and cultural contexts.
  • Openness – of information given, to the highest possible degree to all involved parties.
  • Confidentiality and data protection – measures will be put in place to protect the identity of all participants and any other information that may put them or others at risk.[1]
  • Public access – to the results when there are not special considerations against this
  • Broad participation – the relevant parties should be involved where possible.
  • Reliability and independence – the study should be conducted so that findings and conclusions are correct and trustworthy.

It is expected that:

• Data collection methods will be age and gender appropriate.
• Study activities will provide a safe, creative space where children feel that their thoughts and ideas are important.
• A risk assessment will be conducted that includes any risks related to children, young people’s, or adult’s participation.
• A referral mechanism will be in place in case any child safeguarding or protection issues arise.
• Informed consent will be used where possible.

The study team will be required to obtain approval from a Human Research Ethics Committee. Save the Children will provide assistance with this process

IMPORTANT NOTE TO CONSIDER: For implementation of the study, the consultants would not need to travel to the countries for implementation of the tools as each country will be doing their own translation of the tools developed by the consultant, as well as their own data gathering. For this process, the consultant will be responsible for developing the tools and training the implementors of the tools for each country. The countries will then oversee transcribing the results from the tools and translating them into English for the consultant to use in the analysis.

1. Expected Timeline & Deliverables (note that you can find this information in a table format here.)

The tentative timeline (subject to the commencement date of the study) and study deliverables are outlined below.

1. 1. Study Timeline

Key deliverables in bold

What

Who is responsible

By when

Who else is involved (if applicable)

Selection of consultant

Save the Children US

March 2023

Document review, desk research

Study team

March 2023

Develop research protocol and tools

Study team

April 2023

Review of research protocol and tools

SCUS MEAL team

April-May 2023

Ethics submission

SCUS MEAL team

April-May 2023

Training of local enumerators

Study team

May 2023

SC enumerators

Qualitative Data collection*

Study team; SC Country Office staff; enumerators

May-June 2023

SC enumerators

Data management and analysis (coding, transcriptions, data cleaning, integration and analysis)

Study team

June 2023

First draft of the final study report

Study team

July 2023

Review of first draft report

SC MEAL team; SC program staff

July 2023

Final study report and Validation Call with Countries

Study team, SC Country Office staff, SC MEAL team; SC program staff

August 2023

Presentation of final study

Study team

August 2023

SC MEAL team, SC program staff

*Exact timing of data collection may be adjusted based on school year and availability of participants

1. 1. Deliverables

The study team will produce the following deliverables:

Deliverables

Timeline

Research protocol and data collection tools (April-May 2023) which includes:

• Description of the Program and context relevant to the Study using the literature review and desk research.
• Study objectives, scope and key study questions
• Description of the methodology, including design, data collection methods, sampling strategy, data sources
• Data analysis and reporting plan
• Caveats and limitations of study
• Risks and mitigation plan
• Ethical considerations, including details on consent and data privacy
• Data collection tools (e.g., surveys, FDG/KII guides, etc.)

**Training for local enumerators (**May 2023)

• Provide remote training to local enumerators who will do data collection

**Final Study Report (**July-August 2023) including the following elements:

• Executive summary
• Background description of the Program and context relevant to the Study using literature review and third party data.
• Scope and focus of the study
• Overview of the study methodology and data collection methods
• Findings aligned to each of the key study questions
• Specific caveats or methodological limitations of the evaluation
• Conclusions outlining implications of the findings or learnings
• Recommendations
• Annexes (Project logframe, Inception Report, List of people involved)
• Data and analyses including all encrypted raw data, databases and analysis outputs

**Report brief of 2 to 5 pages (**July-August 2023) including the following elements:

• Brief summary of the project
• Objectives of the research
• Methodology of the study including sample, tools description and field work
• Key findings
• Recommendations

A consolidated set of feedback from key stakeholders will be provided by SC within one week of the submission of the draft report. The final report will incorporate feedback from SC.

**Final presentation (**August 2023) on study report to SC MEAL and program staff

• An oral powerpoint presentation on the key findings from the study

All documents are to be produced in MS Word format and provided electronically by email to SC. Copies of all PowerPoint presentations for the project should also be provided to SC in editable digital format. All deliverables must be approved by SC staff

1. Selection Criteria

To be considered, the study team members together must have demonstrated skills, expertise and experience in:

• Designing and conducting evaluations using mixed methods design.
• Conducting studies in the field of youth livelihoods, education, particularly in relation to life skills, social-emotional learning, digital skills and gender equality and social inclusion.
• Conducting ethical and inclusive studies involving children and child participatory techniques
• Conducting ethical and inclusive studies involving marginalised, deprived and/or vulnerable groups in culturally appropriate and sensitive ways
• Managing and coordinating a range of government, non-government, community groups and academic stakeholders
• Strong written and verbal skills in communicating technical and/ or complex findings to non-specialist audiences (especially report writing and presentation skills)
• A track record of open, collaborative working with clients

There is a high expectation that:

• A team leader will be appointed who has the seniority and experience in leading complex study projects, and who has the ability and standing to lead a team toward a common goal.
• The team has the ability to commit to the terms of the project and have adequate and available skilled resources to dedicate to this study over the period.
• The team has a strong track record of working flexibly to accommodate changes as the project is implemented.

1. Financial Proposal, compensation & schedule of payment

Financial Proposal:

Save the Children seeks value for money in its work. This does not necessarily mean “lowest cost”, but quality of the service and reasonableness of the proposed costs. Proposals shall include personnel allocation (role / number of days / daily rates / taxes), as well as any other applicable costs.

Compensation:

The compensation will be based on number of days worked by the Evaluation Consultant. The total amount for the study will not exceed $53,000.

Schedule of Payment:

The following terms of payments will be applied for this consultancy:

• Upon approval of protocol and data collection tools: [25%]
• Upon completion of the enumerator training and data gathering [25%]
• Upon approval of final study report and presentation [50%]

Annex:

Annex 1: SCI Evaluation Scoring for perspective consultants

Category

Evaluation Quality Criteria (used for internal scoring after completion)

Purpose, Design and Methods

1. Does the expression of interest clearly identify the evaluation’s purpose (including its key objectives, questions and criteria) as set out in the evaluation’s Terms of Reference (ToR)?

2. Are the data collection and analysis methods a clearly justified approach to addressing the evaluation’s purpose and questions? (Do they provide valid, reliable and ethical data?)

3. Is the methodology suitably tailored to the context and population groups to which the evaluation questions relate (e.g. re gender, disability, socio-economic status, geographic location, cultural context, ethnicity)?

4. Is the size and composition of the sample in proportion to the conclusions sought by the evaluation?

5. Does the evaluation build on what is already known, for example existing tried and tested frameworks and tools, existing data/evidence, and previous lessons learned?

6. Are the methods used to collect and analyse data and any limitations of the quality of the data and collection methodology explained and justified?

Please find the RFP posted here.

[1] If any Consultancy Service Provider, Freelancer or Contingent worker will have direct contact with children and/or vulnerable adults and/or beneficiaries and/or have access to any sensitive data on safeguarding and/or children and/or beneficiaries, it is the responsibility of the person receiving the consulting service to contact the local HR team and child safeguarding focal point to ensure vetting checks and on-boarding are conducted in line with statutory requirements, local policies and best practices guidance.

How to apply

Save the Children seeks expressions of interest from evaluator/s. Interested evaluator/s should clearly indicate their background experience and knowledge of the subject area, how they meet the required skills and experience, and their availability. Selection of the evaluator/s will be done by Save the Children and will be based on the strength of the qualifications provided by potential candidates through their:

• Expression of interest/cover letter
• Technical Proposal Summary (5 pages Max.) requiring: 1) Sample size for Qualitative Data collection, 2) Analysis Strategy, and 3) Timeframe.
• CV
• Example of a previous evaluation report with similar scope of work
• Cost estimate (including daily rate and related expenses)

Potential evaluator/s can work remotely and do not need to live in Washington DC or the USA. Submission should include an expression of interest/cover letter, CV, sample evaluation report and cost estimate. The response from the suppliers should be returned as a signed form or on official headed paper. The deadline to submit expression of interest is close of business on Friday, 17th March. All submissions should be sent to Felipe Sarmiento at [email protected] and Mark Meziere at [email protected].