Program Evaluator

Background

ABA ROLI is a non-profit organization that implements legal reform programs in roughly 50 countries around the world. ABA ROLI has nearly 500 professional staff working abroad, and in its Washington, D.C. office. ABA ROLI’s host country partners include judges, lawyers, bar associations, law schools, court administrators, legislatures, ministries of justice and a wide array of civil society organizations, including human rights groups.

ABA ROLI is currently implementing a project on anti-money laundering funded by the U.S. Department of State. The Project seeks to foster communication between private [banks and designated non-financial business and professions (DNFBPs)] and public (supervisor and regulators) sector entities regarding emerging money laundering risks as well as the best way to report these risks via training on Suspicious Activity Reports (SAR) and Suspicious Transaction Reporting (STR) filing. The Project aims to build partnerships between public sector supervisors/regulators and private sector financial institutions and designated non-financial businesses and professions to implement a risk-based approach (RBA), identify emerging money-laundering (ML) threats and improve detection and actionable reporting of suspicious activities and transactions, thereby denying criminal groups access to the funds they use to carry out and profit from transnational crimes.

To achieve these goals, ABA ROLI has partnered with a local think tank and other regulatory bodies to implement a series of activities in furtherance of the following objective: To deny criminal groups, as well as malign state and other non-state actors, access to funds that they use to carry out and profit from transnational crimes in the two program countries, and the South Asia region.

In one of the program countries, ABA ROLI and its local partner has convened a series of Compliance Forum Workshops for Designated Non-Financial Business and Professions (DNFBPs), regulators, and supervisors to discuss emerging ML threats and the RBA; developed four consultation Briefs on Anti-Money Laundering and Countering the Financing of Terrorism (AML/CFT); three How-To-Manuals on issuing advisories; three Handbooks for DNFBPs’ compliance officers; five notice templates for regulators; and held six consultation meetings with DNFBPs’ regulators to review the manuals.

In the other program country, ABA ROLI convened four compliance forum meetings; developed four compliance Briefs; and three advisory and process maps; held three virtual training on the Briefs. On the regional level, ABA ROLI organized a virtual “South Asia Regional Conference on Emerging Money Laundering Threats” for more than 160 participants; and issued newsletters on AML/CFT to facilitate coordination and information sharing among the regional group.

Before the end of the program, ABA ROLI will be conducting the following remaining activities: 1) a legislative review of a program country’s anti-money laundering law; 2) a study tour for a program country’s regulatory agency; and 3) a regional conference on AML/CFT threats, regulation, and responsibilities for the legal profession.

ABA ROLI seeks a Program Evaluator with demonstrated experience in rule of law programming to conduct an endline evaluation of the Project’s achievements toward the above objectives and how local partnerships, as well as the political landscape in the implementation country, have contributed to the Project’s successes, failures, and overall implementation. Knowledge of AML/CFT would be an advantage.

Evaluation Purpose and Objectives

The purpose of the evaluation is to assess whether the project objectives have been achieved and to develop a set of recommendations should future rule of law and AML/CFT programming in similar contexts be considered.

Furthermore, the evaluation should focus primarily on the capacity building activities provided to beneficiaries, as well as on key partnerships that have been critical to project implementation, including partnerships with project beneficiaries and with external stakeholders who are not directly involved with the Project but influence its trajectory.

The main evaluation questions identified by the project team are specified below:

1. To what extent has the project contributed to regulatory agencies ability to detect and refer suspicious activities and transactions linked to ML/TF to the relevant authorities for investigation and legal action?
2. To what extent has the project contributed to creating partnerships between the public and private sectors to combat emerging ML/TF threats in the region?
3. To what extent have key partnerships contributed to project successes, such as securing access to beneficiaries of the Project, and/or setbacks or shortcomings?
4. To what extent has the Project contributed to informing top-level policy making and practices through its partnerships and networks?
5. How has the Project complemented other AML/CFT initiatives, if at all?

Evaluation Approach and Methodology

To respond to the questions identified above, the evaluator will conduct an endline evaluation of the Project that focuses on Project achievements. The evaluator, with assistance from a local enumerator, will gather data from multiple sources, such as project documents, external materials, and key informant interviews, to provide a set of potential responses to the main evaluation questions listed above. The evaluation will provide an opportunity for ABA ROLI and its local counterpart to reflect on the Project’s key milestones, successes, challenges and shortcomings, and any significant changes that may be attributed to the Project. The evaluator is responsible for proposing to ABA ROLI which methodological approaches are most suitable for this evaluation.

Interview and/or focus group discussion participants will be determined in coordination with ABA ROLI, and with consideration of the Project’s context and timeframe and the potential sensitivities of the operating environment.

The evaluation should be guided by the following Organization for Economic Coordination and Development – Development Assistance Committee (OECD-DAC)[1] metrics to assess the identified evaluation questions:

Criteria

Sample Data Collection Methods

1. Relevance

To what extent did the Project respond to the country’s needs to improve its AML/CFT measures to be in accordance with international good practices?

Document review

Project documents; training materials; relevant news articles and reports

Interviews with project beneficiaries, project teams, donors, other relevant stakeholders in the project country’s AML/CFT space

1. Coherence

To what extent has the Project created synergies and been complementary to other in-country AML/CFT initiatives?

How has the Project adapted to and/or been affected by the political climate?

Document review

Project documents; relevant news articles; and if possible, documents/online sources describing other AML/CFT initiatives in country

Interviews with project beneficiaries, project teams, donor community, other relevant stakeholders in the project country’s AML/CFT space

1. Effectiveness

To what extent has the Project achieved/is expected to achieve its objectives?

Document review

Quarterly progress reports; Midterm Evaluation Report; M&E indicator data; pre- and post-tests

Interviews with project beneficiaries, project teams, other relevant stakeholders in the project country’s AML/CFT space

1. Sustainability

To what extent will the Project’s achievements last beyond the life of the Project?

Document review

Project documents, articles or documents reflecting country plans related to AML/CFT, academies’ plans for future AML/CFT training

Interviews with project beneficiaries, donors, project teams, other relevant stakeholders in the project country’s AML/CFT space

Scope of the Evaluation

The evaluation scope covers the entire duration of the project; particular focus should be given to the outputs and activities within the period of October 2020 to May 2023. In addition to identifying achievements toward the project objectives, the evaluation should provide insight into the wider context of the project operating environment, highlighting the following: contextual and political challenges that have affected the Project; partnerships or coordination efforts that have contributed to the Project’s successes and/or shortcomings; and coordination efforts that have complemented other initiatives to advance rule of law and enforce FATF principles and practices to combat money laundering and terrorism financing.

Expected Deliverables

The evaluator will work with ABA ROLI, its partners, and a local enumerator to provide the following deliverables to ABA ROLI:

• Inception report for ABA ROLI, including proposed methodology, data sources, and data collection tools;
• Raw data and analysis results, including coding methodology, etc.;
• Data collection summary report;
• Final evaluation report for ABA ROLI and the State Department;
• Executive summary and redacted version of the report produced for ABA ROLI’s local partner; and
• Presentations of key findings and recommendations for ABA ROLI and its local partner.

ABA ROLI expects the evaluator to hire a local enumerator based in country to assist during the data collection phase. The evaluator will be responsible for coordinating with the enumerator to collect all necessary data from in-country beneficiaries and relevant stakeholders. The combined total number of interviews, focus group discussions, surveys, etc. conducted by the evaluator and the enumerator will be summarized in a single data collection report submitted to ABA ROLI.

Budget & Compensation

The costs for this evaluation will be covered by the project grant.

ABA ROLI will allocate the total compensation amount across a three-month period, the amount of time expected to complete all evaluation activities. The evaluator will be paid based on successful completion of each deliverable.

Logistical Arrangements

The evaluation consultancy is a home-based position and travel to the country of implementation is not expected. ABA ROLI and its local partner will assist the evaluator in arranging online meetings with key stakeholders.

Data collection activities may require the local enumerator to travel to various cities where project activities have taken place. ABA ROLI’s local partner and field team will assist in arranging meetings with key stakeholders in country.

Qualifications, Experience, and Expertise

The evaluator will have the following qualifications:

• A university degree in development, international relations, monitoring and evaluation or other relevant discipline.
• At least seven years of experience in program evaluation, including conducting key informant interviews, qualitative data analysis, and reporting on key findings, particularly for rule of law, governance, or human rights programs.
• Demonstrated knowledge of outcome evaluations.
• Familiarity with the criminal justice system as well as experience conducting evaluations of rule of law and justice sector reform projects is highly preferred.
• Fluency and effective communication skills in English is required.
• Local enumerators should have knowledge of at least one local language. Knowledge of other local languages would be an advantage.
• Familiarity with international laws and standards on AML/CFT, such as the FATF Recommendations would be an advantage.

How to apply

How to Apply

Please send a CV, cover letter describing your relevant experience in program evaluation, a summary of your evaluation approach in response to the TOR (maximum two pages) and how many working days envisioned plus your daily rate to [email protected] by Friday, March 31, 2023.