Norwegian Nurses Organisation
I. Introduction
This is the Terms of Reference (ToR) for the End Term Evaluation (ETE) of the programme CLOSING THE GAP: SUFFICIENT, COMPETENT AND MOTIVATED NURSES AND MIDWIVES (GLO-0759 RAF-18/0044) and its local partner National Organisation of Nurses and Midwives of Malawi (NONM) in Malawi.
The current programme agreement and plan started on 1.1.2019, as an extension of a Partnership with NONM first initiated in 2006. The current programme period ends on 31.12.2023.
This ToR sets out the expectations for this ETE process and deliverables.
II. Background
a) About NNO
The Norwegian Nurses Organisation (NNO) (Norsk Sykepleierforbund, NSF) is a national professional nurses organisation and labour union representing about 120,000 registered nurses, midwives and nursing students in Norway.
NNO was founded in 1912 and focuses on professional, social, and representational policies, concerning areas such as public health, ethical nursing standards, as well as collective bargaining aimed at improving, e.g., working hours, working conditions and pay. NNO is politically neutral and has some 3000 elected union representatives.
NNO has been involved in international development work since the 1980s, with programmes/projects supporting the development of strong nurses’ organisations in Africa. This work has been, and currently is, supported and partly financed by the Norwegian Agency for Development Cooperation (Norad).
b) About NONM
NONM is an organisation with dual mandate; professional organisation, and union. Vested in this is the mandate to support, protect, and speak on behalf of nurses and midwives on a wide range of topics, e.g., quality education and career development, negotiating for working conditions and salaries, and decent work and workplace issues.
As of 2021, NONM counted 4882 members, whereas even more than 400 additional members had signed consent forms to become members. NONMs highest level of decision is set to the National Executive Committee (NEC), of which the President is in lead. All positions in the NEC are elected seats, and most of those sitting in the NEC is having full time positions elsewhere. NONM estimates having around 250 shop stewards, working voluntarily throughout the nation in various health institutions. The shop stewards are NONM’s first point of contact between the organisation and its members. NONM also has its secretariat, with the hired position of Executive Director in lead. The secretariat is sitting in the main office in Lilongwe.
c) NNO’s partnership model
As described in NNO’s Strategy for international development cooperation (2018-2023), civil society is an important driving force in societal development in all countries. Strong and independent organisations can help promote democratisation, realise human rights, and reduce poverty. NNO believes that strong nurses and midwives’ organisations are important contributors in holding governments accountable for having sufficient, competent, and motivated health personnel in a country. This is at the core of NNO’s partnership model.
NNO’s partnership model puts increasing emphasis on technical and professional cooperation with decreasing financial support to administration. This is reflected in collaborative project activities like joint seminars and knowledge exchange in areas like advocacy/lobbying; membership recruitment/administration; strategic communication; and organisational development of democratic member-based organisations.
The division of labour, roles, and responsibilities between NNO and its partners differs slightly depending on the activity. In most cases, the partner is the implementer while NNO has a supporting and/or advisory function. For example, in terms of organisational development, the local partner is responsible for planning, executing, and evaluating local trainings. NNO may be a discussion partner in terms of e.g., the content of the training, and may also contribute with an expert.
In other areas, NNO might have more responsibility for organising and delivering capacity building activities, especially when there is specific expertise or experiences within the NNO structure that is to be shared with the partner. When NNO has a more active role in the activity, emphasis is on making sure that the capacity of the partner is strengthened during the process.
In terms of other technical support, NNO often challenges the partner when it comes to – for example – setting realistic targets, how to assess learning outcomes, how to ensure learning is applied, the documentation of results, etc. This is followed by support, e.g., on how to develop and use simple monitoring and evaluation tools, and how to analyse and apply findings.
d) Brief programme background
The 5-year programme, “Closing the Gap: Sufficient, Competent and Motivated Nurses and Midwives (GLO-0759 RAF-18/0044)” is co-funded by the Norwegian Agency for Development cooperation (Norad) and NNOn and has two local partners: Rwanda Nurses and Midwives Union (RNMU) and the National Organisation of Nurses and Midwives of Malawi (NONM). The latter is subject to this ETE.
In brief, the core of the programme and how it applies to NONM is summarized as follows:
Nurses and midwives are the backbone of the health system. Yet they are overworked, undervalued, and underpaid. Many also lack the skills, equipment, and support to do their job effectively, and safely. These are contributors to the fact that Africa is lacking millions of health personnel.
The planned effect on society of this programme is that universally accessible, equitable, and affordable quality health services are ensured in Malawi. The goal of the programme is to contribute to having sufficient, competent, and motivated nurses and midwives capable of addressing the population’s current and emerging health needs.
Civil society, represented by strong and independent organisations, is an important driving force in development. Hence NNO provides professional, technical, and financial support to NONM in order for them to be: 1) a democratic and accountable organisation for nurses and midwives in Malawi; 2) enabled to provide relevant services to nurses and midwives in Malawi; and 3) effective watchdogs and advocates.
By realising these outcomes, NONM will contribute to better status, pay and working conditions for nurses and midwives; a more competent, qualified, and capable healthcare workforce; greater leadership opportunities for women; and better healthcare and public-sector policy, legislation, and reforms, amongst others. This will again contribute towards the realization of Malawi’s current and emerging health needs.
III. Objective and purpose of the ETE
The ETE is set out as a requirement in the GLO-0759 RAF-18/0044 Special Conditions §9.2.;
“9.2 An end-term review focusing on results achieved in the Project by NONM shall be carried out by 31 August in the fifth year of the grant agreement”.
The ETE aims to:
a) assess and describe the level of progress in NONM and the partnership achieving its planned outcomes and overall objective in the programme period 2019-2023, including cross-cutting issues as gender, inclusion, and equality.
b) assess and describe key milestones and achievements of NONM and the programme collaboration with NNO since the partnership was initiated in 2006.
c) assess and describe internal and external factors that have had an impact on NONM’s development as an organisation (positive and negative), especially since 2006 and to date.
d) assess NONM and the partnership’s alignment with and potential contribution to (Malawi) national plans and targets/goals
e) reflect on potential wider societal impact that the development of NONM may have had in Malawi
f) give examples of positive change that NONM has contributed to, for a) nurses and midwives, b) the nursing- and the midwifery professions, c) patients, d) society at large
g) assess how the programme has been managed by NONM, including financial management (/ anti-corruption), the efficiency and effectiveness of NONM’s work processes, monitoring and evaluation. Internal and external factors affecting this shall be described and explained.
h) assess and reflect on the various aspects of the NNO-NONM partnership, and NNO technical (non-financial) support in particular.
i) assess and describe internal and external risk factors that (may) have had an impact on activities, outcomes, ability to achieve results and the organisation’s development.
j) assess the programme’s alignment with current Norwegian development policies and priorities.
k) identify prioritized areas and recommendations for the continued work of NONM.
l) identify key areas and indicators, internal and external, that affect NONM’s financial sustainability and future ability to operate without the financial support of NNO
m) reflect on the operational context in Malawi, i.e., the space and environment for professional organisations and (labour) unions, the culture for getting organised in member-based organisations, culture of voluntarism, level of trust in the government and level of trust in the society at large.
The ETE may also assess and reflect on other areas that come up during the review process, that are relevant to the objective and purpose of the ETE.
Stakeholder engagement should have a cross-cutting presence throughout the ETE.
IV. Methodology
Background: NNO’s system for result management
NNO’s system for result management is based on Results Based Management (RBM) principles and a human rights-based approach. In practice this means that NNO uses various feedback loops to assess and map progress towards set targets.
The feedback loops are, among others: quarterly activity-, output- and outcome monitoring tools (partner to NNO); data collection and monitoring tools used by the local partners; partnership meetings between NNO and the local partners; local stakeholder meetings; quarterly NNO steering committee meetings; and regular partner-NNO online meetings.
NNO uses external evaluators to perform mid- and end-term reviews and evaluations. In addition, NNO facilitates south-south and partner-to-partner exchanges (e.g., seminars) for additional learning and sharing of experiences and good practices.
For the programme under review, there is a joint logical results framework (consolidating two partners) that uses a number of defined concepts such as overall objective, outcomes, outcome indicators, outputs and output indicators. The framework was developed in close collaboration with NONM and the other partner. It is expected that all activities supported by NNO/Norad, as defined in detailed annual plans developed by NONM, contributes towards achievement of the outcomes and objective in the results framework.
NNO’s system for results management also includes planning and results-based budgeting that involves calculating and proposing resource requirements based on pre-determined results, rather than merely on the basis of scheduled outputs or activities. A central tool is the overall programme budget, which provides a direct link between expected results and resource requirements.
The framework is dependent on critical assumptions about the environment and context, taking into consideration major external and internal factors and conditions as well as clearly defined accountabilities and indicators for results. The budget is reported against in every quarter. When conditions and requirements change, the budget is revised accordingly. This is a partner-NNO process that is usually done once or twice a year.
The human rights-based approach specifies the subjects of the project results, i.e., the rights-holders and the duty-bearers. Typically, the rights-holders are the nurses or their patients, while the duty bearers are different levels of government, or employers. The outputs aim to close capacity gaps while outcomes reflect improvement in the performance of, or the strengthened responsibility of, the rights-holders and duty-bearers. Furthermore, the results specify the realisation of human rights as laid down in the SDGs (and other international instruments).
NNO and local partners have mutual accountability, which applies to both the use of resources as well as the achievement of results. This is done through a shared agenda, ensuring space for dialogue and negotiation, reciprocal trust, transparency, and evidence that is collected and shared amongst all partners.
Methodology for the ETE
The consultant(s) should describe how they will proceed to conduct the evaluation.
The proposed methods must sufficiently address the issues and questions outlined within the ToR, specifying the specific review issues, questions, methods of data collection, and analysis that will be undertaken. Preferably, it should encompass a mixed method approach. It should also allow for wide consultation with all interested partners and stakeholders. Assumptions and limitations must be stated clearly.
Approaches that allow for review over a longer period of time are of particular interest.
It is suggested that the methods should include, but not be limited to the following,
– Document review.
Should include, but not be limited to, the following documents:
– Programme application / proposal
– The NONM-NNO Project Agreement 2019-2023
– Annual work plans
– Results Framework
– Progress and results reports (tertial and annual)
– Audit reports and Management Letters
– Field visits.
During these visits, the consultant(s) may contact, amongst others,
– Zone assistants / offices
– Shop stewards
– Members
– Local or national stakeholders or alliance partners
– Focus Group Discussions (FGDs)
– Shop stewards
– Members
– Interviews.
Key informants can be drawn among FGD participants, stakeholders, members, local communities, workplaces, development partners (current and former), staff, former staff, and others.
V. Qualifications / expertise
The ETE as outlined in this ToR will require competence and experience within several disciplines and areas. The proposal must outline how these requirements are met. If the proposal includes a team of consultants, the team must have a team leader. The team leader shall (besides having a supervisory and coordinating role) be the main point of contact, and responsible for contractual and all other administrative arrangements (including payments).
The consultant/consultant team shall have the following competencies and experience:
– A Master’s degree or higher in relevant subject matter(s)
– At least 10 years of evaluation expertise (evaluating programmes of a similar nature and scope), preferably having conducted evaluations in the areas of labour unions / professional organisations and organisational development. Additional experience from and/or comprehensive understanding of health systems preferred.
– Demonstrated experience in designing evaluation methodology and data collection tools, quantitative and qualitative
– Exceptional analytical abilities
– Excellent written and oral communication skills in English
– In-depth first-hand experience from Malawi, and extensive familiarity and understanding of the Malawian context.
– Excellent inter-personal skills
– Excellent organizational and administrative skills
– Impeccable time management
– Independence and absence of conflict of interests
– Strong references from similar work
All consultants that are contracted for the ETE assignment must adhere to the principles of “do no harm”. Further it is important that the contracted consultant(s) are not representing NNO or NONM.
It is important that all data and information collected for this ETE is collected, treated, and stored in a secure manner.
VI. Deliverables / expected outputs
a) A final report, not exceeding 15 pages including an executive summary, but excluding appendices.
– The report must answer to this ToR.
– The executive sumTmary should not exceed 1 page and should present key findings and recommendations.
– All findings in the report must be supported by evidence. Conclusions must build on the findings, and recommendations must derive logically from the conclusions.
– Appendices may be of such nature that they support the analysis, conclusions, and recommendations in the main report, but are not essential for interpreting the findings.
– The report must follow Norad’s template for evaluation reports, if applicable.
b) An overview of results vs. targets, structured in the same format as the results framework (indicating whether behind, on or ahead of target)
c) Two (2) results examples of 800-1000 words, in a given format
d) Two (2) validation meetings (Lilongwe / virtual meeting)
e) Verbal presentation of final report (Lilongwe / virtual meeting)
VII. Schedule
The specific timeline for the ETE exercise is flexible (depending on the chosen methodology), as long as the schedule is agreed with NONM and NNO.
Tentative time for inception meeting is medio/end January 2023. The field visits and engagements with NONM, and the desk review and engagement with NNO, may start directly after this.
A dissemination meeting in which preliminary findings are presented is to be held medio June 2023.
The final report must be submitted and approved no later than 31 August 2023 (unless otherwise agreed and approved by NNO in writing).
The date for the verbal presentation shall be agreed between NONM, NNO and the consultant(s), but it is expected to occur in September/October 2023 hence the consultant(s) must be available for this after the final report is approved.
VIII. Proposal process: submission of bids (see under “how to apply”)
IX. Financial ceiling and payment modalities
The total financial ceiling for the ETE exercise is NOK 450 000 (equivalent to app. USD 42 850 as of 19 October 2022).
Payment will be made in three instalments upon agreed deliverables.
The contractual arrangement and all payments will be made with one (lead) consultant.
All consultants that are contracted for the ETE assignment are responsible for their own insurance and payment of applicable taxes.
How to apply
VIII. Proposal process: submission of bids
Applicants shall submit the following documents (all in English only):
a) A 1-page cover letter with a brief description of relevant previous experiences
b) Contact details of 3 references expected to support claims of knowledge, skills and experience
c) A technical proposal of no more than 6 pages
d) A detailed budget that specifies fees and all other costs related to the ETE assignment
e) A 1-page resume of each team member’s qualifications and experiences specifically relevant to the current ETE assignment
f) Team composition, roles and responsibilities – if applicable
g) A proposed work plan, with milestones and deliverables
h) A completed and signed Integrity Due Diligence form (one for each consultant), to confirm (amongst others) independence and absence of conflict of interests.
Submission of copies of final reports of comparable evaluations recently carried out is an advantage. If this is not possible, other written work of similar assignments should be made available.
Tender rules
The technical and financial (budget) proposals should be sent electronically (email) to [email protected].
The subject field of the email shall be specified as “PROPOSAL FOR NONM-NNO ETE – [NAME OF APPLICANT]”
The deadline for submission of proposals is 8 December 2022.
Tentative shortlisting is estimated to be completed by 20 December 2022.
Any need for clarification of the ToR should be addressed to [email protected].
Questions related to NONM, local logistics etc. should be addressed to the Executive Director, Peter Mvuma ([email protected]).
Award criteria:
– Demonstrated understanding of the needs of the assignment
– Relevant qualifications and experience of the evaluator(s) including consistency of their experience with eligibility criteria
– Adequacy of the work plan with the technical specifications