THE IDEAL APPROACH TO DEVELOPING AND SUSTAINING RESULTS-BASED MONITORING AND EVALUATION SYSTEMS FOR DEVELOPMENT INTERVENTIONS

 

By Barward Johnson, Chief Managing Officer/Lead Consultant-PERT Consultancy Inc.

​

Results-based Monitoring and Evaluation (M&E) is an effective tool for improving the design and management of development interventions. It provides an effective platform for accountability and learning within development organizations. However, to be able to achieve these objectives, organizations should properly build M&E in their interventions. This article discusses how development organizations and consultants can systematically incorporate results-based M&E in development interventions.

Before delving into the steps and procedures for developing and sustaining results-based M&E systems, let us clearly define two key concepts – “results-based M&E” and “M&E System”. First, let us explore the meaning of “results-based M&E”. Results-based M&E focuses on the outcomes and impacts of policies, programs, and projects rather than just on the inputs used or the activities conducted in development programs and projects (Morra-Imas & Rist, 2009).

​

It is different from traditional M&E in that it goes beyond an emphasis on inputs and outputs to a greater focus on outcomes and impacts. According to Kusek and Rist (2004), traditional implementation-focused M&E systems are designed to address compliance—the “did they do it” question, but the results-based M&E systems are designed to answer the “so what” question that is important for measuring results.

​

The other thing that we should clarify is the meaning of the “M&E system”. Many M&E practitioners consider an M&E system as just a plan that guides the collection, analysis, and reporting of data. However, the use of the word “system” implies that an M&E system includes other components, other than the plan. Simister (2009) defines an M&E system as a ‘series of policies, practices, and processes that enable the systematic and effective collection, analysis, and use of monitoring and evaluation information’. Using system thinking, one should consider an M&E system of comprising the people, practices, processes, plans, technology, and logistics required to systematically and effectively produce and use M&E information.

There are a variety of approaches in designing results-based M&E systems, and there is no one correct way to go about building such systems (Görgens & Kusek, 2009). However, M&E system developers should use a holistic approach to developing systems that suit the development intervention and its context.

​

As such, I have developed and would like to proffer a comprehensive and practical model for developing and sustaining results-based M&E systems for development interventions. My IDEAL model is a results-focused approach that emphasizes problems/needs identification, capacity building, the actual application of knowledge and plans, and the assessment of achievements and challenges.

​

The IDEAL approach does not exemplify a perfect model, but it guarantees a method that suits an organization and the environment and time in which it functions. The application of each of the five steps in the IDEAL model will not only ensure an effective M&E design, but it will also guarantee the strong implementation and sustainability of the M&E system.

Development organizations and other users can customize the approach to conform to their specific needs and situation. Additionally, the development of an M&E system is a gradual process. Organizations can add various components as resources are available and as needs exist, but an M&E system should be comprehensive enough to support the organization's existing needs.

​

An individual or a team with technical M&E knowledge and experience is required to lead the organization through the M&E system development process. M&E system developers may be consultants or internal M&E staff with strong knowledge and experience in M&E. The consultants or internal team serve as technical facilitators and lead the development process, but it is important that they include relevant staff and external stakeholders in the process. In fact, stakeholder participation is at the heart of the IDEAL model. Stakeholders should be engaged at every step of the M&E system development process.

The IDEAL model includes the following five steps to developing and sustaining effective M&E systems.

​

IDENTIFY M&E NEEDS AND CAPACITIES

The first step in developing and sustaining an effective M&E system is to identify the M&E needs and capacities of the organization or intervention through a participatory process. The M&E needs identification stage also helps determine whether the organization or intervention is ready for an M&E system or whether there is an enabling environment for M&E.

Conducting a situational analysis is an effective way of identifying the M&E needs and capacities of an intervention or organization. The situational analysis establishes a clear picture of the existing M&E or data management and reporting practices and defines the scope and purpose of the M&E system.

​

Every M&E system should have a clear scope and purpose. The scope and purpose drive the design of the system. The M&E system developer should hold consultative meetings with key stakeholders to determine how extensive they need the system, and why they need it. Efforts should be made to reach a consensus on the scope and purpose of the M&E system by hashing out the various concerns and interests of stakeholders. Such consensus is necessary for ensuring sufficient buy-in that guarantees the use and sustainability of the M&E system.

​

Accountability and learning are the two most widespread overarching purposes for developing an M&E system for development interventions. Accountability often manifests in the processing and reporting of data to donors and other stakeholders, while learning is often used to inform the designing and planning of new interventions, and budgeting.

Organizations may have some variations, but those differences are often just subsets of the two overarching purposes. For example, some organizations would want an M&E system for planning, but learning in an organization is also intended to inform planning and other initiatives.

​

The scope of an M&E system varies according to the purpose, the available resources, and the context. M&E systems can be developed at different levels and have different components. For example, M&E systems can be developed for an entire organization, a policy, a program or a specific project. Within an organization, the M&E system can be developed at the regional or central level. Additionally, an institutional M&E system may contain more components than a project M&E system.

A situational analysis is conducted through the collection of data, using participatory approaches. Participatory methods of data collection may include consultative meetings, stakeholders’ participatory workshops, and focus group discussions. Data may also be collected through desk reviews, interviews, and site visits.

​

Data should be collected on stakeholders’ motivation, expectations, and needs for M&E; existing M&E practices, capacities, and linkages; M&E Champions and users of M&E information. The data collected should be analyzed to inform the development of the M&E system.

​

DEVELOP THE M&E FRAMEWORKS

The term “M&E Frameworks” as used in this article refers to the policies, plans, guidelines, and manuals required for an effective and functioning M&E system. It includes M&E processes and procedures, roles and responsibilities, and resources.

The design of the M&E frameworks should be informed by findings from the situational analysis and the consultative meetings held with the various stakeholders. Depending on the situation, the framework could be just one document or series of documents. If the system is being developed for an institution, then it is best to first develop an institutional M&E policy framework document and then later the M&E plans for each project. The institutional M&E policy framework contains the structure and general principles and guidelines of how M&E is conducted in the institution. It provides the basis upon which project M&E plans are developed.

​

M&E systems are effective and sustainable when they are developed through a participatory process. Participation should not end with the needs identification stage. It should continue throughout the M&E system development process.

During the design of the framework, different sessions should be held for different stakeholder groups at different times. Technical design sessions should be held with the M&E and program staff of the organization and its partners to agree upon results – impact, outcome, and outputs. The sessions should be used to, among other things, discuss indicators and their targets, disaggregation, data sources, and data collection methods. M&E roles and responsibilities, as well as plans for baselines, evaluations, and learning, should also be discussed during the technical design sessions.

​

These technical design sessions are not meant for writing up the frameworks. Rather, they should be used to gather, agree upon, and document contents for the framework documents. The M&E system developers can also consult other non-technical staff during this stage to explore how synergies could be developed between the M&E system and other related systems like ICT, Finance, Human Resource, and Procurement. For example, the institution may like to link its M&E system to its procurement, or ICT systems.

​

Once the contents of the M&E frameworks are agreed upon, the M&E system developer may commence the actual writing of the framework documents. The M&E framework should have three components – a Monitoring Component, an Evaluation Component, and a Learning Component. The Monitoring Component includes the mechanisms and tools for enabling the routine collection, analysis, and reporting of quality data to measure the progress of the intervention. On the other hand, the Evaluation Component includes, among other things, the evaluation purpose, key research questions, and methodology for conducting evaluations, while the Learning Component includes mechanisms and planned initiatives for learning and use of M&E information.

After the frameworks are developed, technical teams should hold technical sessions to review the documents. Additionally, another working session should be called for all stakeholders to validate the M&E frameworks.

​

EDUCATE OR TRAIN STAFF TO USE THE M&E FRAMEWORKS

Capable M&E staff or structure is an important component of an M&E system. It is important that an M&E system has designated staff or unit who are well trained to implement the M&E frameworks.

However, training is not an end in itself. It needs to go along with other capacity considerations including logistics and staff motivation to be effective. Staff involved in M&E activities should also have clear roles and responsibilities.

The content and nature of training should be informed by the findings of the M&E needs identification process. Generally, as a start, training should be provided to relevant staff in the use of the frameworks as well as in basic M&E concepts and techniques. Training can take several forms, including on-the-job training and external workshops. There could be different training packages for different levels of staff.

Training should be an ongoing activity and should be supplemented by mentoring and coaching initiatives. New staff will benefit from routine training in data management and reporting. This is important for sustaining the M&E system.

​

APPLY THE M&E FRAMEWORKS

Once the M&E frameworks have been developed and staff trained, the frameworks should be implemented to achieve the purpose for which they were developed.

The routine collection, analysis, reporting, and use of monitoring data is the most frequent task during M&E implementation. Additionally, evaluation, research, and learning are also key activities of M&E implementation. Other key activities during M&E implementation include training and the revision of the M&E framework. As a living document, the M&E frameworks should be reviewed periodically during implementation.

​

LOCATE OR ASSESS PERFORMANCE OF THE M&E SYSTEM

Assessing the M&E system is not only vital for measuring the performance of the system, but it is also important for informing M&E plan revision and other adjustments within the M&E system.

As the M&E system functions, there should be periodic assessments to identify the successes and challenges of the system. The assessment should cover all components of the system, not just the M&E framework. Several standardized tools are available for assessing M&E systems. However, organizations may develop their own approaches and tools to carry out the assessment.

 

REFERENCES

Görgens, M., and Kusek, J.Z. (2009). Making Monitoring and Evaluation Systems Work: A

Capacity Development Tool Kit. World Bank. Washington, DC.

Kusek, J.Z. and Rist, R.C. (2004). Ten Steps to a Results-Based Monitoring and Evaluation

System: A Handbook for Development Practitioners. World Bank. Washington, DC.

Morra-Imas, L.G., and Rist, R.C. (2009). The Road to Results: Designing and Conducting Effective

Development Evaluations. World Bank. Washington, DC.

Simister, N. (2009). Developing M&E Systems for Complex Organisations: A Methodology.

M&E Paper 3. International NGO Training and Research Centre. www.intrac.org.