LocationsInnovative Strategies for Measuring Impact: Insights from 14 African Country Offices - Part 2
Connecting Change, Storytelling, and Data to Drive Impactful Results
September 18, 2024
This blog is presented in two parts. The first part shed light on the innovative initiatives driving inclusivity and adaptability within UNDP Country Offices. Now, in Part 2, we look at the intricate realm of impact measurement and explore how 14 UNDP Country Offices in Africa are refining their Monitoring, Evaluation, and Learning (MEL) strategies, tackling key challenges, overcoming bottlenecks, and striving for greater alignment and effectiveness in their development endeavors.
Amidst the backdrop of Kenya's 'Nane Nane' protests, we explored the profound stories behind statistics at our recent regional workshop in Nairobi, which gathered M&E officers from 14 UNDP Country Offices across Africa, from New York Headquarters and from the Regional Service Center in Addis. This compelling context drove us to rethink our approach to development, focusing on addressing root causes and capturing the true impact of our work to ensure it makes a meaningful difference where it is needed most.
A key aspect of this journey and the regional efforts to improve impact measurement is the integration of transformative changes, compelling narratives, and reliable data within a robust Monitoring, Evaluation, and Learning (MEL) framework. Despite several success stories across the region, many lack the necessary solid data to fully support them. Change is closely tied to data, with performance indicators defined both at the level of Country Offices and within the Strategic Plan Integrated Results and Resources Framework (SP-IRRF), all aligned with the trajectory of change outlined in the Theory of Change (TOC). However, there is a pressing need for more dynamic impact measurement systems that can capture the complexities of our development work, while incorporating continuous feedback loops for learning and adaptation.
Countries across Africa are refining their impact measurement strategies by adopting real-time data collection, engaging communities in evaluation, and leveraging digital tools.
Countries across Africa are refining their impact measurement strategies by adopting real-time data collection, engaging communities in evaluation, and leveraging digital tools. The 14 Country Offices in Africa’s Regional Bureau which gathered in Nairobi for an impact measurement workshop, emphasize strategic shifts, digital innovation, and the importance of tailoring approaches in local contexts. These efforts, including collaborations with civil society and the use of citizen-generated data, are transforming challenges into opportunities for meaningful progress.
Figure 1 - MEL System Dynamics and Feedback Loops
By aligning compelling narratives with solid evidence and integrating feedback loops into the MEL system, countries can ensure their development initiatives remain both effective and responsive. The accompanying diagram illustrates the interconnected elements of this approach, highlighting how they collectively create a comprehensive and adaptable framework that links change, data, and storytelling through TOC.
Managing Bottlenecks in MEL Framework
As we continue to refine and implement regional strategies for measuring impacts, managing bottlenecks within the MEL framework is a significant, but often overlooked challenge. Despite the notable progress in data collection, digital innovation, and community engagement, the effectiveness of our MEL systems depends on their ability to function smoothly and adapt to emerging challenges.
An analysis of the MEL framework identifies potential bottlenecks in significant areas, such as Monitoring, Evaluation and Learning, Innovative Reporting, and Capacity Building (see Figure 2). These elements play a pivotal role in the system and, as key connectors, can become critical pressure points that may impede the success of development initiatives if not effectively addressed.
Figure 2 - Significant bottlenecks in the MEL Framework: A systems dynamics perspective
Identifying bottlenecks is not only about spotting inefficiencies; it is also about understanding the points where delays could disrupt the flow of crucial information. Monitoring, Evaluation and Learning are at the heart of a feedback loop. If this area becomes overwhelmed, it can slow the entire cycle of learning, adaptation, and stallprogress across multiple initiatives.
Innovative reporting also facilitates translating data into actionable insights. If it is delayed, timely decision making can be hindered, leading to missed opportunities or slow responses to emerging challenges. Similarly, bottlenecks in capacity-building can have widespread effects. Effective capacity-building ensures that lessons learned through monitoring, evaluation, and learning are implemented. Delays could slow the spread of the essential skills and knowledge needed to measure outcomes and achieve an impact.
The findings from the bottleneck analysis reflect the concerns and needs identified by the 14 pilot countries. While some requirements, such as capacity building, data accessibility, readability, tracking exceptions and outliers, and visualization are already being addressed or refined, tackling these bottlenecks will further enhance the efficiency and effectiveness of our MEL systems, ensuring that our development initiatives are grounded in timely and accurate data.
As we addressed the bottlenecks in our Monitoring, Evaluation, and Learning (MEL) framework, the ongoing debate between attribution and contribution has resurfaced, especially during our recent workshop discussions. This question is not merely technical, but is central to measuring, interpreting, and improving the impact of our initiatives. The challenge lies in determining whether specific outcomes can be directly attributed to specific interventions or whether they result from collective efforts. This debate, which is well known in development evaluation, remains important for refining our impact assessment practices. For further insights, see John Mayne’s overview of Contribution Analysis and Ricardo Wilson-Grau’s work on Outcome Harvesting.
Attribution in the UNDP Framework
Within UNDP’s Integrated Results and Resources Framework (IRRF), there is significant emphasis on clear attribution at the output level and contribution at the outcome level, especially in projects involving joint funding or partnerships. However, identifying the impact of a single intervention becomes increasingly challengingwhen multiple stakeholders are involved. This raises a key question: How can development organizations fairly assign responsibility for outcomes in initiatives with shared efforts?
Figure 3 - Understanding Attribution and Contribution in Evaluating Impact
Harmonizing Approaches Across Development Actors
As more projects involve a mix of actors, including United Nations agencies, governments, and NGOs, harmonizing impact measurement methods is becoming increasingly necessary. The challenge lies in balancing the need for clear accountability, which often requires distinct attributions, with the understanding that most outcomes emerge from collective efforts. A unified approach among partners can ensure that both contributions and collaborative efforts are accurately reflected.
Key Questions Moving Forward:
- How can we develop frameworks that fairly represent both attribution and contribution?
- What strategies can we use to document and communicate the complexities of shared efforts among various development partners?
Embedding impact measurements into every facet of our study is critical. This responsibility doesn't rest solely with MEL specialists, but with each team member, from project managers to field officers. Making impact measurement part of daily routines will build a culture of continuous learning and improvement.
Embedding impact measurements into every facet of our study is critical. Making impact measurement part of daily routines will build a culture of continuous learning and improvement.
Leadership is key to driving this transformation. At the Regional Bureau, we are committed to establishing high standards, demonstrating that impact measurements can be both effective and innovative. We will equip our Country Offices with new tools and and methodologies to cultivate a culture of experimentation, creativity, and growth.
Impact measurement extends beyond just tracking numbers. It is about understanding the real-world effects of our work and using those insights to drive continuous improvement. The true value of our efforts is reflected in the lives we change and the communities we serve.
Our recent workshop in Nairobi underscored the progress achieved, while acknowledging the path that lies ahead. We should view impact measurement as an opportunity for growth rather than a burden. This shift in perspective will allow us to learn, adapt, and enhance our contributions more effectively.
Now is the time to take action. With the right tools, knowledge, and commitment, we can successfully integrate impact evaluation into our work and drive meaningful progress. This is an opportunity to drive significant changes that are clear, measurable, and worth celebrating. Our ongoing pilot phase in 14 countries, supported by key actors, such as the UNDP’s Bureau for Policy, Programme, and Strategy (BPPS), the Chief Digital Office (CDO), the RBA’s Country Oversight Team, the Management Support Team, and the Strategy Analysis and Research Team, offers an opportunity to shape and refine the UNDP corporate Results-Based Management and impact measurement strategy.
As we move forward, it is essential to recognize that embedding impact measurement deeply into our practices is not just a goal but a transformative journey. By shifting our perspective to view impact measurement as a catalyst for growth, we can harness its full potential to drive meaningful and measurable progress. This is our moment to embrace this opportunity, drive significant change, and celebrate the tangible impacts we achieve in the lives of the communities we serve.
Your input and experience are highly valued. We invite you to collaborate on pilot initiatives and/or share your insights. If you are interested in the tools, strategies, or methods discussed in this blog, we would be delighted to connect and explore opportunities for collaboration: comms-africa@undp.org.