Taking the Results agenda to the next level?

The UK’s Independent Commission on Aid Impact’s latest report looks at DFID’s approach to delivering aid impact. The report draws on ICAI’s past reports and other research to explore how well DFID ‘goes about ensuring meaningful and sustainable impact across its portfolio. By ‘impact’, sales we mean positive, discount long-term, transformative change for poor people, who are the intended beneficiaries of UK aid.’ Some of the findings of the report include:

• “[W]e find that the results agenda has helped to bring greater discipline in the measurement of results and greater accountability for the delivery of UK aid – both key objectives. These achievements have, however, involved some important trade-offs. Some of DFID’s tools and processes have had the unintended effect of focussing attention on quantity of results over quality – that is, on short-term, measurable achievements, rather than long-term, sustainable impact.”
• “DFID is, indeed, very influential on development policy…..[i]n recent years, it has been a forceful advocate for the results agenda. We hear from many partners, however, that DFID’s single-minded promotion of better results management has come at the expense of thought leadership on long-term development impact.”
• The DFID Results Framework “conveys the scale of activity rather than the quality of interventions and their impact on people’s lives. We are also concerned that it sends an inaccurate message to both implementing and country partners that DFID cares more about maximising outputs and beneficiary numbers, than achieving transformative change”.
• “Invest in long-term impact rather than short-term results: With average programme length of just three years, transformational impact will often be possible only over several programme cycles. This should be recognised explicitly in programme design. For complex objectives, this means gradually putting in place the building blocks for lasting impact, including the right policies, priorities, institutions and capacities in steps appropriate to the context. While DFID does this well in many instances, its results management processes do not necessarily encourage it.”
• “ A flexible approach to delivery: Good, impactful programming is flexible in pursuit of its goals. Programme designs are ‘best guesses’ as to how to solve complex problems in dynamic environments. They are most effective when they take a problem-solving approach, learning as they go. ….[w]hile annual reviews should provide such a mechanism, they tend to concentrate more on holding the implementer to account for efficient delivery, than on learning how best to achieve long-term impact.

In a nutshell ‘The key message of this report is that it is time to take the results agenda to the next level – to ensure that it focuses not just on the cost-efficient delivery of UK aid but also on achieving genuine and lasting impact for the world’s poor.’ The report goes on to suggest amongst other things that DFID should “ensure that the incentives of staff and implementers encourage an investigative, problem-solving approach to programme implementation and a willingness to adjust programmes as necessary in response to lessons learned or changing conditions”.

In our new book The Politics of Evidence and Results in International Development: Playing the Game to Change the Rules? which is published today, we explore many of these issues and some of the more systemic roots of how, and why, the results agenda can distort attempts to promote transformational change. In particular we look at some specific examples and strategies of how development practitioners and managers have navigated, resisted and used the ‘results agenda’ for more transformational ends. We believe these experiences provide some clues to how organisations which truly seek to promote sustainable impacts might create a more enabling environment for their staff and partners to do so.

Strategies identified

Many of the cases covered in the book noted the key role of individuals’ personal power to question, create and act. It was individuals that adjusted, revised or ignored their particular set of planning protocols, results frameworks, and reporting requirements to create more workable options. Understanding why and how they did this provides examples of not only how these processes might need to be adapted, but what space for human agency and discretion is required if programs are to be flexible and adaptive.

This in turn required people being astute political actors who understood their context, organisational histories and how to leverage organisational values.Those organisations that seem to have navigated well the narrow path between the pressure to meet reporting requirements and continuing to pursue transformational goals drew upon these understandings to resist where necessary, adapt where possible and invent new ways of doing things where feasible. This required a clear identification of working with what is positive about the results and evidence agenda, using certain protocols or methods to advance social justice ideals and being self-critical about what is worth doing, or not.

Another strategy, though less common, was that of helping front line staff and partners to speak up and be heard directly by those making decisions about strategies, priorities and resources. This collapsing of hierarchy seems to have created a space where the authenticity, passion and sheer common-sense of those steeped in local context can be communicated to those with the power to make decisions. These exchanges needed safe spaces in which learning about what works, and does not, could take place, and ideas for influencing developed. Such processes can then lead to collective action to respond to the agenda, although the authors are cautious about how effective that strategy has been for them.

The final strategy which emerged is about being politically opportunistic about fashionable results-related concepts that can provide opportunities to help challenge the power relations that cause inequality, rather than reinforce them. The book documents cases where the current interest in ‘Value for money’, ‘theories of change’, and ‘social accountability’, have all been ‘used’ in this way, and suggests the ‘thinking and working politically’ agenda offers further opportunities to do so.

                                                                Why Politics Matters

Our conclusions note that just as it is clear that ‘politics matters’ in the development process, so too do politics, power, interests and ideas define organisational policies, procedures and behaviours. As another recent publication suggests in respect of indicators, monitoring and evaluation processes smuggle theories of development, human nature and governance into apparently neutral systems of measurement. It is important that this is recognised and understood.

We would argue that understanding and shifting the power dynamics that shape who decides what gets measured, how and why, is a key step to ‘taking the results agenda to the next level’ and helping to get to grips with the evaluation of politics, and the politics of evaluation and how in practice this might be done.

 

This blog was also posted on the Practice for Change website of the Institute for Human Security and Social Change