Contradictions and Gaps in the UK Results Agenda

The UK’s Minister for International Development, recipe Andrew Mitchell, is one of a group of like-minded OECD Ministers (including from Canada, Germany, Denmark and Sweden) placing ‘results’ at the top of their aid policy agenda.  Last week he made a speech ‘results for change’ about his vision of an aid programme that is ‘rooted in evidence and evaluation’ and committed to transparency:  ‘Whether it’s a British person sitting in Manchester or a Kenyan sitting in Kisumu, any individual can hold us to account’.

His speech addressed concerns – including those voiced by the Big Push Forward – that the results agenda is shrinking the space for development aid that can support sustainable and transformative change in complex contexts.  On the contrary, he said, ‘Our approach will be flexible and differentiated, recognising local realities and contexts. We will work to understand and go with the grain of change in complex societies, rather than attempting to impose rigid blue prints’.  He also referred to tackling ‘structural inequalities’ and striving ‘for long-term transformative impact as well as concrete improvements in human lives today’.  This is very welcome.  But, how does this match up with the experiences of so many grantees who find themselves obliged to conform to rigid and burdensome assessment and reporting templates that force them to misrepresent the results they are trying to achieve and makes them even more timid about  honestly noting mistakes and limitations [see Tina wallace’s comment to Irene’s blog of 6 June]?

Yesterday, the Institute of Development Studies and the Overseas Development Institute jointly hosted a meeting of some two dozen or so people – largely from UK development ‘think tanks’  as well as some aid agency staff – to thrash out some of the contradictions in the results agenda. Lawrence Haddad and Simon Maxwell will also be blogging about the discussion.  I highlight five issues here that struck me.

‘Results’ as fuzzword. We all agreed and spoke a lot about the desirability of results but remained pretty vague about the kinds of change we wanted to see or how foreign aid could support such change. We all used the term ‘transformative change’ but skirted around defining what it meant and, importantly, what those definitions imply methodologically It is becoming one of those comfortable buzzwords that Andrea Cornwall refers to as ‘fuzzwords’.  The fuzziness creates a ‘normative resonance’ that makes everyone feel good. It aims to please as many people as possible without revealing which meaning they personally favour. I suspect that if we dug deeper we might find some ideological differences.

Contradictions from seeking different kinds of results We discussed how aid can be used for a wide range of results – from women’s empowerment in local communities to macro-economic stabilisation. We recognised these would therefore require different kinds of methods for assessing the impact of aid. But we did not discuss how when aid achieves one set of results – for example macro-economic stabilisation – it may have a positive or detrimental effect on achieving results in other parts of the portfolio – for example on women’s empowerment.  And although Andrew Mitchell mentioned it in his speech last week, we ran out of time yesterday to look at the wider impact on aid results of any donor government’s broader policy agenda. Yesterday, for example, the Guardian ran a news item that the UK Government is abstaining from voting in favour of the new ILO Convention covering the rights of domestic workers, exactly the kind of impoverished and exploited people that its aid programme is committed to helping.

Modest results vs ambitious public statements. Talking about what foreign aid can achieve also tended to magnify its significance for making a difference – and attributing to aid policies what is very largely achieved by people in countries themselves.  This is partly a problem of how development professionals communicate the role of aid to domestic taxpayers. This is very important with significant risks for achieving results when an over-simplified message shapes the nature of aid interventions that become detached from the complex reality of aid. But even among ourselves, we had to be reminded that these were not our results and that international aid is not the centre of the universe in the countries where money goes.

Your result, my result, their result? Bearing all this in mind, what do we do about the contradiction between on the one hand, official willingness to be flexible and responsive to local realities and to recognise multiple perspectives about what is a result and the methods appropriate for assessing this, and on the other hand, what grantees experience in practice?   At the meeting I read out an email I had had from a grass roots NGO leader:

Sometimes the big donors are themselves ok, but the systems they have set up and the people they have employed to manage their funding have put in a lot of “reporting” initiatives that takes up all the time. We have just had a big transformation within our organization -shifting from big to smaller donors so that we can focus better on the needs in the communities.

Perverse effects of bureaucracy. Others at the meeting also spoke of the negative effect of increasing bureaucratic demands. Up and down the aid chain, managers transmit important messages about accountability with protocols and procedures. These can have perverse consequences if not balanced with an equal interest in grantees reporting about local realities how they are trying to respond to these, including what they have learnt from problems and failures.  Reporting against targets, paradoxically, can inhibit the very innovation and critical reflection needed to make the work more effective from donors in relation to the results agenda.

So what can be done about this? Here are my thoughts. The results agenda has accentuated what had already become an increasingly burdensome process of more frequent upward reporting requirements on growing numbers of quantifiable indicators.  Senior management in aid agencies must take three steps if people are to accept wholeheartedly, rather than comply unwillingly with, the ‘results agenda’ and more systematic and robust approaches to assess performance and effectiveness.

First, senior managers in aid agencies must recognise the power relations that prevent grantees from protesting at the burdens being placed upon them. Then do something about this, such as encouraging and enabling middle management to actively support rather than hinder the commitment and creativity of organisations they (co)fund. And they must be ready to listen to grantees when they come forward and report instances of the mechanistic and counter-productive application of excessive and inflexible procedures. What about a watchdog, complaints option for registering and making public, abuse and rigid use of the results agenda by  aid agencies?

Second, a much-discussed point is using the urgency of the results agenda to review the procedural red tape for planning, monitoring and reporting that has accumulated in the last two decades. Things keep getting added on but nothing ever gets removed, as one meeting participant commented. Some argued that a focus on outcomes could facilitate simplification: But what are the minimal, need to know requirement for different kinds of change that are supported?

Third, organisational learning is not an option for delivering results.Those developing or ensuring compliance of planning, monitoring and reporting procedures must check how these impact on practitioners’ capacities to learn.  Srilatha Batliwala has put this very well:

The first and most important overarching lesson …. is that results assessment is best achieved when it is approached as a learning enterprise for both donors and grantees. This involves something of a paradigm shift – viz., moving from the rather intimidating “prove that you did what you were supposed to” to a more collaborative “let’s learn together “.