Alternatives for Reporting

The Problem

The last decade has seen a narrowing of how the value of international aid and development effectiveness is measured with an increasing emphasis on ‘results’, clinic ‘performance management’, rx and ‘value for money’. Many donor organisations, prostate government ministries and philanthropic foundations demand that performance is assessed against quantifiable time-bound indicators. Taxpayers and donors, they claim, want to know, often in quantitative terms, how their money has been spent. Accountability is therefore largely to the donor, not those that the aid system seeks to benefit.

• What do you think? Who should be reporting to who? What should that reporting focus on? How might it best be done?

It is our challenge to seek out, develop, share and promote innovative reporting processes that are more reflective of the reality of the development process and in which the voices of those usually least heard are central.

•  What experiences do you know of? What promising examples can you share? What ideas do you have?

 Please take the time to share with us your experiences, thoughts, and ideas!

Many development practitioners would argue that the way that this agenda on results and reporting is playing out has become ‘counter-developmental’, negatively shaping the character of development, programming and funding. Most results-based programs assume a linear model with predictable outcomes — a definable problem with a quantifiable solution. Consequently, it is argued, that it has forced a move away from more complex, unpredictable development processes, focused on transformational change, in favour of projects which focus less on more immediate outcomes. Creative pathways developed through collaborative learning between multiple agencies, which incorporate alternative perspectives which may offer better prospects for sustainable change, and a responsiveness to unforeseen consequences are therefore discouraged.  Not least because they make the attribution of results to individual donors harder.

The Accra Agenda for Action states that ‘Aid is about building partnerships for development’. For aid to be truly effective, it must incorporate diverse perspectives and respond to different ideas about how success should be defined and how it is achieved. Consequently, development practitioners often undertake initiatives which are founded upon building effective relationships, risk-taking and learning and adaptation, rather than following the pre-ordained paths of blue-print planning. However, many of these process will be hidden when reporting upwards, transforming their achievements and the processes which underpin them, into linear cause–effect narratives. This deforms the reality of the development experience, renders invisible the importance and effectiveness of relationship-building, and diminishes the impact of non-quantifiable and complex outcomes.

As Rosalind Eyben has noted

“A consultant working on a bilateral project in Africa that explores how formal institutional structures such as parliament interact with networks that aim to strengthen the voice of marginalised communities revealed that when completing an output-to-purpose review, DFID country office staff may have formally represented the project as different from how they knew it to be so as to conform with linear cause-effect thinking about aid effectiveness.  At a seminar in The Hague on complexity approaches to aid effectiveness, officials cited an evaluation of a Tanzania District Development Programme. While it failed to achieve its pre-set objectives, it nevertheless produced very interesting and positive side effects in relation to strengthening civil society – effects that were not reported upon because they were unanticipated.

How can we learn from each other, let alone our beneficiaries, if top-down results-based reporting forces us to hide our most effective achievements?” [1. Eyben, Rosalind 2009. All Party Parliamentary Group for Debt, Aid and Trade Inquiry into Aid Effectiveness, Institute of Development Studies, University of Sussex.]

How can we learn from each other, let alone from the communities in which we work, if top-down results-based reporting forces us to hide our most effective achievements?

On the other hand some people see the focus on results as an opportunity. Claire Melamed of the Overseas Development Institute argues that:

“The results agenda is actually a huge opportunity for people who care about relationships, trust, empowerment, rights and complexity to find ways of getting these things firmly integrated into how we measure development. Then they’d be part of the mainstream……..Imagine if we actually knew what poor people wanted and if they were getting it? Everyone who works in development should surely admit that we don’t know as much as we should about if we are actually delivering ‘value’ as the recipients of our efforts would define it….…..This agenda should be used to encourage a focus on what results poor people themselves (or, more likely, poor women, poor men, poor people in cities, in rural areas and so on, who would all have different priorities) most want to see, and how they’d define ‘value’ or ‘effectiveness” [2. See http://www.oxfamblogs.org/fp2p/?p=4756 ]

In this cluster we want to explore if, and how, aid and humanitarian organisations can push back on a narrowly defined notion of results and push forward to promote different methods of reporting so that the requirement for aggregated data at a Northern policy level does not negatively influence the character of programming in complex development contexts. In addition, we aim to explore how alternative ways of approaching reporting might contribute to, and align with, alternative democratic and decentralised processes of measurement, accountability and transparency in which the voice of those that aid agencies seek to benefit is central.

The way forward…

There are a number of existing alternatives.  People living in poverty are already using their voice to decide their own destiny and hold aid and humanitarian organisations to account.

The rise of story telling. In oral cultures stories have of course been the primary way in which information and knowledge have been shared and passed on to the next generation for millennia. It has taken a bit longer for the development industry to catch up but there have been a number of interesting developments which are relevant to alternative reporting methods including: how Most Significant Change methods are used; how tools like sense maker are attempting to capture anecdotes and ‘water-cooler’ stories and allow the tellers to add levels of meaning about them – see for example how Global Giving are using this method and trying to amplify local voices; and how theatre and video can be used to communicate, and to allow communities to tell their stories. 

The advent of new information and communication technologies (ICTs) and the rapid adoption of mobile phones in developing countries as also given rise to new ways for more engaged citizen participation. New open platforms are building local, regional and international relationships and connections visible to a global audience. Timely and comprehensive information on development organisations’ activities, expenditures, and policies can now be provided to and from stakeholders, ensuring citizens and donors alike can potentially participate more actively in the process of global development.

Citizen journalism (like Global Voices, Small World News), advocacy (Global Witness), and the spread of community radio stations, for example, have led to new ways for the South to engage with the North and create new citizen–citizen communication. Mass media which overlooks or absorbs minority voices is becoming fractured and decentralised leading to smaller-scale localised exchanges of information, stories, and ideas, with some also linked into international networks and communities.

Communities on the receiving end of aid projects and programs possess valuable information that can be used to paint a more complete and holistic picture of the development process. There are a new set of tools that citizens are using to submit real time information on aid projects quickly and easily. Local informational brokering initiatives like Twaweza and its ‘one stop information shop’ Uwazi aim to improve both citizen agency and learning and self-critique in development, and publicly sanction poor performance. Twaweza collects information from citizens in Tanzania through a variety of means, from questionnaires to collecting photographs, on issues as varied as government budget allocations to the deplorable state of primary school toilets. SMS-based systems like uReport inform citizens of their rights and available services while providing a platform for feedback. Open platforms like Huduma aggregate concerns, complaints and observations of citizens, sent through mobile devices, and channel them directly to service providers including aid agencies.

Maps and other data-sharing initiatives can empower citizens to hold agencies and governments accountable. The World Bank’s Mapping for Results (M4R) platform makes it possible to see how much money is going to which of its 16,000 project sites. People in each country now have access to information about what projects are supposed to be happening there.

Crowd-sourcing or crowd-voicing combines the power of mobile phones, mapping technology, and social networking to enable citizens to facilitate aid delivery, bear witness to human rights abuses and corruption, and to hold governments and aid agencies more accountable. The crowd-sourcing platform, Ushahidi, has been utilised to track developments in Afghanistan, document the humanitarian crisis in eastern Congo, track crisis information in disaster-affected Haiti, report sexual harassment in Egypt, and to publicly monitor elections in India and Mexico.

In addition to the cyber-sphere, for many years participatory processes for monitoring development have provided men and women, who are often the ‘objects’ of development, with the ability to become its subjects and to publicly sanction poor behaviour and performance of aid organisations and their governments. Citizen Report Cards  and Community Score Cards have been used to provide direct feedback to service providers. Other social accountability and financial transparency processes, from participatory budgeting and Public Expenditure Tracking Surveys (PETS), to posting financial information on whiteboards, are  being promoted by development agencies to allow primary stakeholders to assess their activities.

Accountability for aid, and issues of global justice, also mean involving citizens from donor countries. New forms of citizen–citizen resource transfer like Kiva are growing in popularity as they are seen to reduce the gap between the donor and receiver. More accessible, multi-dimensional and accountable reporting of development processes using voices from the South like the Guardian’s Katine project have arguably allowed supporters and citizens to understand more effectively the realities of development work. Facilitating direct North–South, citizen–citizen communication (like Refugee Realities Uganda/Australia and ActionAid’s Project Toto) are example of further attempts to narrow the gap.

Some argue these processes not only provide those that aid agencies seek to benefit by having what Albert Hirschman described as the ‘voice’ option, which they so often lack, but also potentially provide aid agencies with the feedback they need to evolve.  Other suggest that many of these efforts are as yet unproven or indeed have been ‘oversold.’