Skip to content

Happy Birthday to … the BPF!

2013 April 21
Comments Off

This week our Politics of Evidence conference gets underway – and we celebrate our second birthday. We are delighted there has been so much interest and sorry that we have inadequate space for all those keen to attend. For those unable to attend, we are live streaming the last session of the conference (15.45- 17.00 UK time on April 24).  We hope to post the conference report on the website at the end of May and have longer term plans for a book.

The conference coincides with the second anniversary of the birth of the Big Push Forward. We posted our first blog on the 26th April 2011. Our aims have stayed consistent – helping create the political space to ensure appropriate approaches in the design, monitoring and evaluation of projects with transformative aims. This is necessary for the international development sector to continue to seize opportunities to support transformation for greater social justice. read more…

Results and Culture: How our Traditions Frame the Agenda

2013 April 18

In my experience, the results agenda is not only emotional in the sense of controversial, but also confusing to many people, NGO staff I work with in Africa, Asia and Germany have difficulties with the concept of results, and much goes wrong. Arguably a lot of the trouble stems from a strong utilitarian influence on the results agenda that does not fit well with other cultural traditions involved in development aid.

Utilitarianism is a philosophical tradition that started in Britain in the 18th century. It deals with the question of how to act morally, and what government action is morally best. Put simply, in a utilitarian view, human behaviour is the more moral, the more it creates happiness. In the words of Bentham, “it is the greatest happiness of the greatest number that is the measure of right and wrong”. The utilitarian idea has been very influential in Britain, and more widely in the Anglo-Saxon world. The push for effectiveness builds on this tradition. “Happiness” is now made to be understood as “results”. Governments are “effective” (read: moral) if they produce lots of “results” (read: happiness). To make effectiveness measurable, results should be pre-defined. I am not sure if the architects of the results agenda are aware of their utilitarian background, but we are all heavily influenced by our traditions, and the forerunners of the results agenda (New Public Management, micro-economics and the logical framework concept) are dominated by North American thinking building on utilitarianism. People from other traditions just do not understand the underlying assumptions and are confused. Being German myself, I have observed that German development agencies found it rather difficult to introduce results frameworks. They experienced a lot of resistance from staff, and people were confused for a long time. They disliked the added bureaucracy that comes with the current results concepts. But, I believe, underlying is that the Anglo-Saxon results concept does not fit into German culture.

Different Paradigms

Many Germans, particularly in the social and cultural sciences, are brought up in very different philosophical traditions than Anglo-Saxons. Two philosophies of German origin are particularly relevant to the effectiveness debate. read more…

Watch out for Payment by Results!

2013 April 11

At the Politics of Evidence conference we will be discussing how certain approaches to accountability may undermine the sector’s potential to support transformative development.  Payment by Results (PBR) is one to watch out for. But we have been here before.

PBR in the 19th Century

As Europe and North America industrialised and proceeded to colonize the rest of the world, the positivist power of numbers appeared to tame uncertainty in an era of such rapid change. In Britain, the fondness  for measurable facts led the introduction of ‘payment by results’ (PBR) into elementary schools in the middle of the 19th Century. PBR (aka Cash on Delivery) is when commissioners of services (e.g. a government) pay the service providers only after a pre-determined result has been achieved and independently verified.  The logic of PBR is that there is a manageable level of risk in achieving the result and that service providers must be incentivised to play a more active role.  150 years ago – like today –   the buzzwords were efficiency, value for money (VfM), competition and a balanced budget. At the time PBR was criticised for its mechanistic approach that impeded children’s educational development and sacrificed long term benefits for short term achievements.  By the end of 19th Century PBR was abolished partly due to the increased bureaucracy and administration costs of verifying the results.  PBR had been proven to be inefficient! Fast forward to 2013 when ‘Public bodies seem to be pursuing the use of payment by results with the vigour of a drunk in search of the next bottle of alcohol’, Jon Tizard writes. read more…

Time for a new, value driven politics of evidence?

2013 April 9
Comments Off

Speakers at IDS and ITAD’s recent launch of the Centre for Development Impact provoked interesting reflections related to evaluation and the politics of evidence. Bob Piccotto, a former Director General of the World Bank’s Independent Evaluation Group, gave an inspiring keynote speech calling for a multi-disciplinary, ‘beyond aid’ evaluation agenda driven by the desire to tackle inequality and contribute to social justice. I found myself wondering whether this could be considered a call for a more reflexive, political evaluation agenda with a new politics of evidence, and whether the development community is equipped to respond. read more…

Mixed Methods to the Rescue? Or Playing the Game to Change the Rules…

2013 April 4
Comments Off

The Politics of Evidence conference will be exploring how people are engaging with problematic practices and protocols, and what alternatives they have found to create spaces for approaches more aligned with transformational development, and which serve their learning purposes. I believe we can learn a lot about this from the experience of ‘front-line’ practitioners who are often subtly playing the game to change the rules.

It would seem that most people believe that ‘mixed methods’, amongst other things, are essential in order to make sensible judgements about the effectiveness of development interventions. This would include agencies such as the World Bank, as well as evaluation specialists. DFID has recently commissioned an important study entitled ‘Broadening the Range of Designs and Methods for Impact Evaluations, which sought to ‘establish and promote a credible and robust expanded set of designs and methods that are suitable to assess the impact of complex development programs’.

Now whilst there is still a great deal of debate about whether there is really a commitment to mixed methods in practice, there are also a number of challenges with implementing ‘mixed methods’ approaches. This post focuses on these issues. read more…

‘The Politics of Evidence’ in my Daily Working Life

2013 March 30
Comments Off

One problem we face in the evidence debates is the use of single examples to assert a generalization or uphold certain positions. This led us to organize the crowdsourcing of experiences as input to the Politics of Evidence conference. With 150 stories shared online and 70+ to be debated at the conference, more grounded discussion becomes possible to generate nuanced insights about ‘the politics of evidence’ and nudge us beyond simplistic yes/no positions.

I’ve been taking a fresh look at my own work with a ‘politics’ lens and see it in small and larger forms in many nooks and crannies. Below are some recent work situations from the past five months in which I am directly involved. They illustrate how I see the ‘the politics of evidence’ playing out in designs and decisions around what Brendan has described as the (small and big) E of evidence– the results and target oriented issues, and those that relate to evidence of what works. read more…

It’s an emotional subject

2013 March 27

In advance of next month’s Conference on the Politics of Evidence, recent weeks have found me speaking at meetings in Abuja, Geneva and Oxford.  Meeting participants were country programme staff, M&E specialists and head office programme/policy people in international NGOs and United Nations agencies.   What was the mood?

First of all relief among many participants to be in a collective space where it’s OK to talk about how they feel about what is happening to them – and to find that others were experiencing similar emotions of frustration of the impact of the results and evidence agenda on their jobs.  ‘All I do now is write reports’ said one. ‘I make up numbers’, said another, bitterly.  There is anger at the absurdity of ‘targetitis’ and the proliferation of tools and protocols.  People mentioned being forced to develop Theories of Change that ignored the ‘politics of how’ and the sheer stupidity of a large global programme that has 300 indicators for measuring change.  The long-standing anxiety was aired about how alternative pathways of change are ignored by the logical framework single linear cause-effect proposition. read more…

Of Devils, Details and Multiple Perceptions

2013 March 12
Comments Off

At the end of February we stopped taking stories from the crowdsourcing survey, having received a total of 151 responses and over 100 individual stories and snippets of experiences. We’ll be working them up into a paper to share at next month’s Conference, but here are a few observations as a trailer.

read more…

Epistemology: the elephant in the room?

2013 February 25

A key take away message for me about the exchange on evidence with Chris Whitty and Stefan Dercon on the From Poverty to Power blog was the challenge of communicating complex ideas simply, particularly, the difficulties of clearly explaining the importance of different ways of looking at the world. In her recent blog  Cathy Shutt pointed out some great comments on this, notably the tweet from @ScepSec ‘Not sure what troubles me most. The twitterati’s inability 2 see the message or Roche & Eyben’s inability to express it’. Inadequate communication can result in those with different perspectives ‘talking past each other’, as suggested by Duncan Green in his summary of the debate.

Understanding different perspectives is important not only to get to grips with ‘evidence’ but also to assess how successfully ‘wicked’ problems are being tackled. For example, understanding the issue of violence against women requires inquiring into the perspectives and world-views of those involved – including victims and perpetrators, as well as citizens, legislators, governments and donors.  And doing this well requires an understanding of epistemology i.e. what kinds of knowledge are considered valid, as well as methodology i.e. how this knowledge can be generated. read more…

One week left to share YOUR experience with the results and evidence agenda: Closes 28 February

2013 February 22
Comments Off

The crowd-sourcing survey for experiences of the results agenda finishes on 28 February. We have already had over a hundred experiences shared.

Evidence on the effects of the results agenda matters, but you need evidence to discuss this! What is happening that is helping your work, and where is it hindering your ability to contribute to transformational development? The nuance of grounded experiences is critical to move beyond unhelpful stereotyping and straw men debates (see Duncan Green’s summary of the wonkwars on evidence).

We need more experiences to start understanding better what is happening from all different corners of international development practice. The survey is designed to identify where the results agenda is showing its best side and where it hurts. What is working, what doesn’t, and under what conditions? The findings will frame discussions on Day 2 at the BPF conference. The collection so far illustrates cautionary tales and stories of good practice. Please add more!

All experiences will be treated confidentially and will contribute to the analysis that will be presented at the conference. Remember – 28 February is the last day to share how the results agenda is affecting your work. Please add more!