Skip to content

Watch out for Payment by Results!

2013 April 11

At the Politics of Evidence conference we will be discussing how certain approaches to accountability may undermine the sector’s potential to support transformative development.  Payment by Results (PBR) is one to watch out for. But we have been here before.

PBR in the 19th Century

As Europe and North America industrialised and proceeded to colonize the rest of the world, the positivist power of numbers appeared to tame uncertainty in an era of such rapid change. In Britain, the fondness  for measurable facts led the introduction of ‘payment by results’ (PBR) into elementary schools in the middle of the 19th Century. PBR (aka Cash on Delivery) is when commissioners of services (e.g. a government) pay the service providers only after a pre-determined result has been achieved and independently verified.  The logic of PBR is that there is a manageable level of risk in achieving the result and that service providers must be incentivised to play a more active role.  150 years ago – like today –   the buzzwords were efficiency, value for money (VfM), competition and a balanced budget. At the time PBR was criticised for its mechanistic approach that impeded children’s educational development and sacrificed long term benefits for short term achievements.  By the end of 19th Century PBR was abolished partly due to the increased bureaucracy and administration costs of verifying the results.  PBR had been proven to be inefficient! Fast forward to 2013 when ‘Public bodies seem to be pursuing the use of payment by results with the vigour of a drunk in search of the next bottle of alcohol’, Jon Tizard writes. read more…

Time for a new, value driven politics of evidence?

2013 April 9
Comments Off

Speakers at IDS and ITAD’s recent launch of the Centre for Development Impact provoked interesting reflections related to evaluation and the politics of evidence. Bob Piccotto, a former Director General of the World Bank’s Independent Evaluation Group, gave an inspiring keynote speech calling for a multi-disciplinary, ‘beyond aid’ evaluation agenda driven by the desire to tackle inequality and contribute to social justice. I found myself wondering whether this could be considered a call for a more reflexive, political evaluation agenda with a new politics of evidence, and whether the development community is equipped to respond. read more…

Mixed Methods to the Rescue? Or Playing the Game to Change the Rules…

2013 April 4
Comments Off

The Politics of Evidence conference will be exploring how people are engaging with problematic practices and protocols, and what alternatives they have found to create spaces for approaches more aligned with transformational development, and which serve their learning purposes. I believe we can learn a lot about this from the experience of ‘front-line’ practitioners who are often subtly playing the game to change the rules.

It would seem that most people believe that ‘mixed methods’, amongst other things, are essential in order to make sensible judgements about the effectiveness of development interventions. This would include agencies such as the World Bank, as well as evaluation specialists. DFID has recently commissioned an important study entitled ‘Broadening the Range of Designs and Methods for Impact Evaluations, which sought to ‘establish and promote a credible and robust expanded set of designs and methods that are suitable to assess the impact of complex development programs’.

Now whilst there is still a great deal of debate about whether there is really a commitment to mixed methods in practice, there are also a number of challenges with implementing ‘mixed methods’ approaches. This post focuses on these issues. read more…

‘The Politics of Evidence’ in my Daily Working Life

2013 March 30
Comments Off

One problem we face in the evidence debates is the use of single examples to assert a generalization or uphold certain positions. This led us to organize the crowdsourcing of experiences as input to the Politics of Evidence conference. With 150 stories shared online and 70+ to be debated at the conference, more grounded discussion becomes possible to generate nuanced insights about ‘the politics of evidence’ and nudge us beyond simplistic yes/no positions.

I’ve been taking a fresh look at my own work with a ‘politics’ lens and see it in small and larger forms in many nooks and crannies. Below are some recent work situations from the past five months in which I am directly involved. They illustrate how I see the ‘the politics of evidence’ playing out in designs and decisions around what Brendan has described as the (small and big) E of evidence– the results and target oriented issues, and those that relate to evidence of what works. read more…

It’s an emotional subject

2013 March 27

In advance of next month’s Conference on the Politics of Evidence, recent weeks have found me speaking at meetings in Abuja, Geneva and Oxford.  Meeting participants were country programme staff, M&E specialists and head office programme/policy people in international NGOs and United Nations agencies.   What was the mood?

First of all relief among many participants to be in a collective space where it’s OK to talk about how they feel about what is happening to them – and to find that others were experiencing similar emotions of frustration of the impact of the results and evidence agenda on their jobs.  ‘All I do now is write reports’ said one. ‘I make up numbers’, said another, bitterly.  There is anger at the absurdity of ‘targetitis’ and the proliferation of tools and protocols.  People mentioned being forced to develop Theories of Change that ignored the ‘politics of how’ and the sheer stupidity of a large global programme that has 300 indicators for measuring change.  The long-standing anxiety was aired about how alternative pathways of change are ignored by the logical framework single linear cause-effect proposition. read more…

Of Devils, Details and Multiple Perceptions

2013 March 12
Comments Off

At the end of February we stopped taking stories from the crowdsourcing survey, having received a total of 151 responses and over 100 individual stories and snippets of experiences. We’ll be working them up into a paper to share at next month’s Conference, but here are a few observations as a trailer.

read more…

Epistemology: the elephant in the room?

2013 February 25

A key take away message for me about the exchange on evidence with Chris Whitty and Stefan Dercon on the From Poverty to Power blog was the challenge of communicating complex ideas simply, particularly, the difficulties of clearly explaining the importance of different ways of looking at the world. In her recent blog  Cathy Shutt pointed out some great comments on this, notably the tweet from @ScepSec ‘Not sure what troubles me most. The twitterati’s inability 2 see the message or Roche & Eyben’s inability to express it’. Inadequate communication can result in those with different perspectives ‘talking past each other’, as suggested by Duncan Green in his summary of the debate.

Understanding different perspectives is important not only to get to grips with ‘evidence’ but also to assess how successfully ‘wicked’ problems are being tackled. For example, understanding the issue of violence against women requires inquiring into the perspectives and world-views of those involved – including victims and perpetrators, as well as citizens, legislators, governments and donors.  And doing this well requires an understanding of epistemology i.e. what kinds of knowledge are considered valid, as well as methodology i.e. how this knowledge can be generated. read more…

One week left to share YOUR experience with the results and evidence agenda: Closes 28 February

2013 February 22
Comments Off

The crowd-sourcing survey for experiences of the results agenda finishes on 28 February. We have already had over a hundred experiences shared.

Evidence on the effects of the results agenda matters, but you need evidence to discuss this! What is happening that is helping your work, and where is it hindering your ability to contribute to transformational development? The nuance of grounded experiences is critical to move beyond unhelpful stereotyping and straw men debates (see Duncan Green’s summary of the wonkwars on evidence).

We need more experiences to start understanding better what is happening from all different corners of international development practice. The survey is designed to identify where the results agenda is showing its best side and where it hurts. What is working, what doesn’t, and under what conditions? The findings will frame discussions on Day 2 at the BPF conference. The collection so far illustrates cautionary tales and stories of good practice. Please add more!

All experiences will be treated confidentially and will contribute to the analysis that will be presented at the conference. Remember – 28 February is the last day to share how the results agenda is affecting your work. Please add more!

 

FAQs ABOUT THE POLITICS OF EVIDENCE CONFERENCE

2013 February 11
Comments Off

Who is it for?  Development practitioners who work towards changing the power relations and structures that create and reproduce inequality, injustice and the non-fulfilment of human rights, and who are interested in the current debates about the positive and negative effects of the ‘evidence’ and ‘results’ discourses on locally-owned and transformational development.  We define development practitioners as people working in the international development sector – in a bilateral/ multilateral agency/ international NGO, as staff member or consultant; or employed by a government department or a non-governmental organisation in a low-income or middle-income country that receives financial aid.  Or s/he might be located in a university/ think tank partially funded by the sector, a private sector consulting company or a philanthropic foundation. read more…

Reflection on politics of evidence debate

2013 February 7

I have been reflecting on the recent ‘wonkwar’  (debate amongst experts) ‘about ‘the political implications of evidence based approaches’. The debate generated considerable interest and activity in the twittersphere; some brilliant comments and quite a large poll that yielded perhaps unsurprising results?

 

 

Excellent comments by several people that I cite here, reminded me that the Big Pushforward’s agenda extends far beyond the realm of evaluation practice and is broadly concerned with the production and use of knowledge by the development community.  Several remarks also emphasised that the Big Push Forward needs to articulate its position on ‘evidence’ more clearly

‘Not sure what troubles me most. The twitterati’s inability 2 see the message or Roche & Eyben’s inability to express it’

It became apparent our critique of the ‘discourse of evidence’ was being read as being anti-evidence!  Nothing could be further than the ‘truth’. We are critiquing the use of the term ‘evidence’ as a particular discourse that influences what is thinkable and do-able in the politics of development. Inequities in power and voice mean that only some approaches to knowledge are judged acceptable and when these are called ‘objective’ it makes them unquestionable. read more…