Skip to content

Conference starts today: Background papers now on line

2013 April 23
Comments Off

The Big Push Forward convenors welcome our hundred and ten participants and student volunteers.  For all those unable to attend, case this evening, viagra Brendan Whitty will be blogging about that day’s highlights and don’t forget tomorrow we are streaming live our final session.

Two papers have been prepared for the Conference and these are now on line. Rosalind Eyben’s Uncovering the Politics of Evidence and Results disentangles the historical threads and origins of results-based management and evidence-based policy/programming discourses. She discovers a strong ‘family resemblance’. Both assume that evidence pertains largely  to verifiable and quantifiable facts, and that other types of knowledge have less or little value; both have a particular understanding of causality, efficiency and accountability. The paper looks at how and why these discourses have entered and influenced the development sector and who is promoting them in which contexts  What has been the effect on the sector’s priorities and practices, and particularly its capacity to support transformative development?

Arguing the importance of being critically aware of how power sustains and reinforces the results-and-evidence discourses, Rosalind examines how these discourses generate artefacts (tools and protocols) such as log frames and theories of change that shape our working practices.   When hierarchical ways of working block communications and dialogue, the artefacts trigger perverse consequences but their power is neither uniform nor constant.  Analysing the politics of accountability and the sector’s internal dynamics, Rosalind suggests there is room for manoeuvre to expand and enable more transformative approaches to results and evidence within the sector.

Brendan Whitty’s paper, Experiences of the Results Agenda, paper analyses the data from an online survey, which invited visitors to the Big Push Forward website to give their perceptions of the impact of the results agenda on their working lives. Brendan analyses the very different experiences and interpretations of the respondents as revealed through both 153 responses to the quantitative survey and 109 qualitative stories. The study discusses the day-to-day practice of small-e evidence –results and targets in management of specific projects – rather than large-E evidence of establishing broader development policies. The stories are about the nuts and bolts of the development processes and artefacts – the theories of change, results frameworks, reporting requirements and value for money rubrics. It is about what ‘e’ is being collected, how it is used, and to what effect.

Respondents disagreed about the effects of these artefacts.   The contradictory perceptions seem to be often in tension. Thus learning is often seen to be(?) in tension with accountability; capturing the complexity in evaluation with harmonisation and reductionism; coordination of partners with constraining their freedom to adapt. How these tensions are resolved and the perceptions play out seems to be dependent on how the artefact is communicated, managed and tailored to its context. The fit appears to be important: the fit of the artefact to the existing systems and capacity of the organisation, and also the fit of the artefact to the specifics of the intervention (e.g. its complexity, the number of partners). Finally, perceptions of an artefact seem to be affected both by staff’s’s own circumstances and their relationship with others. The survey data suggests that those in M&E and management roles, who benefit from better data and more resources for their priorities, tended to be more positive than those in project implementation and mid-level roles.

 

During the conference we will be exploring these ideas and testing these intepretations. Come back tomorrow for the deliberations of Day 1,

Happy Birthday to … the BPF!

2013 April 21
Comments Off

This week our Politics of Evidence conference gets underway – and we celebrate our second birthday. We are delighted there has been so much interest and sorry that we have inadequate space for all those keen to attend. For those unable to attend, ailment we are live streaming the last session of the conference (15.45- 17.00 UK time on April 24).  We hope to post the conference report on the website at the end of May and have longer term plans for a book.

The conference coincides with the second anniversary of the birth of the Big Push Forward. We posted our first blog on the 26th April 2011. Our aims have stayed consistent – helping create the political space to ensure appropriate approaches in the design, monitoring and evaluation of projects with transformative aims. This is necessary for the international development sector to continue to seize opportunities to support transformation for greater social justice. read more…

Results and Culture: How our Traditions Frame the Agenda

2013 April 18

In my experience, try the results agenda is not only emotional in the sense of controversial, but also confusing to many people, NGO staff I work with in Africa, Asia and Germany have difficulties with the concept of results, and much goes wrong. Arguably a lot of the trouble stems from a strong utilitarian influence on the results agenda that does not fit well with other cultural traditions involved in development aid.

Utilitarianism is a philosophical tradition that started in Britain in the 18th century. It deals with the question of how to act morally, and what government action is morally best. Put simply, in a utilitarian view, human behaviour is the more moral, the more it creates happiness. In the words of Bentham, “it is the greatest happiness of the greatest number that is the measure of right and wrong”. The utilitarian idea has been very influential in Britain, and more widely in the Anglo-Saxon world. The push for effectiveness builds on this tradition. “Happiness” is now made to be understood as “results”. Governments are “effective” (read: moral) if they produce lots of “results” (read: happiness). To make effectiveness measurable, results should be pre-defined. I am not sure if the architects of the results agenda are aware of their utilitarian background, but we are all heavily influenced by our traditions, and the forerunners of the results agenda (New Public Management, micro-economics and the logical framework concept) are dominated by North American thinking building on utilitarianism. People from other traditions just do not understand the underlying assumptions and are confused. Being German myself, I have observed that German development agencies found it rather difficult to introduce results frameworks. They experienced a lot of resistance from staff, and people were confused for a long time. They disliked the added bureaucracy that comes with the current results concepts. But, I believe, underlying is that the Anglo-Saxon results concept does not fit into German culture.

Different Paradigms

Many Germans, particularly in the social and cultural sciences, are brought up in very different philosophical traditions than Anglo-Saxons. Two philosophies of German origin are particularly relevant to the effectiveness debate. read more…

Watch out for Payment by Results!

2013 April 11

At the Politics of Evidence conference we will be discussing how certain approaches to accountability may undermine the sector’s potential to support transformative development.  Payment by Results (PBR) is one to watch out for. But we have been here before.

PBR in the 19th Century

As Europe and North America industrialised and proceeded to colonize the rest of the world, mind the positivist power of numbers appeared to tame uncertainty in an era of such rapid change. In Britain, site the fondness  for measurable facts led the introduction of ‘payment by results’ (PBR) into elementary schools in the middle of the 19th Century. PBR (aka Cash on Delivery) is when commissioners of services (e.g. a government) pay the service providers only after a pre-determined result has been achieved and independently verified.  The logic of PBR is that there is a manageable level of risk in achieving the result and that service providers must be incentivised to play a more active role.  150 years ago – like today –   the buzzwords were efficiency, value for money (VfM), competition and a balanced budget. At the time PBR was criticised for its mechanistic approach that impeded children’s educational development and sacrificed long term benefits for short term achievements.  By the end of 19th Century PBR was abolished partly due to the increased bureaucracy and administration costs of verifying the results.  PBR had been proven to be inefficient! Fast forward to 2013 when ‘Public bodies seem to be pursuing the use of payment by results with the vigour of a drunk in search of the next bottle of alcohol’, Jon Tizard writes. read more…

Time for a new, value driven politics of evidence?

2013 April 9
Comments Off

Speakers at IDS and ITAD’s recent launch of the Centre for Development Impact provoked interesting reflections related to evaluation and the politics of evidence. Bob Piccotto, ambulance a former Director General of the World Bank’s Independent Evaluation Group, nurse gave an inspiring keynote speech calling for a multi-disciplinary, look ‘beyond aid’ evaluation agenda driven by the desire to tackle inequality and contribute to social justice. I found myself wondering whether this could be considered a call for a more reflexive, political evaluation agenda with a new politics of evidence, and whether the development community is equipped to respond. read more…

Mixed Methods to the Rescue? Or Playing the Game to Change the Rules…

2013 April 4
Comments Off

The Politics of Evidence conference will be exploring how people are engaging with problematic practices and protocols, cialis and what alternatives they have found to create spaces for approaches more aligned with transformational development, viagra and which serve their learning purposes. I believe we can learn a lot about this from the experience of ‘front-line’ practitioners who are often subtly playing the game to change the rules.

It would seem that most people believe that ‘mixed methods’, amongst other things, are essential in order to make sensible judgements about the effectiveness of development interventions. This would include agencies such as the World Bank, as well as evaluation specialists. DFID has recently commissioned an important study entitled ‘Broadening the Range of Designs and Methods for Impact Evaluations, which sought to ‘establish and promote a credible and robust expanded set of designs and methods that are suitable to assess the impact of complex development programs’.

Now whilst there is still a great deal of debate about whether there is really a commitment to mixed methods in practice, there are also a number of challenges with implementing ‘mixed methods’ approaches. This post focuses on these issues. read more…

‘The Politics of Evidence’ in my Daily Working Life

2013 March 30
Comments Off

One problem we face in the evidence debates is the use of single examples to assert a generalization or uphold certain positions. This led us to organize the crowdsourcing of experiences as input to the Politics of Evidence conference. With 150 stories shared online and 70+ to be debated at the conference, prescription more grounded discussion becomes possible to generate nuanced insights about ‘the politics of evidence’ and nudge us beyond simplistic yes/no positions.

I’ve been taking a fresh look at my own work with a ‘politics’ lens and see it in small and larger forms in many nooks and crannies. Below are some recent work situations from the past five months in which I am directly involved. They illustrate how I see the ‘the politics of evidence’ playing out in designs and decisions around what Brendan has described as the (small and big) E of evidence– the results and target oriented issues, cialis and those that relate to evidence of what works. read more…

It’s an emotional subject

2013 March 27

In advance of next month’s Conference on the Politics of Evidence, viagra recent weeks have found me speaking at meetings in Abuja, cure Geneva and Oxford.  Meeting participants were country programme staff, M&E specialists and head office programme/policy people in international NGOs and United Nations agencies.   What was the mood?

First of all relief among many participants to be in a collective space where it’s OK to talk about how they feel about what is happening to them – and to find that others were experiencing similar emotions of frustration of the impact of the results and evidence agenda on their jobs.  ‘All I do now is write reports’ said one. ‘I make up numbers’, said another, bitterly.  There is anger at the absurdity of ‘targetitis’ and the proliferation of tools and protocols.  People mentioned being forced to develop Theories of Change that ignored the ‘politics of how’ and the sheer stupidity of a large global programme that has 300 indicators for measuring change.  The long-standing anxiety was aired about how alternative pathways of change are ignored by the logical framework single linear cause-effect proposition. read more…

Of Devils, Details and Multiple Perceptions

2013 March 12
Comments Off

At the end of February we stopped taking stories from the crowdsourcing survey, pills having received a total of 151 responses and over 100 individual stories and snippets of experiences. We’ll be working them up into a paper to share at next month’s Conference, viagra but here are a few observations as a trailer.

read more…

Epistemology: the elephant in the room?

2013 February 25

A key take away message for me about the exchange on evidence with Chris Whitty and Stefan Dercon on the From Poverty to Power blog was the challenge of communicating complex ideas simply, cialis particularly, the difficulties of clearly explaining the importance of different ways of looking at the world. In her recent blog  Cathy Shutt pointed out some great comments on this, notably the tweet from @ScepSec ‘Not sure what troubles me most. The twitterati’s inability 2 see the message or Roche & Eyben’s inability to express it’. Inadequate communication can result in those with different perspectives ‘talking past each other’, as suggested by Duncan Green in his summary of the debate.

Understanding different perspectives is important not only to get to grips with ‘evidence’ but also to assess how successfully ‘wicked’ problems are being tackled. For example, understanding the issue of violence against women requires inquiring into the perspectives and world-views of those involved – including victims and perpetrators, as well as citizens, legislators, governments and donors.  And doing this well requires an understanding of epistemology i.e. what kinds of knowledge are considered valid, as well as methodology i.e. how this knowledge can be generated. read more…