Assessing Value for Money of Social Accountability Initiatives

On Wednesday, and I joined a group of INGO practitioners considering undertaking a value for money study of a social accountability programme that uses community scorecards to enable communities to engage with government to demand better quality health facilities. Their monitoring and evaluation systems are producing evidence that suggests the programmes are having positive outcomes on people’s lives and they are interested in exploring value for money aspects more systematically for their own organisational learning and accountability, treatment although it is likely to also help with donor reporting.

We considered whether such a study could generate data that will help build a case for social accountability programmes reducing leakage of government budgets funded by national taxpayers and donors. Obviously this would be useful for beginning to build an argument that civil society actors contributing to changes in the institutional accountability behaviours of governments have the potential to reduce leakage and thus increase the value provided by national and donor tax payers’ money.

Our discussion raised a number of questions about the methodological implications of the different purposes and research questions underpinning such a value for money study. We wondered whether the main problematic should be to establish that improved access to service delivery and health outcomes:

• Offer good value as determined by citizens through a participatory process such as SROI that presumably involves citizens in a) valuing outcomes and b) being involved in the assessment of whether they think those are good value in view of inputs? Would they have to influenced resource allocation decisions for this to make this a truly participatory approach to assessing value?
• Are better than the ‘expected’ or targeted outcomes and thus can be considered to demonstrate value for money in the views of the implementers and funders. I suppose this implicitly suggests a rigorous appraisal process determined that the overall approach proposed was good value?
• Offered better value for money than an established ‘objective’ benchmark for health outcomes?
• Offer better value for money than direct service delivery based on a comparison?
• Or some mix of the above

Does anyone have experience undertaking such studies for social accountability programmes? If so:

• What was the main purpose – was it to satisfy a donor, or your own learning and accountability needs?
• With reference to the questions above, how did you frame the VfM problematic and:
  • What sort of conceptual methodological approach to evaluation did you use? We were concerned about ethical dilemmas around experimental approaches.
  • Has anyone used quasi-experimental methods to establish comparisons?
  • Is there any experience of attempting a value for money exercise for a social accountability programme using developmental evaluation or a complexity lens to not only look at what outcomes were achieved and how they were valued, but also how and why? If we maintain that good management practices, relationships, feedback loops and iterative learning and adaptation is key to achieving value for money, then surely value for money assessments have to explore if how and why such adaptation may have had a bearing on outcomes and thus overall VfM?