Tag Archives: Reflection

Monitoring, evaluation and the search for ‘impact’

It is hard for anyone, be it the general public, donors, governments or non-governmental organisations (NGOs) not to want to commit resources to saving lives. Who wouldn’t? There is nothing more rewarding than knowing that you have contributed towards the preservation of human life, what could be better than something so worthy? Big foundations such as the Bill and Melinda Gates Foundation and NGOs such as Medecins Sans Frontieres celebrate their achievements of saving lives. In fact, it is tempting for any organisation working in development to want to show that they are making a difference.

The football player Didier Drogba, Bono, Nike CEO Mark Parker, RED boss Susan Smith Ellis and Global Fund CEO Michel Kazatchine are all saving lives with football boot laces.
The football player Didier Drogba, Bono, Nike CEO Mark Parker, RED CEO Susan Smith Ellis and former Global Fund ED Michel Kazatchkine are all saving lives with football boot laces.

Due to an increase in calls for accountability in international development, and for direct results from investment to be made clear, the aid community and governments have been trying more and more to show value for money in their projects, to show that there is a true ‘impact’ to their work.

Hence the incredible growth over the last 20 years of monitoring and evaluation in international development; seeing the introduction of the much maligned log-frame, a results-based mechanism for measuring project milestones; and the search for the ever elusive impact of development projects.

Whether this is done by displaying giant pie charts on NGO websites showing that 99% of funds received are spent on programme costs with only 1% spent on administrative costs, surely the worst way to judge a charity, or by claiming that a project supporting vocational teacher training in Manila’s impact was poverty reduction in the Philippines, we are surely summarising achievements in the worst possible way.

It is no coincidence that the largest NGOs by funding are also the best at marketing. ‘Saving lives’ is an easy sell. But let’s think a little more about what it actually means to save a life, and the impact of an aid project on someone’s life.

For instance, if a child is vaccinated against measles, in an area where this is a common disease, could he be grouped into a category of children vaccinated, his life seemingly saved by the reduction in the number of measles cases contracted in this area? This attribution shows some logic, and this is why organisations such as the Bill and Melinda Gates Foundation make videos and campaigns about saving lives.

But let’s take a step back. What about the development of that vaccine? The countless researchers who worked on this? The other researchers who influenced them? If we look at investment in research itself, can we attribute this to a life saved? What about the manufacturer of this product? Are they also saving lives? What about the government that invested in the research? Has their investment saved this life also? Here the attribution starts to break down.

As an end supplier of a ‘product,’ it is easy for a development organisation to claim that they are saving lives, but it took a complex system to develop and deliver this vaccine, and that causality and attribution is something that is much harder to show. Ultimately what all of these actors are doing is contributing towards a reduced chance of mortality, which is hardly sexy. In reality these actors are all part of a system contributing towards a much greater whole.

I’m part of something much bigger?
I’m part of something much bigger?

Measuring impact, which by definition is forceful in its nature, is the antithesis of development, which is never successful as an imposition.

Making less grandiose and glamorous claims will also help us to put development work into perspective, no longer singling out certain parts of the system for recognition, but instead acknowledging a much greater and more complex chain of events and processes that contributes towards development, and the small but significant contribution that development projects and funding play within this.

We need to stop focusing on ‘impact’ because it encourages vertical programming and dramatic marketable programmes with easily attributable end results. That is not to say that we shouldn’t be measuring what the contribution of our work is, but that we need to stop looking at how we ‘impact’ but instead what our contribution is.

This will lead to more intelligent programming and a system-based approach that ultimately makes a much bigger contribution towards development.

Systems approach to development

This could be something as simple as adopting new measures such as contribution mapping or outcome mapping (a more commonly used approach).

If we design and measure our work in terms of our contributions, we will start to understand and see our work in different terms, and this will have a profound and positive effect on development approaches. It might be a tough sell to donors and to the general public, to whom donors are ultimately answerable, but we owe them the truth.