In global development, we are always looking for the most scientific way to show what works. For example, academics heavily criticised the Millennium Villages Project because of the lack of control group. Simply put, we demand more rigorous proof of what does and doesn’t work in development. We want the hard numbers backing up studies. Anecdotal evidence is not enough.
While not refuting the validity of these criticisms, there are some major reasons why anecdotal evidence of what does and doesn’t work in development still gets traction.
The first reason is simple. Human beings are not good with evaluating statistics, percentages and probability. Dan Gardner’s Risk: the Science and Politics of Fear begins with the telling figure that after September 11, an estimated 1,595 Americans were killed when they switched from flying to driving, perceiving the latter to be safer. Our comprehension of numbers, it seems, is poor.
A second and more complex reason can be seen through our desire to make sense of the world through stories, rather than numbers. This desire is outlined in the brilliant book Thinking Fast and Slow, by Nobel Prize-winning author Daniel Kahneman.
In 1975, the social psychologist Richard Nisbett and his student Eugene Borgida, at the University of Michigan, conducted the helping experiment on a cohort of psychology students. The experiment was set up like so: six participants were placed in individual isolation booths, where they were allowed to talk for two minutes at one time about their personal lives and problems. Only one microphone was active at one time. Importantly, one of the participants was a stooge, covertly instructed by the researchers prior to the experiment.
The stooge spoke first, speaking about adjusting to life in New York, and admitting that he was prone to seizures which would could be set off by high-stress situations. Each of the other five participants had their own turn, then it came back to the stooge again. This time, he became agitated and incoherent, told the five others that he felt a seizure coming on, and asked for someone to help him, gasping “C-could somebody-er-er-help-er-uh-uh-uh [choking sounds]. I… Im gonna die-er-er-er Im… gonna die-er-er-I seizure I-er [chokes, then quiet]” as he fell to the ground. Not a further sound was heard from him.
How many of the other people would you expect to rush to the aid of the possibly dying man?
The answer is disturbingly low. Only four of 15 (27%) participants responded immediately. Six never got out of their booth, and five others came only after the “victim” choked to death. This effect is known in psychology as the bystander effect, where the diffusion of responsibility occurs when there are other people around to take action. Decent people, like you and I, are less likely to help someone in need when there are others around that might avoid you dealing with an unpleasant situation.
After describing this experiment, Nisbett and Borgida showed the psychology students video interviews of two people who had supposedly participated in the New York study. The interviews were deliberately bland, where the interviewees talked about their hobbies, plans for the future, and so on. They were designed not to elicit any further information about their propensity to help or not.
Students were then asked to guess whether or not the two interviewees had helped the person in distress. This would help to answer the pressing question: given that students knew the statistical unlikelihood of participants in the helping experiment coming to the aid of the person in distress, would this knowledge affect their guesses about whether the two interviewees had helped?
The answer is both worrying and surprising. They learnt nothing at all. 100% of the class still predicted both interviewees had helped immediately, despite knowing that the probability of anyone helping is only 27%.
This shows that statistical knowledge of human behaviour has very little bearing on our ability to apply that knowledge in predicting human behaviour.
However, all is not lost. The researchers took another class of students, showed them the two video interviews of participants in the helping experiment, and simply told them that these two had not immediately helped the person in distress. They then asked them to predict the global results for the rest of the participants in the helping experiment. The predictions were surprisingly accurate.
This tells us that teaching people a surprising statistic, and then asking them to predict behaviour, is futile. Yet when people were surprised by individual cases, and then asked to generalise from these cases, they do so with relative ease.
Nisbett and Borgida brilliantly summarised the results of this experiment by stating:
“Subjects’ unwillingness to deduce the particular from the general was matched only by their willingness to infer the general from the particular.”
This telling illustration of human behaviour, one of just many within Kahneman’s book, speaks volumes about our desire to prove what works and what doesn’t work in development. We need stories (the particular) to infer information about the world (the general).
This is why ideas like Sachs’ Millennium Villages, although easy to criticize, received so much support from the United Nations and other funders. Not because they are rigorous and scientific, but because they are case studies involving people. Until we recognise that human beings have a bias towards seeking particular stories that we can identify with, we will not be able to convince people of where resources should be allocated.
In a famous study, Paul Slovic, Deborah Small, and George Loewenstein asked people to donate to African relief. One of the appeals showed statistical evidence of the extent of the problem, another profiled a 7-year-old girl, and a third combined statistics and the profile. Unsurprisingly, the profile generated more donations than the statistics, but most surprisingly, it even generated more giving than the combination of profile and statistics. It was like the numbers alone had turned people off the idea of giving.
We still need statistical information, rigorous trials, and solid data in development. The more we can show that development is a science, as opposed to guesswork, the better.
But there is something be learnt from all of this. Even though we push for statistical information to demonstrate to the public the net effect of what works, and what doesn’t, or talk about the need in terms of numbers of people, we still need to keep the message centered around human beings. Without a human story, our ability to empathise and understand is severely hampered.
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.