Big data and humanitarianism: 5 things you need to know

27 June, 2013
Aid agencies are exploiting new technologies to gather near real-time information to prepare for, respond to, and recover from disasters. Credit: QCRI/UNDP
Aid agencies are exploiting new technologies to gather near real-time information to prepare for, respond to, and recover from disasters. Credit: QCRI/UNDP

Consider this: There was more data produced in 2011 than in all the rest of human history combined. Every time we make a phone call, buy something or use social media, we are creating new data. This huge amount of information can, if analysed correctly, be used to answer any number of questions. This massive volume of data created and stored by governments, the private sector (for example telecoms or internet providers) and individuals is known as Big Data.

Humanitarian organizations are trying to come to terms with how this ocean of information can help them deliver better services to vulnerable communities. Here are five things you need to know about big data and humanitarianism.

1. “Finding ways to make big data useful to humanitarian decision makers is one of the great challenges, and opportunities, of the network age,” says OCHA’s Humanitarianism in the Network Age Report. Access to near real-time information can help humanitarian organizations provide more targeted assistance and become more responsive to needs as they evolve.

It could even help the humanitarian community pre-empt crises, or respond to them more quickly. For example, a 2012 study demonstrated that real-time monitoring of Twitter messages in Haiti could have predicted the 2010 cholera outbreak two weeks earlier than it was eventually detected. Cholera deaths, as the HINA Report points out, “are preventable and outbreaks are more easily dealt with in their early stages.” This information in the right hands could have saved lives.

2. Humanitarians can draw inspiration from their development partners. There is already a lot of work being done that humanitarians could easily capitalize on. Robert Kikpatrick is the Director of UN Global Pulse.

“Global Pulse is an initiative that came out of the global financial crisis,” he explains. “There was a recognition that we now live in this hyper-connected world where information moves at the speed of light and a crisis can be all around the world very, very quickly,” he said. “But we’re still using two- to three-year-old statistics to make most policy decisions.”

Earlier this year Global Pulse ran a competition in which they made data from a mobile phone provider in Côte d’Ivoire freely available, and challenged researches to find innovative uses for it.  The winning entries included using the data to map divisions between ethnic groups, and developing a model of how diseases spread.

3. Getting access to data is not necessarily straightforward. In the case of both Haiti and Côte d’Ivoire, organizations needed to negotiate with private telecommunication providers to access their data. Proprietary and privacy concerns mean that many corporations are reluctant to share their massive data reserves. Similarly, many governments are unwilling to make their data accessible to anyone (although in 2011, the Kenyan government started making all of its national data available online).

Social media is one source of big data in which access is easier because much of the information is already public. As a result, it has driven much of the early humanitarian-related big data research and innovation. For example, within 24 hours of Typhoon Bopha hitting the Philippines at the end of 2012, the Digital Humanitarian Network was able to categorize 20,000 social media messages to create a map of the storm’s impact.

4. Big data should complement existing sources of information, not replace them. Patrick Meier of the Qatar Computing Research Institute warns that we should not see big data as a cure for all of our information ills. “The situation is not either/or, but rather a both/and,” he writes. “Big (Crisis) Data from social media can complement rather than replace traditional information sources and methods.”

Big data, especially when generated by social media, has its limitations. There are obvious concerns about bias; in developing contexts where internet access is limited, data drawn from Twitter probably over-represents urban elites. But bias in data is not new, and there are methods of balancing or correcting it.

Concerns about the accuracy of information drawn from social media may also be overstated. For example, the UK’s Guardian newspaper produced a compelling account of how Twitter corrected misinformation during the 2011 London Riots.

5. We cannot assume that better data will necessarily lead to better decision-making. Big Data, the argument goes, should lead to better and more informed decision-making. But unfortunately, decisions are not always driven by evidence. Take the 2011 famine in the Horn of Africa. As early as 2010, many UN and humanitarian organizations were warning about the looming food crisis.

In the year before the famine took its grip on Somalia, the monitoring agency FEWSNET issued more than 70 early warning bulletins and delivered dozens of briefings to donor governments. But these early warnings were not acted on, and the huge amount of support that was needed was only mobilized when the crisis had taken grip. Tens of thousands of people died.