You are here

Big Data for the Rest of Us

May 16, 2013
Photo Credit: 
UN Global Pulse

Recently the news has been full of stories about the potential for `big data’ about customer behavior to revolutionize business, and personal data has been called the `new oil of the internet’.   But what about big data for the average person?  For the poor?

Everywhere in the world and in every business sector we all contribute to create freely available common goods…public roads and schools, water and sewage systems, police and judges…that allow children to grow, business to thrive, and the quality of life to improve.  Now that personal data is becoming so valuable, it makes sense to ask if we can make a data commons that helps individuals, businesses, and government.

On May 1, 2013 we saw the public unveiling of what is perhaps the world’s first true `big data’ commons, with more than 150 research organizations from around the world reporting results from analysis of data describing the mobility and call patterns of the citizens of the African country of Ivory Coast.  This aggregated anonymous data was donated by the mobile carrier Orange, with help from the University of Louvain (Belgium) and MIT (US), and in collaboration with Bouake University (Ivory Coast), the United Nation’s Global Pulse, the World Economic Forum, and the GSMA.     

Highlights of this unveiling include an analysis by IBM’s Dublin laboratory of the public transportation system.  They showed that for very little cost the average commute time in Abidjan, the biggest city, could be cut by 10%.  Other highlights included analysis of disease spreading by groups from Novi Sad University (Serbia), EPFL (Switzerland) and Birmingham (UK) that showed that small changes in the public health system could potentially cut the spread of flu by 20% as well as significantly reducing the spread of HIV and malaria.      Other research groups demonstrated the potential for improvements in government, commerce, agriculture, and finance.     

These research results have demonstrated the great potential for such a data commons for improving society.  From the point of view of Orange, it also demonstrates the potential for new lines of business that combine this data commons with your personal data: imagine a phone app that advises you about which bus will get you to work quickest, or how to reduce your risk of catching the flu.

The work of these 90 research groups also suggest that many of the privacy fears associated with the release of data about human behavior may be generally misunderstood.    In this data commons the data was processed by advanced computer algorithms (e.g., sophisticated sampling and use of aggregated indicators) so that it was unlikely that any individual could be re-identified.   In addition, while the data was freely available for any legitimate research that a group was interested in, it was distributed under a legal contract that specified that it could only be used for the purpose proposed and only by the specific people making the proposal.  No path to re-identification was discovered even though several of the research groups studied this specific question.

The use of both advanced computer algorithms and contract law to specify and audit how personal data may be used and shared is the goal of new privacy regulations in the EU, US, and elsewhere.   Data about human behavior, such as census data, has always been essential for both government and industry.  In this new era of big data, we must make sure that a digital data commons is freely available and yet we must protect the privacy and safety of the individuals whose lives are reflected in that data common.   Indeed we need a `new deal on data’ under which individuals can understand what information about them is used for and the benefits and risks of that that use so that they can choose how data will be shared at both individually and collectively through government.

The D4D initiative was led by Nicolas De Cordes for Orange,  Vincent Blondel for Louvain,Alex Pentland for MIT, Robert Kirkpatrick for the UN Global Pulse, and Bill Hoffman for the World Economic Forum.

Read more....