Big Data: Development tool or Big Brother?

The UN Statistical Commission in New York meets at a moment when the global statistical community is at a “crossroads”.  As we shift towards the post-2015 development agenda there is a need for measurement, monitoring and speed. There appears to be pressure to adapt by moving away from traditional data collecting to using “Big Data”.

Big Data refers to “high volume, velocity and variety of data, which require new tools and methods to capture, curate, manage, and process them in an efficient way”. Currently, Big Data tends to be owned primarily by the private sector (e.g. banks, online retailers, search engines, cellphone providers etc). This information ranges from your financial details to who you call, what you search for online or what you click on. Big Data has a lot of potential to discover subtle patterns.  The private sector mine this data to make the customer’s experience better or tailor marketing to improve their business.

There is no denying that Big Data is part of the future. But it needs to be legislated with transparent governance. There needs to be strict regulations over who gets to interprete it and what training they need.

The UN Statistical Commission recently undertook a survey to analyse the potential role of Big Data in the Post-2015 development agenda context. This attempted to tackle the main concerns such as governance, quality, privacy and skills. It is very interesting because it provides a snapshot of what a number of countries are already doing or considering.

The suggestion that Big Data should then be used for official purposes raises considerations such as what are the boundaries? Should governments have access to that much information about your behaviour patterns without your knowledge or consent? Without adequate training and infrastructure what is to prevent mistakes from happening or inaccurate things from being inferred about you? What prevents this from being abused?

Apart from the very valid concerns of citizens against the state (or big corporations). What about state sovereignty? Can developing governments control data collection about their territory and economy by big corporations or very powerful governments? (See: Big Data: Threat or Revolution?).

It has been suggested that issues such as “quality, confidentiality and partnerships with the private sector are most efficiently handled on the global level”. But shouldn’t civil society at a national level have some control? It is your information afterall….

Read more:

Results of the UNSD/UNECE Survey on organizational context and individual projects of Big Data

Data Revolution

By Sabá Loftus