Big Data in government: challenges and opportunities
We often hear that this is the “age of data” or that data is the raw material of a new industrial revolution. There is truth in this, and there is opportunity. Data can be a catalyst for a society, an economy and a country that works for everyone.
The government holds huge amounts of data. This is not new, but the recent advances in digital technology and data analysis now offer significant new opportunities to improve the experience of the citizen, make government more efficient and boost business and the wider economy. Getting data right is the next phase of public service reform and the transformation of government; and data analytics has the potential to drive this transformation at scale and at speed.
After all, understanding what works and what doesn’t means we can make better decisions, better policy and better services built on reliable data. It’s about identifying more efficient ways of working; listening to what our customers say and delivering smarter services. Whether that’s work to tackle child abuse through image-matching technology, the use of Land Registry data to provide a single house-price index or the ground-breaking medical research which is revolutionising the detection of eye conditions.
Government open data can also fuel an open economy based on more easily accessible information to which entrepreneurs, data start-ups and the public can add value and from which innovative products can be created. To date, government data releases have been turned into over 400 different apps, including services for finding the best dentists, general practitioners, schools and universities.
But government data is public data, so we have a duty to use it well and to open it up where possible.
Public trust in the secure, responsible, appropriate and effective use of data is absolutely critical to a data-driven government. The Government Digital Service and the Office for National Statistics have produced guidelines for data science in government, based on shared key principles including data security, openness, and taking user need and public benefit as the starting point.
Meanwhile, the Digital Economy Bill provides a legal framework for sharing data and applying the powerful digital tools at our disposal. It will allow limited data sharing between public authorities where there is a clear public need and benefit.
There is also a link between public trust and government data capability. That’s why government is modernising its data infrastructure to make data easier to work with. We are introducing developer-friendly open registers of trusted core data; and better tools to find and access personal data, where appropriate, through application programming interfaces – also known as APIs – for transformative digital services.
We also need people with the right skills. We are growing the specialist data science community across government in a variety of ways, from direct recruitment to training to defining new career pathways. And we’re developing a data literacy programme for non-data specialists.
Data is now the lifeblood of 21st century government, driving policy and service development and powering decisions on the frontline. It is transforming the face of government and public services – and we intend to make the most of it.