Whitehall's running of public services could be undermined unless more is done to stop datasets from languishing in departmental "silos", a committee of MPs has warned – as the Government Digital Service (GDS) unveiled plans to try and improve the quality of data held by the civil service.
Spurred on by the rise of digital technology, the government has launched a range of open data initiatives in recent years, including the creation of a single online portal – data.gov.uk – to publish regularly-updated information held by public bodies.
Ministers also last year unveiled plans for a new "steering group", led by Open Data Institute founder Sir Nigel Shadbolt, to try and help departments become "intelligent consumers of their own data".
Whitehall urged to improve "ad hoc" data sharing with governments in Holyrood, Stormont and Cardiff
Government must not get complacent on open data progress, says IfG
Sprint 16: Six things we learned about the future of government digital from the GDS conference
The latest report by the Science and Technology Committee welcomes those initiatives and says there are likely to be "big benefits" to public policymaking from the rise of big data.
But the group of MPs says more could still be done to join up that work across government, with departments urged to ensure that they proactively publish data, share it beyond government, and move away from a situation in which they risk duplication by relying mainly on their own datasets.
"There are enormous benefits in prospect for the economy and for people’s lives from making the nation’s core data infrastructure ‘open’," the report says.
"The government’s work in this area has put the UK in a world-leading position. But there is more to do to break down departmental data silos, to bring data together in order to further improve public services, as well as to improve data quality."
The report says the government should work to establish a "right of access" to data currently held by official stats body the Office for National Statistics, and calls for better scrutiny of the quality of the data held by departments.
It adds: "The government should also establish a framework — to be overseen by the Government Digital Service, the Office for National Statistics or another expert body — for auditing the quality of data within government departments amenable for big data applications, and for proactively identifying data sharing opportunities to break departmental data silos."
Improving the quality of the civil service's datasets – as well as the ability to share that information across government – was one of the major themes highlighted at last week's Sprint 16 conference, organised by the GDS.
The central government digital team is currently leading efforts to build more publicly-available, definitive datasets known as "canonical registers", in a bid to replace what are, in many cases, error-prone digital versions of old, paper-based systems.
"They have to be useful, the one source of information, curated by the experts – that’s the definition of a canonical source of data," the Foreign Office’s digital transformation lead Alison Daniels told delegates.
Her department has just launched the first of Whitehall's open registers, replacing the seven separate lists of countries officially recognised by the UK that previously existed in different parts of government.
"Revolution"
Speaking at the same conference, Cabinet Office minister Matt Hancock said he believed the rise of big data presented one of "the biggest revolutions that the world has gone through" since the invention of the printing press.
"Ultimately, the invention of the printing press brought down the cost of the transmission of information from very expensive to relatively cheap," he said.
"And the data transaction technology that’s been invented over the last 30 years has brought down that cost from very cheap to infinitesimally small. The impact of that, I think, we are only in the foothills of understanding.
"It will have a huge impact on society, on the economy, and certainly, more locally, on the way that public services are delivered. And we have to understand both the power of harnessing data for the benefit of citizens, and also the mission critical requirement to hold it securely and to use it responsibly."
The tension between the potential benefits and dangers of governments holding citizens' data also features in the committee's report, with MPs saying that the scale and pace of data gathering and sharing means public scepticism about privacy and security are "often well founded".
The committee therefore calls on the government to set up a new Council of Data Ethics within the Alan Turing Institute – designated as the UK's national institute for the data sciences – to try and address "the growing legal and ethical challenges associated with balancing privacy, anonymisation, security and public benefit".
The report adds: "Ensuring that such a Council is established, with appropriate terms of reference, offers the clarity, stability and direction which has so far been lacking from the European debate on data issues."
Government should also use its upcoming Digital Strategy to set out plans to ensure that data is anonymised where necessary, although it acknowledges: "There are arguments on both sides of this issue: Seeking to balance the potential benefits of processing data (some collected many years before and no longer with a clear consent trail) and people’s justified privacy concerns will not be straightforward.
"It is unsatisfactory, however, fo rthe matter to be left unaddressed by government and without a clear public policy position set out," the committee says.
Launching the report, committee chair Nicola Blackwood said: "The use of ‘big data’ is already bringing big benefits. Exploited further, big data will be transformative, unlocking new life-saving research and creating unimagined opportunities for innovation. The government has a role in this, in sharing and opening up its own data."
The MPs' recommendations come after a report by the Alliance for Useful Evidence and Institute for Government think tanks warned that the sharing of public service performance data between the government in Whitehall and its devolved counterparts in Scotland, Wales and Northern Ireland remained "underdeveloped".