Data insight is the holy grail of better government decision-making

Government is the mother of all data repositories, but Rohan Malik, Managing Partner for Government and Infrastructure at EY, argues that it needs to use this data well to realise its full potential


29 Mar 2021

In a recent memo, cabinet secretary Simon Case and civil service chief operating officer Alex Chisholm stressed the importance of not returning back to normal post-pandemic. The statement urges the civil service to, be prepared to take risks; seek out additional skills, adopt new methods and, to use data smarter. And if there’s anything that organisations have learned from the Information Age, it’s that data isn’t good enough.

It’s estimated that 85% of big data programmes fail to deliver real business value. Even firms that claim to be ‘data-driven’ and invest huge sums in maintaining that label admit to difficulties connecting analytics to action. They’ve found out the hard way that data without insight isn’t actionable – that unless data can generate realisable benefits, it’s useless.

The heart of the problem is that there’s no inherent value to data. An organisation shouldn’t simply strive to accumulate more and more data, but rather, as former CEO Carly Fiorina put it, to convert data into information and information into insight. For government, this process begins at the end: by identifying the policy outcomes that will then drive insights. Without an outcome in mind, departments are liable to have a wealth of data that isn't being put to good use.

Untapped potential

The pandemic serves as a sobering example. As Reform’s Eleonora Harwich wrote in a recent article for Civil Service World, “the NHS is sitting on a treasure trove of data, but it often struggles to unleash its value.” She writes that, going into the pandemic, information about essential resources such as available ventilators and beds was unknown, and that after almost a full two weeks after the start of the first lockdown there still wasn’t a single data source from which to make key operational decisions. In other words, the UK’s health system was rich with data but short on insight at a time when the latter could have been the differentiator between life and death.

The NHS was forthright about its missteps. In the early days of the pandemic, it admitted it was “unable to move as quickly as the response demands,” and that “information in spreadsheets [was] held by disparate organisations … leading to inaccurate or incomplete understanding of the situation.” While COVID-19 brought the issue to the fore, it was already a longstanding concern of those in the health sector that the NHS’s data wasn’t properly structured. For one thing it’s too siloed: each hospital and even each department within each hospital uses different datasets of varying quality, complexity, and standards for using them. Patient data can get ‘trapped’ in these silos, which impacts health providers’ ability to provide care across organisations. These isolated datasets are also often organised at the patient, population and organisation levels rather than solely at the patient level, which makes it difficult to paint a complete picture of a patient’s health. The NHS is awash with data, it just needs to learn how to use it. 

Case study: poor use of data in safeguarding care

The pressures on safeguarding and care services worldwide are in part due to the sheer number of households and children flowing through the system, and in part because of the complexity of their needs. In our view, one of the main challenges weakening the effectiveness of housing and children’s services is poor use of data.

Many housing services and child safeguarding systems struggle to capture and share information. Poorly linked databases, combined with weak information quality, prevent caseworkers from having the full and accurate picture they need to make the right decisions, with a single view of a child, family or household. The problem worsens when multiple agencies are involved.

Privacy concerns are a barrier to sharing information. Despite guidelines from the UK Government, some partners such as family doctors will not provide vital information because of data protection concerns. Governments need to be clear about the rights and obligations of agencies to share and receive data.

Some services have access to management information but do not use it to its best ability: for prioritising, making decisions, allocating and benchmarking resources or assessing value for money.

Street smart

While the NHS finds its way, other sectors provide inspiration. In urban development, the concept of ‘smart cities’ is demonstrating how more efficient uses of data can reduce costs, carbon, and consumption in populated urban centres.

Manchester, and particularly its Oxford Road Corridor, has been a sort of testing ground for the concept. In 2015 its CityVerve project won a £10m first-place prize in a government-led technology competition that involved 33 other cities. The project used internet of things (IoT) technology,  adding sensors and data analysis to vehicles and home heating equipment, to deliver improvements in city services. For example, air-quality monitoring technology was affixed to lamp-posts to deliver information to residents with health conditions so they can choose the best walking routes.

Just last year the city also reached the completion of the £26 million EU-funded smart cities project “Triangulum” (named after the constellation), with participating cities Eindhoven (NL) and Stavanger (NO). The five-year project used open data platform Manchester-i, which is still in use, to collect large quantities of urban data (e.g., on traffic and hydrology) and provide them to residents in an accessible and usable way.

It is estimated that if IoT pilots such as the ones in Manchester continue to show success, then £8 trillion a year in economic value could be generated by 2025. The IT element of these projects is essential and serves as a model for how data insight can be leveraged. Its recursive nature, whereby insights are used to improve services that in turn generate more insights and so on in a virtuous loop, is itself inspiring. A more practical takeaway is the fact that the technology that harvests data is tacked on to existing physical infrastructure, meaning it makes good use of what’s already there. Departments struggling with legacy IT could probably extract lessons from this, for example in the way they encapsulate or rearchitect existing applications. 

Digital deputies

The brave men and women who police our cities are also showing how it’s done. We tend to think of the police as being a boots-on-the-ground profession, far removed from data. But police today are just as much data analysts as they are street patrols. Data is essential to modern policing – the ability to harvest, integrate, and analyse data in an accurate and efficient way makes all the difference in the war against increasingly sophisticated criminal activity.

In an effort to stay one step ahead, police are now using geocoding, a textbook example of turning data into insight. Geocoding is essentially the process by which text-based descriptions of a location are converted to geographic coordinates on the earth’s surface (e.g., online retailers mapping customer data to develop better marketing strategies). One example is the Home Office Information Sharing for Tackling Violence project, a data-sharing programme between hospitals and other public safety bodies that encourages emergency departments to record data pertaining to injuries suffered by victims of violence. This data is then mapped and shared with public safety bodies to identify “hotspots” that weren’t already known by the police, with the ultimate goal of preventing similar crimes from occurring.

Another example is London Landscape, a project launched by the Mayor's Office for Policing and Crime to help police better understand their communities. London Landscape uses data collated by the open source software R to increase transparency and data availability from 150 crime, demographic and socio-economic datasets. The data, some of which goes back 10 years and some which projects as far as 25 years into the future, is then mapped in an interactive and user-friendly format, providing police with a richer knowledge base from which to plan and detect trends.

Value judgements

Examples such as these illustrate how data can be harnessed and put to good use. Unless actionable insights can be identified to unlock wider socio-economic benefits, then the data itself is worthless – it has no value. In fact, it could have negative value since the costs of collection, storage, and maintenance of worthless data is ultimately a liability.

The volume of data will only continue to grow exponentially, but its utility will remain in line with organisations’ ability to capitalise on it. Until government learns how to unlock the value of the data it owns, it is yet to realise its full socio-economic potential.

Rohan Malik is EY's UK&I Government and Infrastructure Managing Partner. He can be contacted here

About EY

EY is a global leader in assurance, tax, strategy and transactions and consulting services. The insights and quality services we deliver help build trust and confidence in the capital markets and in economies the world over. We develop outstanding leaders who team to deliver on our promises to all of our stakeholders. In so doing, we play a critical role in building a better working world for our people, for our clients and for our communities.

EY refers to the global organization, and may refer to one or more, of the member firms of Ernst & Young Global Limited, each of which is a separate legal entity. Ernst & Young Global Limited, a UK company limited by guarantee, does not provide services to clients. Information about how EY collects and uses personal data and a description of the rights individuals have under data protection legislation are available via For more information about our organization, please visit

Share this page