Whitehall has made big strides in opening up data to public scrutiny since 2010 – but too much remains of "questionable consistency", according to a new report from the Institute for Government (IfG).
The think tank's latest annual "Whitehall Monitor" report offers an in-depth overview of the size and cost of government during the lifetime of the coalition, with the IfG's study based largely on open data sets made available by public bodies.
Among its findings, the think tank points out that the civil service became "both older and more concentrated in senior grades" between 2010 and 2015, as well as becoming more diverse, with 39% of senior civil service roles now occupied by women, compared with 34% in 2010.
Information watchdog pushes back against FoI charges call
Meg Hillier interview: the new public accounts committee chair on HMRC staffing, public service silos – and Jeremy Corbyn
Cyber apprentices and the escalating data arms race
Overall staff numbers are down sharply on 2010, it notes, with nearly 480,000 full-time civil servants at the 2010 Spending Review compared with just over 406,000 at the latest count in March of this year. However, the IfG highlights that this is still some way off the government's own stated projection of having 380,000 staff by this year, giving a "stark illustration of how difficult any further reductions will be".
The number of non-departmental public bodies has also plummeted, from over 680 in March 2010 to 401 in March 2015, while the number of non-ministerial departments and executive agencies dropped from 83 to 62 over the course of the parliament.
But the IfG says that while the UK "is seen as a world leader in open data and transparency" – making a wide range of data available through the data.gov.uk platform – government must not "bask in the glow of such plaudits" and lose sight of the need to ensure data is published in a usable way.
"Too much government data is still published in PDF format, making it harder to reuse," the report's authors say. "There are problems with consistency between different releases of the same data, in terms of when updates are published and in what is included and excluded."
They add: "There are also consistency problems in and across financial datasets. There have been improvements over the past five years – through the consolidation of the Whole of Government Accounts (WGA) and the Clear Line of Sight project – but problems remain. The various sources of government financial data – the WGA, Public Expenditure Statistical Analyses (PESA), budgets and spending reviews, departmental annual reports and accounts – do not always match."
That lack of consistency makes side-by-side comparisons of different areas of government more difficult, the IfG says.
The think tank also warns about the poor quality of some existing data sets, pointing out that several government departments were unable to provide data showing which civil service professions – such as operational delivery; communications; and policy – their officials belonged to.
The report says: "For five departments, the professions for more than 15% of their workforce are given as 'unknown' or something 'other' than the 25 civil service professions, one in eight civil servants overall. Often these numbers do not match those published in departments' own organograms. This should be fairly basic management information."
The report's lead author Gavin Freeguard said: "Looking back also allows us to look forward. The comprehensive picture of government over the last five years built up in Whitehall Monitor allows us – and others – to anticipate and prepare for the challenges over the next five years.
“Wanting to do ‘more for less’ makes it even more important that data is published and used. Departments and parliament must use it to do their job, and the public must be able to access it to hold the government to account.”
Chief data officer
Cabinet Office minister Matt Hancock announced earlier this month that he had drafted in the Open Data Institute's co-founder Sir Nigel Shadbolt to lead a new group ensuring that government is “pushing the boundaries” of open data.
The steering group will advise on data policy and will include Shadbolt as well as and health information expert Dame Fiona Caldicott and members from private sector and academia.
But the IfG points out that the departure of Government Digital Service chief Mike Bracken over the summer left Whitehall without a chief data officer, and says government has gone "from having four advisory bodies involving users of open data to none".
"Government should be clear about the remit of the new chief data officer and ensure they have the political and institutional support to do their job," the think tank recommends.
"They will need sufficient political and institutional clout and will have to work closely with departments, the Government Digital Service, the ONS, and the statistical professions in the civil service."
The report says there is no agreed definition of what constitutes a government department across datasets, so while some will "refer only to the departments directly line-managed from the permanent secretary", others will "swap arm's-length bodies in and out at will".
Public accounts committee chair Meg Hillier welcomed the IfG's report, but called on departments to "up their game and release data in a more manageable form" to aid scrutiny of government spending.
Jeni Tennison, director of ODI, meanwhile said the IfG's report showed how open data could be used to "ultimately to make government more effective".
She added: "The Institute for Government has done a great job at making sense of the data that is available, but it’s clear that higher quality, more timely, more consistent and better documented data could provide even greater insights about how government is working."