Autumn Statement and Budget announcements suffer from “evidence gap”, says charity

Researchers warn about lack of transparency over evidence behind policies announced in big fiscal events


By Suzannah Brecknell

23 Nov 2016

Policy proposals announced in Budgets and Autumn Statements are “particularly non-transparent" about the evidence behind them, according to a new report from the charity Sense about Science.

The study looks at how well government is sharing the evidence which informs policy decisions, and rates a selection of initiatives launched by the Cameron administration using a new tool developed by the charity, the Institute for Government, and the Alliance for Useful Evidence.

The tool looks at how evidence is used in four key areas of the policymaking process: diagnosing a policy problem; proposing and assessing options to solve it; implementing policy and planning for testing and evaluation.


Government “must up its game” on evidence-based policymaking
Open data not enough to ensure evidence-based policy, says new research
Are the What Works Centres working – or is evidence-based policymaking still a pipe dream?


Report author Tracey Brown told CSW that her organisation's review was not looking for departments to set out lengthy documents justifying their policies, but for clarity about the reasoning behind decisions.

“Some of the best, most transparent stuff we saw was not the longest, but it was that which clearly drew a line of reasoning from a problem that government has assessed, through the different options for addressing it, to how this will be implemented, and the likely impact," she said.

But Brown said this line of reasoning was not always clear in policies set out by the Treasury for big, set-piece fiscal events like Budgets and Autumn Statements. 

“There seems to be something of an evidence gap, particularly in policies which are announced by the Treasury and then handed to another department to deliver, " she said.

The report suggests that there could be a systemic problem with such policies which needs be addressed, saying the fact many Budget measures are "developed in secret", with the Treasury not playing "its normal role as policy challenger", may explain "why there are so many transparency issues around Budget announcements".

To address the problem, the report says all Budget announcements should be followed up by a pre-implementation policy proposal. The group found that where public consultation followed Budget announcements, transparency about evidence tended to be better.

"Wrapped up and decided"

Brown told CSW that it was important for departments to publish information about the evidence behind policies at “the right moment” – that is, as soon a policy is publicly announced.

But she said many departments working with Sense about Science on the report had been frustrated that the charity chose to evaluate evidence published at the review stage, by which time "everything’s wrapped up and decided"

"It’s important that the evidence comes at a point where parliament can do its job of scrutinising policy, and the public can look at, understand and engage with policy discussions," she said.

The charity did flag a number of common concerns with the policies it assessed, including a failure to clearly share or reference the research and reviews conducted by officials before making decisions.

Browne said this was “profoundly disappointing”, arguing it undermines proper public discussion, and reduces a department’s institutional memory because officials working on policy in the future will not be able to see clearly the reasoning behind previous decisions.

The report cites examples of where public speculation about government motives has arisen because reasoning is not clearly set out, including when the Cabinet Office announced plans to clamp down on public grants to charities lobbying government with reference to just one think-tank study.

“It’s not as though you have an easy ride by not explaining reasoning,” Brown said. “When people are suspicious about government reasoning, it only goes badly.”

Sense about Science has promised further work on departmental policymaking transparency in July next year, covering all the proposals from the formation of Theresa May's government.That will include a table rating departments according to their results.

Brown said a key area for improvement would be around the plans that departments put in place to test and evaluate polices, saying it would be a “shame” if departments doing well in most areas were dragged down because they did not reveal how they test the effectiveness of proposals.

Share this page