Politicians are fond of bandying around statistics to boost their arguments – but how much of what they say is informed by facts? Suzannah Brecknell meets Full Fact director Will Moy who explains his mission to rid public debate of “nonsense”
In 2011 the coalition government introduced the Health and Social Care Bill, proposing major reforms across the NHS. The bill faced widespread opposition and an enforced pause so that government could “listen, reflect and improve”, as the health secretary put it, before it was passed. The Welfare Reform Bill, introduced in the same year, was equally controversial, enacting sweeping changes to the welfare system, but it had a much smoother ride through parliament.
Why were these bills received so differently? Will Moy, director and co-founder of fact-checking charity Full Fact, suggests that an important factor was the way each department set out the reasons for their bills. While the health bill was something of a bolt from the blue, the work and pensions department, he argues, had done a good job of making the case that the welfare was too expensive and needed reform
"If you look at the most successful government departments in terms of achieving their agenda you see they take seriously the need to engage with the facts from the beginning. They try to marshal an argument that is supported by the facts,” Moy says.
Are the What Works Centres working – or is evidence-based policymaking still a pipe dream?
Open data not enough to ensure evidence-based policy, says new research
Moy's focus on facts stems from his days as a researcher to crossbench peer Lord Low. He describes reading briefings from charities and trade associations which were sometimes "absolute rubbish". While much of the “rubbish” was filtered out before it reached lawmakers, some made it through to Lords debates. “You would see very serious people standing up and making arguments about changing the law, and actually changing the law, based on nonsense. That’s annoying."
It was to better stop the nonsense that Moy co-founded Full Fact. Billing itself as the UK’s independent fact checking organisation, Full Fact scrutinises claims made by politicians and journalists on six topics – the economy, immigration, health, education, crime and Europe. It publishes prompt and simply worded explanations about the veracity and usefulness of individual claims.
It’s an increasingly influential organisation. Lord Justice Leveson described Full Fact's submission to his inquiry on press standards as "comprehensive and extremely helpful". The Department of Health explicitly referred to Full Fact in a letter explaining why it now publishes internal data documents which set out the sources for facts included in ministerial speeches, allowing press officers to respond quickly to queries from journalists. When two fast streamers were seconded to the organisation, they helped to create guidance on how spreadsheets should be published which has now been adopted as official cross-government guidance.
What three things would Moy most like to change about how the civil service and government operate? "First, when you say something in public, back it up in public," he says, explaining that publishing the data behind claims made in official documents or by ministers will allow the public to judge policies more effectively. "Let's enable people to make up their own minds. That's how you build trust in this day and age."
He points to the health, work and pensions and education departments as leading the way in this area – the DWP publishes the data behind claims in ministerial speeches as a matter of course. But other departments are less transparent. The Treasury, he says, refuses to engage both with Full Fact itself (while other departmental press offices answer questions from Full Fact employees, the Treasury argues the charity is not a media outlet and requires it to submit FOI requests for information), and also with the need for transparency over claims made in public.
"Government financial figures are allowed, and sometimes, it seems, encouraged, to be completely opaque" – Full Fact director
"That's very disappointing," says Moy, "because the Treasury is one of the sources of public analysis that can be very influential, and it can be very wrong. More to the point, it often can't be checked.”
Treasury budget documents, for example, don't explain the baselines for spending changes, and don't contain the workings behind the figures they contain. If they were official statistics, Moy says, they would breach the rules which require statistics to have sufficient contextual information to allow external parties to understand them. "Government financial figures are allowed, and sometimes, it seems, encouraged, to be completely opaque."
Despite this, and his daily exposure to false or unsubstantiated claims made by MPs and journalists, Moy is fairly positive about the quality of debate in this country. “We are very lucky to live in a country with quite robust protections, so that our debate is broadly speaking sound and conducted with reference to reality.”
The civil service plays an important role here, both in the way it communicates statistics to ministers and the public, and in upholding the protections around public debate. When a minister makes an incorrect statement to Parliament, for example, the ministerial code requires that he or she makes a prompt correction.
It is for officials to advise on the need for corrections as well as to ensure the corrections are issued, Moy says, and the system often works well. So when David Cameron made an incorrect statement about schools on 2 March this year, he issued a correction on 4 March (in response to a Full Fact prompt, according to Moy).
Moy contrasts this prompt correction with the health department's refusal to issue a correction after health secretary Jeremy Hunt referred to a BMJ article on weekend mortality rates, saying it showed a direct link between a rise in deaths at the weekend and staffing patterns. Full Fact wrote to the department pointing out that the article did not show this link. In response, the department said Hunt's speech had given a "rough approximation of the BMJ article" as well as drawing on other evidence linking the “weekend effect” and service provision.
The UK Statistics Authority – part of the system of protections which Moy praises – asked DH officials to ensure future references to the BMJ article were clear that it did not show a causal link, but said that the decision on whether Hunt should publish a correction "is a matter for the permanent secretary of the department".
"Those decisions that perm secs make about when corrections are needed are part of the informal infrastructure that means we can have a real public debate about public decisions," Moy says.
The second change Moy would like to see is for ministers and civil servants be more rigorous about the facts they accept from others.
"Stop swallowing bullshit from lobbyists," he says. "It is about time government pushed back hard on well meaning and well funded organisations pushing particular agendas. There ought to be a standard people are held to and if you can’t hack it you shouldn’t be listened to on factual questions."
This is fundamental to Full Fact's work, he says. By looking so closely at the claims made by politicians and journalists, he hopes his team will also impact the “largely private, largely unscrutinised and tremendously influential” lobbying community. “Part of what we’re doing when engage with ministers and engage with the media is to try and feed pressure back up the food chain to the ultimate sources of some of these claims, and make it harder for a business, charity or trade association to give information to government and news desks that won’t stand up to scrutiny.”
Underlying all this is the need to make good decisions on the way policies are designed and laws are made, so Moy's final wish is for government to take a fresh look at the information it needs to make public decisions.
"It is about time government pushed back hard on well meaning and well funded organisations pushing particular agendas"
"Let’s think seriously about what information we need to make decisions we’re trying to make to actually run the country," he says, using the example of crime and justice data.
"What information do we need about the police service to make better decisions on policing? Why is that we have better and more detailed data about how many people play golf than how many people are raped?"
Information about golfers is gathered in the Active People Survey, with a sample size of 165,000 and data available down to a local authority level. Official rape statistics, however, are gathered through the Crime Survey of England and Wales (CSEW), with a sample size of 35,000 adults and 3,000 children.
The data is not detailed enough to give reliable information about crime at a local level, and in 2013 MPs on the Public Administration Select Committee heard that it would cost around £13.7m on top of the existing budget of £3.8m to increase the sample size of the CSEW so that it could be used to scrutinise local crime trends. Although MPs at the time described this as a "significant" cost, Moy suggests it would be worthwhile in the context of wider police and justice spending.
"If you think how much we spend on police would it make sense to spend £17m on figuring out whether it works or not? Might that possibly be a really good investment?" he asks.
"It's time to start with a blank sheet of paper on these data questions," he continues. "I’m sure there are people in government who are trying to that, but we must move faster."
Even if we improve the evidence available to policy-makers, will they use it? Moy returns to the point that the most successful policies take facts seriously, referring to a lecture given by Tony Blair at the Institute for Government. "As a prime minister he perhaps won't be thought of by history as the fact checkers favourite," Moy says, "but Blair said he's reached the conclusion that the best long term politics is the best long term policy. If you're going to win a political argument; if you're going to succeed as a politician, then ultimately you need to change what you set out to change, or keep the same what you set out to keep the same. We all hit hard facts eventually."