There’s a comforting myth in British politics: that with the right plan, under the right leadership, the Whitehall machine will whirr into action and change will follow. The Starmer government seemed to believe that by simply resetting relations with the civil service – dialling down the chaos and acrimony of the Johnson and Truss years – the machine itself would start to work differently.
But at some stage – and Labour now seems to be in the midst of this realisation – ugly reality strikes. Things don’t happen as intended. Promising reforms make contact with the government machine and emerge in familiar ways: delayed, diluted, de-scaled or distorted beyond recognition.
Nowhere is this truer than in the vexed but vital territory of public service reform. The government signalled its ambitions in the most recent Spending Review: services that are preventative, joined-up, innovative and centred around people. It’s undeniably the right vision, but also one likely to grind against Whitehall’s most ingrained habits.
As a former Whitehall civil servant – including a stint in its bureaucratic epicentre, the Cabinet Office – I know this all too well. I worked on reforms from prisoner rehabilitation to early years services, and even went through what increasingly feels like a civil servant’s rite of passage: working on a failed attempt to reform social care. I left government after just five years, frustrated less by the absence of ambition than by the system’s inability to turn it into lasting change.
Over the past two years, I’ve interviewed over a hundred civil servants, alongside dozens of reformers familiar with the Whitehall system from beyond its walls, in local government, academia and the voluntary sector. Their stories all bear a striking resemblance to one another. When reforms succeed, it is usually thanks to unicorn conditions: a remarkable minister, a fearless official, a chance pot of funding, or a crisis – like the recent pandemic – that loosens the rules.
When the stars don’t align – which is most of the time – reform doesn’t exactly crash, it stalls: idling in neutral as Whitehall’s tendencies towards caution and centralised control apply the handbrake.
Why? Not – as many of Whitehall’s detractors would have us believe – because civil servants are feckless or politicians uniquely fickle. Instead, most are responding in entirely rational ways to a uniquely unforgiving environment for making change. Five characteristics stand out:
High-stakes politics puts people in self-preservation mode.
Politics is a realm where mistakes can mean scandal – and in some domains, life and death. Add a media culture that punishes failure faster than it recognises improvement, and you get a workforce primed for self-protection. In the civil service, that often looks like hierarchy (six pairs of hands on a piece of policy advice before a minister even sets eyes on it), moderation (grand ambitions sanded down to small-scale pilots) and control (promises of devolution, while quietly burying local leaders under spreadsheets and reporting requirements).
Whitehall rewards “policy heroes” over those who can deliver.
Look at who rises to the top in Whitehall and a pattern emerges. Those who rise fastest are often what I’d call ‘policy heroes’: polished performers who can command a room of politicians, quickly knock up elegant prose and smooth over – if not entirely avert – political crises.
The trouble is that the qualities this system elevates – confidence, polish, speed – often trade off against the ones reform really needs: humility, comfort with uncertainty, and patience. They also sit more naturally with a command-and-control style of governing, where problems are solved by issuing orders from the centre, than with the slower, looser stewardship our public services demand – nurturing capacity over time, collaborating across silos, and staying with problems long after the headlines have moved on.
Scale and separation drive public sector workers into tribes.
Government is a vast nexus of people and activity, and it plays straight into the basic human instinct: to make sense of the world as made up of “us” and “them.” In Whitehall, your ‘ingroup’ usually means the people in your daily working life – your policy team, your profession, your department. Everyone else – operational colleagues, local authorities and frontline practitioners – can all too readily blur into an ‘outgroup’ to be ‘managed’.
Trust is hard to build with people you rarely meet and barely understand. And while these instincts might be understandable, it’s corrosive for reform. The kind of services we need can only be built in partnership with the people delivering them. Distance makes suspicion cheap and collaboration costly.
Whitehall loves tidy answers for messy problems.
Whitehall has a deep bias for standardisation. Faced with real-world complexity, it reaches for the comfort of national programmes with neat rules and metrics. But the most effective reforms are often uneven and adaptive, growing from local relationships, leadership and energy.
Take the Troubled Families programme: its roots lay in small local experiments where close relationships between practitioners and families were central. When it was scaled, much of what made it work locally was stretched thin – quantitative targets, payment-by-results and central monitoring replaced much of the relational work thought to be the ‘gold dust’ of its antecedents. This is a pattern we’ve seen too often: in the rush to scale, Whitehall often smooths out the nuance that makes interventions work in the first place.
Politics craves quick, legible wins.
In today’s political culture, immediacy is everything. Ministers are drawn to outcomes they can count and badge – hospitals built, nurses hired, police recruited.
But the reforms the nation most sorely needs may not show up so cleanly. Prevention is about crises that never happen. Integration spreads credit too thin to badge. People-centred services hinge on relationships that are not reducible to a tick-box. Progress, in other words, is slower, subtler and harder to measure. This inevitably makes it vulnerable to being dismissed as “nothing to show” or dismantled by the next government keen to start afresh.
If all this feels dispiriting, it shouldn’t. These traits aren’t immutable; they’re learned behaviours that make sense in context. And because they’re learned, they can change. Other high-stakes fields have faced the same challenges and adapted. Aviation shifted after a string of fatal accidents in the 1970s, building a “just culture” where near-misses are treated as data to be shared, not careers to be ended. Healthcare has spent decades developing patient-safety systems that separate the domains where variation is dangerous – like surgery protocols – from those where innovation is essential – like research and care pathways. And governments are increasingly embracing some of the agile methods learnt from the tech sector: trialling prototypes with users, iterating in alpha and beta, and improving before scaling.
The problem is not that there are no models for change, but that Whitehall often convinces itself it is uniquely resistant to them. In the long history of pathologising the problems of the British state, “culture” is invoked as a diagnosis only to shrug and move on - a polite way of conceding defeat. Change in Whitehall is possible – and change it must. If Whitehall can’t prove it can deliver change, the public will stop believing politics can deliver anything at all. The fragile legitimacy of our political system hangs in the balance.
Amy Gandon is a fellow at the cross-party think tank Demos. Demos will be exploring how to overcome the above-mentioned barriers as part of its Powering Public Service Reform programme. Find out how to get involved in shaping these conversations by following Demos on LinkedIn and Bluesky