An under-development artificial intelligence tool designed to analyse consultation responses on behalf of civil servants has passed its first real-world test, the Department for Science, Innovation and Technology has announced.
The Consult tool forms part of the “Humphrey” suite of civil service AI applications for officials announced by DSIT earlier this year. Some elements are already in use.
Today DSIT said Consult had been used to analyse responses to a recent Scottish Government consultation on the regulation of non-surgical cosmetic procedures, such as lip fillers and laser hair removal.
It was tasked with identifying key themes in feedback across six qualitative questions. The tool then sorted individual responses into themes, giving Scottish Government officials more time to delve into the detail of the feedback received and evaluate policy implications.
As it was the first time Consult had been used on a live consultation, Scottish Government officials also manually reviewed each of the exercise’s 2,000 responses.
DSIT said the results showed that Consult and the human reviewers agreed “the majority of the time” and that differences in view had a “negligible” impact on how themes were ranked overall.
The department said the approximately 500 consultations that the UK government runs every year take up around 75,000 days of human analysis – and that using Consult across all of them could save an annual £20m in staffing costs.
Technology secretary Peter Kyle said using Consult rather than officials to analyse consultation responses was a no-brainer.
“No one should be wasting time on something AI can do quicker and better, let alone wasting millions of taxpayer pounds on outsourcing such work to contractors,” he said.
“After demonstrating such promising results, Humphrey will help us cut the costs of governing and make it easier to collect and comprehensively review what experts and the public are telling us on a range of crucial issues.
“The Scottish Government has taken a bold first step. Very soon, I’ll be using Consult, within Humphrey, in my own department and others in Whitehall will be using it too – speeding up our work to deliver the Plan for Change.”
DSIT said Scottish Government officials had been “pleasantly surprised” by Consult, however it added that more evaluation on the tool’s “accuracy and efficiency” would be required before it is rolled out across departments.
The department said Consult had secured an “F1 score” of 0.76, which it said was “widely considered ‘good’” in evaluating the performance of AI tools.
DSIT first announced the creation of the Humphrey suite of tools in January. It is named after senior civil servant Sir Humphrey Appleby, star of classic BBC situation comedy Yes, Minister.
At the time, DSIT said better digital services had the potential to provide ministers with £45bn a year in annual productivity gains. The department subsequently confirmed that automating routine tasks conducted by civil servants accounted for £36bn of that figure in ministers’ calculations.
In addition to Consult, the Humphrey package tools include “Parlex”, which will help policymakers search through and analyse parliamentary debate to shape their thinking and better manage bills; “Minute”, a secure AI transcription service for meetings; “Redbox”, a generative tool to help civil servants with summarising policy and preparing briefings; and legal-analysis tool “Lex”.