What’s ‘human in the loop’ and why does it matter for government AI?

Louis Pattison

People often mention the importance of keeping a “human in the loop” when discussing government AI use. But what does that actually mean? Here’s how following user-centred design principles keeps that human oversight and makes AI safer.

There is immense pressure on public sector leaders to embrace artificial intelligence. Generative AI, machine learning, and automation are frequently promised as magic bullets to drive efficiency, clear backlogs, and streamline processes.

But as anyone who has experimented with large language models knows, AI is inherently unpredictable. It hallucinates, can perpetuate bias, and rarely shows its workings. In a commercial setting, a dud AI recommendation might mean a missed product recommendation.

In the public sector, it can mean a wrongly denied benefit, an unfair immigration decision, or a breach of data privacy. If the why behind an important decision can’t be explained later on, the damage to public trust could be severe.

All this makes for a big headache for digital leaders. Yes, you can procure an AI tool from a third-party vendor – but you can’t outsource accountability. 

The legal shift

Recently, the legal landscape shifted fundamentally. The old UK GDPR rule was a strict prohibition – solely automated decision-making on critical issues was banned by default. Now, under the updated legislation, the framework has flipped to permission, provided that suitable measures are taken to safeguard the data subject’s rights, freedoms, and legitimate interests.  

This means the law no longer acts as an automated backstop. Instead, the legal and ethical burden is entirely on how you design the system. If you automate a decision, you are legally required to build robust mechanisms for citizens to understand the decision, challenge it, and easily trigger human intervention. If you’re a digital leader, the way your systems are designed is your business. In a very real sense, the buck stops with you. 

So, how do you benefit from the operational value of AI while minimising your department’s exposure to risk? Choosing the right model is part of the equation. But to really address risk, you need to consider the user. 

At Zaizi, we believe user-centred design is a key way to de-risk important services. That’s why when we build AI into government services, we consider an important design concept: a human in the loop. 

READ: Is ‘vibe design’ ready for public sector projects?

Defining intervention: Human in, on, or out?

The AI Playbook for the UK Government explicitly states departments must understand the level of human oversight required for any AI use case. This isn’t optional: it’s an important governance standard. 

Broadly, there are three levels of human intervention: 

The level of intervention you choose should be defined by what the AI is doing – as well as the consequences of failure. In areas of high risk or serious consequences – for example, decisions affecting rights, benefits or legal status – it is important for a human to verify and sign off a final decision. 

However, a full human-in-the-loop approach removes some of the benefits of automation by adding friction into the process. In areas of high throughput, we might consider if a human-on-the-loop approach is more appropriate – for example a dashboard that alerts a responsible human when a confidence boundary is breached. Think of this approach as a “cognitive speed bump” – a timely intervention designed to make a human pause, think critically, and actively confirm a choice. 

While the updated UK GDPR makes full automation – human out of the loop – a tempting path to clear backlogs, it brings with it risk. If AI operates completely out of the loop on a critical service, you must intentionally design those legally mandated “escape hatches.” How does a citizen dispute a bad AI calculation? How quickly can a civil servant step in to review it? Under the new legal framework, these escape hatches aren’t just optional.

Human in the loop: designing for the user

We’re talking a lot about humans – let’s reframe that as users. 

To meet the Service Standard, we need to ensure that a service meets the needs of the people using it. At Zaizi, we’ve built flexible, reusable service patterns around key user interactions to ensure that when we’re integrating AI into a service, it works well for the humans providing that crucial decision step. 

But what do we need to tell the human in the loop?

How do we check if our approach works? We test with users, looking out for things like automation bias (users blindly trusting the system), misplaced trust (waving through AI suggestions without scrutiny) or alert fatigue (ignoring critical warnings because the system is too noisy) and iterating where necessary. Only by testing with the people who use the system can we make sure that these safety guardrails survive contact with the real world.

The safeguard is the strategy

With the safety nets of the old GDPR gone, the burden of protecting citizens now falls entirely on how we design our systems. Integrating AI into public services is no longer a binary choice between lightning-fast automation and absolute safety. Achieving both means designing with intention.

When dealing with consequential decisions that impact people’s rights, livelihood, or legal status, human intervention can’t be treated as a bureaucratic afterthought. It is your primary risk-management strategy.

By designing “human-in-the-loop” and “human-on-the-loop” interfaces as deliberate, user-tested patterns, you do more than just meet the legal threshold for safeguards. You actively protect your users, insulate your department from risk, and build public services that are both highly efficient and trustworthy.

In the age of automated public services, system design is your accountability.

What next?

If you would like to speak to the author about the topic, feel free get in touch

Download our whitepaper “AI readiness roadmap for government decision makers” to get practical insights on to start and scale your AI operations. 

How prepared is your organisation for AI? Take our interactive assessment and find out your AI maturity score.

Related content

Thanks for joining us! We’ll keep you informed with regular updates.

[gravityform id=”16″ title=”true” ajax=”true”]