Beyond the cloud: What edge AI and SLMs could mean for government services

Seyed Razavi-Nematollahi

Given that many government organisations grapple with connectivity, security, and privacy challenges, could SLMs and edge AI offer the answer? Seyed Razavi-Nemtollahi explores.

For the past two years, most public sector conversations around artificial intelligence have focused on large cloud-hosted models, including foundation models, enterprise AI platforms and how government can safely adopt generative AI at scale.

But many government challenges look very different from the environments these systems are designed for.

For example, what happens when: 

Addressing these challenges means looking beyond traditional AI architectures to find new ways of addressing the problem.

Exploring how SLMs and edge devices can solve real-world problems

Edge devices are everyday devices such as tablets, smartphones, or laptops that process data locally, rather than sending it to a remote cloud service for analysis.

Unlike large foundation models, SLMs are designed for specific tasks and can run locally on these edge devices. This approach can reduce dependency on connectivity, improve resilience and offer stronger control over where sensitive information is processed.

As part of our internal learning programme, our innovation team scoped a real-world use case and ran a six-week sprint to learn more.

In the scenario, field officers visit rural households to help residents complete national surveys. In Wales, recruiting enough bilingual officers to do the job is difficult and online translation tools are not always practical due to poor connectivity in remote areas.

We set up a multidisciplinary team to explore how edge devices could support those frontline operations but also apply more broadly to other government challenges.

What are the lessons for government from our SLM and edge exploration?

The technology is still emerging but our exploration highlighted four lessons for government organisations. 

Fix real operational problems: user experience matters

In government, you need to understand the realities of frontline staff before selecting the tech. Our project focused on a real-world challenge. And user research was vital in building that understanding. 

Once you’re designing a solution, operational usability and trust heavily influences whether a workflow (AI or otherwise) is actually viable. A technically accurate system that disrupts human interaction will fail in practice.

Design for reality, not the demo

Many AI systems perform impressively under controlled conditions. Operational environments are different. In our use case, background noise, poor weather, intermittent connectivity, accessibility needs and human behaviour all influenced outcomes. Success depends as much on understanding the context in which a service operates as it does on model accuracy.

This led us to focus heavily on user journeys, environmental constraints and trust. We found that questions around confidence, oversight and usability emerged far earlier than questions about model performance.

Smaller models are becoming surprisingly capable

A few years ago, running useful AI workloads on commodity mobile devices would have been unrealistic. Today, narrowly focused language models can perform specialised tasks on relatively modest hardware. While they cannot match the breadth of larger cloud-based systems, they do not necessarily need to.

For many public sector use cases, the goal is not to build a model that knows everything. The goal is to solve a specific problem reliably, securely and cost-effectively.

Make data and governance a priority from day-one

Our biggest challenge in the project was not necessarily model architecture or device performance. It was access to high-quality linguistic data, coverage of regional dialects, and the ability to validate outputs with subject matter experts.

This is particularly relevant for minority languages, specialist domains and highly regulated environments. 

Organisations often discover that their AI strategy depends as much on data availability and stewardship as it does on technology choices.

The work also reinforced that governance cannot be added later.

Questions around human oversight, auditability, confidence scoring and acceptable levels of risk appeared almost immediately. These were not implementation details. They shaped the design of the solution itself.

Cloud v edge

As government moves from AI experimentation towards operational deployment, the most important lesson may be that the most successful AI initiatives are unlikely to be those with the largest models. They will be the ones that balance technical capability, operational reality, user trust and governance from the outset.

Edge AI will not replace cloud AI. Nor should it.

But for organisations dealing with connectivity challenges, security constraints, or privacy requirements, it offers an increasingly credible option worth exploring.

The future of government AI may not always live in the cloud. Sometimes it may sit in the hands of the people delivering services.

If your organisation is seeking to adopt secure AI capabilities in high-security environments, we’d love to share our findings. Get in touch using the form below

Latest content

Thanks for joining us! We’ll keep you informed with regular updates.

[gravityform id=”16″ title=”true” ajax=”true”]