Aveni: After Mills Review, AI agents need targeted assurance

The Mills Review established that firms must hold AI agents to the same standards as human staff, while ensuring accountability remains firmly with humans, and Aveni's Chief Commercial Officer, Kent Mackenzie, emphasised the need for small models to include specialist, relevant input.

The review, led by Financial Conduct Authority executive director Sheldon Mills, made seven recommendations to the regulator's board, including building and adopting an AI-enabled agentic supervisory model and strengthening system-wide coordination and oversight.

Mackenzie said the regulator has been consulting, running sandbox experiments and consulting the industry for a long time, so the review came as no surprise, but also reflects expectations on consumer outcomes and wellbeing that regulators have held for years, with the mentality that the same level of expectation applied to humans dealing with humans ought to apply to machine-based interactions.

"It's not outlining new regulatory obligations that suddenly need to be upheld. Mills is being very clear about the shifting landscape and the fact that the same set of obligations and expectations on senior managers are there, even when they start using agentic routines and machines to interface with human beings," said Mackenzie.

What has changed, he said, is that agents and chatbots now deal with customers in a front-facing function and are spreading from start to finish across the average customer journey. The question for firms is how to keep oversight and assurance across every one of those interactions, he said.

In general, firms currently have two answers, Mackenzie argued: hire more people to oversee what the machines are doing, or slow the rollout of agents and chatbots. He said neither works, because there are not enough people and manual checking is inefficient, and firms will not hold back deployment while rivals race ahead.

Aveni's response is what he called a machine line of defence, delivered in two of the company's two products. Agent Approve is a pre-deployment test environment where a client can plug in an agent and simulate thousands of customer conversations, then check whether any cross regulatory lines against a taxonomy Aveni has built. Agent Assure monitors every conversational turn once an agent is live, flagging emerging financial and non-financial risk. Mackenzie said it does the work of a second-line risk and assurance person in an automated setting.

Mackenzie said well-orchestrated, domain-specific small language models are more effective for overseeing non-financial risk than large frontier models. A frontier model not trained on how risk shows up in a specific domain will not deliver the result needed, he said, and analysing a complicated case can be slow and costly because the time before the model starts responding is high.

"Small language, domain specific, orchestrated in the right way has to be the answer here. Sovereignty can be far more controlled, domain specificity can be defined, and cost can be controlled, and that's massively important," said Mackenzie.

The skills the industry asks for have changed over Mackenzie's career. He looks to the question of "What are the skill sets of the future that we need? "Fifteen years ago, he would have named data scientists as the tech "rock stars," along with coders, prompt engineers, and, more recently, orchestration engineers. He said his conclusion is that small, domain-specific models are the answer, which rests partly on what comes next.

"[There's] the need for anthropological understanding and social science understanding to make sure that, as we're managing this brave new world, the societal understanding is built into the heart of these small language models. Quite frankly, that has to be true to identify vulnerabilities, bias, hardship and emerging concerns."