AI adoption must not outpace governance

INSURANCE is not short of enthusiasm for artificial intelligence (AI). Firms are exploring where automation can remove friction, improve speed and support better operational decision-making.

In claims in particular, expectations are moving quickly, with 90% of senior insurance professionals in the UK and Europe expecting end-to-end claims administration to be managed by AI within the next 24 months according to EIP’s recent research.

But confidence in the technology doesn’t necessarily translate to confidence in the outcomes it produces. The same research found that 87% of respondents are concerned about bias or unfair outcomes, while 99% believe some form of human oversight should remain in place.

This contradiction is at the centre of conversations around deploying AI in insurance. Adoption of the technology is happening at speed, but difficult questions are being asked about whether governance, compliance and trust can keep pace.

Adoption versus trust
Insurance is not the only sector experimenting with AI, but it is one of the least able to treat the technology as a low-risk productivity tool. Insurers make decisions daily that can impact customers financially and personally, which brings about a different standard of scrutiny.

A claims decision must be fair, consistent and capable of being explained. It must be possible to show why a particular outcome was reached, which rule was applied and how the decision aligns with the relevant policy terms. A system that produces an answer without a clear audit trail may be useful in some contexts, but it is not enough for regulated insurance decisioning.

The key challenge for insurers deploying AI lies in its probabilistic nature. AI systems identify patterns, generate outputs and make predictions based on probabilities. That can be highly valuable when the task is to extract data, flag anomalies, detect possible fraud or prioritise work. It is much less comfortable when the task requires a transparent, repeatable and defensible decision.

Regulators and customers will not accept “the model said so” as a sufficient explanation and nor should insurers. It is telling that 39% of the industry says transparent algorithms and decision logs would help reassure them about the use of AI in insurance. The market turning its attention towards whether AI outputs can be understood and defended.

Claims require a harder line
Nowhere is this tension more obvious than in claims. The research found that the industry feels least comfortable automating claims submissions, with 40% identifying it as an area they would not feel comfortable handing over to AI, ahead of underwriting recommendations and customer interactions.

Claims decisions are among the most sensitive moments in the insurance workflow. Customers need speed but they also need confidence that the outcome is fair. Insurers need efficiency but not at the expense of consistency or regulatory confidence.

This doesn’t mean that AI should be kept out of claims altogether. Used properly, it can play an important supporting role by turning unstructured documents into usable data, identifying inconsistencies, enriching a file with relevant external information and routing cases for faster assessment. These are valuable uses because they improve the quality and speed of the process without making AI the final arbiter of a customer outcome.

The line is crossed when AI moves from assisting the process to deciding the result. At that point, insurers risk creating outcomes that may be difficult to explain, audit and defend.

Rules, oversight and auditability
For claims decisioning, the safer model is not unconstrained AI but AI working alongside deterministic rules-based systems. A rules engine configured and controlled by the insurer applies explicit business logic to each claim. The decision can then be mapped back to policy terms, rules and compliance requirements.

That distinction matters. Machine learning models are probabilistic and rules engines are deterministic. They do not remove the need for good governance, but they provide the structure insurers need to show how a decision was reached and why it was applied consistently.

AI can still add value around that framework. It can improve the inputs, accelerate triage and help human teams focus on the cases that need judgement. But the final decisioning architecture must remain transparent and auditable. In practical terms, that requires human oversight and systems that are designed for regulated environments rather than retrofitted into them.

Disciplined procurement
The industry’s approach to AI procurement is already reflecting this shift. Insurers are not simply chasing the cheapest tools or the boldest automation claims, as evidenced by the fact only 10% of senior professionals said cost would strongly influence their decision when assessing AI solutions.

That suggests a market becoming more selective, not less ambitious. Insurers want systems that integrate cleanly, come with strong vendor support and give them enough visibility and control to manage the risks.

AI should be embraced by insurance. The potential gains in efficiency, speed and customer experience are too significant to ignore. But responsible adoption requires discipline. It means knowing where AI should support a process, where it should not make the final call and how every important decision can be explained after the fact.

The industry does not need to choose between innovation and control. It needs systems that combine AI’s ability to process information with the oversight and auditability that regulated insurance demands.

Ross Sinclair is founder and chief executive of EIP