Cutting claims triage from days to minutes

An insurer's adjusters were spending the first hour of every complex claim doing the same thing: opening a policy document of eighty to four hundred pages, finding the handful of clauses that governed the claim, and deciding whether it was covered.

The documents were not standardised. Policies from three acquisitions sat alongside each other with different section numbering, and some of the older ones existed only as scans.

The problem

The obvious framing was "summarise the policy", and that was the wrong one. Adjusters did not want a summary. They wanted the two or three clauses that decided this claim, with enough context to defend the decision to a regulator.

That distinction changed the whole build. A summary can be judged loosely. A citation is either correct or it is not, which meant we could score it.

What we built

A retrieval system over the policy library, tuned on the client's own documents rather than on a benchmark.

• Hybrid retrieval. Semantic search alone kept missing clauses that hinged on exact terms of art. Keyword search alone missed paraphrases. Running both and reranking the union recovered most of what either missed on its own.
• Chunking by clause, not by token count. The section structure was inconsistent, so we parsed each document into clauses first and chunked those, which stopped answers straddling two unrelated provisions.
• Answers as citations. Every output is a set of clause references with the quoted text and a one-line reason. The model never paraphrases the policy in the field the adjuster reads.
• Scanned documents handled separately. OCR quality varied enough that we routed those through a different path with a lower confidence ceiling, rather than pretending the text was as good as the rest.

How we knew it worked

Four adjusters built a test set with us over two days: 180 real claims, with the clauses each of them would have cited.

That set became the thing every change was measured against. When we replaced the reranker, we could say it moved clause recall from 0.81 to 0.94. When a prompt change looked better in a demo but scored worse, we did not ship it.

The set also settled the disagreements. On eleven of the 180 claims, the four adjusters disagreed with each other. Those cases were removed from scoring and sent back to the underwriting team, which turned out to be a useful finding on its own.

What changed

Adjusters now open the policy on about four claims in ten, and when they do it is because the system flagged low confidence and told them why.

The hour at the start of a complex claim became a few minutes of checking citations. The decision stayed with the adjuster, which was a requirement from the compliance team on day one and, as it turned out, the reason the rollout went quickly.

What we would do differently

We spent the first week building a general policy question-answering interface, because that was what had been asked for. It demoed well and nobody used it.

The build that mattered started when we watched three adjusters work through a claim and saw that they never asked a question in a sentence. They looked for clauses. We should have watched first and built second.