What we check before building anything

Most of the AI work that fails does not fail in production. It fails in the decision to build it, months earlier, when nobody could say what "working" would mean. By the time the system is live the argument is about model choice, and that was never the problem.

So before we write anything, we go through four questions. They take about a week. Two of them have ended engagements before they started.

Can you show me the last fifty?

Not the data model. The last fifty actual cases, as your team saw them.

If the answer is a spreadsheet somebody maintains by hand, that is fine and useful. If the answer is that the cases exist across three systems and nobody has ever looked at them together, that is the project, and it is not an AI project yet.

The point of asking for fifty is that it is small enough to actually produce and large enough to be embarrassing. Ten looks clean by accident. Fifty shows you the edge cases the team has quietly been absorbing for years.

What does a wrong answer cost?

There is an enormous difference between a system that drafts a reply for someone to check and one that sends it. The second needs to be right far more often, and needs a way to notice when it was not.

We ask people to put a number on it. Not a precise one. An order of magnitude is enough to decide whether a human stays in the loop, and that single decision changes the architecture more than any model does.

If nobody can say what a wrong answer costs, nobody can say what accuracy is worth paying for. That conversation does not get easier later.

How will we know it got better?

This is the one that ends projects.

If we cannot agree on a way to score the output before we start, then every future change is a matter of opinion, and every release becomes an argument about whether the new prompt is better than the old one. We have watched teams spend a quarter on that argument.

What we want is a test set: real cases, with the answers your experts would give, that we can run against any version of the system. It does not need to be big. A hundred cases you actually agree on beats ten thousand you have never read.

Building that set is work, and it is work the client has to do with us, because it encodes their judgement and not ours. When a client is not willing to spend the days on it, that is real information about how the project will go.

What happens when it breaks at 3am?

Every system we have run in production has had a bad day. A provider degrades, an upstream format changes, someone pastes in a document three times longer than anything we tested.

The question is not how to prevent that. It is what the system does when it happens: whether it fails loudly or quietly, whether anyone is paged, whether there is a way to fall back to the old process for a day without a deployment.

Teams that have run software before answer this immediately. Teams that have only ever run pilots have usually not thought about it, and that is fine, but it means the operate phase is real work rather than a line in the proposal.

The two that end projects

Question three ends more engagements than any other. If there is no way to score the thing, we will not build it, because we would have no way to tell you whether it works, and neither would you.

Question two ends the rest. Sometimes the honest answer is that a wrong answer costs almost nothing and a right one saves almost nothing, and the sensible build is forty lines of ordinary code and a rule table.

We would rather say that in week one than in month six. It costs us a project and it buys the kind of reputation that brings back the next one.