Making AI skills deterministic in data applications

The question we get most often from technical leads: "It gives a different answer every time I ask. How do we make it reliable enough to put in front of a customer?"

The answer is not a lower temperature. It is a different division of labour.

The rule: the model chooses, the code computes

A language model is good at deciding which operation applies to a messy, ambiguous request. It is bad at being an arithmetic engine, and it is bad at being a source of truth for figures.

So split the work:

• The model decides: which report, which period, which filter, which customer the vague phrasing refers to.
• The code computes: the sum, the margin, the ranking, the date arithmetic.

Every number a customer sees should have been produced by a function you could unit-test, not generated as text. If you can point at a figure in an answer and not name the function that produced it, that figure is a guess.

Give the model tools, not tables

Handing a model a wide table and asking for a total invites it to do the sum itself. Handing it a revenue_by_period(customer, from, to) tool does not. Narrow, well-named tools with typed arguments are the mechanism — they make the deterministic path the easy path.

Three properties make a tool reliable in practice:

1. Typed arguments with bounds. Validate at the boundary and reject bad input with a message the model can act on. A tool that silently accepts a malformed date produces a confident wrong answer.
2. A recoverable error shape. When a lookup misses, return an error result the model can read and retry from — not an exception that aborts the turn.
3. No hidden state. The same arguments must give the same answer. A tool that depends on conversation history cannot be tested, and cannot be trusted.

Pin what you can, measure what you cannot

Temperature zero reduces variation but does not eliminate it, and it is not available in every deployment. Treat it as a small improvement, not a guarantee.

What actually gives you confidence is a regression suite: a fixed set of questions with known-correct answers, run on every change to the prompt, the tools, or the model version. Model upgrades are the most common cause of a skill that "used to work". Without a suite you find out from a customer.

What to accept as non-deterministic

Wording will vary, and that is fine. Fix the facts, not the prose. Insist on:

• the same figures,
• the same set of records,
• the same decision or recommendation.

Accept that the sentence around them will be phrased differently each time. Teams that try to pin the prose spend months and get brittleness in exchange.