Four Habits That Break When You Start Building AI Systems

You run a test a second time in a row, without touching a single line of code - and it fails. Not because of the network, not because of a random timeout: the response simply came back different. Same facts, different words, different order of sentences. Your first reaction is to hunt for the bug. There is no bug. What broke isn't the program's behavior - it's your idea of what a "correct answer" is.

I came to AI systems from ordinary development: 19 years in IT, from software engineer to architect. I haven't forgotten how to write code, and almost everything I knew about engineering still holds. But a few working habits had to be reconsidered from scratch, and it's better to do that deliberately than in the aftermath of an incident. By the end of this piece you'll understand how working on an AI system differs from the development you're used to, and which four habits break first.

Two definitions to start. A language model is a program that produces response text from request text: it predicts the continuation rather than computing it from a formula. A token is a chunk of text roughly the size of a word or part of a word; tokens are the unit for both the size of your request and the bill from your provider.

Change one: the result stops being deterministic

An ordinary program is deterministic: the same input gives the same output. The whole familiar notion of a test rests on this - you compare what you got against what you expected, and it either passes or it doesn't.

With a language model that doesn't work. The same request gives different answers, and "correct" stops being a point and becomes a range. Comparing a response against a reference string is pointless: the test will go red on correct answers and green on wrong ones that happened to match the wording.

Hence the first habit to change: check the properties of the response, not its text. Not "did it match word for word," but:

• does the response contain a link to a source
• do the numbers in the response agree with the numbers in the source
• does it follow the format the next step of the system expects.

Checks like these survive rephrasing. And they double as a gate - the place where a response that fails the check doesn't move on, but goes back for a retry.

flowchart LR
  A["Запрос пользователя"] --> B["Ответ модели"]
  B --> C{"Есть ссылка на источник?"}
  C -- нет --> F["Не отдавать, повторить запрос"]
  C -- да --> D{"Числа сходятся с источником?"}
  D -- нет --> F
  D -- да --> E{"Формат соблюден?"}
  E -- нет --> F
  E -- да --> G["Показать пользователю"]
  F --> B

Change two: quality lives in the data, not in the code

Before, when the result was bad, I went to the algorithm. In an AI system I go to the data - and almost always find the cause there.

The clearest case is a search assistant: a system that finds an answer in your documents and responds from what it found, rather than out of the model's head. Almost every error in it was fixed not by code but by two things: how documents were cut into fragments, and how those fragments were labeled.

It sounds boring until you see the mechanics. A document gets cut into pieces because it won't fit into the request whole. A piece is found correctly by the query - the words matched, search did its job. But it's understood incorrectly: there's no context in it. A paragraph with a condition or a deadline is correct inside its own section and misleading when torn out of it: it's unclear what the condition refers to and who it applies to. The model receives a fragment without that frame and honestly answers based on what it sees.

That's why the boundary of a chunk is an engineering decision, not a technical triviality, and every fragment needs labeling: where it's from, which section, what it relates to. Change the chunking and the quality of the answers changes, even though you touched nothing in the code.

flowchart LR
  A["Очередь задач пуста"] --> B["Служебный опрос уходит в модель"]
  B --> C["Запрос: около 29 700 токенов"]
  C --> D["Ответ: 90 токенов, смысл: задач нет"]
  D --> E["Доля в счете за модели: 45 процентов"]
  A -.-> F["Тот же ответ обычным кодом: 0 токенов"]

Change three: every call has a price

In ordinary development an extra function call costs almost nothing. A call to a model always costs money, and the cost isn't obliged to be proportional to the benefit.

My most annoying example: a housekeeping poll - the system periodically asking itself whether there's work in the queue. With an empty queue it still went to the model with the full context: about 29,700 tokens of request for a 90 token response whose meaning was "no tasks." A call like that looks cheap: it rarely catches your eye and doesn't slow the user down. In the end it ate 45 percent of all model spend.

The conclusion is simple: look not at the total on the bill, but at the breakdown by call type. The total tells you nothing - it just grows. The breakdown shows that nearly half the money goes to a question you can answer without the model: the queue is empty, and ordinary code can check that.

Change four: the human becomes part of the system

The familiar role of the human is the user: they're outside, they press buttons. In an AI system they become a link in the system itself, on par with the queue and the storage.

It works like this: actions with consequences require their confirmation, and publishing anything requires their explicit decision. Not "the model usually doesn't get it wrong," but "without confirmation the step physically doesn't execute."

An important detail: this is designed in advance, not added after an incident. Bolting confirmation onto a finished pipeline means cutting it in half and rethinking what happens to a task while it waits for a human: where it sits, what happens if there's no answer for a day, who sees it next. Building in a confirmation point from the start is cheaper than retrofitting one.

What stayed the same and turned out to matter most

None of the ordinary engineering discipline was repealed. On the contrary, layered on top of a nondeterministic component it became more valuable:

• timeouts on external calls - the model responds more slowly than your database, and you can't wait for it forever
• the order of operations on failure - what has already happened and what to roll back if the system dies between steps
• reproducibility - the ability to repeat a specific run and get a clear picture instead of guessing
• logs you can use to reconstruct an incident - what exactly went to the model and what came back.

Without that last point, the post-mortem turns into an argument about memories. I found the housekeeping poll story precisely in the logs: without them I'd have had only a line reading "spend went up."

And one temptation

Almost any problem tempts you to solve it by switching models: it's one line in a config, and trying is faster than digging in. Sometimes it helps. But almost always the fix lies a level below - in how the data is chunked, in a redundant call, in a missing check. Switching models masks these things, and they come back in the next release.