The Agent Said: 'Done'. The Database Disagreed.

The agent reports that the task is complete. The database shows otherwise. Microsoft described in a Hugging Face blog the ThinkingBox system: it evaluates agents not by their responses, but by what they actually changed in backend systems.

graph TD
    A[Агент запускает действие] --> B{Вызов инструмента};
    B -- Успешно --> C[Агент сообщает: "Готово"];
    C -- Не проверяет --> D[База данных: Состояние не изменилось / Ошибка];
    D -- Обнаружено ThinkingBox --> F{Расхождение: "Готово" vs "Не готово"};

How to measure agent reliability: Microsoft ThinkingBox

ThinkingBox looks at how the system state actually changed, not at the correctness of responses or tool calls. The tool runs tasks multiple times and verifies their actual execution.

Why a tool is not a result

The agent called the tool correctly - is the task done? ThinkingBox shows the opposite. Final responses and valid tool calls are just proxies. The agent can sound correct while leaving an incorrect value, changing the wrong record, or creating an unintended side effect. Only the records in the system answer the question of whether the task was completed.

Microsoft conducted 121,680 attempts with 12 LLMs, 79,853 attempts failed execution checks. Of these failures, 67.24% completed cleanly, called a state-changing tool, and reported no errors. Most problems lie not in the agent's logic, but in its interaction with external systems.

graph LR
    A[Общая производительность (Pass@1)] -- Не всегда = --> B[Надежность (Observed 20/20)];
    subgraph Метрики ThinkingBox
        A1[Pass@1: Доля успешных попыток] --> A;
        A2[Pass@20: Хотя бы один успех из 20] --> A;
        A3[Observed 20/20: Все 20 попыток успешны] --> B;
    end
    style A fill:#fff,stroke:#333,stroke-width:2px;
    style B fill:#f9f,stroke:#333,stroke-width:2px;

One success doesn't mean reliability: Pass@1, Pass@20, Observed 20/20 metrics

Agent reliability cannot be measured by one successful run. ThinkingBox runs each task 20 times from an identical clean backend. Important metrics:

• `pass@1`: The proportion of all attempts that were successful (how well does it work normally?).
• `pass@20`: The proportion of tasks solved at least once in 20 attempts (can it do this *at all*?).
• `Observed 20/20`: Tasks that actually passed all 20 recorded attempts (can it *always* be correct?).

The difference between what a model does once and what it does always - that's reliability. Only three models retain most of their `pass@1` after 20 runs: GPT-6 Astra retains 78%, Claude Opus 5.5 and Claude Opus 5 both 71%. GLM-5.1, Kimi-K2.6, and DeepSeek-V4-Pro retain around 8%.

What reliability costs: hidden costs of consistency

Reliability costs more than one-time success. The authors calculated (this is an estimate, not real cloud bills) the difference between the cost of a successful task and the cost of a *reliable* task (one that passes all 20/20 attempts).

• The cheapest reliable task is GPT-5.4: $6.80, but it only has 128 such tasks.
• Claude Opus 5.5 achieves the highest number of reliable tasks (241) at a price of $7.80.

The cheapest way to get a correct answer is not the cheapest way to get a reliable answer. In 79.9% of cases, agent failures are related to tool handling, not logic.

pie title Причины сбоев AI-агентов
    "Обработка инструментов" : 79.9
    "Некорректное обновление состояния" : 10.3
    "Незавершенные действия пользователя" : 7.0
    "Отсутствие действия, изменяющего состояние" : 2.9

Failure reasons: more tool than logic

About four out of five failures are related to tool handling, not reasoning logic. Agents typically reach the workflow execution stage, then cannot recover from tool errors, unfulfilled preconditions, or empty search queries. This is a retry and error recovery problem before it becomes a model problem itself. Complexity varies significantly by domain: retail averages 59.52% `pass@1`, while auto insurance is only 33.83%.

Sources

Microsoft ThinkingBox: The Agent Said It Was Done. The Database Disagreed.