Answer
What makes an AI agent unreliable?
Almost always the setup rather than the model: no completion test, no state read-back, too much context and too many tools.
Four setup defects account for most of it: no testable completion condition, no read-back of actual state, an overgrown context, and too many tools. All four are visible in a run log and none requires a different model.
When an agent behaves inconsistently, the first hypothesis reached for is the model, and it is the least useful one, because it points at a change nobody can make this week. The more productive hypothesis is that the same model is being asked to operate in a setup that provides no way to check itself, and the evidence for this is that the same model becomes markedly more consistent when the setup changes.
The first defect is the absent completion test. Without something checkable, the decision to stop is made against an impression, and impressions vary. This produces exactly the pattern people describe as unreliability: the same task done well on Monday and half-done on Tuesday, with no visible difference in the request. The variability is real and its source is that nothing was measuring completion in either run.
The second is acting on assumed state. An agent that issues an instruction and reports it as done, without observing the result, produces a run whose correctness depends entirely on nothing having gone wrong. Most of the time nothing does, which is why this defect hides. When something does — a rejected write, a lock, a path that moved — the run continues on a false premise and everything after it is unsound.
The third is context that has grown past usefulness. A long session accumulates material, and the instruction that governs the current step becomes a small fraction of what is present. Behaviour then drifts, and drift over a long run looks like randomness because nobody can point to the moment it changed. This is why the same work divided into stages with written outputs is more consistent than the same work done in one continuous session, with no change to the model at all.
The fourth is tool sprawl. Selection accuracy falls as the tool list grows and as entries become harder to distinguish, and a wrong tool choice early in a run propagates through everything after it. This one is easy to check: the run log shows the calls, and repeated first-choice errors are a configuration signal rather than a run signal.
There is a fifth cause that is genuinely about variability rather than setup, and it should be stated so the page is not overclaiming. These systems are not deterministic, and identical inputs can produce different routes. What the four defects above do is remove the checks that would otherwise contain that variability, which is why a well-constructed setup produces consistent outcomes from a non-deterministic process, in the same way that a manufacturing process does.
Unreliability is usually a property of the harness, and blaming the model is comfortable precisely because it is the part you cannot change.
Siddharth Sharma, Context Theory
Related questions
How do you tell whether the model or the setup is the problem?
Look at whether the failures are detectable in the run. If the agent went wrong and nothing in the system could have noticed, that is a setup problem regardless of what caused the error. If the checks were in place, fired, and the agent still could not complete the work, that is closer to a capability limit and a different task design is the answer.
Does a more capable model reduce unreliability?
It reduces the rate of certain errors and does not add the checks that catch the rest, so the improvement is real and bounded. Teams that upgrade without changing the setup usually report that things are better and the same failures still occur occasionally, which is exactly what you would predict: the errors got rarer and nothing new is detecting them.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Sub-15-minute compliance — automated routing vs manual only | 62.5% vs 39.1% | Category-wide |
| Close rate — response under 5 minutes vs over 24 hours | 32% vs 12% | Category-wide |
2026 speed-to-lead benchmark · verified
Optifai speed-to-lead benchmark · n=939 companies · Q2 2025–Q1 2026 · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | Inconsistent completion across identical requests is produced by the absence of a testable stopping condition rather than by model variance, because the stopping decision is then made against an impression that legitimately differs between runs. | Running the same task repeatedly with and without a checkable completion condition and comparing the variance in coverage. |
| Software | An agent that reports an action from having issued it rather than from observing the resulting state produces runs whose correctness depends on nothing having failed, which conceals the defect until an operation is rejected and the run continues on a false premise. | Introducing a silently rejected write into a run and checking whether subsequent steps proceed as though it succeeded. |
| Response | The same work divided into staged sessions with written outputs behaves more consistently than the same work performed in one continuous session, with no change to the model, because the governing instruction retains a larger share of the attention at each stage. | Running an identical body of work as one long session and as staged sessions and comparing adherence to the original brief. |
| Constraint | These systems are not deterministic and identical inputs can yield different routes, so the function of setup controls is to contain variability rather than to remove it, which is why a well-constructed harness yields consistent outcomes from a variable process. | Running the same task several times under identical configuration and diffing the tool-call sequences. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one