Answer
Why do AI systems get worse when nothing has changed?
Because the system is not the only thing involved. The inputs, the sources it reads and the platform underneath all move independently.
Because the workflow is only one of the moving parts. What arrives changes, the documents it reads change, the platform underneath changes, and the business changes around it. Nothing about the workflow itself has to move for its results to.
The observation is real and the framing is misleading. A workflow sits between inputs it does not control and a platform it does not control, and it depends on material that other people maintain. Any of those can move without notice, and the workflow will continue applying yesterday's logic to today's situation with no indication that the situation is different.
The most common cause is input drift. The business starts serving a new kind of customer, a channel begins sending differently formatted messages, a form field is added, a seasonal pattern arrives. The classifier keeps classifying, the extractor keeps extracting, and the proportion of items handled correctly falls. This is not a defect in the workflow; it is a workflow being used outside the distribution it was built for, and it is by far the most frequent explanation.
The second is source material. A workflow grounded in documents inherits every change to those documents, including the ones that make its previous behaviour wrong. A policy is updated, a price list is revised, a page is restructured. Retrieval keeps working, the answers change, and if nobody is comparing outputs across time the change looks like inconsistency rather than like an update.
The third is the platform. Model versions move, defaults change, a provider adjusts how something is handled. Some of this is announced and some is not, and even announced changes rarely come with a statement about how they affect your particular workflow, which is not knowable by the announcer. The practical response is not to resist the change but to have an evaluation set that can be re-run when one occurs.
The fourth is the surrounding process. People change how they use the output, a downstream step is modified, a human check that was catching errors is removed because the system seemed reliable. This last one is worth naming specifically: removing a check because a workflow has been performing well is a change to the system, and it is the change most likely to be described as not changing anything.
The response to all four is the same and it is unexciting. Keep a fixed evaluation set and re-run it on a schedule and after any known change. Watch the input distribution. Record downstream corrections. None of these prevents drift, and together they turn a mysterious decline into an attributable one, which is the difference between fixing something and adjusting it until the symptom stops.
Nothing changed is a statement about the part you own, and it is usually the only part that did not.
Siddharth Sharma, Context Theory
Related questions
How quickly does this happen?
Faster for input drift than for anything else, and it tracks how quickly the business changes rather than any property of the technology. A workflow in a stable process can run for a long time; one in a growing or seasonal business meets a materially different input mix within months. The rate is a fact about your business rather than about the system.
Does pinning the model version solve it?
It removes one of the four causes and is worth doing where the platform allows. It also has a cost: a pinned version eventually stops being available or supported, and a workflow that has never been tested against anything else then has to move all at once. Pinning plus periodic testing against the current version is the arrangement that avoids both problems.
METHOD
Every figure below carries its source and the date it was verified. Nothing on this page is asserted.
The numbers on this page.
| What | Value | Specific to |
|---|---|---|
| Close rate — response under 5 minutes vs over 24 hours | 32% vs 12% | Category-wide |
| Sub-15-minute compliance — automated routing vs manual only | 62.5% vs 39.1% | Category-wide |
Optifai speed-to-lead benchmark · n=939 companies · Q2 2025–Q1 2026 · verified
2026 speed-to-lead benchmark · verified
What is specific to this page.
| Kind | Claim | Check it against |
|---|---|---|
| Workflow | An unchanged workflow sits between inputs, source material and a platform it does not control, any of which can move without notice while the workflow continues applying its original logic without indication that conditions differ. | Listing every dependency of a workflow that is maintained by someone other than its owner. |
| Response | Input drift is the most frequent cause and is not a defect in the workflow but a workflow operating outside the distribution it was built for, which is why the input mix is the first thing to check when results decline. | Comparing the current input distribution against the distribution present when the workflow was validated. |
| Software | A workflow grounded in maintained documents inherits every change to them, so an updated policy or revised price list changes answers while retrieval continues to function, which reads as inconsistency rather than as an update. | Checking the modification dates of source documents against the period in which output changed. |
| Constraint | Removing a human check because a workflow has performed well is itself a change to the system, and it is the change most likely to be described as nothing having changed. | Reviewing what checks existed when the workflow was first deployed against what exists now. |
Each row would be wrong on another industry's page. Where a sourced figure exists it is in the table above instead; these are the constraints that shape the work and do not happen to be numbers.
Start with the measurement.
Reading about a benchmark is not the same as knowing your own number. The audit produces yours, measured rather than estimated.
$497 · delivered in 5 business days · credited against month one