What does reliable mean for an AI system?

Applied to machinery, reliable means it works the same way every time. Applied to a system whose output varies, that definition produces the conclusion that nothing here is reliable, which is not useful. The workable definition comes from engineering rather than from manufacturing: a component is reliable when its failure modes are known, its failure rate is measured, and the system around it handles those failures without producing an unacceptable outcome.

That gives three requirements, and each is a question a business can answer. What does it get wrong, and how often — which requires measurement against real inputs rather than an impression. How would you know — which requires a detection path that does not depend on someone happening to notice. And what happens then — which requires that the consequence of an undetected error is one you would accept before it occurs rather than after.

The first is where most efforts stop short, because it means running the thing against a period whose answers are known and counting. The result is usually neither as good as the enthusiasts claim nor as bad as the sceptics assume, and it is specific to the workflow and the inputs rather than to the technology. A rate measured on your own data is worth more than any published figure, because published figures measure a different population.

The second is where systems fail in practice. A workflow with a genuine error rate and no detection is not unreliable in a way anyone can see; it is silently wrong at a steady rate, and the discovery event is a customer, an auditor or a reconciliation. Detection can be a check, a sample, a downstream reconciliation or a complaint path, and any of them beats the common arrangement of none.

The third converts the first two into a decision. An error rate is only tolerable relative to what an error costs, and the same rate is fine in one process and unacceptable in another. Stating the consequence in advance is what turns reliability from a feeling into a threshold, and it also identifies the processes where no achievable rate is good enough and the design has to change instead.

There is a fourth property worth adding because it is what people actually mean when they complain: consistency. A system that is right in the same way each time is easier to work with than one that is right at the same rate but differently, because the second cannot be reasoned about. Most of the practices that improve consistency — a fixed completion condition, a checked output shape, a stable context — are the same ones that improve the measured rate, which is convenient.