Before you can judge output you need to know what going wrong looks like. AI failures are not random noise; they fall into a small number of recognizable patterns, and learning to name them turns a vague feeling that something is off into a specific, checkable defect.
Content Failures: Invented, Missing or Stale
The most consequential pattern is confident invention: a figure, a name, a citation or a clause that appears nowhere in the source. Close behind are omissions, where a condition in the source never reaches the output, and staleness, where the answer reflects something other than the material supplied. All three read fluently, which is what makes them dangerous.
Instruction Failures: Format, Scope and Drift
A second family ignores the brief rather than the facts. Output arrives in the wrong format, exceeds a length limit, answers a broader question than the one asked, adds commentary nobody requested, or silently drops a required section. On longer inputs, instructions given early are sometimes applied at the start and abandoned later in the same response.
Consistency Failures: Same Input, Different Answer
Run the same input twice and the answers may differ in substance, not just wording. Related items may be classified inconsistently, and a small change in phrasing can flip a judgment. This matters most where outputs are compared with each other, such as triaging tickets, and it is invisible unless you test for it.
Action Step
Collect twenty recent outputs from one AI task you rely on. Label each failure you find with one of the three families above, then write down which family is most common and what it would cost you if it reached a customer.
This lesson is educational and reflects general practice. Results depend on your situation, market and effort; adapt what you learn to your own context.