Hallucination

Hallucination is the reason the rest of the safety and red-teaming theme exists before a single adversary shows up — a model’s default behavior is to sound certain about claims it never checked, no attack required. That makes this topic the theme’s entry point, not one item among many: understanding why confidence and truth come apart here changes how you read every other safety test downstream. What remains unsettled is not whether it happens, but how far detection and grounding can push it down, and who answers for what still gets through.

Reading hallucination in order: mechanism, ceiling, then defense

Start with what AI hallucination is and how next-token prediction creates confident falsehoods — it is the one explainer this whole topic defers to, and every later claim about detection or grounding assumes you already have this picture. From there, the intrinsic vs. extrinsic, closed vs. open domain taxonomy gives you the four failure modes the detection tooling actually targets — without it, “reduce hallucination” stays a slogan rather than a checklist. Why zero-hallucination LLMs remain impossible then sets the honest ceiling, so you stop promising a stakeholder a number no architecture can deliver.

Once the mechanism and the ceiling are both clear, the DeepEval, RAGAS, and RAG-grounding detection guide turns that understanding into a pipeline you can actually run. For the stakes behind getting it wrong, Finix-S1’s 1.8% error rate against courtroom fabrications documents the two-tier split between specialized and general models already in production. Close with the unsolved ethics of liability and disclosure — if your system’s output ever reaches a courtroom or a customer’s decision, read it before it does, not after.

How hallucination differs from red teaming, safety evaluation, and leakage

Three neighbouring practices get credited with catching hallucination, and none of them do the job directly.

• Hallucination is not something red teaming exists to find. Red teaming is deliberate: a human or an automated probe sets out to break the model. Hallucination needs no adversary at all — it is the default behavior of a well-formed, friendly prompt, which is why it sits in the theme’s foundation tier rather than its attack tier.
• Passing toxicity and safety evaluation says nothing about factual accuracy. Guard models score outputs against a harm taxonomy — violence, hate, self-harm — and a response can clear every one of those categories while still fabricating a case citation or a statistic.
• Data leakage fakes the metric before deployment; hallucination fakes the answer after it. Leakage inflates an offline score by letting test information reach training. Hallucination arrives at generation time, with no equivalent split to audit — the fabrication happens live, in front of the user.

Common questions about hallucination

Q: Does upgrading to a larger, newer model reduce hallucination enough that I can skip grounding entirely? A: No — the honest ceiling on zero-hallucination models holds regardless of scale, because the failure is structural to autoregressive prediction, not a capacity gap. Bigger models shrink the rate; none has closed it. Budget for grounding and detection at any model size.

Q: I already ground my RAG pipeline in retrieved documents — which single hallucination article is worth reading before I ship? A: The DeepEval, RAGAS, and RAG-grounding guide — grounding narrows where hallucination can hide, it does not remove the need to measure generation fidelity and retrieval accuracy as two separate checks before launch.

Q: Why do specialized models report hallucination rates under 2% while general frontier models still sit above 10%? A: Narrow, purpose-built models trade generality for a smaller output space to verify, which is exactly what Finix-S1’s 1.8% error rate against courtroom-grade fabrications demonstrates against frontier models tested on harder benchmarks.

Q: Who is liable when an LLM hallucinates false information inside a professional deliverable? A: The doctrine is still unsettled — courts have sanctioned the human who submitted fabricated citations, not the model, but the ethics of liability and disclosure argues that answer stops working once no human meaningfully reviewed the output.

Part of the safety and red-teaming theme · closest neighbour: red teaming for AI. New to safety testing from a software background? Start with the story: AI Safety Testing for Developers: What Maps and What Breaks.