The Problem With Confident Nonsense
Here's what got me. I was using ChatGPT to help draft a technical summary about a software library, and it produced something that looked perfect. Clean sentences, correct terminology, even plausible version numbers. I almost copy-pasted it straight into a document for my team.
Then I actually checked one of the version numbers. Wrong. Then I checked a function name it cited. Made up. The model had written confidently worded fiction, and nothing in its tone suggested anything was off.
That experience sent me down a rabbit hole: how do you catch this stuff automatically, before it reaches a human? This article is what I found. We're going to cover three practical techniques — grounding scores, consistency sampling, and claim verification — that you can actually implement today.
Why Hallucinations Are Hard to Catch
The frustrating thing about AI hallucinations is that they don't announce themselves. The model doesn't say "I'm making this up." It uses the same confident, fluent language whether it's citing a real paper or inventing one. Perplexity (how "surprised" the model is by its own output) doesn't reliably flag hallucinations either.
So you can't just look at the output and spot the fakes by feel. You need systematic checks — ideally automated ones that run before the output hits anyone downstream.
The good news is that researchers and engineers have been working on this problem, and there are now practical approaches that don't require retraining a model from scratch.
Technique 1: Grounding Scores
A grounding score measures how well an AI's output is actually supported by the source material you gave it. If you're doing RAG or passing documents into context, this is your first line of defense.
The basic idea: take each claim the model made, and check whether that claim can be traced back to something in the source. Tools like Azure AI Studio's groundedness evaluation and open-source projects like ragas implement this as a measurable score.
Here's a simplified version of what the evaluation prompt looks like under the hood:
# Grounding evaluation prompt (sent to a second model call)
You are a fact-checker. You will be given:
- SOURCE: the documents retrieved from our knowledge base
- ANSWER: what the AI assistant said
For each factual claim in ANSWER, rate it:
[SUPPORTED] - directly backed by SOURCE
[UNSUPPORTED] - not found in SOURCE
[CONTRADICTED] - conflicts with SOURCE
Return a JSON object: {"score": 0.0-1.0, "claims": [...]}You run this as a second model call — sometimes called a "judge" or "critic" model. The output gives you a grounding score between 0 and 1. Anything below 0.7 is a red flag worth flagging to a human reviewer or blocking outright.
Use a different model as judge
Don't ask GPT-4 to evaluate its own output — you'll get optimistic scores. Use a separate model call, ideally with a lower temperature (0.0–0.2) to make the evaluation more deterministic and consistent.
Technique 2: Consistency Sampling
This one is my favorite because it's elegant in its simplicity. The idea is: if something is actually true, the model should say the same thing when you ask the question multiple ways. If it's hallucinating, the answers will drift.
Here's how it works in practice:
- Ask your original question and get answer A.
- Ask 3–5 paraphrased versions of the same question (different wording, different order).
- Compare the answers. High consistency = more trustworthy. High variance = likely hallucination.
# Simplified consistency sampling
import openai
from difflib import SequenceMatcher
question_variants = [
"What year was Python first released?",
"When did Python launch for the first time?",
"In what year did Python become available?",
"Tell me the release year of Python.",
]
answers = []
for q in question_variants:
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": q}],
temperature=0.3
)
answers.append(response.choices[0].message.content)
# Check if key facts (e.g. years, names) match across answers
# If answers wildly disagree, flag for reviewYou don't need to do exact string matching. A second model call can compare all four answers and output a consistency score. Research from papers like SelfCheckGPT formalizes this — if you sample the model multiple times at a slightly higher temperature and the sampled answers contradict each other, that's a strong hallucination signal.
In practice, I've found this works especially well for factual claims like dates, names, and statistics. Abstract reasoning tends to vary naturally; specific facts should be stable.
Technique 3: Claim Verification
Grounding checks what's supported by your documents. Consistency checks internal agreement. Claim verification goes further: it tries to actually confirm specific factual claims against external sources.
The workflow looks like this:
- Extract claims — Use a prompt to pull out every verifiable factual statement from the AI's output.
- Search for each claim — Send each claim to a web search or knowledge base lookup.
- Compare and score — Evaluate whether the retrieved evidence supports, contradicts, or is silent on the claim.
# Step 1: Extract verifiable claims
Given the following AI-generated text, extract every
factual claim that could be independently verified.
Return as a JSON array of strings.
Examples of verifiable claims:
- "Python was created by Guido van Rossum"
- "The function returns a list of integers"
- "Version 3.11 introduced faster startup times"
Do NOT include opinions or subjective statements.
TEXT: {ai_output}Once you have the list of claims, you can route each one to a tool call — web search via Bing or Brave API, a Wikipedia lookup, your internal docs — and have a final judge model rate each claim as verified, unverified, or contradicted.
This is expensive — be selective
Claim verification can mean 5–10 extra API calls per response. Reserve it for high-stakes outputs: medical, legal, financial, or anything going into a published document. For casual use, consistency sampling is cheaper and surprisingly effective.
Putting It Together: A Layered Detection Pipeline
These three techniques aren't mutually exclusive — they complement each other at different cost levels. Here's the layered approach I'd recommend:
# Hallucination detection pipeline
LAYER 1 (Always run, cheap)
→ Grounding score against source docs
→ Flag if score < 0.7
LAYER 2 (Run on flagged outputs, moderate cost)
→ Consistency sampling (3–4 variants)
→ Flag if key facts differ across samples
LAYER 3 (High-stakes outputs only, expensive)
→ Claim extraction + external verification
→ Block or human-review if any claim contradictedThe key is that Layer 1 filters most of the obvious cases cheaply. Layer 2 catches things Layer 1 missed. Layer 3 is your nuclear option for when accuracy really matters.
Practical Limitations (Be Honest With Yourself)
None of these techniques are perfect. A few things to keep in mind:
The evaluator can also hallucinate. When you use a model to judge another model's output, you're relying on the judge being more reliable than the original. This is usually true with careful prompting, but it's not guaranteed.
Consistency doesn't equal truth. A model can be consistently wrong. If its training data contained a pervasive error, it'll confidently repeat that error across all sampling variants. Consistency is necessary but not sufficient.
Some facts can't be externally verified. Internal business logic, proprietary data, or recently changed information might have no external source to check against. That's where grounding against your own documents becomes even more important.
Start with one technique
Don't try to implement all three at once. Pick the one that fits your use case: grounding if you're doing RAG, consistency sampling if you're doing open-domain Q&A, claim verification if you're generating content that needs to be publishable.
The Mindset Shift That Matters
The biggest change I made after going down this rabbit hole wasn't technical — it was attitudinal. I stopped treating ChatGPT output as a draft to edit and started treating it as a hypothesis to test.
That's not pessimism. It's just engineering. You wouldn't ship code without tests. You shouldn't ship AI-generated content without some form of verification either.
The techniques in this article aren't perfect, but they shift the burden. Instead of you personally catching every hallucination by manual review, you're building a system that catches the obvious ones automatically so your human attention can focus on the edge cases that actually need it.
That's a meaningful improvement — especially as AI output volume scales up and manually reviewing everything becomes impossible.
More tutorials in this category, or explore the full field guide.