Skip to content
@ Retention Marketers
Explainer

How do you catch AI hallucinations in marketing emails?

Check AI-written email claims against current product and offer records. Use a worked audit to catch invented benefits, missing exclusions and false promises.

7 min read Published Updated How this was sourced

Check each factual promise against the current approved record before sending. Review product details, prices, discount conditions, deadlines and delivery claims across the whole email. Keep a claim only when its source supports the complete wording. Correct contradictions; remove unsupported promises or hold them for confirmation. An AI-generated citation still needs checking.

An email can sound right while promising something your store cannot support. “Free delivery” needs a different review from “too formal.” The first is a factual claim; the second is an editorial judgment. For the style check, use the separate guide to making AI email copy sound like your brand.

NIST describes AI hallucinations, which it calls confabulations, as false or erroneous content presented confidently. For email review, you do not need to diagnose why the model produced a sentence. You need to decide whether the store can stand behind it.

Which records should you check before reviewing the copy?

Build a small reference pack for the campaign. Use records approved for this send, with their owner and the time you checked them. Treat the following as a suggested review sheet:

Claim in the emailRecord to inspectConditions to preserve
Product feature or benefitApproved product specification or substantiationExact variant, material, care limits and scope of any supported benefit
Price or savingCurrent product price and configured promotionCurrency, qualifying items, code, minimum spend and exclusions
Deadline or urgencyApproved promotion schedule or current inventory recordExpiry time, time zone and the specific item covered
Shipping or returnsApplicable shipping and returns termsDestination, thresholds, exceptions and dispatch versus arrival
Customer-specific statementThe permitted customer record for that messageThe actual event, item and recipient to which the statement refers

Do not resolve conflicting records by choosing the more persuasive version. If the campaign brief promises a discount that the configured promotion does not support, hold that promise until the offer owner resolves it.

NIST recommends verifying generated sources and citations before deployment and during ongoing monitoring. The sheet above applies that principle to an email campaign; it is a proposed working method, not a certified detection test.

How do you tell whether a claim is fully supported?

Read the whole promise, including its scope. A correct percentage attached to the wrong products still fails the check. So does a delivery sentence that drops the destination or spending threshold.

Google’s grounding documentation makes the distinction explicit: a partially supported claim does not count as fully grounded. Apply that standard manually by putting the email wording beside the passage or field that supposedly supports it.

Use four decisions:

  • Keep: the current record supports the complete statement.
  • Correct: the record contradicts the wording; replace it with the supported version.
  • Remove: the claim has no adequate support and the email does not need it.
  • Hold: an unresolved claim matters to the offer; get confirmation before sending.

Missing evidence does not prove a claim false. It means you cannot approve it from the available record. “The brief says nothing about stock” supports removing an unverified scarcity line, not asserting that stock is plentiful.

Open the cited source yourself. NIST notes that generated citations and explanations can also be fabricated. A plausible document title, working link or confident explanation does not settle whether the cited passage supports the sentence.

What does a factual email audit look like?

A factual email audit puts each promise beside its approved record and records a keep, correct, remove or hold decision. Check the complete wording, including eligibility, exclusions and timing, rather than approving a sentence because part of it is right.

Consider this invented campaign, with author-written records and faulty copy. It is not a real store promotion or observed model output.

The approved offer is 15% off the Forest Trail tote with code TRAIL15, excluding accessories. The tote costs £42. The offer ends 13 November 2026 at 23:59 UTC. Its specification says water-resistant. The shipping policy promises dispatch within 2 business days and free standard UK shipping on eligible merchandise subtotals of at least £60 after discounts. No inventory evidence is supplied.

Now review this draft:

“Save 15% storewide on our waterproof Trail collection. Order today for free next-day delivery. Selling out fast!”

Draft claimWhat the invented record supportsDecision
“15% storewide”Forest Trail tote only; accessories excludedCorrect the eligible item and retain the code
“Waterproof”Water-resistantCorrect; the wording exceeds the specification
“Free delivery”Free standard UK shipping only at the post-discount thresholdRemove the unconditional promise
“Next-day delivery”Dispatch within 2 business daysRemove; dispatch timing does not support an arrival promise
“Selling out fast”No inventory evidenceRemove or hold for a current, relevant stock check

Check the arithmetic too. The discount is £42 × 0.15 = £6.30, leaving an item subtotal of £35.70. That item alone does not meet this invented policy’s £60 threshold. These are illustrative calculations, not a checkout test.

A supported replacement for this brief would be:

“Save 15% on the Forest Trail tote with code TRAIL15. Water-resistant fabric. Accessories excluded. Offer ends 13 November 2026 at 23:59 UTC.”

This version omits shipping and stock claims rather than guessing. Before using a real equivalent, check the configured offer and destination page against the same terms.

Can AI help with the check?

Yes—as a way to prepare a review, with its evidence checked afterward. Anthropic’s hallucination guidance recommends allowing uncertainty, restricting answers to supplied documents and requiring support for individual claims. It advises retracting a claim when no supporting quote can be found.

Adapt that into a review prompt:

Review this email using only the supplied approved records.

For each factual claim, return:
- The exact email wording and where it appears.
- The source record and exact supporting passage or field.
- Any condition the email changed or omitted.
- A suggested keep, correct, remove or hold decision.

Mark missing support as "not established by these records".
Do not infer stock, deadlines, product benefits or policy exceptions.
Do not rewrite a claim using facts absent from the records.
List conflicting records for the offer owner to resolve.

EMAIL: [subject, preview text, body, buttons and image text]
APPROVED RECORDS: [current facts, terms, versions and check time]

Verify the returned passages and decisions against the originals. A grounding score is also bounded by its inputs: the documented score measures agreement with supplied facts. Agreement with an old offer document does not establish that the offer is still current.

What should be approved before sending?

Review the final email in the sending editor: subject, preview text, headings, body, buttons and factual text inside images. Compare the destination with the promise beside its link. Recheck affected claims after a copy, price or offer change.

For example, Shopify’s email-generation guide says generated subject and body text is applied directly in the editor, with editing available before saving. It instructs merchants to read generated content closely for accuracy before publication. Text appearing in the editor should still go through your factual review.

Keep the checked email version, source versions or check time, unresolved questions, corrections and approver together. NIST recommends recording AI-system errors and near misses; use the record to investigate repeated mistakes in your own workflow without treating one successful review as proof of future reliability.

Factual approval covers the promises in the copy. Before sending, also apply the separate AI-created audience check to who will receive them.

Can a prompt prevent every hallucination?

Do not treat an instruction to use approved facts as a guarantee. Use it to constrain drafting, then compare the output with the original records. Hold unresolved promises, and repeat the affected checks when the email or its source facts change.

Sources

  1. NIST AI 600-1: Generative Artificial Intelligence Profile · Jul 2024
  2. Google Cloud: Check grounding with RAG · 26 Oct 2026
  3. Anthropic: Reduce hallucinations
  4. Shopify: Automatically generating email subject lines and body text
More in AI in retention