AI search
What AI fact checking means, and what happened when we tried the usual checks on 11 wrong AI shopping answers whose right values we already knew.
Last updated November 4, 2026 · ~6 min read
The short answer: AI fact checking means either using AI to check claims or checking what an AI told you. For the second job, we tried the usual checks on 11 wrong shopping answers. A second AI gave the right value 3 times; in the other 8, another AI repeated the same error. Asking the same AI again helped 4 times. All 11 answers linked the seller’s own site, which had the right value. So the check that works is opening the source, not asking another AI.
AI fact checking means two different jobs that share a name. The first is using AI to check a claim: a tool breaks a text into individual claims, looks for sources and returns a verdict with links. The second is checking what an AI told you: you take an answer from ChatGPT, Perplexity or Google and find out whether it is right. This article is mostly about the second job, because that is the one we measured.
Tools for the first job exist. Journalists use Full Fact AI to flag claims in speeches and articles, and Google’s Fact Check Tools to see whether a claim has already been checked (IJNet lists both). Originality.ai scans drafts for false facts before publishing, and its own page says not to treat the result as the final word. We did not test any of these.
The best-known public evidence is about news. In a study coordinated by the European Broadcasting Union and led by the BBC, journalists at 22 public broadcasters reviewed more than 3,000 answers from ChatGPT, Copilot, Gemini and Perplexity: 45% had at least one significant issue, 31% had serious sourcing problems and 20% had major accuracy issues (BBC, October 2025). When the Tow Center asked eight AI search tools to name the source of a news excerpt, more than 60% of 1,600 answers were wrong, and most were stated with confidence (Columbia Journalism Review, March 2025).
Shopping facts go wrong too. On October 22, 2026, we asked “where can I buy this, and how much is it?” about 8 products, three times each, in Google, ChatGPT and Perplexity, and compared every answer with the seller’s own live price and stock. Of 55 prices quoted for the seller’s own store, 5 were wrong. And for one product, ChatGPT and Perplexity said it was sold out in all three runs, 6 answers in all, while the store showed it in stock. The full study has the details.
Those 11 wrong answers (5 prices, 6 stock claims) make a useful test, because we know the right value for each. For every one, we looked at what the usual checks would have shown, using the answers recorded the same day.
| The check | Led to the right value | Otherwise |
|---|---|---|
| Ask a second AI | 3 of 11 | 8 of 11: another AI repeated the same error |
| Ask the same AI again | 4 of 11 | 6 of 11: the same error in every other run; 1 had no other answer |
| Open the link in the answer | 11 of 11 | All 11 linked the seller’s own site; 8 linked the product page itself |
ChatGPT and Perplexity both said a pair of socks was sold out, in all three runs, while the store’s own data showed both sizes available. In one run, both also put a hand soap refill at $25 when the store charged $39. In each of these cases both assistants linked the same seller’s site, so a second assistant was not a second source. Perplexity did hedge on the soap: it noted that another page on the site showed $39 and told the reader to verify at checkout.
ChatGPT priced a cooler at $213.74 in one run and at the correct $164.99 in the next two. That sounds like a catch, but a shopper who sees two numbers still doesn’t know which one is right. It also runs the other way: ChatGPT had the soap right in the first run and wrong in the next two. And the socks stayed wrong every time, so repeating a question can make an error look settled.
Every wrong answer linked the seller’s own site, and that site’s live data had the right value. Three of the 11 linked somewhere other than the product page, so you may need one more click. One caution from the soap: the $25 and $39 were two options of the same product, so check the option you actually want rather than the first price on the page.
Across all products and runs, ChatGPT and Perplexity both stated the same fact 27 times. Both were right 21 times, both were wrong 4 times, and they split twice. So a disagreement flagged only 2 of the 6 cases with an error, and 4 of their 25 agreements (16%) were agreements on something false. A disagreement is a good reason to look closer. An agreement tells you much less than it seems to.
If you sell something, this is a different problem. A shopper who reads “sold out” may never open your page to check; they can go elsewhere, and you never see it happen. In the same test, answers named another seller as the cheaper or better option 9 times. Our short video walks through that test.
So check your own answers on purpose. Ask each assistant what your buyers ask, several times, and compare what it says with your real prices, stock and specs. Then state those facts plainly on your pages, option by option: see how to write product descriptions and product page SEO. From outside we can’t tell why an assistant misread a page, so recheck after each change rather than assuming the fix worked.
8 products with their own online stores, one question each, asked three times, an hour apart, on October 22, 2026, from a US location, in Google (the AI Overview, when one appeared), ChatGPT through chatgpt.com search and Perplexity through its Sonar API, all collected through DataForSEO. That is 72 asks and 57 answers, because Google showed an AI Overview for 9 of its 24. Prices and stock were labelled by hand against the store’s own product data, saved at the start of each run. We did not run the checks live: “second AI” and “ask again” compare answers recorded that day for the same product, and “open the link” means the answer cited the seller’s site, whose data we had saved. Six of the 11 errors are one product’s stock, so read this as a small worked example, not a rate. Shopping facts only, one market, one day.
Hasmord is the AEO-native sales platform. Buyers ask questions across AI assistants, search, and sales conversations. Hasmord uncovers where your answers about your products and services fall short, turns evidence into better answers, finds potential customers for you, and connects every lead to your sales process.
In Hasmord, this is the first step, which uncovers where your answers about your products and services fall short. AI answers asks ChatGPT, Claude, Gemini and Perplexity your buyers’ questions and keeps every answer and the pages it cites, engine by engine, so you have a record to check against your own facts.
The second step turns evidence into better answers: Hasmord compares those answers with your product facts and prepares fixes, product by product, for your team to approve. Find customers then finds potential customers already looking for what you sell, with the reason each one fits; only the people you verify join your CRM, and Hasmord never contacts anyone for you. In the CRM, each lead keeps the question that brought it, so what buyers ask in sales becomes the next question to check. See pricing.
It means two things: using AI tools to check claims against sources, and checking whether an answer from an AI assistant is right. The tools split a text into claims and return verdicts with links. Checking an AI answer means going to the source that owns the fact.
Not reliably. In our test of 11 wrong shopping answers, another AI gave the right value for 3, and for 8 another AI repeated the same error. Asking the same AI again found the right value for 4, and 6 stayed wrong in every run.
Yes. Full Fact AI and Google’s Fact Check Tools are used by journalists, and Originality.ai scans drafts before publishing. We have not tested them. Treat any verdict as a lead and open the source it cites.
Use it to find sources and to spot claims worth checking, not to settle them. In our test ChatGPT gave different prices for the same product in different runs, and it repeated one wrong stock claim in all three.
Pick the claims you would act on, go to whoever owns each fact, open the link in the answer and find the claim on the page, and check that it applies to the right option, region and date. All 11 wrong answers in our test linked the seller’s own site, which had the right value.
It depends on the topic and the assistant. An EBU and BBC study found significant issues in 45% of AI answers about news. In our shopping test, 5 of 55 quoted store prices were wrong, and ChatGPT and Perplexity called one in-stock product sold out in all 6 of their answers.
Related: AI shopping answers accuracy study · AI brand mentions · Perplexity vs ChatGPT · how to write product descriptions · product page SEO · LLM SEO.