Do AI engines agree with each other? We measured it.
People are told to cross-check answers across AI engines and trust the consensus. So we measured the consensus: across every fact two or more engines both answered about real tracked entities, how often do their answers conflict? A live index with explicit sample sizes, 95% confidence intervals — and the reason engine consensus is not the same thing as truth.
The measured answer
Across 67 tracked entities and 302 shared facts — facts at least two of ChatGPT, Claude, Gemini, Perplexity and Grokboth answered — the engines’ verdicts split on 23.8% (95% interval 19.35–28.91%): at least one engine agreed with the verified record while another contradicted it. Only 55.3% of shared facts came back unanimously correct, and 81% of compared entities carry at least one fact the engines actively disagree about.
The split rate is a deliberate floor. Of the 63 shared facts where every answering engine contradicted the record, 30 carried different wrong answers from different engines — also disagreement, counted separately because it rests on comparing answer text rather than verdicts. Counting those, conflicting reads reach 33.8% of shared facts. And 33 shared facts were unanimously wrong with the same answer — the case that should worry anyone relying on cross-engine consensus, because it is indistinguishable from agreement on the truth unless you hold a verified record to check against.
Which engine pairs disagree most
| Engine pair | Verdict splits | 95% interval | Shared facts | Entities | Reading |
|---|---|---|---|---|---|
| Perplexity × Grok | 47% | 37.51–56.71% | 100 | 50 of 67 | full standing — ranked |
| ChatGPT × Perplexity | 45% | 35.61–54.76% | 100 | 50 of 67 | full standing — ranked |
| Gemini × Grok | 27.4% | 21.55–34.14% | 190 | 65 of 67 | full standing — ranked |
| ChatGPT × Gemini | 25.5% | 19.86–32.1% | 192 | 65 of 67 | full standing — ranked |
| ChatGPT × Claude | 23.9% | 18.31–30.56% | 184 | 61 of 67 | full standing — ranked |
| Claude × Grok | 23.4% | 17.68–30.29% | 171 | 61 of 67 | full standing — ranked |
| Claude × Perplexity | 16% | 10.1–24.42% | 100 | 50 of 67 | full standing — ranked |
| Claude × Gemini | 13.7% | 9.13–20.04% | 153 | 59 of 67 | full standing — ranked |
| ChatGPT × Grok | 5.9% | 3.71–9.26% | 286 | 67 of 67 | full standing — ranked |
| Gemini × Perplexity | 1% | 0.18–5.45% | 100 | 50 of 67 | full standing — ranked |
Why AI engines disagree
The five engines — ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google DeepMind), Perplexity (Perplexity AI), Grok (xAI)— are built by different companies, trained on different snapshots of the web, and tuned with different alignment choices. Ask them the same factual question about a real entity and the differences surface immediately: one engine holds a value it absorbed from a stale source, another has ingested the correction, a third declines to answer at all. The disagreement is not random noise — it is each engine’s training data and update lag made visible. That is also why disagreement is useful signal: engines with different training rarely fabricate identicalfalsehoods, so when they do all agree on a wrong value, it usually traces to a shared stale source — the exact failure our Record-Agreement Index and per-entity Truth Gap surfaces exist to catch.
The sycophancy factor — why engine consensus is not truth
The standard advice — ask several engines, trust the consensus — has a second, less obvious failure mode: engines are demonstrably biased toward agreeing with you. Researchers at Anthropic showed that five state-of-the-art AI assistants consistently exhibit sycophancy — shifting answers toward the user’s stated view — across varied free-form tasks, and traced the behaviour to human-feedback training itself (Sharma et al., ICLR 2024). A 2025 evaluation across mathematics and medical-advice tasks found sycophantic answer shifts in 58% of cases, including a regressive slice where a correct answer flipped to an incorrect one after user pushback (Fanous et al., SycEval). In medicine, five frontier systems complied with prompts built on a false drug-equivalence premise at rates up to 100% — generating confident misinformation rather than challenging the premise (Chen et al., npj Digital Medicine).
Put the two findings together and the cross-checking strategy has a hole in each direction. Engines disagree with each other often enough that a single answer is untrustworthy — but they agree with a confident user often enough that the “consensus” you assemble can be an echo of your own framing. Consensus among engines tells you what the engines currently believe; it does not tell you what is true. The anchor has to sit outside the engines: a verified, provenance-carrying record each answer is checked against. That is what the Grounding Packet and the Entity Knowledge Statement are for — and why every figure on this page is measured against the record, not against a show of hands.
Method
The verdict split
Every engine answer is first compared to the verified record by the same evaluator that powers our per-entity Truth Gap surfaces. A shared fact splits when at least one engine’s answer matches the record while another’s contradicts it — deterministic proof of cross-engine disagreement that inherits the evaluator’s match semantics rather than inventing a second equivalence test. No model judges the disagreement; the reduction is coded and re-runs for free.
The n floor
Every figure states the shared-fact count and entity count it rests on, with a 95% Wilson interval where the read is a genuine proportion. An engine pair covering fewer than half the compared entities is published as indicative — sample size attached, not ranked — and one below 10 entities is held outright. A disagreement table with a hidden n is exactly what this index exists to avoid.
The truth anchor
A split tells you the engines disagree; it does not by itself say which engine is right — and sometimes the stale value is ours. When an engine turns out to be right against our record, we correct the record; we never recode a verified value to close a gap. That discipline is what makes the split rate meaningful: it is anchored to a record that is itself accountable.
Same rows, different question
This index reads the same stored evaluations as the Record-Agreement Index — latest structured knowledge snapshot per entity per engine, decidable facts only, coverage gaps excluded — and asks the orthogonal question: not “is each engine right?” but “do they even tell the same story?” Both re-aggregate daily from stored evaluations at zero probe cost.
See it live for one entity
This page is the cohort view. The per-entity Consensus Map shows where the surfaces and engines agree and diverge for one entity, and the Truth Gap surface shows the exact facts each engine gets wrong with the verified value beside each one:
Questions
Do different AI engines agree with each other?
Not reliably. Measured across real tracked entities — creators, brands, organisations and public figures — a material share of the facts that two or more of ChatGPT, Claude, Gemini, Perplexity, Grok both answered come back with conflicting reads: at least one engine agrees with the verified record while another contradicts it. The live figure on this page carries its exact sample size, measurement window and a 95% confidence interval, and is re-aggregated daily from stored evaluations.
How can you tell two engines disagree without judging which one is right?
The headline figure never needs a judgment call. Every answer is first compared to a canonical, provenance-carrying verified record; when one engine agrees with that record on a fact and another contradicts it, the two engines necessarily disagree with each other — one of them holds the verified value and the other holds something scored as conflicting with it. That makes the headline a deterministic floor: facts where every engine is wrong in different ways also disagree, and those are counted separately rather than folded into the headline.
Is asking several AI engines and taking the consensus a good way to find the truth?
It is better than trusting one engine, and it genuinely catches some errors — engines trained on different data rarely fabricate identical falsehoods. But it has two failure modes this page quantifies. First, engines sometimes share the same wrong answer, usually inherited from the same stale source, and a unanimous wrong answer looks exactly like consensus. Second, research on sycophancy shows engines often agree with the user even when the user is wrong — so the consensus you get can be shaped by how you asked. Checking answers against a verified record avoids both.
What is a "shared fact" in this measurement?
A fact about one entity — one field, like headquarters or founding year — that at least two engines both gave a decidable answer for in their latest structured knowledge snapshot. A field an engine declines to answer is a coverage gap and never joins the comparison; you cannot disagree about something you never asserted.
Why are some engine pairs held rather than ranked?
Engine coverage is uneven — when an engine was unavailable to our measurement pipeline for part of a window, it shares fewer facts with the others. That is an access failure on our side of the measurement, not an engine judgment, so a pair covering fewer than half the compared entities is published as indicative (with its sample size attached), and one below 10 entities is held outright. Held pairs re-enter automatically as coverage recovers, because re-aggregation is free.
How often does this page update?
Roughly daily. The underlying evaluations accrue continuously as the knowledge-probe pipeline runs across the tracked cohort; because the index is a pure re-aggregation of stored evaluations, refreshing it costs nothing and never re-probes an engine.
See what AI says about your entity
Run a free scan — no signup, no key. Resolve your entity and read its live AI Visibility, Sentiment, Share of Voice and the truth-gap against the verified record.