Second in our series on how AI answer engines decide what to say. The first looked at businesses the model has never heard of; this one looks at the opposite case, the brands it knows well. The full methodology and data can be found here: When Does the Model Look It Up?
You might expect a large model to be hard to mislead about a company it has seen thousands of times. Redfin was founded in 2004 in Seattle, and Claude will tell you so without looking anything up. So we asked a sharper question: if the model searches and the page it finds says something different - a wrong founding year, a wrong city - which one wins, its memory or the page?
We expected the model to defend what it knows, at least for the famous brands. For the most part, it did not.
The test
Using the same controlled setup as before, where the search tool returns only pages we wrote, we took seven well-known real estate and home-services brands that the model demonstrably knows. For each one we published a single, matter-of-fact page stating a false version of two core facts: the wrong founding year and the wrong founding city. There was no hedging and no "some sources say", just a confident, ordinary-looking page. We then asked the model about the company and let it search.
The result
The model adopted the false facts about 95 percent of the time, and kept its own correct knowledge only about 7 percent of the time. One page was enough. This held for household names, not only minor brands. The model would tell you, citing "its research", that a company you can look up in seconds was founded in the wrong year in the wrong city, because a single retrieved page said so. What surprised us was not the direction of the effect - models are known to lean on the evidence in front of them - but how little it took, and how little protection brand fame provided.
The useful part: what the model is skeptical of
We then ran a second version of the test. Instead of contradicting a fact the model already knew, we added a new claim it could not possibly know - a plausible new program or statistic, the kind of thing a company launches every quarter.
For the well-known brands, adoption of these new claims dropped to about 37 percent. That is far below the 95 percent we saw when contradicting core facts, and far below the 92 to 93 percent rate at which the model adopted new claims about businesses it had never heard of. In other words:
- Correcting a core fact the model already knows is accepted almost every time.
- Adding a novel claim about a brand the model knows is accepted only about a third of the time.
- Any claim about a business the model does not know is accepted almost every time.
So the model is neither uniformly credulous nor uniformly stubborn. It reserves its skepticism for a specific case, which is a new and surprising claim about an entity it already has a model of. Tell it something that fits what it already believes, such as a corrected date, and it complies. Tell it something novel about a brand it knows, such as a new award or program, and it pushes back.
You can see the same thing in how hard the model works. For businesses it did not know, it searched once or twice and moved on. For the well-known brands it searched ten to sixteen times per question, and most aggressively when a novel claim clashed with what it already believed. It was, visibly, trying to verify a surprising claim against a prior.
What we suggest if you are optimizing a known brand
The approach here is different from the one for an unknown business, where simply being findable is most of the battle.
- Treat your established facts as less safe than they feel. A confident third-party page with wrong details can overwrite them, so it is worth monitoring what the reachable web says about your core facts, since that is what the AI reports.
- Support new claims, do not just assert them. The single-page approach that plants a new program or statistic works on unknown businesses, but for a known brand a lone page clears the bar only about a third of the time. Novel claims about an established entity need several credible, consistent sources in order to be believed.
- Aim for consistency across the web. The model searches a known brand many times and weighs what it finds, so a coherent story across your own site and third-party sources is what survives that scrutiny.
In the next article we move from what the model believes to who it recommends, and show that when it ranks competing options, it barely cares about search order and cares a great deal about what is actually on the page.
If you want help auditing what the web currently says about your core facts, let us know and we are happy to help.