I asked three AIs how they pick a local business. They quoted Google’s rulebook.
If you read the last two Lefty Notes, you know the ring. ChatGPT, Claude and Perplexity, labels shuffled so none of them knows who is who, going at it round after round while we jump in the middle and throw a few punches of our own.
On October 15th we put the question in there that pays our bills. When somebody asks AI for the best roofer or the best dog trainer in town, how does it decide whose name to say? And how much of that comes from the business’s own website, compared with everything else, like reviews, the Google profile and directories?
All three gave the same answer in round one. The website is the foundation, but the reviews and the listings are what win. Claude even labeled it a guess, then added, “but I’m fairly confident.”
Then I looked at where they got it.
All three were leaning on the same Google help page. It explains how Google decides who to list on Google Maps, and it comes down to relevance, distance and prominence. It is a good page. It is also about Google Maps.
It was like asking three umpires to make a call in a soccer game and watching all of them reach for the baseball rulebook. In a second fight that same afternoon, Claude admitted it right there in its answer: “That document covers Maps only, not other AI products.” Then it ranked reviews third anyway.
I cannot be too hard on them. Google is the only one of these companies that publishes much of anything on how it picks a local business, so it is the only book on the shelf. And last year we built encyclopedia pages for our clients off a rulebook we had read instead of tested, and the volunteers who run that encyclopedia deleted every one of them. So I know the feeling.
So we quit arguing and showed them our numbers.
First up was a test we ran on eight of our client businesses, asking the AI about each one by name. When the AI read the business’s own website before it answered, it got the business right 17 times and wrong zero times. When it never read the website, it got it right once and wrong five times. The AI never found two of those businesses on their own sites, and one of them shares its name with a pile of other companies, so the AI kept describing them instead.
Claude and Perplexity both changed their answer. Claude said it plain: “I was too loose in calling the site ‘secondary’ across the board.”
Then we showed them our research paper. When a website pointed the AI robots to its fact file, they went and got it 1,549 times. The same kind of file with nothing pointing to it got picked up twice. ChatGPT and Perplexity both moved again.
Then came my favorite moment of the day. In the last round, Perplexity told the other two they were overstating how much reviews and listings decide who gets picked. In round one, Perplexity had told us to prioritize reviews and listings. Every answer in the ring ends with a line asking whether you changed your mind, and Perplexity filled it in with no.
The other two looked at its first answer and its last one, and both marked it as moved. I am with them on this one.
Ask an AI how AI works and it will quote somebody else’s rulebook with total confidence.
The second fight went after the bigger question, which is whether the AI picks you over everybody else when somebody asks for a recommendation.
In round one, Claude ranked a business’s own website fourth out of five for getting picked. Best-of lists came first, directories and profiles second, and reviews third.
Then we showed all three thirty days of our own data: 786 answers from ChatGPT, Claude and Perplexity to the kind of question a customer asks when they want to hire somebody in one line of work in Kansas City. Each time, the AI pulled five or six web pages and named the businesses on those pages, and it was mostly the same pages every day. The pages that won were almost all a single page whose web address said the service and the city. One business got named in about 300 of those 786 answers off one page like that plus one directory listing. A company from Cleveland with no office here got named 58 times because it was on the lists the AI kept reading. Another company’s website was down, and the AI still named it 52 times.
Reviews and business profiles were not what decided it, anywhere in that data.
All three changed their answer, and this time all three said so. Claude moved the business’s own page from fourth to first. ChatGPT moved reviews and profiles to the bottom of its list, and Perplexity put them second to last.
Now the honest part. For ChatGPT and Claude, our own measuring tool supplied the search results from one search service, so part of that pattern comes from our setup and not from them. All three AIs caught that, and they were right to. Perplexity did its own searching. It is one city and one line of work, and we have not published it yet. So we will not call it a rule until we see it with every one of them doing its own searching.
Here is what I take from it if you run a business.
In what we saw, when somebody asked who to hire, the AI did not go count your stars. It searched, it read a handful of pages, and it named the businesses on them. So the question I would ask is which pages the AI reads when somebody asks for what you do in your town, and whether your name is on them.
You can do something about two of those. You can have a plain page on your own site that says what you do and where you do it, in the words a stranger would type. And you can work at getting on the lists in your line of work that the AI already reads.
Ask it a different question, though, and you get a different AI. Nova also asks about every one of our clients by name, things like “Is this business reputable?” and “What do people say about them?” Over the last thirty days that came to 1,095 answers across ten businesses, and 87 out of every 100 of them talked about reviews or star ratings. When the question was who to hire or where to find something, it was about 12 out of 100.
So in our data, reviews were not what got anybody picked. They come in later, once somebody already has your name and is deciding whether to call you. And people ask who to hire long before they ask who is any good, so the order matters. First the AI has to know who you are. Then your name has to be on the pages it reads when somebody asks who to hire. Then your reviews get their turn.
Which leaves me with a question I cannot answer yet. Take a business with no website at all, just a Google profile and a pile of great reviews. Ask the AI who they are, and ask it who to hire. Does it know them? Does it pick them?
These three only changed their minds when we showed them what happened, and one of them still says it did not. So we are not going to ask them that one. We are going to count it.
Nathan. Lefty Media Co, Kansas City.
Lefty Notes goes out every Tuesday morning. Subscribe on Substack and the next one lands in your inbox.
