How do you figure out AI when AI hasn't figured out itself yet

Reputable research says the work does not work. I watched it work anyway.

There is a quiet war going on right now over how businesses show up inside AI, and almost nobody outside of it knows the war is even happening.

On one side you have a stack of files you can put on a website. Things with names like llms.txt and structured data and schema. The idea is simple. You hand the AI a clean, honest, machine readable version of who you are, so when someone asks about you, it has the truth ready to go.

And then the research showed up.

Some of the most reputable, disinterested names in this space went and looked at the actual server data across hundreds of thousands of websites. Their finding was blunt. Almost nobody’s AI files are ever getting read. The bots barely touch them. Ninety some percent of these files get requested zero times. Not rarely. Zero.

Now, one of the loudest voices saying do not bother was Google. And here is where I have to be honest with you, because this is the part most people will not say out loud. Google’s opinion on this one is close to irrelevant. Google owns search. Their AI, Gemini, mostly reads the same old search index Google has always owned. Of course Google is going to tell you that you do not need a new kind of file. A new file that Google does not control is a threat to the thing Google does control. So when Google says it does not work, hear it for what it is. It does not work for Google, because Google is still just doing Google.

But the rest of the research is not Google. It is independent, it is careful, and it has no reason to lie. And it is saying the same thing. These files are barely getting touched.

So I should believe it. I want to believe good research.

Except.

I was sitting in front of a potential client a few days ago. They had ChatGPT and Claude open on their own screens. I asked them to ask a simple question about a company I already work with. Who is this company in Kansas City.

Both AIs answered by reading our words back. Word for word. The exact language we wrote and placed on that company’s site. That is the part we call GEO, which is just whether the AI knows who you are. In that moment, it knew. Cold.

Then I asked them to ask the harder one. Who is the best in this whole category in Kansas City. No name. No hint. Wide open.

My client came up first. That is the part we call Pick Rate. Not just being known, but being chosen. And it chose us.

So sit in that with me for a second.

The best research in the world, the honest kind, says these files do not get read. And I am standing in a room watching two of the biggest AIs on earth read them back word for word and then pick my client as the best.

I cannot fully reconcile that yet. And I am not going to pretend I can.

Here is the honest theory I am working on. I do not think the AI crawls your files ahead of time and files them away. The research is probably right about that. I think it goes and reads your site live, in the second someone asks, and grabs whatever is clearest. So the file may not be getting crawled in advance, but the words in it are still right there when the AI comes looking. Both things can be true. The file sits unread for months, and then the moment of truth arrives and it gets read in real time.

And then there is the other half. When I asked a different AI that same best in the category question, it did not just trust the website. It went and pulled a government safety record. A real, third party source. It wanted proof from outside the business before it would name anyone.

That is the whole paradox in one room. Be clear and you get read. Be backed up by the world outside your own walls and you get picked.

I want to be careful here, because this is the part I actually care about. I am not telling you this to sound smart. We did not crack a code. Nobody has cracked this code, and anybody who tells you they have is selling you certainty that does not exist yet.

What I actually believe is stranger and more humbling than that. AI is not meant to be figured out right now. Not because we are not trying hard enough, but because it is still figuring itself out. It is changing under our feet while we study it. The map is being drawn by something that is also still deciding where the roads go.

So that is what we are really doing. Not standing on a stage claiming we know. Standing in real rooms, watching real moments, holding two true things that should not both be true, and writing it all down honestly as it unfolds.

The research says it does not work. I watched it work. Both are real. That contradiction is not a problem to hide. It is the most interesting thing happening on the internet right now, and we get to live inside it.

Next week: the machine I built to watch the files caught something I was not looking for. The exact moment a stranger asks AI about you.

Lefty Notes goes out every Tuesday morning. Subscribe on Substack and the next one lands in your inbox.