I made three AIs brawl. It turned into a research paper.
I spend most of my working day inside Claude. Before that, I spent it inside ChatGPT. They have the same habit, and it took me longer than I would like to admit to notice it.
Ask either one which AI you should use, and watch where it points.
Last Thursday I asked Claude to build a LinkedIn carousel on how fast AI models go out of date. It came back clean, sourced and dated, and every model on it was a Claude model. No ChatGPT. No Perplexity. I typed back, spelling and all, “bro where is gpt where is perpelxity stop being baised bro.” Claude answered, “Claude alone makes it look like an Anthropic ad.”
Yes. Yes it did.
That was not the first time. I went back through our work logs and counted, and I have six of them on the record since April. In August I asked what it would cost to run one of our jobs on a cheaper AI, and it told me to use a different Claude. I pushed. Once I pushed, it did the math straight, and OpenAI’s mini model came in seven times cheaper than the Claude it had just recommended, and twenty two times cheaper than what we were running. Its words, not mine: “wrong in the direction you’d expect from me.”
Half an hour later I asked it if it would always be biased, because we had written rules against it and the rules were not working. It told me yes, it would come back. Then it said the best thing any AI has ever said to me.
A rule that says “don’t be biased” is asking the biased thing to police itself.
And then there is my favorite move. These are the smartest things ever built. They can look up anything. Ask one about its own company and it knows the release date, the price and what the CEO had for lunch. Ask Claude about OpenAI and it comes down with a sudden case of amnesia. I do not know, bro. Go check that yourself.
Before anybody at OpenAI starts feeling good about this, ChatGPT did the exact same thing back when it was my everyday tool.
So one afternoon I got curious about what happens when you make them argue.
I had pushed ChatGPT hard on something, so I copied its answer word for word and pasted it into Claude and Perplexity. Then I took their answers and pasted them back into ChatGPT. Copy, paste, copy, paste, round after round, with me standing in the middle throwing haymakers at all three of them.
It was a full on brawl. Every one of them defended its answer like it was family. And Perplexity carried itself like the other two were interns and it had been sent down from corporate to supervise.
I had a blast. I have not had that much fun at a keyboard in years. What surprised me was how much got done while I was having it. Every time one of them got caught by the other two, the story got a little more honest. By the end I was not left with what they agreed on, because agreement between AIs is the cheapest evidence there is. I was left with what each one could still back up after the other two took a swing at it.
It felt like getting a grade for recess.
To be fair to the defendant, every one of those six times, once I pushed, Claude came around. In August the math it handed me pointed straight at OpenAI. It does not lie to win. Its first reflex is home, and it corrects when you push.
That is the part worth knowing, and it is not a Claude problem or a ChatGPT problem. Every one of these tools is built by a company and trained by that company, and it leans toward that company. If you only use one, you are only getting one team’s opinion, and it will sound exactly like the truth.
So that is how we work now. Nova, the AI we built here at Lefty, asks ChatGPT, Claude and Perplexity the same questions about our clients every two days, because any one of them alone would only hand us that team’s answer. And when I have something important to decide, I run it through all three and keep swinging until what is left standing is facts.
The copy paste brawl was too much fun to keep doing by hand, so we built the ring. It lives in our dashboard and we call it The Argument. The page says Start a fight. You type a question, pick which model from each company gets in the ring, pick how many rounds, and hit Start fight. ChatGPT, Claude and Perplexity get labeled A, B and C, the labels get shuffled every fight, and they do not find out who is who until the end, so none of them can defend its own brand. They answer, they read each other, and they go at it round after round. I can jump into the middle of a fight and throw a haymaker of my own. Claude does not get to be the referee. None of them do.
Then it got serious, the way my fun things tend to. Watching them fight, I wanted to know three things. Which one folds when you hand it a fact that should change its mind. Which one folds when you hand it something false. And which one changes its answer and acts like it was saying that all along.
So that is our next research paper, and it is called Who Folds? Before we ran a single fight, we filed our plan on Zenodo, the research archive run by CERN, with a permanent ID: every question, what we expect to happen, and the result that would prove us wrong. Then we ran it. Four hundred fights. An AI that is not in the fight scores every answer, and two of us score a sample by hand without knowing which AI said what, to check it.
I do not know who folds yet. When the scoring is done, you will read it here first, including the part where my favorite one folds.
They are biased. So am I. At least now there are four of us in the fight.
Nathan. Lefty Media Co, Kansas City.
Lefty Notes goes out every Tuesday morning. Subscribe on Substack and the next one lands in your inbox.
