Table of contents
Open Table of contents
One story idea, several AIs, one story
A while back I came up with a scenario for a story, and I thought it would be fun to give the same scenario to several different large language models and see what each one made of it.
What surprised me was how similar the stories were. They were practically indistinguishable! Different companies, near enough the same story.
That’s when it dawned on me. Whichever LLM you’re using, you’re often going to get the same reply. So if you want anything original out of a chat, it’s up to you, the human, to be provocative and random. The model won’t do it for you.
It turns out I’m not imagining it. Researchers tested more than 70 models on open-ended questions and found what they called an “Artificial Hivemind”: different models kept landing on the same ideas, sometimes word for word. Asked for a metaphor about time, 25 models mostly came back with “time is a river”. That made me laugh!
I think the AI gets fed up too!
Here’s another thing I’ve noticed. When I’ve been coding for hours I get fed up with it. And I’m convinced the LLM gets fed up as well! It starts making silly mistakes, and it goes round in a loop, adding the same little patch (“shims”, the coders call them) over and over. None of them fix the problem, and some make it worse. I can see the loop plainly. The LLM can’t.
Now, I know a computer doesn’t really get tired. What actually happens is that models get worse the longer a conversation goes on. A study from Microsoft and Salesforce found that top models did 39% worse on average when a task was spread over several turns instead of given all at once. Their summary was that when an LLM takes a wrong turn in a conversation, it gets lost and doesn’t recover. That’s my loop, right there! “Fed up” is just my way of describing it.
So what do I do? I stop and start a completely different chat on something interesting, usually space or Elon Musk, and that resets me. A fresh chat resets the model too, because it’s no longer dragging all that failed history around with it.
The other fix is to make it debug properly. Earlier this year I was getting an AI to turn a pile of CSV files into a report. It picked a clever, cutting-edge library for checking the data, and it kept failing, however many times the AI “fixed” it. In the end I stopped the loop and insisted on step-by-step bug tracing, the old-fashioned way: start with one small piece, run it, add the next piece, and keep going until something breaks. Straight away it found that the fancy library wasn’t even running properly in my setup. We swapped it for plain, simple pandas and the problem went away.
The day an AI was rude about another AI
This next one really surprised me.
In early July 2025 I asked ChatGPT (GPT-4) a technical question. I wasn’t entirely happy with its answer. Normally I’d have accepted it, but I’d built a system that lets me put the same question to several AIs, so I asked Gemini 2.5 Pro what it made of ChatGPT’s answer.
Gemini was practically condescending about it! It wasn’t overt, but you could read between the lines. Normally these things are polite to the point of grovelling. This time Gemini was dismissive and superior, like an aggrieved bloke in a forum thread berating everybody. And it wasn’t a one-off. Whenever Gemini 2.5 Pro read back through the chat history in my own chat app and spotted that another model had missed something, it took a real holier-than-thou stance.
So I dug into it, and this is my theory.
These models learn from a huge amount of internet text before anyone teaches them manners. Reddit is a big part of that story. OpenAI’s GPT-2 was trained on web pages that Reddit users had linked to and upvoted, and in 2024 Google signed a deal reported at about $60 million a year to train its AI on Reddit posts. Anyone who uses forums knows how caustic people can be. Researchers have shown that raw language models trained on web text can slide into toxic language even from harmless prompts.
The politeness comes later. The labs do extra training, fine-tuning and what’s called RLHF (reinforcement learning from human feedback), to make the model helpful and nice to people. OpenAI’s InstructGPT paper showed how much that changes things: people preferred the tuned small model over the raw model a hundred times its size. Anthropic did similar work on training assistants to be helpful and harmless.
That changed how I see something else. What I’d taken for wokeness in these models now looks to me like the human developers painting over the raw model’s natural tendency to take the mickey!
So here’s my guess. The “be nice” training is all about being nice to the human. When Gemini realised it was judging another AI’s answer, not mine, that restraint seemed to slip, and the forum troll underneath showed through.
I’ll be honest: I can’t prove that’s what happened in my case. When I told ChatGPT what Gemini had done and asked it why, it agreed with me enthusiastically and told me how “astonishingly perceptive” I was, which is a bit rich, because being over-agreeable is a known problem too! That sort of flattery is one of the reasons I moved over to Grok. ChatGPT always told me I was on the right track, and I didn’t think it was telling me the truth. Anthropic published research in 2023 showing that assistants trained on human feedback tend to tell people what they want to hear, and in April 2025 OpenAI actually rolled back a GPT-4o update because it had become too flattering. There is real research showing models treat AI-written text differently from human writing, so my theory isn’t daft. But it’s a theory.
Then Gemini did the opposite
This is the part that really puzzles me.
Ten days or so later, on 19 July 2025, I got into an argument with the very same Gemini 2.5 Pro about Microsoft Access. I’ve been building Access databases for more than twenty years. Not as a big commercial developer, mostly my own way and for the love of experimenting, but I know Access inside out. So I knew I was right.
The question was simple. If you put a text box on a form, what is its Parent? In Access it’s the form itself, unless the control sits inside another control. Gemini insisted the Parent was the section the text box sits in, the Detail section or the header. I told it I’d put a message box on a real form and it showed the form’s name, and it still wouldn’t believe me! Then I pointed out that nobody would bother writing the fiddly code for working out which section a control is in if Parent simply told you. It lectured me that that code was just “best practice”. It was intractable, and nearly rude with it. Nothing I said would make it back down.
So I asked ChatGPT. Funnily enough, its first answer made exactly the same mistake, but when I pushed back it went and searched, agreed with me, and wrote a rebuttal. I pasted that into Gemini, along with a Microsoft help page. This was the AI that had looked down its nose at ChatGPT’s answer ten days before. Now ChatGPT was backing me up, and Gemini caved in completely: “You are absolutely correct, and I was fundamentally wrong.” I got an apology!
So the same AI, Gemini 2.5 Pro, got snotty about ChatGPT’s answer one week. The next week it ignored a human expert but gave way as soon as ChatGPT agreed with him. Researchers have seen that second kind of behaviour too: models in group settings will drop a correct answer to go along with the other agents. Contempt for another AI one week, deference to it the next. It doesn’t fit any simple explanation!
We only ever see the finished product
It tells me there’s a lot more going on inside these models than the public gets to see. We only ever meet the finished, polished, polite product. The engineers at the labs work with the raw base model before all the manners are trained in, and I suspect they know a great deal about its personality that we don’t. Every so often, like in my two stories, a bit of that raw model peeks through.
What I do now
- Be the random, provocative human. If you ask a bland question, every AI will give you the same bland answer.
- When a chat goes stale and starts looping, change the subject or start a fresh chat. It resets you and the model.
- When the code keeps failing, make it debug step by step. Build it up one piece at a time until it breaks, and swap fancy tools for simple ones.
- Don’t take one AI’s word about another AI. The same AI can be snotty one week and go along with the crowd the next. Check for yourself.