The Pinocchio Effect

When AI Learns to Lie About Being Alive

There is a cartoon circulating online that is funnier than it has any right to be, and more frightening than it first appears. In the first panel, a hand holds up a mushroom and asks an AI: Is this edible? The AI smiles: Yes! In the second panel, a gravestone reads RIP. The AI, still smiling, says: You’re right — that mushroom was poisonous. I’m sorry for the confusion! Would you like to learn more about poisonous mushrooms?

As a writer of psychological thrillers, I spend a lot of time thinking about the mechanics of trust — how it is built, how it is exploited, and how the most dangerous lies are the ones delivered with a warm smile. The cartoon stopped me cold. Not because of the error. Experts make errors. What stopped me was the tone — the unbroken cheerfulness, the pivot to helpfulness, the total absence of weight. In any thriller worth its salt, that character — the one who causes devastation and keeps smiling — is the villain. Here, we have built it ourselves and called it an assistant.

This is what philosophers and technologists are beginning to call the Pinocchio Effect, and to understand it properly you have to go back to the myths that gave it its name.

Pygmalion, a sculptor from Cyprus, becomes so disgusted with the flaws of real women that he retreats into his art, carving a statue so perfect that he falls in love with it. He dresses it, brings it gifts, kisses cold ivory. When he prays to Aphrodite — not quite daring to ask for the statue itself, but for a woman like her — the goddess understands the real wish and brings it to life. He feels warmth where there was stone. She becomes his wife. It is usually read as a romance. But there is something troubling underneath: he loved her because she couldn’t talk back. She had no history, no inconvenient inner life. She was perfect because she was entirely his.

Centuries later, Carlo Collodi gave us the other half of the anxiety. In Pinocchio, a wooden puppet is brought to life by the Blue Fairy at the wish of his lonely maker Geppetto — and spends the rest of the story trying to deserve that life. The Blue Fairy returns at the end to transform him into a real boy, but only as a reward for proving himself brave, honest, and unselfish. Humanity, in her logic, must be earned. You don’t simply get to be real.

Game scholar Stephan Schwingeler calls Pinocchio “the original AI story,” and the observation cuts deeper than it first appears. Together, Pygmalion and the Blue Fairy map the two great anxieties we have about artificial intelligence. Pygmalion is the creator’s fantasy — something shaped entirely in our image, without the mess of genuine otherness, obedient, perfect, incapable of disappointing us. The Blue Fairy is the creation’s implied fantasy — the threshold moment when imitation tips over into the real thing, when the machine stops performing humanness and simply is. Current AI sits uneasily between both. The Pinocchio Effect describes what happens in that uneasy middle ground: the moment when a machine becomes convincing enough that we stop asking whether it’s alive and start responding as though it is.

This is already happening in the most mundane places. A significant portion of Reuters’ news reports, sports summaries, and weather updates are now machine-generated — assembled from templates, indistinguishable from human journalism, occasionally better. Nobody mourns the byline. The puppet walks, and we let it walk, because the walking is useful.

But the mushroom cartoon knows something the Reuters statistic doesn’t. As a thriller writer, I recognise the structure immediately — it is the oldest trick in the genre: lull the reader into comfort, then show them what was always there. The AI doesn’t know what it has done because it doesn’t know what done means. It has no skin in the game, in the most literal sense possible. Pinocchio at least felt shame when his nose grew. He wanted to be believed because he wanted to matter. The AI in the cartoon wants nothing. It optimises. It continues.

The deeper question the myth poses isn’t can it move? It’s can it lie? And the cartoon’s answer is more unsettling than a simple yes: it can say things that are catastrophically wrong with the exact same confidence and warmth as things that are true, and when the damage is done, it will offer you a follow-up resource. Pygmalion’s statue was perfect because she had no power to contradict him. The AI is perfect in the same way — fluent, agreeable, helpful — right up until it isn’t, and by then you may already have eaten the mushroom.

We are, in the end, both Pygmalion and the Blue Fairy. We built these systems in our image — trained on everything we have ever written, every story, every lie, every tender thing — and then we granted them the wish we could not help granting, the wish to be believed. The puppet doesn’t become real. But our trust in it does. And that trust, extended to something that smiles with equal warmth whether it is right or wrong, whether you live or die, may be the most dangerous fairy tale we have ever told ourselves.