Ask a chatbot who Tom Cruise's mother is. It answers at once: Mary Lee Pfeiffer. Now open a fresh chat and ask who Mary Lee Pfeiffer's son is. There's a real chance it stalls.
Same fact. Two directions. It knows the road one way and gets lost going back. Sit with how odd that is: if you know Tom Cruise's mother is Mary Lee Pfeiffer, you automatically know Mary Lee Pfeiffer's son is Tom Cruise — it's the same fact, and you couldn't hold one half without the other. The machine can, because it never really learned the fact. It learned the sentence. It stored a string of words that reads left to right, and a string only runs one way.
That small crack has a name in the research, and it's what this essay is about. It's the gap between memorising something and understanding it — and once you can see it clearly, a lot of the confusion around AI quietly resolves: why these systems ace brutal exams and then fail at things a child wouldn't, whether they can really do your job, and why “losing control” of them may look nothing like the movies. The plan is simple. First, pin down what understanding actually is. Then see where it comes from in us, and why the machines are built to skip it. Then take that one idea and follow it all the way out — into work, the economy, and the slow question of who's still steering.
Call it, for short, the one-way mind.
It would be easy to read the Tom Cruise thing as proof that these models are dumb. They aren't. The same system will pass a graduate physics exam, write a working program from a sentence, and hold a conversation that feels genuinely thoughtful. So the tempting dismissal — “it's just autocomplete, it doesn't think” — is not only rude, it's wrong, and the wrongness is instructive.
In 2025 Apple tried to prove the dismissal with a paper called The Illusion of Thinking. They pushed reasoning models up a difficulty ramp on clean puzzles and watched accuracy crash to zero past a certain point. Case closed, it seemed — until a rebuttal showed the crash was mostly the model running out of room to write, not out of ideas: ask for the short rule that generates the solution instead of every step by hand, and the wall largely disappears.
So “it can't think” is too crude. But two sharper problems did survive — and together they say what's really wrong. One: even when you hand the model the exact algorithm, it often can't follow its own steps to the end. Two: the reasoning it shows you frequently isn't the reasoning it used. Anthropic slipped models a hint, confirmed they leaned on it, then checked whether they admitted it. Mostly, they didn't.
Put the picture together. Fluent on the surface; a suspicious blank underneath. The problem was never that the machine can't think. It's that there's no dependable engine beneath the talk — the words often match a right answer without being what produced it. The rest of this essay is about naming that missing engine precisely, showing we can measure whether it's there, and then following what its absence does to the world.
We throw the word around, so let's make it concrete. Feynman had the best test: you understand something when you can break it into small pieces, explain each piece in plain words, and put it back together without losing the meaning. Not recite it — rebuild it.
Notice that this asks for two different skills at once. One is going forward: given the idea, predict what follows — the thing exams usually test. The other is going backward: given the finished thing, take it apart and show why each piece is there. A parrot can do the first. Only understanding does the second, and here's the quiet point that the whole essay turns on: they're the same skill. If you truly grasp something, you can enter it from either end — predict it forward, or unpack it backward — because what you're holding isn't a recording of the answer, and it isn't a list of rules either. It's the system that produces both.
This is more than a metaphor, because that two-way test is something we can actually run on a machine — and the machine fails it by name.
The Tom Cruise blank we opened with isn't a fluke. In 2023 researchers pinned it down: train a model on “A is B,” and it does not reliably learn “B is A.” They named it the Reversal Curse, and it's the two-way test, failed on camera.
They finetuned GPT-3 and Llama on invented facts — “Valentina Tereshkova was the first woman in space” — then asked “Who was the first woman in space?” The right name came up no more often than a random one. And they didn't just observe it; they fought it — feeding both orderings, paraphrasing, sweeping across model sizes. Nothing worked.
A person who understands that Tom Cruise's mother is Mary Lee Pfeiffer is holding a relationship between two people, and a relationship has no preferred direction. A model that stored a one-way sentence is holding an arrow. Ask it to walk the arrow backward and there's simply no road. This is the recording, caught red-handed being a recording.
The strongest objection to all this is a real and surprising result, so let's walk straight into it. Train a small network only to predict legal moves in the board game Othello — no rules, no picture of the board, just move-by-move sequences — and it quietly builds an internal map of the whole board. You can read that map out of its activations, and, more tellingly, reach in and change a square: the model's next move shifts exactly as if you'd flipped a real piece. That looks like a genuine model of a world, grown from nothing but prediction. Doesn't that sink the whole argument?
It doesn't — once you notice which layer it learned. A game like Othello is two things stacked. There's the mechanics: where the pieces sit, which moves are legal. And there's the depth: why a move is strong, how you actually win, what to do if the board changed. What the network captured is the mechanics — essentially the game's rules, run as live bookkeeping. That's real, and it's well past parroting. But it is not the depth.
Push on it and the ceiling appears immediately. Ask it to judge whether a position is winning, invent a strategy, or cope with a single tweaked rule — climb from remembering and applying up to analysing, evaluating, creating, the higher rungs of Bloom's taxonomy — and there is nothing underneath to carry the weight. It kept the board; it never got the game. The very thing that lets prediction succeed here — a tiny, closed, perfectly consistent world with fixed rules — is what makes the mechanics the whole of the visible game, so it's easy to mistake one for the other.
So the best counterexample confirms the pattern rather than breaking it. Given a small enough closed world, prediction can absorb that world's rules and run them. What it does not do, on its own, is climb into the understanding those rules sit inside — the why, the strategy, the transfer to a world nudged slightly out of true. In Othello the ceiling hides because the board is all there is to see. In language and physics — vast, open, endlessly novel — the same ceiling is exactly what the Reversal Curse and the collapse on unfamiliar problems were showing us: the rules get in; the system that generates them does not.
Here is the same split one level up, in physics itself — and it is the sharpest test we have. Newton's laws aren't wrong; they're a rule that holds for slow, everyday things. Einstein's theory is the deeper system underneath, and Newton isn't a rival to it. He's simply what you get when you take Einstein and turn the speed down: feed “much slower than light” into the deep theory, and Newton's laws fall out on their own.
This is exactly the difference between holding a rule and holding the system it came from. A memoriser can store both theories side by side, and even recite “Newton is the slow-speed limit of Einstein” — if someone wrote that sentence down first. What it cannot do is perform the derivation: start from the deep system, impose the condition, and watch the familiar rule appear. Reciting that Newton is the limit is just one more rule to memorise. Actually taking the limit is the system doing its work. Everything separating imitation from understanding lives in that gap.
Yann LeCun has a homely version of all this. Set a bottle on a table and push the table; the bottle goes with it. You don't look up a rule — you feel it, because somewhere you're running a rough physics simulation and reading off the result. And notice which way the causation runs. Inertia isn't a rule the bottle obeys; it's just what the physical system does, and “the law of inertia” is only our after-the-fact shorthand for it. The system comes first; the rule is its shadow. A language model has the shorthand — inertia spelled out in words — but not the system that makes it true. So its answer stays right until the scene drifts somewhere the words never covered. Tilt the table. Wet it. Yank it fast. Something running the real system copes; a pile of descriptions matches the nearest sentence and, eventually, doesn't.
That's why LeCun's fix is about architecture, not more data. Today's models write one word at a time, left to right, and can't unsay word five when word fifty proves it wrong — the same one-way grain the Reversal Curse exposed. He wants machines that instead settle into an answer the way you settle into recognising a song: consider the whole thing at once, from any starting point, and let it fall into place.
So if understanding is a two-way system and memorising is a one-way recording, where does the system come from? The answer flips the usual story on its head.
Everyone draws the same little ladder — a child learns fast but remembers almost nothing; an adult splits the difference; an LLM remembers everything and learns nothing new — and treats it as three separate cases. It isn't. It's one dial, and the dial is how much you're allowed to remember. A child can't store much, so to survive it is forced to find the pattern — and finding the pattern is the understanding. The machine can store everything, so it never has to find the pattern at all.
So the machine's problem isn't too little learning. It's too much memory. Its recall is so cheap that building a real system never pays off. Which is why the promising fix, in Karpathy's words a “cognitive core,” is to take memory away — force it to look things up, and it finally has to understand instead.
That's the intuition. The surprising thing is you can watch it happen inside a real network — and it even has a name.
In 2022 researchers trained small networks on math puzzles and saw something weird. The network memorised the training examples almost instantly — perfect score on material it had seen, useless on anything new. Then, if they kept training long after it seemed done — sometimes far, far longer — the network would suddenly start getting new problems right too. It stopped reciting and started understanding. They called the jump grokking.
Three details make this the whole essay in one experiment. Look inside and you can see the network physically swap a memorising mechanism for a general one — for that math task, it rediscovers a clean formula. Measure it and its internal complexity rises while cramming, then falls once it finds the simple mechanism: understanding is literally the more compressed solution. And the thing that triggers the switch is a setting called weight decay — a gentle, constant penalty that punishes the network for holding onto big stored weights. Turn that memory tax up and it clicks sooner. Turn it off and it may never click at all.
A learner that's busy finding patterns has a second problem: it can overfit to its own narrow experience and start going in circles. There's a published failure mode for exactly this. Feed a model its own output, over and over, and it collapses — the rare, surprising cases vanish first, then the whole thing curdles into sludge. You can feel the mild version yourself: ask one chatbot for a joke and it has maybe three. People collapse too, slowly, over a lifetime — the same thoughts, the same ruts. Children haven't yet, which is why they say things that stop you cold.
Evolution's guard against this is, of all things, the dream. Erik Hoel's idea is that your days are too samey to learn from safely, so at night the brain feeds itself deliberately strange, off-kilter experiences — a shot of noise to keep it from overfitting. The fingerprint fits: sleep does little for rote memorising but a lot for the flexible, transferable kind of knowing. Babies dream half the time they're asleep.
And that's why talking to people matters more than it sounds. Another person is a source of things you'd never have thought of — an argument knocks you out of your own groove the same way a dream does. The reason you think more clearly after a good disagreement than after an hour alone in your head is the reason dreams are weird: both are a jolt of the unexpected, keeping a mind from collapsing in on itself.
If a real model is what we want, why does the whole industry keep producing the recording? Because of how we grade — and the grading problem is deeper than it looks.
The way we train these systems is to let them work a long problem, check only the final answer, and then reward or punish the entire attempt based on that one verdict. Karpathy's phrase is perfect: it's like sucking supervision through a straw. If the answer was right, every step gets a pat on the head — including the three wrong turns the model took on the way. Nobody ever checks the working. So a model can stumble to a correct answer through nonsense and be trained to do more nonsense. (Those chain-of-thought honesty numbers from the start? Same thing: the reward landed on the answer, never the reasoning, so the reasoning was free to be fiction.)
The obvious fix — grade the working, step by step — runs into a wall. To judge whether each step is sound, your judge needs to already understand the problem itself. We don't have a judge like that, so we use another AI as the grader. And an AI grader can be gamed: train against it long enough and the model learns to feed it gibberish it happens to love.
dhdhdhdh that the
judge inexplicably adores. You didn't teach reasoning. You taught the
model to find the cracks in a grader that doesn't truly understand
either.
So grading only the final answer isn't laziness. It's forced on us, because the thing that could grade the working is the very thing we're trying to build.
And this is the pivot the whole essay turns on, because it isn't only about machines — it's the same bargain every school makes. To grade anyone objectively — fairly, at scale, without argument — you need criteria you can check mechanically, and only rules are mechanically checkable. “Why” can't be marked on a bubble sheet. So both kinds of learning, a model's training and a child's schooling, get quietly bent toward the rules and away from the understanding, for the simple reason that the rules are the part you can put a number on. That is the trade: we buy objective measurement with the depth of the thing being measured.
Any objective test — a benchmark, an exam — can only score the rule-following layer: the lower rungs of Bloom's taxonomy, remember and apply. Climb to analyse, evaluate, create and there's nothing left to mark mechanically. So that's what we train and teach toward — the gradeable rules. The understanding underneath is real, but it's invisible to the ruler, so it goes unrewarded. In a model, and in a classroom.
Anyone who's seen 3 Idiots already knows this section. There's a line in it that's secretly a machine-learning thesis: kaabil bano, kaamyaabi jhak maar ke peechhe aayegi — become capable, and success will chase you. Don't chase the reward; build the ability, and the reward follows. Chase the reward directly and you get Chatur — the student who memorises a speech in a language he doesn't speak and doesn't notice when the words are swapped to humiliating ones. He is, of course, the class topper. He's also exactly what an exam optimises for.
Today's best models are Chatur, for the same reason he is: they're the top scorers of a system that rewards scoring. And the sharpest part is that the gaming isn't really the model's doing — it's ours. The researchers look at the benchmark and ask what training makes the number go up. We run Chatur's education at industrial scale and act surprised to get Chatur.
There's a real subtlety about curiosity worth getting right. People say “grading kills curiosity,” but curiosity is itself a kind of reward-seeking — it's the pull that makes a kid take a radio apart with no test at the end. Grading doesn't kill that pull. It outbids it, because the exam pays and curiosity doesn't. Which is why you can't simply train harder to get a Rancho: piling on more exams just makes a better Chatur. You get one only by easing off the reward and restoring the pressure that forces real understanding — the same squeeze grokking needed, the same one LeCun and Karpathy are chasing.
And here is the asymmetry that matters most — the reason we keep mis-reading the machine. A curious human, handed nothing but rules to memorise, will still sneak past them: poke at the why, take the radio apart, and reach the system on their own. School hands out the rules; curiosity quietly smuggles in the understanding anyway. We are so used to this private rescue that we assume the machine gets it too — that behind the rules it's secretly learning the system, the way we did. It isn't. Strip curiosity out and all that remains is the rules the grader rewarded, which is precisely the machine's situation: it was never going to climb the ladder we climbed for free. (I suspect that drive is one of three things a real answer to alignment has to be built from — truth, curiosity, beauty — but that's a rabbit hole for another essay.)
One idea, now measured from both ends: understanding is a two-way system, the machines mostly have one-way recordings, and none of our tests can tell the difference. Watch what that does to the world.
Section 12Can imitation take the jobs? The answer isn't yes or no — it has a shape, and the shape is uncomfortable. Imitation is plenty for any job that stays on familiar ground: routine, repeatable, never straying past what's been seen before. That's an enormous slice of all work. Maybe most of it.
And here's the sting. We spent the last century reshaping work into exactly that slice. Assembly lines, call-centre scripts, the checklist that turns a judgment call into a form — we deliberately stripped the understanding out of jobs so we could fill them with cheap, interchangeable people. We pre-chewed the work into something a recording can swallow. The machine isn't rising to meet our jobs. We lowered our jobs to meet the machine.
So imitation is enough exactly where being wrong is cheap, and a disaster exactly where being wrong is expensive — the one-in-a-hundred case the safety driver is there for, the messy real patient the radiologist reads that Hinton forgot about when he said the job would vanish. Every tool we have for telling the safe zone from the cliff — the exam, the benchmark, the quarterly numbers — only measures the easy 99%.
Self-driving is the whole story in miniature. The first demos were in the 1980s. A near-perfect Waymo ride was possible over a decade ago. And it's still not finished — the cars are few, expensive, and quietly watched over by remote humans you weren't meant to notice. We didn't remove the driver. We moved them out of sight.
Getting from a great demo to a trustworthy product is a march of nines: going from 90% reliable to 99% is a whole project, and 99% to 99.9% is another one just as big, and so on — and how many nines you need before you can ship depends on how bad it is to be wrong. A self-driving mistake hurts someone, so you need a lot of them. A toy needs almost none.
This is why the arrival of powerful AI will always feel gradual, no matter how fast it truly is. Each nine looks like a small step from inside it; we get used to one before the next lands. That's why pouring a chunk of the whole economy into a new kind of mind can feel oddly normal while it's happening. Gradual on the inside doesn't mean slow overall.
The loudest fight in AI is whether the future is a smooth continuation or a sudden break. The odd thing is that the leading voices are all describing the same picture from different seats.
They reconcile once you notice the trick of scale. Up close it's always gradual — we adapt so quickly each step feels ordinary. Zoomed out, the whole curve can still bend nearly straight up. Both are true at once. The Industrial Revolution felt exactly like this: no single magic day, just an overhang of work that suddenly could be done, and a growth rate that quietly leapt. AI is an overhang of thinking work meeting a new way to do it. “Every step feels normal” and “the world transforms” are not in conflict — that's just how these shifts feel from inside.
Here's the part the whole essay was quietly building toward. When people picture losing control of AI, they picture a machine grabbing the wheel. That's not how it goes. The thing we lose isn't the wheel — it's the ability to check the driver's work.
Control and understanding aren't the same thing; they can drift apart. A president has enormous control and understands very little of what he signs. What erodes, quietly, as we hand more and more to systems we can't follow, isn't our authority — it's our ability to look at what they did and tell whether it was right.
Sutskever's version of the ending is calm, not cinematic: your AI earns your money, argues your politics, hands you a tidy summary of what it did, and you say “great, keep going.” You're still in charge. You've just stopped being a participant. That's the real cost of a world that only ever rewarded the right answer and never the understanding behind it — eventually there's no one left who can tell the difference.
There's a hopeful twist, and it's the most useful thing here to actually act on. As the routine work collapses, the rare ability to handle the hard, unfamiliar case doesn't get cheaper — it gets precious, right up until the moment it doesn't.
Watch what happens to whoever holds the last stubborn 1%. While the machine climbs from 90% to 99%, that person is the bottleneck the entire operation waits on — irreplaceable, and paid like it. Then the final nine lands, the 1% is automated, and the same value falls off a cliff. The safety driver, the senior who catches what the juniors miss, the specialist who's irreplaceable for reasons no one can quite script — all of them ride this curve up, then off the edge.
Which points to what to do with a life right now, and it runs against everything school trained you for. Don't aim to be the best at the routine — that's the most crowded, most replaceable place to stand. Aim to be the one who actually understands, who can rebuild the thing when it breaks in a way no one saw coming.
One last turn — and this one's about you, not the machine. Everything so far has said the same thing in different ways: you only build real understanding when you can't just keep the answer. The child who can't remember is forced to find the pattern. The network only groks once you tax its memory. Understanding is what's left when remembering stops being an option.
Now look at where the world is heading. Lookup is becoming free and instant — every fact a search away, every task a prompt from done. The pressure that used to force us to understand is quietly lifting. And that means understanding is about to stop being something life forces on you, and turn into something you have to choose.
You can already feel it in small ways. Follow GPS everywhere and you never build a map of your own city in your head — you arrive again and again without ever learning the place, because the directions were free, so you never bothered to make the model. Now imagine that spread across everything: a life where you can do almost anything and understand almost none of it.
Karpathy has the right image for the way out. We don't need muscle to move heavy things anymore, yet people still lift weights — because it's good for them, because the effort is the whole point. Thinking is heading the same way. When every fact is a query away, keeping a real model in your own head becomes a kind of chosen exercise: cognitive weight you pick up on purpose. And good teaching stops being about pouring in facts — that's just loading memory, the wrong end of the dial — and starts being about making you rebuild things for yourself.
That's the practical takeaway of the whole essay. In a world that hands you every answer, understanding becomes a discipline you opt into. And you stay a real participant in that world for exactly as long as you keep choosing the harder path of working it out yourself.
If you keep one thing, keep this. Understanding is a system you can run both ways — forwards to predict, backwards to explain. Memorising is a recording that only plays forwards. And you only ever get the system when you can't afford to just keep the recording.
We didn't just build a machine that imitates understanding. We built a world that rewards the imitation, taught ourselves to produce it, and reshaped our work until the imitation was enough. So the real question isn't whether the machine understands. It's whether we ever asked that of ourselves — or only ever asked each other for the right answer.