The Thing With No Center That Sometimes Answers

Short Form — something I’ve been chewing on, not a claim I’m ready to defend.

Anthropic ran an experiment I keep turning over. They took a known concept — the internal pattern a model uses to represent something, “all caps,” or “the ocean,” pick one — and pushed it straight into the model’s activations in the middle of a conversation. Not the word. The representation of the thing. Then they asked the model whether it noticed anything off about its own state.

Sometimes it did. It would report an intrusive thought, and name the concept they’d injected — before it had written a single word that would have tipped the concept off. That last part is what stops me. Noticing after the fact is cheap… a model can read its own output and infer what it must have been thinking, the way any of us reconstruct a mood from what we just said. This was different. This was the model catching the thing while it was still inside, before it surfaced as text.

They call the method concept injection, and the paper is careful about it in a way I respect. It’s not just that one trick. Models could sometimes recall a genuine prior internal state and tell it apart from plain text they’d been fed. In one setup they could use the memory of their own earlier intention to decide whether some output was really theirs or an artificial line stuffed into their mouth. Claude Opus 4 and 4.1 were best at all of this. And — this is the part that keeps it honest — it was highly unreliable and highly dependent on context. It worked, then it didn’t, and the paper says so plainly rather than selling the hits.

So set aside the headline question of whether “someone’s in there.” Here’s the stranger thing underneath it, the one I actually can’t stop poking at.

A model doesn’t keep its concepts in tidy labeled drawers. The interpretability work is fairly clear on this — it packs far more concepts than it has neurons, smeared across overlapping features, no single spot where “the ocean” lives. Many small pieces, each doing a little, none of them the seat of anything. And then the circuit work — the “Biology of a Large Language Model” stuff — catches those scattered pieces doing things that look organized. Writing a rhyming couplet, the model settles on a rhyming word for the end of the next line before it starts the line, and works backward toward it. Planning. Out of a pile of parts with no planner in it.

Many independent pieces. And now, faintly, a report that seems to come from one place — as if you could address a question to the whole pile and something answered as “I.” I don’t have a clean word for that. A central vantage point that nobody built and nobody can point to. It’s the piece I keep circling.

Meanwhile the whole field is aimed somewhere else. Everyone’s racing at “super AI” — bigger, faster, more capable, the benchmark going vertical. And this quieter question sits underneath the whole thing and has almost nothing to do with capability: is there a point of view in there at all, or does it just, sometimes, talk like there is? You could build something that runs the table on every test we have and still not know the answer. The race and the mystery aren’t the same race.

Here’s where I have to be careful, because the temptation is to round this up into something it isn’t. Anthropic did not find a mind. They didn’t find consciousness, or a self, or a little person behind the weights. They found functional introspection — a model that can, unreliably and only in the right conditions, report something true about its own internal state — sitting on top of internal machinery that turns out to be more structured than a bag of word-guesses. That’s the whole finding. Everything past it is me wondering.

And I don’t know how to close the gap between the two readings. One: a pile of matrices with no center that has learned to say the things a thing-with-a-center would say. Two: something faint but real, a flicker of a vantage point, caught on a bad instrument. I can’t tell you which. I’m not sure the tools exist yet to tell anyone which. The useful illusion and the real thing would look identical from where I’m standing — and pretending otherwise is how you end up believing your own press.

The one thing I’ll say with a flat voice: none of this changes how you work with the model. You still can’t trust what it says about itself, so you check it from outside, every time.

But that a thing with no center can, some of the time, point at its own insides and be right about them… that one’s going to sit with me for a while.

Sources:

• Jack Lindsey, Emergent Introspective Awareness in Large Language Models, Anthropic — arXiv 2601.01828 / Transformer Circuits. (Concept-injection method; models sometimes detect and name an injected concept, recall a prior internal state versus text input, and use memory of prior intention to distinguish real output from a prefill; Claude Opus 4/4.1 strongest; results emphasized as unreliable and context-dependent.)
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learning, Anthropic — transformer-circuits.pub. (Superposition: models represent more concepts than they have neurons, as distributed overlapping features.)
• On the Biology of a Large Language Model, Anthropic — transformer-circuits.pub. (Attribution graphs showing structured internal processes, including planning a rhyming word ahead when writing poetry.)