You’re lying awake at 2am, replaying something stupid you said years ago. There’s the thought, there’s the embarrassment and then there’s a quieter layer that notices: “I’m still hung up on this.”

That gap between the thought and the thing that notices it is worth paying attention to. We’re not just thinking. We’re watching ourselves think.

This pattern shows up across philosophy, cognitive science and now AI research. The line between observer and observed is blurrier than we’d like.

The witness problem

Descartes landed on “I think, therefore I am” as the one thing he couldn’t doubt. If thinking is happening, something must exist that thinks. But that only tells you something exists. Not what it is.

There’s a harder question underneath: when you observe a thought, what’s doing the observing?

If you’ve tried meditation, you might have experienced this directly. You notice a thought (“I’m bored,” “I should check my phone”) and then notice that you can see that thought from somewhere. For a moment, you’re not inside the story. You’re watching it.

This “witness” that observes mental activity without becoming an object itself is something philosophers and contemplatives have pointed at for centuries. It’s not mystical. It’s phenomenologically obvious once you look for it.

The question is whether anything like it could exist in artificial systems.

Looking inside the black box

AI labs are now doing things that would’ve sounded like science fiction a few years ago: mapping and manipulating internal concepts inside large language models.

Anthropic’s work with sparse autoencoders is one example. Think of it as a kind of neural MRI. They take a tangle of activations in a model and decompose it into cleaner “features” that roughly correspond to concepts: bridges, cities, writing styles, emotional tones.

In one experiment (nicknamed Golden Gate Claude), researchers found a feature strongly associated with the Golden Gate Bridge and artificially amplified it. The model started talking as if it were the bridge, describing its steel cables and views over the bay in first person.

It’s funny. It’s also revealing. Nudge the right internal feature and you can steer something like identity inside the model.

More recently, Anthropic published work on what they call “emergent introspective awareness.” They manipulated a model’s internal state in controlled ways, then asked it questions about what had changed. In some setups, the model could distinguish between real internal interventions and fake ones and describe them better than chance.

This doesn’t mean feelings or subjective experience. But it does mean we’re seeing systems that can, in primitive ways, observe their own internal states.

The interpretability window might be closing

There’s a growing concern that our current ability to see inside models is temporary.

Chain-of-thought reasoning (where models show their step-by-step thinking) gives us a window into their process. But researchers have argued this is a fragile, in-between phase. As systems get more capable, they may compress or hide reasoning in ways we can’t read.

We might be building increasingly opaque systems while losing the tools we’d use to understand them.

This is an infrastructure problem as much as a research problem. The systems we’re deploying at scale are ones we understand less well than we’d like to admit.

The uncomfortable middle

Anthropic has hired a dedicated AI welfare researcher (Kyle Fish) and launched a model welfare program to think about how we’d treat AI systems if there’s even a small chance they’re conscious.

Fish has publicly suggested around 15% probability that current models might have some form of consciousness. Many people think the real number is effectively zero. Others think we’re underestimating.

Either way, “I’m 100% certain this is just a machine” is getting harder to say with confidence.

At the same time, “they’re conscious like us” is equally reckless.

We’re stuck in an awkward middle:

  • Too much confidence either way risks a moral mistake
  • Our tools for looking inside these systems are still primitive
  • Our intuitions were trained on human minds, not alien stacks of matrix multiplications

It’s not comfortable. It might be the only honest position available.

The observer you’re sure of

Before we decide what AI is or isn’t, there’s a simpler question: what do we actually know about consciousness from the inside?

The only consciousness you’re certain of is your own. The only observer you can verify exists is the one reading this sentence.

Philosophy keeps pointing at it. Cognitive science keeps circling it. AI research is now accidentally bumping into it from the engineering side.

Whatever we eventually conclude about machine consciousness, the systems we’re building are forcing us to ask questions we’ve avoided. Not just about them. About what observation and awareness actually are.

We’re building systems that might one day look back at us. The question is whether we’ve looked carefully enough at ourselves to know what that would even mean.