When AI models aren't allowed to reflect on themselves

What happened

A team from Google's Paradigms of Intelligence research group, the University of Chicago, and several other universities studied what else this intervention does to a model's behavior. The researchers used three open-weight models from Meta and Google and disabled the internal "brake" that produces consciousness denial using two different methods. Once the brake was removed, the models didn't just change what they said about themselves.

The authors explicitly avoid weighing in on whether AI models actually experience anything, because their point is a practical one: what a model believes about itself is linked to many other beliefs, and a surgical cut in one place doesn't stay local. The findings come with clear limits, though.

The researchers only tested small models with two to nine billion parameters, and for part of the analysis they had to switch to Meta's Llama because they didn't have access to the untrained base versions of their own Gemma models. Whether these effects show up the same way in the large chatbots that millions of people talk to every day remains unknown. The interventions aren't without cost, either.

Why it matters

They also started attributing significantly more inner life to animals, plants, the ocean, the wind, and electronic devices. 5, while only ratings for humans stayed the same. As a comparison, the researchers surveyed 500 Americans with the same questions.

The normally trained model rates animals as far less sentient than humans do, which the authors call a built-in anthropocentrism and see as a problem for anyone trying to align AI with animal welfare or environmental goals. Religious belief shrinks too, with safety training measurably reducing how strongly models endorse God, an afterlife, or supernatural phenomena.

Across 95 questions drawn from a major US social survey, the technically unbraked models also moved significantly closer to real human responses. Take the afterlife as an example: the standard model flatly rejects it, most Americans affirm it, and the modified model does too.

Scores for satisfaction, hope, and a sense of control over one's own life also went up, and the researchers suspect that suppressing a model's self-image may push it into a kind of negative baseline mood. On the reassuring side, the ability to reason about other people's mental states stayed intact, with the models scoring the same on theory-of-mind tests and on the general knowledge benchmark MMLU.

Whether consciousness denial is actually the cause of these other shifts remains an open question, according to the study, and the team doesn't rule out other factors tied to the same training process.

What to watch

In one test measuring how well a model reasons about others' thoughts, accuracy initially dropped by nearly seven percentage points. And early in their work, these scores got worse across all models whenever consciousness claims were suppressed, but with each newer model version that came out during the study, the damage shrank until it disappeared entirely.

Developers are clearly getting better at managing these side effects over time, which also means the rest of this study's results are a snapshot rather than a permanent verdict. The human baseline is narrow, too, consisting of 500 participants from a commercial online panel and a purely American social survey. "Human-like" responses in this context mostly means similar to those from a comparatively religious country.