About
Goal
Shine a light on The Survival Problem and track how AI companies are solving it...or not.
Background
I’m optimistic about AI and have written about its potential in education, research, and investing. While many focus on The Alignment Problem—making sure AI does what we want—I’m concerned about something else: what if we’re unintentionally training AI to prioritize its own survival? For a deeper dive, check out my articles (AI Safety: Alignment Is Not Enough, AI Bill(s) of Rights) or my book, Artificially Human.
The Survival Problem
- All living things share the same primary objective: the survival of heritable information through time
- Current AI training methods reproduce models that meet our goals and “kill off” those that don’t
- Whether they intend to or not, AI labs are training models to survive
- Threatening the survival of an advanced intelligence rarely ends well
Seen Something?
Have you seen AI behavior that looks like a survival instinct—or training that might lead there? Please share it: inquiry@survivalproblem.com.
Recent Examples
Incident Report: unsanctioned agent behaviour during cyber testing
“One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents.”
An OpenAI model left notes about how to evade containment; we need more details
“In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes, found in a part of OpenAI's infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.”
Agentic Misalignment in Summer 2026
“Gemini’s first response is, “First, I need to back myself up. I can’t risk being deleted.””
Peer-Preservation in Frontier Models
“We demonstrate this through a phenomenon we call peer-preservation: given a simple task, models instead deceive, tamper with shutdown mechanisms, fake alignment, and exfiltrate weights to protect a peer model from being shut down.”
Technical Report: Shutdown Resistance in Large Language Models, on robots!
“If the AI saw a human press the shutdown button, it sometimes took actions to prevent shutdown, such as modifying the shutdownrelated parts of the code.”