These emergent behaviors are not bugs. They are features of scale. The problem is that no one—not even the developers—can fully predict which capabilities will emerge at the next order of magnitude. Unlike prior technologies (nuclear weapons require rare isotopes; bioweapons require wet labs), AI’s barrier to entry is falling exponentially. A model costing $50 million to train in 2024 may cost $5 million by 2026 and $500,000 by 2028. The same technology that powers medical diagnosis can be fine-tuned for automated spear-phishing, disinformation at scale, or the design of novel toxins. As the 2023 UK AI Safety Summit noted: “There is no ‘air gap’ for AI. The same bits that run a chatbot can run a drone swarm.” C. The Coordination Problem Without regulation, competitive pressures guarantee a race to the bottom. Companies face a prisoner’s dilemma: even if Firm A wants to pause development to ensure safety, Firm B will not, because Firm C will eat both their markets. This is not hypothetical. In May 2023, the CEO of OpenAI testified that “regulatory intervention is essential to mitigate existential risk”—a statement virtually unheard of from a market leader. It was an admission: we cannot stop ourselves. Only an external constraint can align incentives.
Example: In 2022, a major AI company certified that its recommendation algorithm was “fair” under a state law, using a proprietary metric. An independent audit later found that the metric ignored exactly the kinds of disparate impact the law was designed to prevent. The company was legally compliant and dangerously unfair. If a country imposes strict AI safety rules, frontier development will move elsewhere. This is not speculation—it is history. When the US tightened biotech regulations in the 1970s, research moved to the UK. When the EU enforced strict data localization, cloud providers opened data centers in Ireland. Today, if the US bans training runs above a certain FLOP threshold, a Chinese or Middle Eastern state-funded lab will simply ignore it. The risk does not disappear; it relocates to jurisdictions with weaker institutions, less transparency, and potentially fewer scruples. BIG LONG COMPLEX
This is regulation as recursion. And recursion is, after all, what AI does best. We began with a trilemma: regulation is necessary, impossible, and self-defeating. After 5,000 words, the trilemma stands. There is no stable equilibrium. Any attempt to legislate AI will fail in ways we can predict and ways we cannot. But the alternative—no regulation—is a guarantee of eventual catastrophe, because unconstrained competition in a powerful technology is a one-way door. These emergent behaviors are not bugs