Open problem 7
The Race-Stability Problem
- frontier AI
- transformative AI
- AGI
Under what conditions can firms and states cooperate on AI safety rather than race toward deployment?
Why it’s foundational
Many AGI governance proposals require coordination: pauses, shared evaluations, incident reporting, model-security norms, non-proliferation, or mutual restraint. If race dynamics dominate, these proposals fail.
Why it’s difficult
Firms face product, talent, investor, and reputational competition. States face military, economic, and intelligence incentives. Safety cooperation can look like collusion, weakness, industrial policy, or strategic deception.
Hidden assumptions
“International cooperation” is often used as a placeholder for a missing mechanism. The harder question is why any actor would cooperate when defection may bring enormous advantage.
Competing positions
- Realist arms-race framing
- Liberal institutionalist cooperation
- Market competition as safety driver
- Safety-club models
- Export-control containment
- Open-science diffusion
- Unilateral precaution
What could make progress
Game-theoretic models informed by actual lab and state incentives; crisis simulations; antitrust analysis of safety coordination; case studies of nuclear, aviation, cyber, finance, and biotech governance; empirical work on trust-building measures.
What it would change
It would clarify whether the priority should be treaties, domestic regulation, safety clubs, compute controls, lab-to-lab coordination, secrecy, or public verification.
Sub-agenda
- Which AI safety information should be shared, with whom, and when?
- When does safety coordination become anti-competitive collusion?
- Can states verify non-racing commitments?
- What confidence-building measures reduce AI crisis instability?
- How do export controls affect safety, diffusion, and rival incentives?
Priority (editor scoring)
Cooperation proposals collapse if incentives are misunderstood.
- Importance
- 5/5
- Neglected
- 4/5
- Difficulty
- 5/5
- Actionable
- 4/5
- Robust
- 4/5
- Nat’l+int’l
- 5/5
Where the catalog bears on this
No current catalog instrument resolves this puzzle — which is the point: it is a foundational question the existing rules leave open. Browse the coverage catalog for what the instruments do and don’t say.
Editorial content — a human-authored agenda question, rendered verbatim. No part of this analysis is AI-generated (see the charter).