?asOf= parameter to see the current catalog state.Catastrophic & Existential Risk
catastrophic_risk · AI-governance topic
Catastrophic & Existential Risk is governance of model capabilities that could cause mass casualties or civilisational-scale harms (CBRN uplift, autonomous replication, deceptive alignment). Distinct from EU AIA 'systemic risk' which targets market-scale rather than catastrophic-scale harms. Across 29 tracked AI-governance instruments, 11 address this topic explicitly, 6 via general principles, and 12 are silent.
Definition and scope
Governance of model capabilities that could cause mass casualties or civilisational-scale harms (CBRN uplift, autonomous replication, deceptive alignment). Distinct from EU AIA 'systemic risk' which targets market-scale rather than catastrophic-scale harms.
The cross-jurisdiction picture below shows how each of 29 tracked instruments treats this topic. The patterns vary substantially — and 12 regimes are silent, leaving gaps that future policy work could address.
Historical primacy & cross-jurisdiction tension
First addressed by NIST AI Risk Management Framework on (implicit). Subsequent regimes have either codified, diverged from, or remained silent on this baseline.
- Forum-shoppingExecutive Order 14110 on Safe, Secure, Trustworthy AI↔Executive Order 14179 — Removing Barriers to American Leadership in AI
- Forum-shoppingG7 Hiroshima AI Process Code of Conduct↔Interim Measures for Generative AI Service Management
- Forum-shoppingBletchley Declaration on AI Safety↔OECD AI Principles (Recommendation)
Cross-jurisdiction coverage
At a glance — same legend as the hub matrix
Silent regimes — gap signal
Instruments that do not address Catastrophic & Existential Risk — candidates for future policy work.
- Executive Order 14179 — Removing Barriers to American Leadership in AIUS
- Interim Measures for Generative AI Service ManagementCN
- OECD AI Principles (Recommendation)OECD
- Council of Europe Framework Convention on AIcouncil_of_europe
- India Digital Personal Data Protection Act + AI Advisory (MEITY)IN
- ASEAN Guide on AI Governance and EthicsASEAN
- African Union Continental AI StrategyAfrican_Union
- Singapore Model AI Governance Framework for Generative AISG
- Japan METI AI Guidelines for BusinessJP
- NYC Local Law 144 of 2021 (Automated Employment Decision Tools)US
- Colorado Artificial Intelligence Act (SB 24-205)US
- Illinois HB 3773 / Public Act 103-0804 (AI Employment Discrimination)US
References
- EU-AIA-2024: Art. 51 + Recital 32 — systemic risk overlaps with but does not fully cover catastrophic-risk framing
- US-EO-14110: §4.2(a)(ii) — CBRN + autonomous replication explicitly named
- UK-WHITEPAPER-2023: AISI remit covers frontier-model evaluation; not in white paper text
- G7-HIROSHIMA: Code §1 + §3 — explicit risk-identification including CBRN
- UN-RES-2024: Notes 'shared concerns' but no operative catastrophic-risk text
- NIST-AI-RMF: Map 1.1 risk classification covers catastrophic via 'societal' impact tier; GenAI Profile (2024) adds explicit content
- BLETCHLEY-2023: Declaration §3-5 (substantial risks from frontier AI, including catastrophic harm)
- SEOUL-2024: Frontier AI Safety Commitments §1: identify thresholds for severe risks pre-deployment
- NIST-AI-RMF-GENAI: NIST AI 600-1 §3.1 CBRN Information Uplift; §3.3 Dangerous, Violent, or Hateful Content
- CA-SB-1047: Cal. SB-1047 §22602 — defines 'critical harm' including mass casualties, $500M+ damage
- BR-AIBILL-2024: PL 2338/2023 Art. 14 (excessive-risk AI applications — explicit prohibition + risk-tier framework)
- ANTHROPIC-RSP-2024: RSP v2 §3 — ASL-3 / ASL-4 capability thresholds explicitly target CBRN uplift + autonomous-replication
- OPENAI-PREPAREDNESS-2023: Preparedness Framework risk-tier matrix — Critical tier explicitly targets CBRN, cyber, persuasion, autonomy
- DEEPMIND-FSF-2024: FSF Critical Capability Levels (CCL) — explicit thresholds for autonomy, biosecurity, cyber, persuasion
- META-FRONTIER-2024: Framework critical-risk tier — commit to halt training pre-mitigation if reached
- UK-US-AISI-MOU-2024: Joint evaluation scope encompasses CBRN + autonomy uplift questions; MoU text does not enumerate explicit thresholds
- WH-VOLUNTARY-2023: Commitments §1 references CBRN + bio risks via 'most significant societal risks'; not threshold-explicit
Cite this article
6 formats · 1-click copyPersistent identifier: https://policywindow.org/wiki/catastrophic-risk — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Take this further — sign up free
Save, compare, or get alerts when Catastrophic & Existential Risk changes. Policy Window is the analyst workbench layered on top of this wiki — free for researchers, civil society, and verified policymakers.
17 instruments tracked.