Google DeepMind Frontier Safety Framework
DEEPMIND-FSF-2024 · US
Critical Capability Levels (CCL) regime spanning autonomy, biosecurity, cybersecurity, and persuasion domains. Distinct vocabulary from Anthropic ASL + OpenAI Preparedness — designed for cross-domain elicitation; each CCL triggers domain-specific mitigations including model-weight access controls + enhanced red-teaming. Seoul Frontier AI Safety Commitments signatory. Alphabet-published; effective across Google DeepMind frontier-model releases. NOTE (iter-314): the FSF is a versioned-evolving artefact; this row pins v1 (May 2024) as the load-bearing reference, but DeepMind publishes incremental updates on the deepmind.google blog. Citers tracking specific CCL definitions or mitigation requirements should confirm against the current published version — the catalog re-pins on the next Coverage Games event. Currency (2026-06-21): The catalog pins FSF v1 (May 2024), but DeepMind has since published v2.0 (4 Feb 2025), v3.0 (22 Sept 2025, adding a harmful-manipulation Critical Capability Level plus expanded misalignment and ML-R&D protocols), and v3.1 (17 Apr 2026, introducing Tracked Capability Levels); citers should confirm CCL definitions against the current version at deepmind.google/blog/strengthening-our-frontier-safety-framework/.
Background & scope
Google DeepMind Frontier Safety Framework addresses 3 contested AI-governance topics explicitly, 4 via general principles.
Provisions & coverage
- governsFoundation Models / GPAIFSF applies to Google DeepMind frontier-model releases[10]
- implicitTransparency ObligationsFSF publication discloses framework + thresholds; per-evaluation outputs not consistently public[10]
- governsCatastrophic & Existential RiskFSF Critical Capability Levels (CCL) — explicit thresholds for autonomy, biosecurity, cyber, persuasion[10]
- implicitInternational CoordinationSeoul Frontier AI Safety Commitments signatory; UK AISI pre-deployment evaluation cooperation[10]
- governsAgentic AI GovernanceFSF Critical Capability Levels — Autonomy is one of four named CCL domains[10]
- implicitOpen-Weight Frontier ReleaseFramework applies to Google DeepMind deployments (mostly closed); third-party open release not addressed[10]
- implicitCompute + Model-Weight Export ControlsFSF mitigations include model-weight access controls + restricted-deployment options[10]
What the Framework Commits To
The Frontier Safety Framework (FSF), published 17 May 2024 and effective across Google DeepMind frontier-model releases, is a voluntary corporate code, not statute — it carries no formal article or section numbers, so its obligations are stated as protocol paragraphs rather than enumerated provisions. Its load-bearing construct is the Critical Capability Level (CCL): a capability threshold across four named domains — autonomy, biosecurity, cybersecurity, and persuasion — that, once a model approaches it, triggers domain-specific mitigations including model-weight access controls and enhanced red-teaming. The enhanced red-teaming the autonomy CCL contemplates is exactly the kind of agentic stress-testing operationalised by harm benchmarks such as AgentHarm 1, which measures whether tool-using LLM agents resist harmful multi-step tasks. The autonomy CCL squarely engages agentic-AI governance, the very surface that 2 argues demands agency-law-grounded infrastructure for visibility and liability. By committing to halt or restrict deployment absent adequate mitigations, the FSF operationalises a precautionary posture toward catastrophic capability.
Standing Relative to Binding Law
As an Alphabet-published voluntary code, the FSF has no legal force and no external enforcement: DeepMind both writes and audits its own CCL thresholds, a self-certification posture that contrasts sharply with the EU AI Act's binding general-purpose-AI regime (Regulation (EU) 2024/1689). The terminological gap is itself analytically significant — 3 documents how the Act's text oscillated among 'AI system', 'GPAI', and 'foundation model', and the FSF's bespoke CCL vocabulary maps onto none of those legal categories cleanly, complicating any future conformity assessment. 4 notes the Act's risk-based model strains where autonomous generation 'challenges legal categories of authorship, accountability, and control'. The FSF supplies engineering thresholds where law supplies duties; the two are complementary but not interchangeable, and the framework's commitments remain revocable at the publisher's discretion.
Critiques and Coverage Gaps
Three gaps recur. First, transparency: the FSF discloses the framework and its thresholds, but per-evaluation outputs are not consistently public, so external parties cannot verify that a CCL was correctly assessed — a visibility deficit 5 frames as the core missing 'agent infrastructure' for attributing actions and remediating harms. Second, scope: the framework governs DeepMind's own (largely closed) deployments and does not address third-party open-weight release, leaving the highest-irreversibility distribution channel uncovered. Third, the four-domain CCL taxonomy targets decisive, threshold-crossing capability, but 6 warns that 'accumulative' existential risk — gradual societal erosion — escapes any single-model gate. The biosecurity CCL also inherits the dual-use mapping problems catalogued by 7, where AI-bio convergence outpaces governance pathways.
Adoption and Versioning Trajectory
The FSF is a versioned, evolving artefact: this entry pins v1 (May 2024), but DeepMind has since published v2.0 (4 Feb 2025), v3.0 (22 Sept 2025 — adding a harmful-manipulation CCL plus expanded misalignment and ML-R&D protocols), and v3.1 (17 Apr 2026, introducing Tracked Capability Levels); citers tracking specific CCL definitions must confirm against the current published version. As a Seoul Frontier AI Safety Commitments signatory engaged in UK AISI pre-deployment cooperation, DeepMind ties the FSF to an emerging international scaffold — yet this remains soft coordination, short of the compute-threshold treaty with binding halt-authority proposed by 8. Whether voluntary frameworks converge toward the precautionary state obligation argued by (Druzin et al. 2025, Mich. J. Int'l L. vol. 46 iss. 2) or remain industry self-governance is the open trajectory question; multi-agent deployment risks 9 will test the autonomy CCL most.
Enforcement & impact
Cross-jurisdiction comparison
How peer instruments treat the topics Google DeepMind Frontier Safety Framework governs.
| Topic | EU-AIA-2024 | US-EO-14110 | US-EO-14179 | UK-WHITEPAPER-2023 | CN-GENAI-2023 | G7-HIROSHIMA | OECD-AI-PRIN | COE-AI-CONV | UN-RES-2024 | NIST-AI-RMF | BLETCHLEY-2023 | SEOUL-2024 | NIST-AI-RMF-GENAI | CA-SB-1047 | IN-DPDP-2023 | BR-AIBILL-2024 | ASEAN-AI-GUIDE-2024 | AU-AI-STRATEGY-2024 | ANTHROPIC-RSP-2024° | OPENAI-PREPAREDNESS-2023° | META-FRONTIER-2024° | UK-US-AISI-MOU-2024 | WH-VOLUNTARY-2023 | SG-MODEL-AI-2024 | JP-METI-AI-2024 | EU-GDPR-2016 | EU-GPAI-COP-2025 | OMB-M-24-10 | GSA-AI-GUIDE-2024 | DOD-RAI-2022 | FEDRAMP-AI-2024 | DFARS-252-204 | CA-SB-53 | CA-SB-243 | CA-SB-942 | EU-PLD-2024 | UNESCO-AI-ETHICS-2021 | EU-PWD-2024 | CN-DEEPSYN-2022 | NY-RAISE-2025 | US-TAKEITDOWN-2025 | IT-AILAW-2025 | JP-AIPROMO-2025 | UN-GDC-2024 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Foundation Models / GPAI | governs | governs | silent | implicit | governs | governs | implicit | implicit | silent | governs | governs | governs | governs | governs | implicit | governs | implicit | silent | governs | governs | governs | governs | governs | governs | governs | silent | governs | implicit | governs | implicit | implicit | implicit | governs | silent | implicit | silent | silent | silent | silent | governs | silent | silent | implicit | implicit |
| Catastrophic & Existential Risk | implicit | governs | silent | implicit | silent | governs | silent | silent | implicit | implicit | governs | governs | governs | governs | silent | governs | silent | silent | governs | governs | governs | implicit | implicit | silent | silent | silent | governs | silent | silent | implicit | silent | silent | governs | silent | silent | silent | silent | silent | silent | governs | silent | silent | silent | implicit |
| Agentic AI Governance | implicit | silent | silent | silent | implicit | implicit | silent | implicit | silent | implicit | implicit | governs | governs | silent | silent | implicit | silent | silent | governs | governs | implicit | implicit | silent | silent | silent | silent | silent | silent | silent | silent | silent | silent | implicit | silent | silent | implicit | silent | implicit | silent | implicit | silent | silent | silent | silent |
°= industry self-imposed voluntary framework. Comparing a voluntary code's "governs" tint with a binding regulation's "governs" tint flattens the legal-force distinction; use the instrument-page banner for the operative status of each.
See also
Per-audience views
- Provisions →Article-by-article obligation breakdown for procurement + RFP authors.
- Disclosure form →Vendor-disclosure questionnaire derived from this instrument's operative obligations.
- Harm narratives →Documented harms relevant to this instrument's topics, for civil-society advocacy.
- Briefing pack →Journalist-ready summary with quotes + dates + primary-source links.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here
Further reading
105 academic & grey-literature sources on the topics this instrument addresses (not commentary on the instrument itself) — catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways Peer-reviewed✦ AIReviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats.
- Governing AI Agents Preprint✦ AIUses "agency law and theory to identify and characterize problems arising from AI agents" and proposes governance infrastructure built on inclusivity, visibility, and liability.
- Infrastructure for AI Agents Peer-reviewed✦ AIProposes "agent infrastructure": external technical systems for attributing actions "to specific agents, their users, or other actors," shaping interactions, and remediating harms.
- Multi-Agent Risks from Advanced AI Research institute✦ AIIdentifies three failure modes of advanced multi-agent systems — "miscoordination, conflict, and collusion" — plus seven risk factors, posing challenges distinct from single-agent AI.
- China's semiconductor conundrum: understanding US export controls and their efficacy Peer-reviewed✦ AIArgues "America's chokepoint strategy is increasingly proving to be a fallacy": Chinese chipmakers have "managed to circumvent these measures" in four ways, accelerating domestic innovation.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- Export Controls and Innovation in Sanctioned Countries Working paper✦ AIUsing the 2007 US 'China Rule', finds sanctioned Chinese firms raised R&D by ~49% and patenting by ~41% — evidence export controls can accelerate the target's indigenous innovation.
- Two types of AI existential risk: decisive and accumulative Peer-reviewed✦ AIDistinguishes 'decisive' (sudden takeover) from 'accumulative' AI existential risk, arguing governance must address gradual societal erosion as well as abrupt scenarios.
- Confronting Catastrophic Risk: The International Obligation to Regulate Artificial Intelligence Peer-reviewed✦ AIArgues international law imposes a precautionary-principle obligation on states to regulate AI to mitigate the threat of human extinction.
- Artificial Intelligence and Nuclear Weapons Proliferation: The Technological Arms Race for (In)visibility Peer-reviewed✦ AIAnalyzes how AI-driven detection/concealment in nuclear arsenals reshapes strategic stability and proliferation risk, with governance implications.
+ 93 more across this instrument's topics — see the literature index.
References
Sources cited inline in the analysis (linked from the superscript markers), then the primary instrument sources behind the classifications.
- Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Eric Winsor, Jerome Wynne, Yarin Gal, Xander Davies (UK AISI / Gray Swan) (2025) AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents, ICLR 2025. arXiv:2410.09024 — Provides a 440-task benchmark across 11 harm categories measuring whether LLM agents resist or comply with harmful multi-step tool-use tasks, grounding safety-evaluation regimes for agents. ↩
- Noam Kolt (2025) Governing AI Agents, Notre Dame Law Review (forthcoming). arXiv:2501.07913 — Uses "agency law and theory to identify and characterize problems arising from AI agents" and proposes governance infrastructure built on inclusivity, visibility, and liability. ↩
- David Fernández-Llorca, Emilia Gómez, Ignacio Sánchez, Gabriele Mazzini (2025) An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI, Artificial Intelligence and Law. 10.1007/s10506-024-09412-y — Traces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime. ↩
- Martina Hulok (2025) The EU model of AI governance: regulating artificial intelligence through law and policy, ERA Forum. 10.1007/s12027-025-00869-1 — Analyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'. ↩
- Alan Chan, Kevin Wei, Sihao Huang, Nitarshan Rajkumar, Elija Perrier, Seth Lazar, Gillian K. Hadfield, Markus Anderljung (2025) Infrastructure for AI Agents, Transactions on Machine Learning Research. arXiv:2501.10114 — Proposes "agent infrastructure": external technical systems for attributing actions "to specific agents, their users, or other actors," shaping interactions, and remediating harms. ↩
- Atoosa Kasirzadeh (2025) Two types of AI existential risk: decisive and accumulative, Philosophical Studies. 10.1007/s11098-025-02301-3 — Distinguishes 'decisive' (sudden takeover) from 'accumulative' AI existential risk, arguing governance must address gradual societal erosion as well as abrupt scenarios. ↩
- Kirolos Eskandar (2026) Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways, AI and Ethics (Springer). 10.1007/s43681-025-00872-9 — Reviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats. ↩
- Rebecca Scholefield, Samuel Martin, Otto Barten (2025) International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty, arXiv (cs.CY). arXiv:2503.18956 — Proposes a conditional AI safety treaty with a compute threshold triggering mandatory audits by an international network of AI Safety Institutes empowered to halt development if risks are unacceptable. ↩
- Lewis Hammond, Alan Chan, Jesse Clifton, et al. (Cooperative AI Foundation) (2025) Multi-Agent Risks from Advanced AI, Cooperative AI Foundation. arXiv:2502.14143 — Identifies three failure modes of advanced multi-agent systems — "miscoordination, conflict, and collusion" — plus seven risk factors, posing challenges distinct from single-agent AI. ↩
- Google DeepMind Frontier Safety Framework (May 2024)
- FSF applies to Google DeepMind frontier-model releases
- FSF publication discloses framework + thresholds; per-evaluation outputs not consistently public
- FSF Critical Capability Levels (CCL) — explicit thresholds for autonomy, biosecurity, cyber, persuasion
- Seoul Frontier AI Safety Commitments signatory; UK AISI pre-deployment evaluation cooperation
- FSF Critical Capability Levels — Autonomy is one of four named CCL domains
- Framework applies to Google DeepMind deployments (mostly closed); third-party open release not addressed
- FSF mitigations include model-weight access controls + restricted-deployment options
How to cite this article
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/deepmind-fsf — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Does this instrument’s approach work? — the social-science evidence
Aggregated over the 7 topics this instrument governs: whether each harm is empirically real, and whether the peer-reviewed evidence shows governance reduces it. The badge is the epistemic status of the evidence— “thin”/“absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.
Of the 7 governed topics with a social-science evidence review, evidence that governance reduces the harm is established for 0, contested for 0, thin for 0, and absent for 7 — for most, no replicated study yet shows this instrument's approach works (the "second silence").
Agentic AI Governance
The capability that agentic governance targets — autonomous multi-step action — is real and rapidly, measurably advancing: METR finds the task length AI agents complete at 50% reliability has doubled roughly every seven months for the past six years (about 50 minutes for frontier 2025 models), and the UK AI Security Institute's first Frontier AI Trends Report (Dec 2025, >30 systems) reports models now finish hour-long software tasks >40% of the time versus <5% in late 2023. The distinct realized HARM from agency (as opposed to the underlying model) is, however, thinly documented: on consequential real-world tasks agents still fail the majority — Gemini 2.5 Pro completed only 30.3% of TheAgentCompany's 175 professional tasks (OpenHands scaffold, project leaderboard) — so the agency-specific harm magnitude is early and context-dependent rather than established at scale.
Sources: Kwa, West, Becker et al. 2025 (METR; arXiv:2503.14499, 'Measuring AI Ability to Complete Long Tasks'); UK AI Security Institute 2025 (Frontier AI Trends Report, Dec 2025); Xu, Song, Zhou et al. 2024 (TheAgentCompany, arXiv:2412.14161); 30.3% figure per TheAgentCompany leaderboard (OpenHands)
There is no impact-evaluation evidence that agent-specific governance reduces agentic harm: the operative regimes — the EU GPAI Code of Practice (published July 2025, voluntary/non-binding), the Seoul Frontier AI Safety Commitments (2024, voluntary), and AISI agent evaluations — are 2024-25 vintage and have never been measured against an outcome. The scholarship itself has not settled the contested unit of regulation: Kolt (2025) argues for governing the agentic relationship via principal-agent and agency-law tools, while Chan, Ezell, Kaufmann et al. (2024) propose agent-specific visibility mechanisms (identifiers, real-time monitoring, activity logging) that remain proposal-stage and unevaluated — meaning the field has design proposals but, as with most frontier-AI rules, the evidence that any of them works is absent rather than merely thin.
Sources: Kolt 2025 ('Governing AI Agents', 101 Notre Dame L. Rev., forthcoming; arXiv:2501.07913); Chan, Ezell, Kaufmann et al. 2024 ('Visibility into AI Agents', ACM FAccT 2024, pp. 958-973; DOI 10.1145/3630106.3658948); EU AI Office 2025 (GPAI Code of Practice, July 2025); Seoul Frontier AI Safety Commitments 2024
Catastrophic & Existential Risk
The catastrophic-uplift premise is genuinely contested: the empirical uplift studies that exist find current frontier models add little. RAND's red-team study found no statistically significant difference in the viability of bioweapon-attack plans produced with vs. without LLMs (Mouton, Lucas & Guest 2024), and OpenAI's 100-participant trial found GPT-4 gave at most a mild, non-significant accuracy uplift (mean +0.88 out of 10 for PhD experts, +0.25 for students; Patwardhan et al. 2024). Honest caveat: the harm is forward-looking, not yet observed — expert opinion on the catastrophic tail is sharply split (median AI researcher puts ~5% on extremely-bad/extinction outcomes, mean ~9-16% across differently-framed questions, n=2,778; Grace et al. 2024), and forecasters underestimated how fast risk-relevant capabilities (e.g. virology troubleshooting) actually arrived (Forecasting Research Institute 2025), so the relevant capabilities are a moving target rather than a settled magnitude.
Sources: Mouton, Lucas & Guest 2024 (RAND RR-A2977-2, Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study); Patwardhan et al. 2024 (OpenAI, Building an Early Warning System for LLM-aided Biological Threat Creation); Grace et al. 2024 (Thousands of AI Authors on the Future of AI, arXiv:2401.02843); Forecasting Research Institute 2025 (Forecasting LLM-enabled Biorisk and the Efficacy of Safeguards)
There is essentially no impact evidence that catastrophic-risk governance reduces catastrophic risk, and structurally there cannot yet be: the harm is a low-probability civilisational tail event, so no controlled trial or before/after evaluation of a realised catastrophe is possible. The dominant instruments are recent, voluntary developer frameworks (Anthropic's Responsible Scaling Policy 2023; OpenAI's Preparedness Framework 2023) built on if-then capability thresholds the developers themselves describe as speculative and qualitative rather than validated risk thresholds. The closest evidence is adjacent and indirect: trained-in deceptive behaviours can persist through standard safety training (Hubinger et al. 2024) — a demonstration that current mitigation may be insufficient, not that any governance regime works — and Anthropic's documented loosening of earlier commitments (RSP 2025 dropped the original pledge to define higher-tier ASL evaluations before developing the corresponding models) illustrates that even the strongest voluntary regimes lack external enforcement or measured efficacy.
Sources: Anthropic 2023 (Responsible Scaling Policy); OpenAI 2023 (Preparedness Framework); Hubinger et al. 2024 (Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, arXiv:2401.05566); Hendrycks, Mazeika & Woodside 2023 (An Overview of Catastrophic AI Risks, arXiv:2306.12001)
Compute + Model-Weight Export Controls
That compute controls materially constrain China's frontier-AI hardware access is empirically real and measured: by mid-2025 the US hosted ~75% of catalogued AI-supercomputer performance versus China's ~15% (Pilz, Sanders, Rahman & Heim 2025), corroborated by the Federal Reserve's estimate of ~74% US vs ~14% China high-end AI compute share (Haag, FEDS Notes, Oct 2025), and US prosecutions document large diversion networks (e.g. the ~$160M Alan Hao Hsu / Hao Global H100/H200 case — the first 'AI diversion' conviction, guilty plea Oct 2025, SDTX). The honest caveat is that the magnitude of the binding constraint is genuinely contested: DeepSeek's V3/R1 reached near-frontier capability at an order of magnitude less reported compute via algorithmic efficiency, and analysts argue the controls have simultaneously accelerated Chinese state-backed indigenization, so whether the controls slow capability (the actual aim) versus merely shift its cost structure remains unsettled.
Sources: Pilz, Sanders, Rahman & Heim 2025 (Trends in AI Supercomputers, arXiv:2504.16026 / Epoch AI) — VERIFIED, gives US ~75% / China ~15%; Haag 2025 (FEDS Notes, 'The State of AI Competition in Advanced Economies', Federal Reserve, 6 Oct 2025) — VERIFIED, gives US 74% / China 14% / EU 4.8% of high-end AI compute; US v. Alan Hao Hsu / Hao Global 2025 (DOJ, SDTX; ~$160M H100/H200 diversion, guilty plea Oct 2025, first AI-diversion conviction) — VERIFIED
There is no rigorous impact evaluation showing that compute or model-weight export controls achieve their stated strategic aim of durably slowing frontier-AI capability diffusion to China — the regime is too recent, the counterfactual is unidentified, and the most ambitious instrument (the Jan 2025 BIS 'Framework for AI Diffusion', ECCN 4E091 covering model weights of closed models trained on >10^26 operations) was rescinded on 12-13 May 2025 before it ever took effect (its enforcement date was 15 May 2025), so the evidence that the rule works is itself missing. The closest analogue evidence base, the economic-sanctions evaluation literature, is sobering: Hufbauer, Schott, Elliott & Oegg (2007) coded roughly a third of their historical cases as 'successful' (their database covers ~170-200 cases since WWI; the disputed coding was ~40 of 115, ~34%), but Pape's reanalysis (1997/1998) argued the genuinely sanctions-attributable success rate was far lower (he recoded it to ~5 of 115, under 5%), and the broader literature finds efficacy decays as targets adapt and substitute — the precise dynamic export-control critics attribute to Chinese indigenization and smuggling. This is an analogue, not direct evidence on export controls.
Sources: Hufbauer, Schott, Elliott & Oegg 2007 (Economic Sanctions Reconsidered, 3rd ed., Peterson Institute for International Economics) — VERIFIED; Pape 1997/1998 (Why Economic Sanctions Do Not Work, International Security 22(2), 1997; Why Economic Sanctions Still Do Not Work, International Security 23(1), 1998) — VERIFIED; BIS 2025 (Framework for Artificial Intelligence Diffusion, Federal Register doc 2025-00636 / 90 FR, eff. 13 Jan 2025; ECCN 4E091 model-weight control; rescinded 12-13 May 2025 before its 15 May effective date) — VERIFIED
Foundation Models / GPAI
Whether the foundation-model category maps to a coherent capability/risk tier is genuinely contested. The original case rests on scale-driven 'emergent abilities' that appear unpredictably above a size threshold (Wei et al. 2022; Ganguli et al. 2022 documented capabilities that are smoothly predictable in aggregate loss yet locally surprising), but Schaeffer, Miranda & Koyejo (2023, a NeurIPS Outstanding Paper) showed many 'emergent' jumps are artefacts of discontinuous metrics and dissolve under linear/continuous scoring — implying capability scales more smoothly than a sharp tier would suggest. Honest caveat: this is a live empirical disagreement about measurement, not a settled finding either way, and compute (the regulatory proxy) is an imperfect stand-in for capability or risk regardless of which side is right.
Sources: Wei et al. 2022 (Emergent Abilities of Large Language Models, TMLR; arXiv:2206.07682); Schaeffer, Miranda & Koyejo 2023 (Are Emergent Abilities of Large Language Models a Mirage?, NeurIPS 2023, Outstanding Paper; arXiv:2304.15004); Ganguli et al. 2022 (Predictability and Surprise in Large Generative Models, ACM FAccT; DOI 10.1145/3531146.3533229)
There is no impact evaluation showing that GPAI/foundation-model governance reduces harm — the rules are too new (EU AI Act GPAI obligations and the 10^25-FLOP systemic-risk presumption only began binding on 2 August 2025) and the central regulatory lever is itself contested: Hooker (2024) argues compute thresholds are a shortsighted proxy because compute does not reliably track capability or risk, and the thresholds already diverge across jurisdictions (EU 10^25 vs. the now-rescinded US EO 14110's 10^26 operations, rescinded 20 January 2025). The mandated mitigation methods also lack validated efficacy: model evaluation and red-teaming face well-documented coverage limits and an 'audit gap' in the survey/position literature (behavioural testing cannot establish the absence of untested failure modes), and adversarial red-teaming repeatedly defeats deployed safeguards — the UK AI Safety Institute reports finding universal jailbreaks for every frontier system it has tested, and a large public agent-injection competition elicited policy violations across all 22 frontier models tested from ~1.8M attacks (Zou et al. 2025). Even compliant evaluation therefore cannot yet certify the safety the rules demand. (Caveat: this is an absence-of-evidence claim — no efficacy study has been done — not evidence the rules are ineffective.)
Sources: Hooker 2024 (On the Limitations of Compute Thresholds as a Governance Strategy, arXiv:2407.05694); EU AI Act Arts. 51 & 55 (GPAI systemic-risk presumption, 10^25 FLOP; binding 2 Aug 2025); US EO 14110 (10^26-operation reporting threshold, rescinded 20 Jan 2025 by EO 14148); Zou et al. 2025 (Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition / Gray Swan Arena, arXiv:2507.20526 — 22 frontier agents, ~1.8M attacks); UK AI Safety/Security Institute, Frontier AI Trends Report (universal jailbreaks for every system tested); METR, Common Elements of Frontier AI Safety Policies (2024)
International Coordination
The DESCRIPTIVE premise is well-established: IR scholarship now treats global AI governance as a fragmented 'regime complex' of partially overlapping G7/G20/OECD/GPAI/UN/standards-body arrangements with no central hierarchy (Tallberg et al. 2023 — verified verbatim: 'the emerging governance architecture for AI can be described as a regime complex'; Cihon, Maas & Kemp 2020). But the implied HARM — that forum-shopping and regulatory arbitrage cause a measurable race-to-the-bottom or relocate AI development to lax jurisdictions — is largely theorized/anticipated rather than empirically demonstrated for AI; Tallberg et al. explicitly flag forum-shopping as a dynamic whose presence in the AI regime complex is an open empirical question ('Establishing whether these patterns and dynamics are key features also of the AI regime complex stand out as important priorities in future research'). Honest caveat: the strongest empirical arbitrage evidence comes from analogue footloose digital markets (e.g., ICO reallocation after US securities enforcement) — itself a mixed/contested literature — not from AI firms, so the magnitude of coordination-failure harm in AI specifically remains contested and under-measured.
Sources: Tallberg, Erman, Furendal, Geith, Klamberg & Lundgren 2023 (International Studies Review 25(3): viad040); Cihon, Maas & Kemp 2020 (Should AI Governance be Centralised?, AIES '20: 228-234); Lancieri, Edelson & Bechtold 2025 (AI Regulation: Competition, Arbitrage & Regulatory Capture, Theoretical Inquiries in Law 26(1): 239-262)
There are essentially no impact evaluations showing that the negotiated-coordination mode (AI Safety Institute network MoUs, forum-shifting, multilateral declarations) actually produces regulatory convergence or reduces arbitrage — the AISI Network began only as a statement of intent at the Seoul Summit (Seoul Statement of Intent, 21 May 2024) and held its first operational meeting in November 2024, with no defined metrics or outcome studies, so these soft-law instruments are too new to have measurable effects. The closest analogue evidence is mixed and works through DIFFERENT mechanisms than this topic describes: Bradford's Brussels Effect documents de-facto convergence driven by market access rather than negotiated coordination, and the FATF transgovernmental-network literature shows peer-review mutual evaluation can drive AML convergence — but neither evaluates voluntary AI MoU networks, and FATF's effects come with well-documented unintended consequences (de-risking, financial exclusion). The plain finding: the evidence that AI-governance coordination 'works' is itself missing.
Sources: Bradford 2020 (The Brussels Effect: How the European Union Rules the World, Oxford University Press); Nance 2018 (The regime that FATF built: an introduction to the Financial Action Task Force, Crime, Law and Social Change 69(2): 109-129; cf. Slaughter 2004, A New World Order, Princeton University Press); International Network of AI Safety Institutes — Seoul Statement of Intent toward International Cooperation on AI Safety Science (21 May 2024; network's first meeting San Francisco, Nov 2024)
Open-Weight Frontier Release
The empirical picture splits into two well-separated questions. (1) The MECHANISM that distinguishes open-weight release — that safety guardrails can be cheaply and irreversibly stripped once weights are public — is established: Qi et al. (2024) removed GPT-3.5 Turbo safety alignment by fine-tuning on only ~10 adversarially designed examples for under $0.20 (and the attack generalizes to Llama-2), and even purpose-built tamper-resistant safeguards (Tamirisa et al. 2025, TAR) were subsequently shown to be defeatable by adaptive fine-tuning (Qi et al. 2024, durability critique). (2) Whether this mechanism produces real-world CATASTROPHIC uplift is genuinely contested and, for the headline biosecurity case, currently unsupported: RAND's red-team study found no statistically significant difference in the viability of bioweapon attack plans produced with versus without LLM assistance (Mouton, Lucas & Guest 2024), and OpenAI's 100-participant trial found at most mild uplift over an internet baseline (Patwardhan et al. 2024). Honest caveat: these null/mild results are time-stamped to 2023-2024 frontier capability and to biothreats specifically; the marginal-risk framework (Kapoor, Bommasani et al. 2024) concludes the evidence base is too thin to characterize marginal risk across most misuse vectors, so 'no measured harm yet' is not 'no harm.'
Sources: Kapoor, Bommasani, Klyman, Longpre et al. 2024, 'Position: On the Societal Impact of Open Foundation Models', PMLR 235 / ICML 2024 (arXiv 2403.07918); Mouton, Lucas & Guest 2024, RAND RR-A2977-2, 'The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study'; Qi, Zeng, Xie, Chen, Jia, Mittal & Henderson 2024, 'Fine-tuning Aligned Language Models Compromises Safety', ICLR 2024 (arXiv 2310.03693); Tamirisa et al. 2025, 'Tamper-Resistant Safeguards for Open-Weight LLMs', ICLR 2025 (arXiv 2408.00761); Qi, Wei, Carlini, Huang, Xie, He, Jagielski, Nasr, Mittal & Henderson 2024, 'On Evaluating the Durability of Safeguards for Open-Weight LLMs' (arXiv 2412.07097); Patwardhan et al. 2024, 'Building an early warning system for LLM-aided biological threat creation', OpenAI
There is no impact evaluation showing that any specific weight-release governance regime reduces downstream harm, because no binding regime has been implemented and measured: California SB-1047's release-conditioning framework was vetoed in September 2024, and the EU AI Act's open-source carve-outs (Recital 102, Art. 53(2)) exempt most open-weight models (those below the systemic-risk compute threshold) from the documentation obligations that would generate evaluable conduct. The structural obstacle is also documented: Kapoor, Bommasani et al. (2024) characterize open-weight release as effectively irreversible and poorly monitorable once weights are public, so post-release governance has little to act on. The closest analogue evidence — technology export controls — is mixed and points to circumvention: commentators argue blanket export controls on freely copyable open-source models cannot work (Just Security 2024), and independent analyses of the post-2022 semiconductor controls document displacement to less-regulated channels (smuggling, threshold-tuned chip variants, cloud access) rather than disappearance of activity (e.g., CSIS, FPRI 2024), suggesting recipient-restriction regimes face the same leakage problem for weights. (Caveat: this is analogical, not direct evidence about weight-release governance, which remains unmeasured.)
Sources: Kapoor, Bommasani, Klyman, Longpre et al. 2024, 'Position: On the Societal Impact of Open Foundation Models', PMLR 235 (arXiv 2403.07918); California SB-1047 (2024, vetoed by Gov. Newsom 29 Sep 2024); EU AI Act Regulation (EU) 2024/1689, Recital 102 & Art. 53(2) open-source exemptions; Just Security 2024, 'Export Controls on Open-Source Models Will Not Win the AI Race'; CSIS, 'The Limits of Chip Export Controls in Meeting the China Challenge' and FPRI 2024, 'Breaking the Circuit: US-China Semiconductor Controls' (export-control circumvention analogue)
Transparency Obligations
Documentation artifacts (model cards, datasheets) are well-specified as proposals and are genuinely adopted, but the empirical premise that mandated disclosure produces meaningful transparency is contested. Selbst & Barocas (2018) argue inscrutability and non-intuitiveness are distinct problems and that disclosing rules does not resolve the latter, and large-scale audits find documentation is sparsely and unevenly completed: a systematic analysis of 32,111 Hugging Face model cards (Liang et al. 2024) found environmental-impact, limitations and evaluation sections least often filled, and Bhat et al. (2023, 45 practitioners) found a substantial gap between the documentation proposal and actual practice. Honest caveat: the documentation frameworks themselves are real and adopted, so the dispute is about whether disclosure conveys decision-relevant information, not whether the artifacts exist.
Sources: Selbst & Barocas 2018 (Fordham Law Review 87:1085-1139); Liang et al. 2024 (Nature Machine Intelligence, s42256-024-00857-z, 'Systematic analysis of 32,111 AI model cards'); Bhat et al. 2023 (CHI '23, 'Aspirations and Practice of ML Model Documentation', DOI 10.1145/3544548.3581518); Mitchell et al. 2019 (FAccT, Model Cards for Model Reporting); Gebru et al. 2021 (CACM 64(12):86-92, Datasheets for Datasets)
There is no rigorous impact evaluation showing that AI transparency mandates (model cards, training-data summaries) measurably reduce bias, misuse or accidents — the central regulatory assumption is empirically untested, partly because flagship mandates like EU AI Act Art. 53(1)(d) GPAI training-data summaries are only subject to AI Office enforcement/verification from 2 August 2026 (the obligation itself began 2 August 2025 for new models). The closest analogue, mandated consumer disclosure, shows small and context-dependent effects: Bollinger, Leslie & Sorensen (2011) found mandatory calorie posting cut average calories per transaction by about 6%, while Loewenstein, Sunstein & Golman (2014) review evidence that disclosure effects are frequently diminished or even reversed by limited attention and often change provider rather than recipient behavior. These are analogues, not AI studies; no study demonstrates that AI transparency disclosure achieves its stated downstream safety aims.
Sources: Bollinger, Leslie & Sorensen 2011 (AEJ: Economic Policy 3(1):91-128); Loewenstein, Sunstein & Golman 2014 (Annual Review of Economics 6:391-419, 'Disclosure: Psychology Changes Everything'); EU AI Act Art. 53(1)(d) GPAI training-data summary (obligation from 2 Aug 2025; AI Office enforcement from 2 Aug 2026)