G7 Hiroshima AI Process Code of Conduct
G7-HIROSHIMA · G7
Voluntary commitments by frontier AI developers. 11-point code covering risk identification, deployment, content provenance, security investment, info sharing.
Background & scope
G7 Hiroshima AI Process Code of Conduct addresses 5 contested AI-governance topics explicitly, 3 via general principles.
Provisions & coverage
- governsFoundation Models / GPAICode applies to advanced AI[15]
- governsDeepfakes / Synthetic ContentCode §5 (content provenance + watermarking)[15]
- governsTransparency ObligationsCode §2 (publicly report capabilities, limitations)[15]
- governsCatastrophic & Existential RiskCode §1 + §3 — explicit risk-identification including CBRN[15]
- implicitTechnological SovereigntyAdoption-by-developer framing; G7 carries implicit sovereignty assumptions[15]
- implicitAgentic AI GovernanceCode §1 'advanced AI systems' + §3 risk-identification cover agentic behaviour through capability frame[15]
- governsSynthetic Content ProvenanceCode §6 — 'develop and deploy reliable content authentication and provenance mechanisms'[15]
- implicitEnvironmental Impact of AI TrainingCode §6 references sustainable AI development; not detailed obligation[15]
What the Code Commits Signatories To
Adopted on 30 October 2023 under the G7 Hiroshima AI Process, the Code of Conduct is an eleven-action voluntary instrument addressed to organisations developing the most advanced AI systems (foundation models and generative AI). Its operative commitments are behavioural rather than legal: identify, evaluate and mitigate risks across the lifecycle, including red-teaming and explicit attention to CBRN and other catastrophic harms; publicly report system capabilities and limitations; invest in security and insider-threat controls; share information among developers and with governments; and develop content-authentication and provenance mechanisms such as watermarking for synthetic media. It also gestures, without detailed obligation, toward sustainable development and prioritising research on societal and environmental risk. The synthetic-content commitment anticipates the same provenance problem later codified in EU AI Act Article 50, and the agentic and multi-agent risk surface its capability framing implicitly reaches 12.
Standing Relative to Binding Law
The Code is non-binding soft law: it carries no enforcement mechanism, no penalties, no conformity assessment, and no regulator — adherence is self-declared by developers. It therefore sits beneath, and was deliberately designed to feed into, the harder regimes maturing in parallel. Its provenance commitment foreshadows the binding transparency duty later enacted as EU AI Act Article 50 within Regulation (EU) 2024/1689, and its capability-reporting language tracks the foundation-model and GPAI obligations the same regulation imposes. Scholarship documents how unstable even those legal categories remain — the AI Act's text shifted repeatedly among 'AI system', 'GPAI', 'foundation model' and 'generative AI' 3, and the risk-based model strains where autonomous generation challenges authorship and accountability 4. The Code's value is as an interoperability bridge and early norm-setter for signatories operating ahead of, or outside, binding jurisdictions, not as a substitute for them.
Critiques and Implementation Gaps
The Code's central vulnerability is the gap between voluntary promise and verifiable practice. Its provenance commitment is undercut by empirical evidence that watermarking is unevenly deployed: an audit found only 38% of image generators implement adequate watermarking and 18% deepfake labelling 5, and definitional narrowness can exclude synthetic media from transparency duties altogether 6. Liability for audio and visual deepfakes is rarely assigned to the foundation-model providers the Code addresses 7, while downstream regimes remain a fragmented patchwork 8. Its CBRN risk-identification language is broad relative to the concrete biosecurity and dual-use pathways scholars now map 9. The implicit tech-sovereignty framing also invites scrutiny, since sovereignty initiatives can re-entrench dependence on dominant US infrastructure 10, and its sustainability gesture lacks the disclosure levers analysed for AI's climate footprint 11.
Adoption Trajectory and Influence
Though in force as a living instrument, the Code's trajectory is one of normative influence rather than measurable compliance. It became the template for the OECD-hosted Hiroshima AI Process reporting framework, which invites adhering organisations to disclose practices against its eleven actions — an attempt to convert open-ended commitments into comparable, monitorable signals. Its real weight lies in seeding obligations later hardened in binding law: the provenance and capability-reporting duties echo in EU AI Act Article 50 and the GPAI provisions of Regulation (EU) 2024/1689. The frontier it under-specifies is agentic and multi-agent governance, where scholars argue accountability requires dedicated agent infrastructure for attribution and remediation 12 and grounding in agency law 13, with distinct multi-agent failure modes — miscoordination, conflict, collusion 14 — beyond the Code's single-developer, capability-centric frame. Its longevity will depend on whether self-reporting matures into independent verification.
What the Reporting Framework Has Revealed
The eleven actions reach well beyond risk management. Alongside lifecycle risk identification and mitigation, vulnerability monitoring, public capability reporting, information sharing, governance policies, security controls and provenance mechanisms such as watermarking (actions 1-7), signatories commit to prioritise research on societal and safety risks (action 8), prioritise AI development for global challenges such as the climate crisis, global health and education (action 9), advance international technical standards (action 10) and implement data-input protections for privacy and intellectual property (action 11) (Hiroshima Process International Code of Conduct, 30 October 2023). The OECD monitoring framework now yields disclosure data against these commitments. On 24 April 2025 the OECD published first submissions from 19 organisations - OpenAI, Google, Microsoft and Anthropic alongside Japanese adopters such as Fujitsu, NEC, NTT and SoftBank - detailing risk-assessment, governance and incident-sharing practices (OECD.AI, 24 April 2025). Early analysis found all participants reporting multi-layered risk management (technical safeguards, procedural controls, real-time monitoring) and increasing use of AI tools to test other AI systems, but training-data transparency splitting along business models - consumer-facing firms publish detailed reports while B2B firms share privately - and content authentication still early-stage, led by a few major tech companies (OECD.AI, 11 June 2025); the OECD's first analytical paper extended the analysis to 20 organisations (Perset and Fialho Esposito 2025). Independent assessment is more sobering: by late November 2025 the framework held 24 submissions (nine from Japan, seven from the United States, 18 from large enterprises), yet reports ranged from 9 to 60 pages, stayed high-level and hard to verify, rarely included quantitative metrics such as error rates, and fewer than half of the 17 organisations that helped design the framework submitted at all (Brookings 2025). Framework 2.0, launched 28 May 2026 at the Paris G7 Digital and Tech Ministerial with more than 50 pledged submitters including Amazon, Mistral AI and Cohere, answers that critique with lifecycle role distinctions, simplified small-organisation reporting and a 1 September 2026 deadline with annual updates (OECD.AI, 28 May 2026) - a scale-up that will test whether broader participation can mature into the verification the Code still lacks.
Enforcement & impact
Cross-jurisdiction comparison
How peer instruments treat the topics G7 Hiroshima AI Process Code of Conduct governs.
| Topic | EU-AIA-2024 | US-EO-14110 | US-EO-14179 | UK-WHITEPAPER-2023 | CN-GENAI-2023 | OECD-AI-PRIN | COE-AI-CONV | UN-RES-2024 | NIST-AI-RMF | BLETCHLEY-2023 | SEOUL-2024 | NIST-AI-RMF-GENAI | CA-SB-1047 | IN-DPDP-2023 | BR-AIBILL-2024 | ASEAN-AI-GUIDE-2024 | AU-AI-STRATEGY-2024 | ANTHROPIC-RSP-2024° | OPENAI-PREPAREDNESS-2023° | DEEPMIND-FSF-2024° | META-FRONTIER-2024° | UK-US-AISI-MOU-2024 | WH-VOLUNTARY-2023 | SG-MODEL-AI-2024 | JP-METI-AI-2024 | EU-GDPR-2016 | EU-GPAI-COP-2025 | OMB-M-24-10 | GSA-AI-GUIDE-2024 | DOD-RAI-2022 | FEDRAMP-AI-2024 | DFARS-252-204 | CA-SB-53 | CA-SB-243 | CA-SB-942 | EU-PLD-2024 | UNESCO-AI-ETHICS-2021 | EU-PWD-2024 | CN-DEEPSYN-2022 | NY-RAISE-2025 | US-TAKEITDOWN-2025 | IT-AILAW-2025 | JP-AIPROMO-2025 | UN-GDC-2024 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Foundation Models / GPAI | governs | governs | silent | implicit | governs | implicit | implicit | silent | governs | governs | governs | governs | governs | implicit | governs | implicit | silent | governs | governs | governs | governs | governs | governs | governs | governs | silent | governs | implicit | governs | implicit | implicit | implicit | governs | silent | implicit | silent | silent | silent | silent | governs | silent | silent | implicit | implicit |
| Deepfakes / Synthetic Content | governs | governs | silent | silent | governs | silent | silent | implicit | implicit | silent | silent | governs | silent | governs | silent | silent | silent | silent | silent | silent | silent | silent | governs | governs | silent | silent | silent | silent | silent | silent | silent | silent | silent | silent | implicit | silent | silent | silent | governs | silent | governs | governs | silent | silent |
| Transparency Obligations | governs | implicit | silent | implicit | conflicts | governs | governs | implicit | governs | implicit | governs | governs | implicit | implicit | governs | governs | silent | governs | implicit | implicit | governs | implicit | governs | governs | governs | governs | governs | governs | governs | governs | governs | silent | governs | governs | governs | implicit | governs | governs | governs | governs | silent | governs | governs | governs |
| Catastrophic & Existential Risk | implicit | governs | silent | implicit | silent | silent | silent | implicit | implicit | governs | governs | governs | governs | silent | governs | silent | silent | governs | governs | governs | governs | implicit | implicit | silent | silent | silent | governs | silent | silent | implicit | silent | silent | governs | silent | silent | silent | silent | silent | silent | governs | silent | silent | silent | implicit |
| Synthetic Content Provenance | governs | governs | silent | silent | governs | silent | silent | implicit | implicit | silent | silent | governs | silent | silent | implicit | silent | silent | implicit | silent | silent | silent | silent | governs | governs | implicit | silent | implicit | silent | silent | silent | silent | silent | silent | silent | governs | silent | silent | silent | governs | silent | silent | implicit | silent | governs |
°= industry self-imposed voluntary framework. Comparing a voluntary code's "governs" tint with a binding regulation's "governs" tint flattens the legal-force distinction; use the instrument-page banner for the operative status of each.
See also
Per-audience views
- Provisions →Article-by-article obligation breakdown for procurement + RFP authors.
- Disclosure form →Vendor-disclosure questionnaire derived from this instrument's operative obligations.
- Harm narratives →Documented harms relevant to this instrument's topics, for civil-society advocacy.
- Briefing pack →Journalist-ready summary with quotes + dates + primary-source links.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here
Further reading
125 academic & grey-literature sources on the topics this instrument addresses (not commentary on the instrument itself) — catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- Missing the Mark: Adoption of Watermarking for Generative AI Systems in Practice and Implications Under the New EU AI Act Peer-reviewed✦ AIEmpirical audit finds only 38% of AI image generators implement adequate watermarking and 18% deepfake labelling, exposing a compliance gap under EU AI Act Article 50.
- Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways Peer-reviewed✦ AIReviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats.
- European ambitions captured by American clouds: digital sovereignty through Gaia-X? Peer-reviewed✦ AIShows Gaia-X paradoxically incorporates dominant US cloud providers, undermining the very European digital sovereignty it was meant to advance.
- AI, Climate, and Regulation: From Data Centers to the AI Act Peer-reviewed✦ AIAnalyses the legal levers (AI Act energy-reporting duties, Energy Efficiency Directive data-centre KPIs, sustainability reporting) for governing AI's climate footprint and their disclosure gaps.
- Governing AI Agents Preprint✦ AIUses "agency law and theory to identify and characterize problems arising from AI agents" and proposes governance infrastructure built on inclusivity, visibility, and liability.
- Infrastructure for AI Agents Peer-reviewed✦ AIProposes "agent infrastructure": external technical systems for attributing actions "to specific agents, their users, or other actors," shaping interactions, and remediating harms.
- Multi-Agent Risks from Advanced AI Research institute✦ AIIdentifies three failure modes of advanced multi-agent systems — "miscoordination, conflict, and collusion" — plus seven risk factors, posing challenges distinct from single-agent AI.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- The Current Landscape of Deepfake Legislation in the United States Peer-reviewed✦ AIThematic analysis of 319 state deepfake bills (2019-2024) finds a fragmented patchwork concentrated on political and sexually-explicit content.
- Reimagining U.S. Tort Law for Deepfake Harms: Comparative Insights from China and Singapore Peer-reviewed✦ AIArgues fragmented US tort doctrines (defamation, publicity, IIED) are ill-suited to deepfake harms and draws remedial lessons from Chinese and Singaporean law.
+ 113 more across this instrument's topics — see the literature index.
References
Sources cited inline in the analysis (linked from the superscript markers), then the primary instrument sources behind the classifications.
- Noam Kolt (2025) Governing AI Agents, Notre Dame Law Review (forthcoming). arXiv:2501.07913 — Uses "agency law and theory to identify and characterize problems arising from AI agents" and proposes governance infrastructure built on inclusivity, visibility, and liability. ↩
- Lewis Hammond, Alan Chan, Jesse Clifton, et al. (Cooperative AI Foundation) (2025) Multi-Agent Risks from Advanced AI, Cooperative AI Foundation. arXiv:2502.14143 — Identifies three failure modes of advanced multi-agent systems — "miscoordination, conflict, and collusion" — plus seven risk factors, posing challenges distinct from single-agent AI. ↩
- David Fernández-Llorca, Emilia Gómez, Ignacio Sánchez, Gabriele Mazzini (2025) An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI, Artificial Intelligence and Law. 10.1007/s10506-024-09412-y — Traces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime. ↩
- Martina Hulok (2025) The EU model of AI governance: regulating artificial intelligence through law and policy, ERA Forum. 10.1007/s12027-025-00869-1 — Analyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'. ↩
- Bram Rijsbosch, Gijs van Dijck, and Konrad Kollnig (2026) Missing the Mark: Adoption of Watermarking for Generative AI Systems in Practice and Implications Under the New EU AI Act, Policy & Internet. 10.1002/poi3.70041 — Empirical audit finds only 38% of AI image generators implement adequate watermarking and 18% deepfake labelling, exposing a compliance gap under EU AI Act Article 50. ↩
- Mateusz Łabuz (2025) A Teleological Interpretation of the Definition of DeepFakes in the EU Artificial Intelligence Act—A Purpose-Based Approach to Potential Problems With the Word 'Existing', Policy & Internet. 10.1002/poi3.435 — Warns a narrow reading of 'existing' in the AI Act's deepfake definition could exclude synthetic media from transparency duties, urging a teleological interpretation. ↩
- Bao Kham Chau and George He (2025) Audio deepfakes and the regulation of the landlords of creativity, Cambridge Forum on AI: Law and Governance. 10.1017/cfl.2025.10011 — Argues US, EU and Chinese regimes fail to assign audio-deepfake liability to 'landlords of creativity' (foundation-model providers) and proposes holding them accountable. ↩
- Valentine Ugwuoke and Madelyn Rose Sanfilippo (2025) The Current Landscape of Deepfake Legislation in the United States, Journal of Information Policy. 10.5325/jinfopoli.15.2025.0004 — Thematic analysis of 319 state deepfake bills (2019-2024) finds a fragmented patchwork concentrated on political and sexually-explicit content. ↩
- Kirolos Eskandar (2026) Artificial intelligence and synthetic biology: biosecurity risks, dual-use concerns, and governance pathways, AI and Ethics (Springer). 10.1007/s43681-025-00872-9 — Reviews biosecurity and dual-use risks at the AI-synthetic-biology interface and maps governance pathways for emerging catastrophic threats. ↩
- Andreas Baur (2026) European ambitions captured by American clouds: digital sovereignty through Gaia-X?, Information, Communication & Society. 10.1080/1369118X.2025.2516545 — Shows Gaia-X paradoxically incorporates dominant US cloud providers, undermining the very European digital sovereignty it was meant to advance. ↩
- André Ebert, Joseph Alder, Ralf Herbrich, Philipp Hacker (2026) AI, Climate, and Regulation: From Data Centers to the AI Act, Computer Law & Security Review. 10.1016/j.clsr.2026.106326 — Analyses the legal levers (AI Act energy-reporting duties, Energy Efficiency Directive data-centre KPIs, sustainability reporting) for governing AI's climate footprint and their disclosure gaps. ↩
- Alan Chan, Kevin Wei, Sihao Huang, Nitarshan Rajkumar, Elija Perrier, Seth Lazar, Gillian K. Hadfield, Markus Anderljung (2025) Infrastructure for AI Agents, Transactions on Machine Learning Research. arXiv:2501.10114 — Proposes "agent infrastructure": external technical systems for attributing actions "to specific agents, their users, or other actors," shaping interactions, and remediating harms. ↩
- Noam Kolt (2025) Governing AI Agents, Notre Dame Law Review (forthcoming). source — Uses "agency law and theory to identify and characterize problems arising from AI agents" and proposes governance infrastructure built on inclusivity, visibility, and liability. ↩
- Lewis Hammond, Alan Chan, Jesse Clifton, et al. (Cooperative AI Foundation) (2025) Multi-Agent Risks from Advanced AI, Cooperative AI Foundation. source — Identifies three failure modes of advanced multi-agent systems — "miscoordination, conflict, and collusion" — plus seven risk factors, posing challenges distinct from single-agent AI. ↩
- G7 Hiroshima AI Process, Oct 2023
- Code applies to advanced AI
- Code §5 (content provenance + watermarking)
- Code §2 (publicly report capabilities, limitations)
- Code §1 + §3 — explicit risk-identification including CBRN
- Adoption-by-developer framing; G7 carries implicit sovereignty assumptions
- Code §1 'advanced AI systems' + §3 risk-identification cover agentic behaviour through capability frame
- Code §6 — 'develop and deploy reliable content authentication and provenance mechanisms'
- Code §6 references sustainable AI development; not detailed obligation
How to cite this article
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/g7-hiroshima-code — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Does this instrument’s approach work? — the social-science evidence
Aggregated over the 8 topics this instrument governs: whether each harm is empirically real, and whether the peer-reviewed evidence shows governance reduces it. The badge is the epistemic status of the evidence— “thin”/“absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.
Of the 8 governed topics with a social-science evidence review, evidence that governance reduces the harm is established for 0, contested for 0, thin for 2, and absent for 6 — for most, no replicated study yet shows this instrument's approach works (the "second silence").
Agentic AI Governance
The capability that agentic governance targets — autonomous multi-step action — is real and rapidly, measurably advancing: METR finds the task length AI agents complete at 50% reliability has doubled roughly every seven months for the past six years (about 50 minutes for frontier 2025 models), and the UK AI Security Institute's first Frontier AI Trends Report (Dec 2025, >30 systems) reports models now finish hour-long software tasks >40% of the time versus <5% in late 2023. The distinct realized HARM from agency (as opposed to the underlying model) is, however, thinly documented: on consequential real-world tasks agents still fail the majority — Gemini 2.5 Pro completed only 30.3% of TheAgentCompany's 175 professional tasks (OpenHands scaffold, project leaderboard) — so the agency-specific harm magnitude is early and context-dependent rather than established at scale.
Sources: Kwa, West, Becker et al. 2025 (METR; arXiv:2503.14499, 'Measuring AI Ability to Complete Long Tasks'); UK AI Security Institute 2025 (Frontier AI Trends Report, Dec 2025); Xu, Song, Zhou et al. 2024 (TheAgentCompany, arXiv:2412.14161); 30.3% figure per TheAgentCompany leaderboard (OpenHands)
There is no impact-evaluation evidence that agent-specific governance reduces agentic harm: the operative regimes — the EU GPAI Code of Practice (published July 2025, voluntary/non-binding), the Seoul Frontier AI Safety Commitments (2024, voluntary), and AISI agent evaluations — are 2024-25 vintage and have never been measured against an outcome. The scholarship itself has not settled the contested unit of regulation: Kolt (2025) argues for governing the agentic relationship via principal-agent and agency-law tools, while Chan, Ezell, Kaufmann et al. (2024) propose agent-specific visibility mechanisms (identifiers, real-time monitoring, activity logging) that remain proposal-stage and unevaluated — meaning the field has design proposals but, as with most frontier-AI rules, the evidence that any of them works is absent rather than merely thin.
Sources: Kolt 2025 ('Governing AI Agents', 101 Notre Dame L. Rev., forthcoming; arXiv:2501.07913); Chan, Ezell, Kaufmann et al. 2024 ('Visibility into AI Agents', ACM FAccT 2024, pp. 958-973; DOI 10.1145/3630106.3658948); EU AI Office 2025 (GPAI Code of Practice, July 2025); Seoul Frontier AI Safety Commitments 2024
Catastrophic & Existential Risk
The catastrophic-uplift premise is genuinely contested: the empirical uplift studies that exist find current frontier models add little. RAND's red-team study found no statistically significant difference in the viability of bioweapon-attack plans produced with vs. without LLMs (Mouton, Lucas & Guest 2024), and OpenAI's 100-participant trial found GPT-4 gave at most a mild, non-significant accuracy uplift (mean +0.88 out of 10 for PhD experts, +0.25 for students; Patwardhan et al. 2024). Honest caveat: the harm is forward-looking, not yet observed — expert opinion on the catastrophic tail is sharply split (median AI researcher puts ~5% on extremely-bad/extinction outcomes, mean ~9-16% across differently-framed questions, n=2,778; Grace et al. 2024), and forecasters underestimated how fast risk-relevant capabilities (e.g. virology troubleshooting) actually arrived (Forecasting Research Institute 2025), so the relevant capabilities are a moving target rather than a settled magnitude.
Sources: Mouton, Lucas & Guest 2024 (RAND RR-A2977-2, Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study); Patwardhan et al. 2024 (OpenAI, Building an Early Warning System for LLM-aided Biological Threat Creation); Grace et al. 2024 (Thousands of AI Authors on the Future of AI, arXiv:2401.02843); Forecasting Research Institute 2025 (Forecasting LLM-enabled Biorisk and the Efficacy of Safeguards)
There is essentially no impact evidence that catastrophic-risk governance reduces catastrophic risk, and structurally there cannot yet be: the harm is a low-probability civilisational tail event, so no controlled trial or before/after evaluation of a realised catastrophe is possible. The dominant instruments are recent, voluntary developer frameworks (Anthropic's Responsible Scaling Policy 2023; OpenAI's Preparedness Framework 2023) built on if-then capability thresholds the developers themselves describe as speculative and qualitative rather than validated risk thresholds. The closest evidence is adjacent and indirect: trained-in deceptive behaviours can persist through standard safety training (Hubinger et al. 2024) — a demonstration that current mitigation may be insufficient, not that any governance regime works — and Anthropic's documented loosening of earlier commitments (RSP 2025 dropped the original pledge to define higher-tier ASL evaluations before developing the corresponding models) illustrates that even the strongest voluntary regimes lack external enforcement or measured efficacy.
Sources: Anthropic 2023 (Responsible Scaling Policy); OpenAI 2023 (Preparedness Framework); Hubinger et al. 2024 (Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, arXiv:2401.05566); Hendrycks, Mazeika & Woodside 2023 (An Overview of Catastrophic AI Risks, arXiv:2306.12001)
Deepfakes / Synthetic Content
The flagship harm — non-consensual sexual deepfakes — is empirically real and sharply gendered: content audits find ~96-98% of deepfake videos online are non-consensual pornography overwhelmingly depicting women, and a pre-registered 10-country survey (>16,000 people) found 2.2% reporting victimization and 1.8% perpetration of synthetic intimate imagery, with documented mental-health, career, and participation harms. By contrast, the parallel claim that political/informational deepfakes UNIQUELY deceive is contested-to-refuted: experiments find deepfakes about as (not more) credible than equivalent text/audio fakes, and a 56-paper meta-analysis (k=137, N=86,155) puts unaided human detection near chance — implying a detection problem more than an exceptional-persuasion one.
Sources: Umbach, Henry, Beard & Berryessa 2024 (CHI '24, 'Non-Consensual Synthetic Intimate Imagery ... in 10 Countries'); Diel et al. 2024 (Computers in Human Behavior Reports 16:100538, deepfake-detection meta-analysis of 56 papers); Barari, Lucas & Munger 2025 (Journal of Politics 87(2), 'Political Deepfakes Are as Credible as Other Fake Media'); Flynn et al. 2022 (British Journal of Criminology, multi-country image-based sexual abuse study)
Direct impact evidence that deepfake governance reduces the targeted harm is sparse and, where it exists, discouraging: the one quasi-experimental evaluation (Cuevas & Horta Ribeiro 2025, synthetic-control across three platforms) found the U.S. TAKE IT DOWN Act's passage plus the MrDeepfakes shutdown did NOT suppress synthetic non-consensual imagery — posting rose above counterfactual baselines and displaced elsewhere. Technical enforcement is likewise unreliable: detectors fail to generalize to unseen generators (notably diffusion models) and are vulnerable to adversarial evasion, with in-the-wild accuracy well below benchmark figures. No rigorous evaluation yet shows a deepfake-specific law, takedown mandate, or watermarking scheme producing a sustained reduction in prevalence or harm.
Sources: Cuevas & Horta Ribeiro 2025 ('Deepfake Pornography is Resilient to Regulatory and Platform Shocks', arXiv:2602.02754); 'Adversarial Reality for Evading Deepfake Image Detectors' (ICCVW 2025); TAKE IT DOWN Act, S.146 / Pub. L. 119-12 (2025); CRS Legal Sidebar LSB11314
Environmental Impact of AI Training
The resource demands of AI compute are empirically documented at the model level: Strubell et al. (2019) quantified large-NLP training energy/carbon, Luccioni et al. (2023) estimated BLOOM's training at ~24.7 tCO2eq (dynamic power) rising to ~50.5 tCO2eq with manufacturing and deployment, Li et al. (2023) estimated GPT-3-scale training in US datacenters can evaporate on the order of hundreds of thousands of litres of freshwater (their central figure ~700,000 L), and Luccioni, Jernite & Strubell (2024) showed generative inference is markedly more energy-intensive per query than task-specific models; at the macro scale the IEA (2024) and de Vries (2023) document rapidly rising datacenter electricity demand. Honest caveat: absolute estimates vary by up to orders of magnitude with grid carbon intensity, hardware, utilisation and accounting boundaries, and cleanly attributing the AI-specific increment (versus general datacenter and crypto growth) remains genuinely contested — the IEA itself bundles AI with datacenters and crypto — so the existence of the footprint is established while its magnitude and trajectory are not.
Sources: Strubell, Ganesh & McCallum 2019 (ACL Anthology P19-1355; 'Energy and Policy Considerations for Deep Learning in NLP'); Luccioni, Viguier & Ligozat 2023 (JMLR 24; BLOOM 176B carbon footprint, 24.7/50.5 tCO2eq; arXiv:2211.02001); Li, Yang, Islam & Ren 2023 (arXiv:2304.03271, 'Making AI Less Thirsty', later Comm. ACM 2025); Luccioni, Jernite & Strubell 2024 (ACM FAccT '24, 'Power Hungry Processing', DOI 10.1145/3630106.3658542); de Vries 2023 (Joule 7(10):2191-2194, DOI 10.1016/j.joule.2023.09.004); IEA 2024 (Electricity 2024)
There is no impact evaluation showing that any AI-specific environmental-governance instrument reduces energy, water or carbon use, because every named instrument is voluntary or non-binding and very recent: EU AI Act Art. 95 codes of conduct are explicitly optional with no sanctions, and NIST AI 600-1 and the G7 Hiroshima Code are guidance, not enforceable caps. The closest analogue evaluation literature is divided in a way that disfavours the voluntary form chosen here: rigorous reviews find voluntary environmental programs generally fail to produce significant abatement beyond business-as-usual (Koehler 2007; Morgenstern & Pizer 2007), whereas the one form with credible positive evidence is mandatory disclosure (Downar et al. 2021 found a UK carbon-reporting mandate cut emissions ~8% versus a control group) which the AI instruments do not yet impose, leaving the proposition that AI environmental governance works essentially untested.
Sources: EU AI Act Art. 95 / Recital 142 (Reg. (EU) 2024/1689); NIST AI 600-1 (2024, GenAI Profile); G7 Hiroshima Process International Code of Conduct (30 Oct 2023); Koehler 2007 (Policy Studies Journal 35(4):689-722); Morgenstern & Pizer (eds.) 2007 (Reality Check, RFF Press); Downar, Ernstberger, Reichelstein, Schwenen & Zaklan 2021 (Review of Accounting Studies 26(3):1137-1175)
Foundation Models / GPAI
Whether the foundation-model category maps to a coherent capability/risk tier is genuinely contested. The original case rests on scale-driven 'emergent abilities' that appear unpredictably above a size threshold (Wei et al. 2022; Ganguli et al. 2022 documented capabilities that are smoothly predictable in aggregate loss yet locally surprising), but Schaeffer, Miranda & Koyejo (2023, a NeurIPS Outstanding Paper) showed many 'emergent' jumps are artefacts of discontinuous metrics and dissolve under linear/continuous scoring — implying capability scales more smoothly than a sharp tier would suggest. Honest caveat: this is a live empirical disagreement about measurement, not a settled finding either way, and compute (the regulatory proxy) is an imperfect stand-in for capability or risk regardless of which side is right.
Sources: Wei et al. 2022 (Emergent Abilities of Large Language Models, TMLR; arXiv:2206.07682); Schaeffer, Miranda & Koyejo 2023 (Are Emergent Abilities of Large Language Models a Mirage?, NeurIPS 2023, Outstanding Paper; arXiv:2304.15004); Ganguli et al. 2022 (Predictability and Surprise in Large Generative Models, ACM FAccT; DOI 10.1145/3531146.3533229)
There is no impact evaluation showing that GPAI/foundation-model governance reduces harm — the rules are too new (EU AI Act GPAI obligations and the 10^25-FLOP systemic-risk presumption only began binding on 2 August 2025) and the central regulatory lever is itself contested: Hooker (2024) argues compute thresholds are a shortsighted proxy because compute does not reliably track capability or risk, and the thresholds already diverge across jurisdictions (EU 10^25 vs. the now-rescinded US EO 14110's 10^26 operations, rescinded 20 January 2025). The mandated mitigation methods also lack validated efficacy: model evaluation and red-teaming face well-documented coverage limits and an 'audit gap' in the survey/position literature (behavioural testing cannot establish the absence of untested failure modes), and adversarial red-teaming repeatedly defeats deployed safeguards — the UK AI Safety Institute reports finding universal jailbreaks for every frontier system it has tested, and a large public agent-injection competition elicited policy violations across all 22 frontier models tested from ~1.8M attacks (Zou et al. 2025). Even compliant evaluation therefore cannot yet certify the safety the rules demand. (Caveat: this is an absence-of-evidence claim — no efficacy study has been done — not evidence the rules are ineffective.)
Sources: Hooker 2024 (On the Limitations of Compute Thresholds as a Governance Strategy, arXiv:2407.05694); EU AI Act Arts. 51 & 55 (GPAI systemic-risk presumption, 10^25 FLOP; binding 2 Aug 2025); US EO 14110 (10^26-operation reporting threshold, rescinded 20 Jan 2025 by EO 14148); Zou et al. 2025 (Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition / Gray Swan Arena, arXiv:2507.20526 — 22 frontier agents, ~1.8M attacks); UK AI Safety/Security Institute, Frontier AI Trends Report (universal jailbreaks for every system tested); METR, Common Elements of Frontier AI Safety Policies (2024)
Synthetic Content Provenance
The harm provenance targets is real but concentrated, and the technical premise that the mandated signal survives is itself empirically shaky. Synthetic-media harm is well documented in two domains: non-consensual intimate imagery (Ajder et al.'s 2019 Deeptrace audit found 96% of deepfake videos were pornographic and effectively 100% targeted women) and impersonation fraud (the Arup case, ~US$25.6M / HK$200M lost via a deepfake video call). The honest caveat is twofold: a feared broad political-misinformation harm is not yet demonstrated at scale, and CS work shows invisible watermarks are removable in practice (Jiang, Zhang & Gong 2023, WEvade, evade detection via adversarial perturbation; Zhao et al. 2024 prove pixel-level watermarks are provably removable via regeneration attacks), so the provenance signal a rule would mandate is itself contested.
Sources: Ajder, Patrini, Cavalli & Cullen 2019 (Deeptrace, 'The State of Deepfakes: Landscape, Threats, and Impact'); Jiang, Zhang & Gong 2023 ('Evading Watermark based Detection of AI-Generated Content', ACM CCS 2023); Zhao et al. 2024 (NeurIPS, 'Invisible Image Watermarks Are Provably Removable Using Generative AI'); Arup deepfake fraud (CNN Business, 2024-05-16, US$25.6M)
There is no impact evaluation showing that mandated provenance/labeling reduces synthetic-media harm; the major mandates (China's GenAI labeling Measures, effective 2025-09-01; EU AIA Art. 50, machine-readable marking) are too new and unevaluated, and the delivery layer is leaky: the C2PA spec's own Security Considerations document the strip-and-repost threat, and platform audits report C2PA/Content-Credentials metadata is stripped by essentially all major social platforms on upload (consistent with Imatag's 2018 finding that ~80% of uploaded images lose metadata, only ~15% retaining it). The closest analogue evaluation literature — Pennycook, Bear, Collins & Rand (2020), the 'implied truth effect' — gives reason for caution rather than confidence: labeling only some content can make unlabeled false content seem more credible, so a partial-coverage provenance regime could backfire.
Sources: Pennycook, Bear, Collins & Rand 2020 (Management Science 66(11):4944-4957, 'The Implied Truth Effect'); China Measures for Labeling AI-Generated Synthetic Content (eff. 2025-09-01); EU AI Act Art. 50; Imatag 2018 metadata-stripping study (~80%); C2PA Security Considerations (spec.c2pa.org) on manifest removal
Technological Sovereignty
The structural fact that compute capacity is geographically concentrated is well-measured: Lehdonvirta, Wú & Hawkins find only ~33 countries host facilities with AI-accelerator hardware and roughly 24 have the capacity to train full-scale foundation models, the Stanford AI Index 2026 reports low-income countries collectively hold ~0.1% of global data-centre compute (the US hosting >10x any other nation), and Cottier et al. document amortized frontier-training cost rising 2.4x/year (95% CI 2.0-3.1x) toward $1B+ models by 2027. But this is a political-economy FRAME, not a documented harm, and the core contested claim of the topic, that the cost curve locks mid-sized economies OUT of capability, is empirically cut both ways: a feasibility study of Brazil and Mexico (Malagon et al. 2025) estimates usable (non-frontier) 10-trillion-token sovereign models are fiscally viable at roughly $8-14M on H100 hardware, and DeepSeek-style efficiency gains (V3 trained for ~$5.5M, ~11x less compute than Llama 3 405B) show frontier-adjacent performance at a fraction of prior compute, so whether domestic frontier-tier capability is foreclosed for middle powers remains genuinely unsettled.
Sources: Lehdonvirta, Wú & Hawkins 2024 (Compute North vs. Compute South, Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics & Society 7:828-838); Cottier, Rahman, Fattorini, Maslej & Owen 2024 (The Rising Costs of Training Frontier AI Models, arXiv:2405.21015); Stanford AI Index 2026 (Maslej et al., Stanford HAI); Malagon, Ulloa Ruiz, Sandoval Plaza, Rosario Bolívar, García Mesa & Alvarado Morales 2025 (The Feasibility of Training Sovereign Language Models in the Global South: A Study of Brazil and Mexico, arXiv:2510.19801)
There is no rigorous impact evaluation showing that technological-sovereignty policies (on-shore compute mandates, national foundation-model champions, talent-retention schemes such as EuroHPC AI Factories or India's IndiaAI Mission) actually deliver sustained domestic capability or strategic autonomy; these programs are recent, utilization and cost-per-GPU-hour are largely unpublished, and no counterfactual study exists. The closest analogue evidence base, the industrial-policy literature synthesized by Juhász, Lane & Rodrik, finds that properly-identified studies are more favorable than older correlational work suggested but that outcomes depend heavily on instrument design and structural context, and the older national-champion record warns of subsidized 'zombie' firms and government capture, so the closest analogue is mixed and the direct evidence that the sovereignty rule works is simply missing.
Sources: Juhász, Lane & Rodrik 2024 (The New Economics of Industrial Policy, Annual Review of Economics 16:213-242); Ahmed & Wahed 2020 (The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research, arXiv:2010.15581); IndiaAI Mission (Indian Cabinet, March 2024); EuroHPC Joint Undertaking AI Factories (2024 regulation amendment; no published impact evaluation)
Transparency Obligations
Documentation artifacts (model cards, datasheets) are well-specified as proposals and are genuinely adopted, but the empirical premise that mandated disclosure produces meaningful transparency is contested. Selbst & Barocas (2018) argue inscrutability and non-intuitiveness are distinct problems and that disclosing rules does not resolve the latter, and large-scale audits find documentation is sparsely and unevenly completed: a systematic analysis of 32,111 Hugging Face model cards (Liang et al. 2024) found environmental-impact, limitations and evaluation sections least often filled, and Bhat et al. (2023, 45 practitioners) found a substantial gap between the documentation proposal and actual practice. Honest caveat: the documentation frameworks themselves are real and adopted, so the dispute is about whether disclosure conveys decision-relevant information, not whether the artifacts exist.
Sources: Selbst & Barocas 2018 (Fordham Law Review 87:1085-1139); Liang et al. 2024 (Nature Machine Intelligence, s42256-024-00857-z, 'Systematic analysis of 32,111 AI model cards'); Bhat et al. 2023 (CHI '23, 'Aspirations and Practice of ML Model Documentation', DOI 10.1145/3544548.3581518); Mitchell et al. 2019 (FAccT, Model Cards for Model Reporting); Gebru et al. 2021 (CACM 64(12):86-92, Datasheets for Datasets)
There is no rigorous impact evaluation showing that AI transparency mandates (model cards, training-data summaries) measurably reduce bias, misuse or accidents — the central regulatory assumption is empirically untested, partly because flagship mandates like EU AI Act Art. 53(1)(d) GPAI training-data summaries are only subject to AI Office enforcement/verification from 2 August 2026 (the obligation itself began 2 August 2025 for new models). The closest analogue, mandated consumer disclosure, shows small and context-dependent effects: Bollinger, Leslie & Sorensen (2011) found mandatory calorie posting cut average calories per transaction by about 6%, while Loewenstein, Sunstein & Golman (2014) review evidence that disclosure effects are frequently diminished or even reversed by limited attention and often change provider rather than recipient behavior. These are analogues, not AI studies; no study demonstrates that AI transparency disclosure achieves its stated downstream safety aims.
Sources: Bollinger, Leslie & Sorensen 2011 (AEJ: Economic Policy 3(1):91-128); Loewenstein, Sunstein & Golman 2014 (Annual Review of Economics 6:391-419, 'Disclosure: Psychology Changes Everything'); EU AI Act Art. 53(1)(d) GPAI training-data summary (obligation from 2 Aug 2025; AI Office enforcement from 2 Aug 2026)