Deepfakes / Synthetic Content

Regulatory approaches: four distinct mechanisms

Although the coverage matrix records eight instruments as governing deepfakes, they reach the topic through markedly different legal modalities, and the verdict labels alone obscure these design choices. Four mechanisms recur. (1) Deployer disclosure: the EU AI Act, Art. 50(4) obliges the deployer of a system generating deepfake image, audio or video to disclose that the content is artificial, with carve-outs for law-enforcement and for artistic, satirical or fictional work (AI Act, Art. 50(4)). (2) Machine-readable marking at the provider level: AI Act Art. 50(2) separately requires providers to mark synthetic outputs in a robust, machine-readable, state-of-the-art format — a technical-provenance duty distinct from the human-facing label (AI Act, Art. 50(2)). (3) Labelling plus distribution-platform verification: China's Measures for Labeling of AI-Generated Synthetic Content (effective 1 Sept 2025) mandate perceptible explicit labels and metadata-embedded implicit labels, and uniquely place a verification-and-flagging duty on content-distribution platforms, not only generators (Measures, Art. 6). (4) Notice-and-takedown: the US TAKE IT DOWN Act requires covered platforms to remove notified non-consensual intimate imagery, including digital forgeries, within 48 hours, enforced by the FTC (TAKE IT DOWN Act, Pub. L. 119-12 (2025), §3). These statutory choices sit atop a fragmented sub-national layer: a thematic analysis of 319 US state deepfake bills (2019-2024) finds a patchwork concentrated on political and sexually-explicit content rather than a coherent federal scheme ¹. Takedown-style regimes are themselves contested as insufficient: the UK Online Safety Act 2023 has been read as inadequately addressing non-consensual intimate deepfakes as image-based sexual abuse, leaving enforcement and removal gaps ². Voluntary provenance commitments (G7 Hiroshima Code §5; White House Voluntary Commitments §5) layer atop these but lack binding force. The mechanisms are not interchangeable: a labelling regime governs honest disclosure; a takedown regime governs removal of identified harm. China's labelling regime in fact predates the 2025 Measures: the 2022 Provisions on the Administration of Deep Synthesis already required deep-synthesis providers to place conspicuous labels on face-generation, face-swapping, face-manipulation and pose-manipulation outputs that significantly alter identity features (Deep Synthesis Provisions, Art. 17). India layers a takedown-and-advisory approach onto data-protection law, with MEITY's March-2024 advisory and the IT Rules 2021 §3(1)(b)(v) deepfake-takedown obligations directing intermediaries to remove synthetic impersonations (MEITY Mar-2024 Advisory + IT Rules 2021 §3(1)(b)(v)). Singapore's non-binding Model AI Governance Framework for Generative AI similarly addresses synthetic media through its content-provenance and synthetic-content-disclosure dimension (Framework Dimension 7).

Definitional contestation: what counts as a deepfake

A recurring fault line precedes enforcement: the term "deepfake" lacks a settled legal meaning, and the boundary it draws determines which content is regulated. The EU AI Act offers the most consequential statutory definition — Art. 3(60) defines a deepfake as AI-generated or manipulated image, audio or video content "that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful" (AI Act, Art. 3(60)). Each qualifier is contested. Łabuz (2025, Policy & Internet) argues that a literal reading of "existing" risks excluding wholly fabricated but realistic personas — synthetic faces of no real individual — from the Art. 50(4) transparency duty, and urges a teleological, purpose-based interpretation to avoid that gap ³. Meding and Sorge (2024) press a complementary problem: the line between "legitimate processing" (routine retouching, colour correction, denoising) and regulated "manipulation" is underspecified, leaving the threshold of artificiality indeterminate ⁴. The definitional choice has downstream effects: a narrow scope under-includes harmful synthetic media, while a broad scope sweeps in ordinary edits and burdens benign use. US instruments sidestep the abstraction by defining narrowly around harm — the TAKE IT DOWN Act targets "digital forgeries" of intimate imagery rather than deepfakes generally (Pub. L. 119-12 §2). The field has therefore not converged on whether deepfake regulation should key on technique, resemblance, deceptive intent, or downstream harm.

Key fault lines: where jurisdictions and experts diverge

Beyond definition, governance is divided on several structural questions. First, the durability of technical provenance. Policy has converged on watermarking and content credentials, yet the technical literature questions whether they survive adversarial conditions: Zhao et al. (2024, NeurIPS) prove that invisible pixel-level watermarks are removable via diffusion-based regeneration attacks ⁵, and provenance manifests under the C2PA standard — adopted by Adobe, Google, Meta, OpenAI and camera makers — are stripped by routine re-encoding and screenshotting, recording asserted history rather than verifying authenticity (NSA/CISA, "Content Credentials," 2025). Detection fares no better as a backstop: Harris (2024) argues detector-based solutions depend on scarce institutional trust and risk undermining epistemic autonomy, so purely technological fixes are dim ⁶. Second, the point of obligation: the EU splits duties between provider (marking) and deployer (disclosure); China additionally conscripts distribution platforms to verify and surface labels on redistributed content (Measures 2025, Art. 6); the US TAKE IT DOWN model loads the duty onto hosting platforms post-publication — a focus on intermediaries that critics say still misses upstream foundation-model providers, the "landlords of creativity" who escape audio-deepfake liability across all three regimes ⁷. The active US federal lever has also narrowed: Executive Order 14110's §4.5 content-authentication and watermarking directive — which the coverage matrix still records as a (now historical) governing provision — was rescinded on 20 January 2025 by EO 14148, and the successor EO 14179 is silent on synthetic media, leaving the statutory TAKE IT DOWN Act and persisting NIST provenance guidance (rather than a binding federal labelling mandate) as the operative US instruments. Third, transparency versus prohibition: critics argue the AI Act's placement of deepfakes in the limited-risk transparency tier leaves no ban and no victim remedy, treating an information harm as a labelling problem ³. Pending US federal bills — the NO FAKES Act (S.1367, 119th Cong.) on digital replicas and the DEFIANCE Act on non-consensual intimate deepfakes — would instead create property-like and civil-remedy rights, signalling an unresolved divergence over whether deepfakes are best governed as disclosure failures or as actionable wrongs.

Historical primacy & cross-jurisdiction tension

First addressed by Provisions on the Administration of Deep Synthesis of Internet Information Services on 2022-11-25 (governs). Subsequent regimes have either codified, diverged from, or remained silent on this baseline.

• Forum-shoppingEU AI Act↔Executive Order 14179 — Removing Barriers to American Leadership in AI
• Forum-shoppingExecutive Order 14110 on Safe, Secure, Trustworthy AI↔UK Pro-Innovation Approach to AI Regulation (White Paper)
• Forum-shoppingInterim Measures for Generative AI Service Management↔OECD AI Principles (Recommendation)

Silent regimes — gap signal

Instruments that do not address Deepfakes / Synthetic Content — candidates for future policy work.

• Executive Order 14179 — Removing Barriers to American Leadership in AIUS
• UK Pro-Innovation Approach to AI Regulation (White Paper)UK
• OECD AI Principles (Recommendation)OECD
• Council of Europe Framework Convention on AIcouncil_of_europe
• Bletchley Declaration on AI Safetyglobal
• Seoul Declaration on Safe, Innovative and Inclusive AIglobal
• California SB-1047: Safe and Secure Innovation for Frontier AI Models ActUS
• Brazil AI Bill (PL 2338/2023)BR
• ASEAN Guide on AI Governance and EthicsASEAN
• African Union Continental AI StrategyAfrican_Union
• Anthropic Responsible Scaling Policy (RSP) v2US
• OpenAI Preparedness FrameworkUS
• Google DeepMind Frontier Safety FrameworkUS
• Meta Frontier AI FrameworkUS
• UK-US AI Safety Institute Memorandum of Understandingglobal
• Japan METI AI Guidelines for BusinessJP
• General Data Protection Regulation (GDPR)EU
• EU General-Purpose AI Code of PracticeEU
• OMB Memorandum M-24-10 (Advancing Governance, Innovation, and Risk Management for Agency Use of AI)US
• GSA Generative AI and Specialized Computing Infrastructure Acquisition Resource GuideUS
• DoD Responsible AI Strategy and Implementation PathwayUS
• FedRAMP AI Cloud Procurement GuidanceUS
• DFARS Subpart 252.204 (Safeguarding Covered Defense Information and Cyber Incident Reporting)US
• California SB-53: Transparency in Frontier Artificial Intelligence Act (TFAIA)US
• California SB 243: Companion ChatbotsUS
• Revised Product Liability Directive (Directive (EU) 2024/2853)EU
• UNESCO Recommendation on the Ethics of Artificial IntelligenceUNESCO
• Directive (EU) 2024/2831 on improving working conditions in platform workEU
• New York RAISE Act: Responsible AI Safety and Education ActUS
• Japan AI Promotion Act (Act on the Promotion of Research, Development and Utilization of AI-Related Technologies)JP
• UN Global Digital CompactUN

Related enforcement cases

Documented enforcement actions where regulators invoked a rule against a deploying organisation, with the primary-source case record and the lesson the field has drawn from each.

• CAC Doubao deep-synthesis enforcementCN · 2024–2024
  Cyberspace Administration of China (CAC) v. ByteDance (Doubao) — Doubao deployed deepfake-generation feature without prior CAC algorithm registration; failed to embed mandatory deep-synthesis content-provenance label per Deep Synthesis Provisions Art. 16. CAC requested feature removal pending registration.
  Lesson: First named CN-GENAI-2023 enforcement against a frontier-tier Chinese AI developer. Demonstrates: (1) CN registration regime applies pre-deployment + has bite even against domestic champions; (2) cross-instrument enforcement (GenAI rules + Deep Synthesis rules) is operative; (3) remediation is rapid (weeks not years) but private — no consent decree text published.
  Source record →
• MEITY deepfake takedown advisoriesIN · 2023–2024
  Ministry of Electronics and Information Technology (MEITY) v. Multiple intermediaries — Meta, YouTube/Google, X, several Indian social platforms — Failure to take down political deepfake content within statutory windows (36 hours under IT Rules 2021). MEITY's 1 Mar 2024 advisory additionally required pre-deployment-approval for AI models above unspecified capability thresholds; superseded 15 Mar 2024 by a revised advisory dropping the approval requirement after industry pushback.
  Lesson: India's compressed legislative cycle: a sweeping pre-deployment-approval requirement (closer to CN registration than US sectoral) was rescinded within 5 weeks after industry + civil-society pushback. Demonstrates that Global South AI regulation is in active design AND that even nationally-coercive states face frontier-lab leverage. Indian regulatory approach now favours post-deployment incident reporting + IT-Rules takedown.
  Source record →
• CN CAC algorithm-recommendation rectification campaignCN · 2022–2023
  Cyberspace Administration of China (CAC) v. Multiple platforms: Douyin, Kuaishou, Xiaohongshu, Weibo, Taobao (named in CAC public action) — Unregistered or non-compliant algorithm-recommendation systems. Failure to provide opt-out mechanisms. Failure to register algorithms with CAC under the Algorithm Recommendation Provisions.
  Lesson: First operational implementation of pre-deployment AI registration regime. Demonstrated that CAC has enforcement bandwidth + technical capability to audit recommender algorithms at scale. Platforms responded by significantly altering algorithm transparency + opt-out flows. Cited as the working counter-example to the 'registration regimes are unenforceable' claim.
  Source record →

Does governance work? — the social-science evidence

What the peer-reviewed social science shows: whether the harm this topic addresses is empirically real, and whether governance of it works. The badge is the epistemic status of the evidence(not the policy debate) — “thin” or “absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.

• Is the harm real?evidence: established

  The flagship harm — non-consensual sexual deepfakes — is empirically real and sharply gendered: content audits find ~96-98% of deepfake videos online are non-consensual pornography overwhelmingly depicting women, and a pre-registered 10-country survey (>16,000 people) found 2.2% reporting victimization and 1.8% perpetration of synthetic intimate imagery, with documented mental-health, career, and participation harms. By contrast, the parallel claim that political/informational deepfakes UNIQUELY deceive is contested-to-refuted: experiments find deepfakes about as (not more) credible than equivalent text/audio fakes, and a 56-paper meta-analysis (k=137, N=86,155) puts unaided human detection near chance — implying a detection problem more than an exceptional-persuasion one.

  Sources: Umbach, Henry, Beard & Berryessa 2024 (CHI '24, 'Non-Consensual Synthetic Intimate Imagery ... in 10 Countries'); Diel et al. 2024 (Computers in Human Behavior Reports 16:100538, deepfake-detection meta-analysis of 56 papers); Barari, Lucas & Munger 2025 (Journal of Politics 87(2), 'Political Deepfakes Are as Credible as Other Fake Media'); Flynn et al. 2022 (British Journal of Criminology, multi-country image-based sexual abuse study)

• Does governance work?evidence: thin

  Direct impact evidence that deepfake governance reduces the targeted harm is sparse and, where it exists, discouraging: the one quasi-experimental evaluation (Cuevas & Horta Ribeiro 2025, synthetic-control across three platforms) found the U.S. TAKE IT DOWN Act's passage plus the MrDeepfakes shutdown did NOT suppress synthetic non-consensual imagery — posting rose above counterfactual baselines and displaced elsewhere. Technical enforcement is likewise unreliable: detectors fail to generalize to unseen generators (notably diffusion models) and are vulnerable to adversarial evasion, with in-the-wild accuracy well below benchmark figures. No rigorous evaluation yet shows a deepfake-specific law, takedown mandate, or watermarking scheme producing a sustained reduction in prevalence or harm.

  Sources: Cuevas & Horta Ribeiro 2025 ('Deepfake Pornography is Resilient to Regulatory and Platform Shocks', arXiv:2602.02754); 'Adversarial Reality for Evading Deepfake Image Detectors' (ICCVW 2025); TAKE IT DOWN Act, S.146 / Pub. L. 119-12 (2025); CRS Legal Sidebar LSB11314

Related topics

Topics with similar coverage patterns across the tracked instruments.

• Synthetic Content Provenance