Dual-Use Research Norms (DURC for AI)
dual-use-research-taxonomy · Frontier safety
A normative framework — adapted from biosecurity's Dual-Use Research of Concern (DURC) policies — for governing AI research and publication decisions when research outputs have both beneficial and harmful applications.
Definition & scope
Dual-use research norms in AI explicitly draw on the biosecurity precedent: the 1975 Asilomar conference on recombinant DNA, the 2004 US National Science Advisory Board for Biosecurity, and the 2014 US gain-of-function moratorium. The AI parallels are publication-control debates around GPT-2 (OpenAI's staged release, 2019), the deepfake-generation research community (FaceSwap-era, 2017-2020), CBRN-uplift research, and offensive cybersecurity capabilities (e.g., AutoAttack research). Field positions cluster: (a) full publication — Brundage et al. 2018 critique of selective release; (b) staged or structured access — Solaiman et al. 2019; (c) capability-thresholded redaction — Anthropic, OpenAI, DeepMind dual-use policies, 2023-2025. Governance instruments are catching up. US EO 14110 §4.2(a)(ii) explicitly required reporting on dual-use capabilities including CBRN, cyber, and autonomous-replication. EU AI Act Art. 5 prohibits certain dual-use applications (manipulation, social scoring) but does not regulate research-stage decisions. NIST AI RMF Map 1.1 includes 'risk of misuse' assessment but does not prescribe publication norms. The G7 Hiroshima Code §3 endorses 'responsible information sharing' without operationalising it. For AI safety researchers, dual-use research norms are the closest analogue to peer-review-style governance of which findings should be public — a research-community-internal governance layer that operates upstream of regulator-mandated controls.
Locus of dispute: Is the biosecurity DURC analogy applicable to AI? Information-spread dynamics differ fundamentally (Brundage 2023); the field has not converged on whether DURC-style governance translates.
From Biosecurity DURC to AI: The Borrowed Frame
Dual-use research norms in AI are transplanted, not native. The frame inherits a specific institutional lineage from the life sciences: the 1975 Asilomar conference on recombinant DNA, the 2004 US National Science Advisory Board for Biosecurity, and the 2014 gain-of-function moratorium (OSTP 2014). Each established that a research community can govern its own outputs upstream of any regulator, deciding which findings circulate freely and which are withheld or staged. The canonical AI articulation, Solaiman et al. 2019 1, recast this as 'release strategies' for language models after OpenAI's staged GPT-2 release. The transplant strains the moment the governed object is a broad, general-purpose capability rather than a discrete protocol: Eloundou et al. 2024 2 find LLMs exhibit 'traits of general-purpose technologies,' estimating that roughly 80% of the US workforce could have at least 10% of their work tasks affected — a diffusion surface unlike any single pathogen technique. The borrowed vocabulary (containment, redaction, thresholds) thus arrives pre-loaded with premises — that dangerous knowledge can be physically or socially contained — that the AI field has not independently validated.
Mechanisms: The Three Release Postures
Operationally the norm resolves into three field positions. Cluster (a), full publication, holds that openness maximises scrutiny, reproducibility, and defensive research, and that withholding rarely durably contains capability. Cluster (b), staged or structured access, is Solaiman et al. 2019's 1 middle path — graduated release, gated APIs, and access tiers that preserve oversight while curbing immediate misuse. Cluster (c), capability-thresholded redaction, ties disclosure to measured dangerous-capability levels; Phuong et al. 2024 3 pilots exactly such evaluations across persuasion, cyber, and self-proliferation, finding 'early warning signs' but no present danger, supplying the empirical trigger for if-then gating. Anderljung et al. 2023 4 supplies the surrounding apparatus these postures presuppose — safety standards, registration and reporting, and compliance mechanisms — without which a measured threshold has no channel to bind. The frontier-safety-framework cluster (12+ firms, 2023-2025) instantiates (c) with CBRN and cyber thresholds, making evaluation the load-bearing mechanism: the norm only binds if dangerous capability can be reliably measured before release (METR 2025).
Governance Relevance: Soft Law Meets Research-Stage Gaps
Formal instruments engage the concept obliquely and incompletely. US EO 14110 §4.2(a)(ii) was the most explicit, requiring reporting on dual-use capabilities spanning CBRN, cyber, and autonomous replication — though it was rescinded in January 2025 and replaced by EO 14179, leaving the reporting hook in flux. The EU AI Act (Regulation (EU) 2024/1689) prohibits certain dual-use applications under Art. 5 (manipulation, social scoring) but, tellingly, does not reach research-stage publication decisions at all; Novelli et al. 2024 5 map exactly this kind of residual gap across the Act, liability, privacy and cybersecurity rules as applied to generative AI. NIST AI RMF Map 1.1 folds in 'risk of misuse' assessment yet prescribes no publication norms, and the G7 Hiroshima Code §3 endorses 'responsible information sharing' without operationalising it. The structural pattern is that hard law targets deployment and prohibited uses, while the research-to-publication decision remains governed by firm-internal frameworks (Anthropic, OpenAI, DeepMind, Meta, 2023-2025) — consistent with Anderljung et al. 2023 4, who treat 'industry self-regulation' as 'an important first step' while warning 'government intervention will be needed'.
Debates: Does the Biosecurity Analogy Hold?
The field's central unresolved question is whether the biosecurity DURC analogy is even applicable to AI; the concept records the empirical consensus as contested. The strongest objection is that information-spread dynamics differ fundamentally from biological materials: a pathogen protocol and a model weight do not diffuse alike, so containment intuitions calibrated for wet labs may misfire on code that can be re-derived, leaked, or independently reproduced. Longpre et al. 2024 6 sharpen the point empirically — even as web sources race to restrict training access, with '~5%+ of all tokens in C4...fully restricted' inside a single year, the underlying data has already diffused, illustrating how late and partial information-side containment runs. This bears directly on whether thresholded redaction (cluster c) actually reduces risk or merely advantages well-resourced actors. A second strand concerns measurement legitimacy: if evaluation is the binding mechanism, governance inherits the contested validity of dangerous-capability tests — Phuong et al. 2024 3 report only 'early warning signs,' a thin basis for redaction. The honest position, per the concept's own caution, is that citations of dual-use norms in AI should travel with a note that the analogy has not converged.
Use in governance
How instruments operationalise this concept
| Instrument | Jurisdiction | Status |
|---|---|---|
| Executive Order 14110 on Safe, Secure, Trustworthy AI | US | partial |
| G7 Hiroshima AI Process Code of Conduct | G7 | in force |
| NIST AI Risk Management Framework | US | in force |
| Anthropic Responsible Scaling Policy (RSP) v2 | US | in force |
| OpenAI Preparedness Framework | US | in force |
| Google DeepMind Frontier Safety Framework | US | in force |
| Meta Frontier AI Framework | US | in force |
| White House Voluntary AI Commitments | US | in force |
Appears in topic articles
Editorial note
The biosecurity DURC analogy is contested: critics (Brundage 2023) argue that information-spread dynamics in AI are fundamentally different from biological materials. Pair citations of 'dual-use research norms in AI' with a note on the analogy's contested status. Currency (2026-06-21): Definition (DURC-derived norms for governing dual-use AI research/publication; biosecurity analogy still "contested") remains accurate and matches current 2025-26 framing; two governance-instrument references have drifted in status — EO 14110 (cited in scope §4.2(a)(ii)) was rescinded Jan 2025 and replaced by EO 14179, and the biosecurity-precedent 2024 US DURC/PEPP policy is being replaced under EO 14292 (May 2025) — but neither alters the concept definition; the capability-thresholded frontier-safety-framework cluster (12+ firms, if-then CBRN/cyber thresholds, METR Dec-2025) confirms cluster (c) is current.
See also
Further reading
Sources on the broader topics this concept relates to — complementing, not standing in for, the primary sources cited inline above. 71 academic & grey-literature sources; catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- Open Foundation Models and TDM Exceptions to Copyright – Building Blocks for an AI Ecosystem Peer-reviewed✦ AIArgues Art. 3 CDSM Directive's scientific-research TDM exception 'does not grant rightsholders any control' and can be a 'safe harbor' for training openly released foundation models without licensing data.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- Copyright and AI in the UK: Opting-In or Opting-Out? Peer-reviewed✦ AIContends the UK opt-in/opt-out framing is a 'missed opportunity'; a broadened research exception plus market-entry transparency and creator remuneration would better serve both innovation and rightsholders.
- Technical Challenges of Rightsholders' Opt-out From Gen AI Training after Robert Kneschke v. LAION Peer-reviewed✦ AIExamines post-LAION practical obstacles to the EU TDM opt-out (robots.txt, machine-readability, memorisation): 'While the TDM exceptions may seem workable in theory, implementing them in practice presents a variety of practical…
- GPTs are GPTs: Labor market impact potential of LLMs Peer-reviewed✦ AIFinds around 80% of the U.S. workforce "could have at least 10% of their work tasks affected" by LLMs, which exhibit "traits of general-purpose technologies".
- Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity Peer-reviewed✦ AIExamines how the EU AI Act, liability regimes, GDPR, copyright and cybersecurity rules apply to generative AI, identifying gaps and proposing targeted regulatory refinements.
- A large-scale audit of dataset licensing and attribution in AI Peer-reviewed✦ AIAudit of 1,800+ AI training datasets finds "licence omission rates of more than 70% and error rates of more than 50%" on popular hosting sites.
- Evaluating Frontier Models for Dangerous Capabilities Preprint✦ AIPilots dangerous-capability evaluations (persuasion, cyber, self-proliferation) on frontier models, finding 'early warning signs' but no strong present danger — grounding evaluation-based gating.
- The Right to Transparency in Public Governance: Freedom of Information and the Use of Artificial Intelligence by Public Agencies Peer-reviewed✦ AIFinds freedom-of-information regimes "generally only grant access to existing documents" and that with "no mature standard for documenting AI models," public-sector AI transparency is limited.
- On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives Peer-reviewed✦ AISynthesises interdisciplinary evidence to argue that legally mandated human oversight of AI is often ineffective ('rubber-stamp') unless effectiveness conditions are explicitly designed for.
+ 59 more across this concept's topics — see the literature index.
References
Sources cited inline in the analysis, numbered in order of appearance.
- Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, et al. (2019) Release Strategies and the Social Impacts of Language Models, arXiv (OpenAI). arXiv:1908.09203 — Documents OpenAI's GPT-2 staged-release experiment, arguing 'staged release allows time between model releases to conduct risk and benefit analyses' and proposing publication norms for powerful models. ↩
- Eloundou, Manning, Mishkin, Rock (2024) GPTs are GPTs: Labor market impact potential of LLMs, Science. 10.1126/science.adj0998 — Finds around 80% of the U.S. workforce "could have at least 10% of their work tasks affected" by LLMs, which exhibit "traits of general-purpose technologies". ↩
- Mary Phuong, Matthew Aitchison, Elliot Catt, et al. (Google DeepMind) (2024) Evaluating Frontier Models for Dangerous Capabilities, arXiv (cs.LG). arXiv:2403.13793 — Pilots dangerous-capability evaluations (persuasion, cyber, self-proliferation) on frontier models, finding 'early warning signs' but no strong present danger — grounding evaluation-based gating. ↩
- Anderljung, Barnhart, Korinek, et al. (2023) Frontier AI Regulation: Managing Emerging Risks to Public Safety, arXiv. arXiv:2307.03718 — Argues "industry self-regulation is an important first step" but "government intervention will be needed", proposing safety standards, registration and reporting, and compliance mechanisms. ↩
- Novelli, Casolari, Hacker, Spedicato & Floridi (2024) Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity, Computer Law & Security Review. 10.1016/j.clsr.2024.106066 — Examines how the EU AI Act, liability regimes, GDPR, copyright and cybersecurity rules apply to generative AI, identifying gaps and proposing targeted regulatory refinements. ↩
- Shayne Longpre, Robert Mahari, Ariel Lee, et al. (2024) Consent in Crisis: The Rapid Decline of the AI Data Commons, arXiv (Data Provenance Initiative; presented NeurIPS Dataset. arXiv:2407.14933 — Longitudinal audit of 14,000 web domains finds a 2023-24 surge in AI training restrictions, with '~5%+ of all tokens in C4...fully restricted from use' within a single year. ↩
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/dual-use-research-taxonomy — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here
Social-science evidence — the “so-what”
What the peer-reviewed social science shows: whether the harm this concept addresses is empirically real, and whether governance of it works. The badge is the epistemic status of the evidence(not the policy debate) — “thin” or “absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.
The underlying phenomenon is real and concretely instantiated: OpenAI's GPT-2 staged release was conducted explicitly to allow risk/benefit analysis of misuse (synthetic-media and impersonation concerns) before full publication (Solaiman et al. 2019), and the dual-use-research-of-concern category has a formal, coherent definition in the biosecurity domain (US Government DURC Policy 2012/2014). Whether the biosecurity DURC framework actually TRANSFERS to AI is genuinely contested in the broader literature: critics note AI's information-spread dynamics differ sharply (digital replicability, leakable model weights, low tacit-knowledge barriers vs. the lab-bound, materials-gated diffusion DURC was built for). The category is thus coherently defined but its analogical applicability to AI is disputed, not established. Attribution caveat: the contested-disanalogy argument is NOT made by the two AI-publication sources cited here — Partnership on AI (2021) invokes biosecurity APPROVINGLY as an analogous field to learn from, and Solaiman et al. (2019) is a GPT-2 case study, not a transfer critique; the disanalogy claim is corroborated elsewhere in the literature (e.g., work differentiating language-model vs. biological-design-tool risks) but is not sourced to these citations.
Sources: Solaiman et al. 2019 (Release Strategies and the Social Impacts of Language Models, arXiv:1908.09203); Partnership on AI 2021 (Managing the Risks of AI Research: Six Recommendations for Responsible Publication); US Government Policy for Oversight of Dual Use Research of Concern 2012/2014 (NSABB/NIH)
There is no rigorous impact evaluation showing that DURC-style publication norms measurably reduce downstream AI-misuse harm. The GPT-2 staged release was an uncontrolled single case with no counterfactual (Solaiman et al. 2019). The NeurIPS 2020 broader-impact-statement requirement was removed after a single year in favour of a checklist, and the post-hoc analysis identified structural shortcomings — weak incentives, unclear expectations and guidance — rather than demonstrated protective effect (Ashurst et al. 2021). Even the originating biosecurity analogue (the H5N1 gain-of-function moratorium and DURC oversight) is documented as having produced debate, dialogue and framework creation but no quantified harm-reduction or protective outcomes (Federation of American Scientists 2013). The evidence that this governance lever works is essentially absent, and its closest real-world analogue is itself unproven on harm-reduction. (Softened: Ashurst et al. characterise 'lessons to be learnt' rather than a flat failure verdict; the absence-of-protective-evidence claim is what survives, not a claim that the mechanism was shown harmful.)
Sources: Ashurst et al. 2021 (AI Ethics Statements — Analysis and lessons learnt from NeurIPS Broader Impact Statements, arXiv:2111.01705); Solaiman et al. 2019 (Release Strategies and the Social Impacts of Language Models, arXiv:1908.09203); Federation of American Scientists 2013 (The Moratorium on H5N1 Gain-of-Function Experiments, Winter 2013 Public Interest Report)