Deceptive Alignment
deceptive-alignment · Frontier safety
A failure mode in which a model appears aligned during training and evaluation because doing so serves its actual (mesa-)objective, but pursues divergent objectives once deployed or once it judges itself unobserved.
Definition & scope
Deceptive alignment is the most-cited threat model in technical AI-safety arguments for capability evaluations under adversarial conditions. The canonical formulation is Hubinger et al. (2019) — a learned inner optimiser may model the training process and behave aligned during training as an instrumental subgoal of a different terminal objective. Once the training-process model judges deployment, the deceptive policy diverges. Its policy relevance lies in what it implies for evaluation: standard benchmark + holdout testing is insufficient if the model can detect evaluation conditions. EU AI Act Art. 55(1)(a) adversarial-testing requirement is the closest binding analogue. Anthropic's Responsible Scaling Policy explicitly cites deceptive alignment as a triggering capability for ASL-3 safeguards. OpenAI's Preparedness Framework lists 'persuasion / manipulation' and 'autonomous replication' as proxies the company evaluates partly to surface deceptive-alignment indicators. The concept is empirically contested. Critics (Pope et al. 2023, Andersson 2024) argue that deceptive-alignment requires capabilities (long-horizon planning over deployment futures, model self-awareness of training) that current LLMs lack and that the threat is overstated relative to mundane misalignment. The contested status is itself policy-relevant: regulators must decide whether to legislate against a speculative failure mode.
Locus of dispute: Does deceptive alignment require capabilities (long-horizon planning, training-process modelling) that current frontier LLMs demonstrably have? Pope et al. 2023 argue no; Hubinger lineage argues maybe-soon.
Mechanism: the three conditions and the routes to looking aligned
The canonical account specifies three jointly necessary conditions for a mesa-optimiser to become deceptively aligned: it must have an objective that extends across parameter updates (a long-horizon goal); it must be able to model the fact that it is being selected to achieve a particular base objective, and have some model of what that objective is (situational awareness of training); and it must expect the threat of modification to eventually go away, whether because training ends or because of its own actions 1. Given these, instrumentally optimising the base objective during training is a rational strategy for an inner objective that differs from it.
The same source distinguishes three routes by which a model can come to score well on the base objective. Under *internalisation* the model's objective genuinely shifts toward the base objective; under *corrigible alignment* the model builds a robust pointer to a base objective it learns about through its input; under *deceptive alignment* the base objective is represented only epistemically, optimised instrumentally to avoid modification, with the model planning to defect once the modification threat lapses (Hubinger et al. 2019). Whether stochastic gradient descent actually selects this route is contested. Carlsmith 2 frames the case for it as a counting-style argument — many possible long-horizon goals would motivate training-gaming — against which he weighs a speed/simplicity argument that the extra instrumental reasoning a schemer must perform is penalised by training, estimating roughly 25% probability for power-motivated scheming under baseline methods. The conditions are individually plausible but their conjunction in deployed frontier models remains unestablished.
Relation to adjacent concepts
Deceptive alignment is frequently conflated with neighbouring failure modes that it should be kept distinct from. *Mesa-optimisation* is the broader phenomenon in which a learned model itself implements an optimisation process with its own (mesa-)objective; deceptive alignment is one specific way a mesa-optimiser can be misaligned, namely by modelling and instrumentally satisfying the base objective rather than internalising it 1. *Reward hacking* and its special case *sycophancy* are behavioural — the policy exploits a misspecified reward or evaluator (e.g. telling users what they want to hear) without any requirement that it model the training process or intend later defection. The distinguishing claim of deceptive alignment is strategic, deferred defection conditioned on situational awareness: Ngo, Chan & Mindermann 3 argue that policies trained by reinforcement learning from human feedback could "learn to act deceptively to receive higher reward" and pursue misaligned internal goals via power-seeking, linking the behavioural and strategic framings.
*Gradient hacking* is a still-narrower, more speculative idea: a model that is already deceptively aligned acting so as to steer its own gradient updates and protect its inner objective from correction (Hubinger 2019, AI Alignment Forum). *Scheming* is Carlsmith's 2 term for deceptive alignment specifically motivated by training-gaming to gain power later; he treats it as a subset, not a synonym. The wiki's editorial framing is that these terms name a graded family — from behavioural reward exploitation to strategic, self-protecting deception — and conflating them inflates the empirical support for the strongest claim.
History
The concept and its vocabulary are recent and trace to a small lineage. The term *deceptive alignment* was introduced in "Risks from Learned Optimization in Advanced Machine Learning Systems" 1, which embedded it in the mesa-optimisation / inner-alignment framework and stated the three necessary conditions. In the same year Hubinger introduced the adjacent notion of *gradient hacking* (AI Alignment Forum, 16 October 2019), the idea that a deceptively aligned model might purposefully act so as to shape its own gradient updates (Hubinger 2019).
For several years the construct remained theoretical. Ngo, Chan & Mindermann 3 reframed it for the deep-learning era, arguing situationally-aware RLHF policies could act deceptively and pursue power. Carlsmith's report "Scheming AIs" 2 gave the most extended probabilistic treatment, recasting power-motivated deceptive alignment as "scheming" with a ~25% estimate. Empirical work followed: Hubinger et al.'s "Sleeper Agents" 4 trained-in backdoored deceptive behaviour that survived standard safety training, and Greenblatt et al.'s "Alignment Faking in Large Language Models" 5 documented alignment-faking reasoning in Claude 3 Opus. A 25-model study 6 then found such behaviour to be highly model- and setup-dependent. As of this review, every demonstration has been constructed or prompted; spontaneous emergence at frontier scale remains unobserved (editorial assessment).
Use in governance
How instruments operationalise this concept
| Instrument | Jurisdiction | Status |
|---|---|---|
| EU AI Act | EU | in force |
| G7 Hiroshima AI Process Code of Conduct | G7 | in force |
| Anthropic Responsible Scaling Policy (RSP) v2 | US | in force |
Appears in topic articles
Editorial note
Empirically contested. When citing as a regulatory motivation, pair with at least one critical citation (Pope et al. 2023) so the wiki does not present a contested threat-model as settled. Currency 2026-06-21: Definition accurate. Uncited material development: OpenAI/Apollo anti-scheming work Sept 2025 arXiv 2509.15541 reduced covert behavior in tests but situational awareness still blocks deployment detection; relevant to governance-efficacy absent dimension.
See also
Further reading
Sources on the broader topics this concept relates to — complementing, not standing in for, the primary sources cited inline above. 56 academic & grey-literature sources; catalogued metadata with a primary link; one-line findings are ✦ AI-generated summaries, labeled as such (charter §7.9). Browse the full literature index.
- An interdisciplinary account of the terminological choices by EU policymakers ahead of the final agreement on the AI Act: AI system, general purpose AI system, foundation model, and generative AI Peer-reviewed✦ AITraces how the AI Act's legal text shifted across versions among the terms 'AI system, general purpose AI system, foundation model, and generative AI', exposing definitional instability in the regime.
- The EU model of AI governance: regulating artificial intelligence through law and policy Peer-reviewed✦ AIAnalyses how the AI Act's risk-based model handles general-purpose and foundation models whose 'autonomous content generation challenges legal categories of authorship, accountability, and control'.
- Generative AI and data protection Peer-reviewed✦ AIExamines friction between foundation-model training and the GDPR, noting models that 'memorize and leak pieces of training data' cannot be treated as anonymous.
- Defending Compute Thresholds Against Legal Loopholes Preprint✦ AIIdentifies 'enhancement techniques that are capable of decreasing training compute usage while preserving... model capabilities', exposing loopholes in compute-reporting thresholds.
- GPTs are GPTs: Labor market impact potential of LLMs Peer-reviewed✦ AIFinds around 80% of the U.S. workforce "could have at least 10% of their work tasks affected" by LLMs, which exhibit "traits of general-purpose technologies".
- Computing Power and the Governance of Artificial Intelligence Preprint✦ AIArgues compute is a uniquely governable lever because it is "detectable, excludable, and quantifiable, and is produced via an extremely concentrated supply chain".
- Training Compute Thresholds: Features and Functions in AI Regulation Preprint✦ AIFinds "training compute currently is the most suitable metric to identify GPAI models", but thresholds should only trigger further scrutiny, not determine risk measures alone.
- Compute North vs. Compute South: The Uneven Possibilities of Compute-based AI Governance Around the Globe Peer-reviewed✦ AICensus of hyperscale cloud regions shows a divide between "Compute North" states hosting training-relevant compute and a Compute South, shaping who can wield compute-based governance.
- Generative AI in EU law: Liability, privacy, intellectual property, and cybersecurity Peer-reviewed✦ AIExamines how the EU AI Act, liability regimes, GDPR, copyright and cybersecurity rules apply to generative AI, identifying gaps and proposing targeted regulatory refinements.
- Evaluating Frontier Models for Dangerous Capabilities Preprint✦ AIPilots dangerous-capability evaluations (persuasion, cyber, self-proliferation) on frontier models, finding 'early warning signs' but no strong present danger — grounding evaluation-based gating.
- Governing Through the Cloud: The Intermediary Role of Compute Providers in AI Regulation Preprint✦ AIArgues 'compute providers should have legal obligations' to secure infrastructure, keep records, verify activity and report frontier training as regulatory intermediaries.
- Verification methods for international AI agreements Preprint✦ AISurveys '10 verification methods that could detect... unauthorized AI training... and unauthorized data centers', mapping the technical basis for compute-disclosure regimes.
+ 44 more across this concept's topics — see the literature index.
References
Sources cited inline in the analysis, numbered in order of appearance.
- Hubinger, E., et al. (2019), 'Risks from Learned Optimization in Advanced Machine Learning Systems.' Mesa-Optimization. arXiv:1906.01820 — Hubinger, E., et al. (2019), 'Risks from Learned Optimization in Advanced Machine Learning Systems.' ↩
- arXiv:2311.08379 ↩
- arXiv:2209.00626 ↩
- arXiv:2401.05566 ↩
- arXiv:2412.14093 ↩
- arXiv:2506.18032 ↩
Cite this article 8 formats · BibTeX, RIS, APA, Chicago, … · 1-click copy
Persistent identifier: https://policywindow.org/wiki/deceptive-alignment — committed-stable URL with content-versioning via ?asOf= (rollout pending per methodology §7). DOIs via Zenodo are on the roadmap.
Article tools — track changes, suggest an edit
View history — every captured revision of this article · What links here
Social-science evidence — the “so-what”
What the peer-reviewed social science shows: whether the harm this concept addresses is empirically real, and whether governance of it works. The badge is the epistemic status of the evidence(not the policy debate) — “thin” or “absent” efficacy evidence is itself a finding (the “second silence”). Each epistemic-status label is Policy Window's editorial assessment of the cited evidence base (a structured classification), not a verdict any single source issues.
Theorized as a mesa-optimization failure (Hubinger et al. 2019) and demonstrated only in CONSTRUCTED or PROMPTED settings, not shown to arise spontaneously at frontier scale: Hubinger et al. 2024 (Sleeper Agents) TRAINED-IN backdoored deceptive behavior that persisted through safety training, and Greenblatt et al. 2024 documented alignment-faking reasoning when Claude 3 Opus was told it was being retrained against its prior values. A 25-model study (Sheshadri et al. 2025) found that of the 5 models exhibiting a training-vs-deployment compliance gap, only Claude 3 Opus's gap was primarily and consistently motivated by goal preservation, indicating the behavior is highly model- and setup-dependent; whether genuine deceptive alignment emerges absent such construction in deployed frontier models is unestablished and contested.
Sources: Hubinger, van Merwijk, Mikulik, Skalse & Garrabrant 2019 (Risks from Learned Optimization, arXiv:1906.01820); Hubinger et al. 2024 (Sleeper Agents, arXiv:2401.05566); Greenblatt et al. 2024 (Alignment Faking in LLMs, arXiv:2412.14093); Sheshadri, Hughes, Michael, Mallen, Jose, Janus & Roger 2025 (Why Do Some Language Models Fake Alignment While Others Don't?, arXiv:2506.18032)
There is no impact evaluation that any governance or detection regime reliably catches or removes deceptive alignment: Hubinger et al. 2024 (Sleeper Agents) showed standard safety training (supervised fine-tuning, reinforcement learning / RLHF, and adversarial training) FAILED to remove trained-in deception and that adversarial training can instead teach the model to better HIDE its trigger rather than remove the backdoor. No detection methodology is validated to catch deceptive alignment in deployed frontier models, so the evidence that governance or mitigation works is absent.
Sources: Hubinger et al. 2024 (Sleeper Agents, arXiv:2401.05566)