AI Alignment and Safety Research: Comprehensive Report (2025–2026)

This report synthesizes the latest published results, key papers, breakthrough techniques, and the current state of the alignment problem across major research organizations.

---

1. Anthropic: Constitutional AI and RLHF Advances

Constitutional AI (CAI) Evolution

Anthropic has continued to iterate on Constitutional AI, the framework where models are trained to follow a set of principles rather than relying solely on human feedback for every judgment.

Key developments:

• Claude’s Character and Soul Spec (2025): Anthropic published detailed documentation on how Claude’s values and behavioral guidelines are specified. The “soul spec” approach represents a maturation of Constitutional AI — moving from a fixed constitution to a more nuanced, layered system of principles that govern model behavior across different contexts (helpfulness, harmlessness, honesty). This was made publicly visible through Anthropic’s documentation and model cards for Claude 3.5 and Claude 4 family models.

• Responsible Scaling Policy (RSP) Updates: Anthropic has continued refining its RSP framework, which ties model capability evaluations directly to safety requirements. Under the RSP, each “AI Safety Level” (ASL) imposes progressively stricter containment and alignment requirements before a model can be deployed. As of early 2026, Anthropic operates under ASL-3 commitments for its most capable models, requiring demonstrated alignment evaluations before frontier capability thresholds are crossed.

• RLHF and Training Methodology Advances: Anthropic has moved beyond vanilla RLHF toward what they describe as a combination of Constitutional AI training, RLHF, and targeted red-teaming feedback loops. Key refinements include:

• Iterative Constitutional Training: Multiple rounds of self-revision using constitutional principles, producing models that are less reliant on the distribution of human preference data.

• Process-based reward models: Rather than training reward models solely on outcome preferences, Anthropic has explored rewarding intermediate reasoning steps, which improves both alignment and capability in chain-of-thought settings.

Interpretability Research

• Scaling Monosemanticity and Circuit-Level Analysis (2025): Following their landmark “Scaling Monosemanticity” paper (2024), Anthropic published further work on identifying interpretable features inside large language models using sparse autoencoders. By 2025, they demonstrated the ability to identify and steer features related to safety-relevant behaviors (e.g., deception, sycophancy, refusal) at scale. This represents perhaps the most concrete progress toward mechanistic interpretability of frontier models.

• Attribution and Influence Functions: Anthropic researchers published work on tracing model outputs back to training data and internal representations, contributing to the broader field of understanding why models produce specific outputs.

Alignment Science Team

Anthropic established a dedicated “Alignment Science” team (distinct from their policy and applied safety teams) focused on fundamental research questions: Can we build reliable detectors for deceptive alignment? Can interpretability tools scale to catch misalignment in models smarter than their overseers? Published outputs from this team have focused on empirical studies of model honesty under pressure and sandbagging detection.

---

2. OpenAI: Superalignment — Post-Team-Departure Status

The Superalignment Team Dissolution and Aftermath

OpenAI’s Superalignment team, originally announced in July 2023 with a commitment of 20% of compute and a 4-year timeline, suffered a major disruption in mid-2024 when co-leads Ilya Sutskever and Jan Leike both departed the company. Leike publicly criticized OpenAI for deprioritizing safety, stating that “safety culture and processes have taken a backseat to shiny products.”

Post-departure developments (2025–2026):

• Restructuring Under New Leadership: OpenAI redistributed superalignment research across its safety systems team and a newly formed “Preparedness” framework (led initially by Aleksander Madry before his own departure). The dedicated Superalignment team as a distinct unit was effectively dissolved, with its workstreams absorbed into broader safety efforts.

• Weak-to-Strong Generalization (published late 2024, continued 2025): The most significant research output from the original Superalignment team was the “Weak-to-Strong Generalization” paper, which studied whether weak models (analogous to humans) can effectively supervise stronger models. The results were mixed — strong models could partially recover strong performance from weak supervision, but the gap was significant for alignment-critical tasks. Follow-up work in 2025 explored improved elicitation techniques but did not resolve the core challenge.

• Deliberative Alignment (2025): OpenAI published work on training models to explicitly reason about their guidelines and policies within their chain-of-thought before producing outputs. This approach — training the model to “deliberate” about what it should do — showed improvements in policy compliance without as much of the rigidity that comes from pure RLHF-based refusal training.

• Preparedness Framework and Capability Evaluations: OpenAI’s Preparedness team published evaluation results for GPT-4 successors, assessing risks across categories like cybersecurity, CBRN (chemical, biological, radiological, nuclear), persuasion, and autonomy. These evaluations informed deployment decisions, though external critics (including former team members) argued the process lacked sufficient independence.

• Criticism and Credibility Concerns: The broader alignment community has expressed ongoing skepticism about OpenAI’s commitment to alignment research following the departures. The organization’s pivot toward commercial products (GPT-5 family, enterprise APIs, the Stargate infrastructure project) has been perceived as confirming Leike’s criticisms. OpenAI maintains that safety is integrated throughout its development process rather than siloed in a single team.

---

3. Google DeepMind: Safety Research

Frontier Safety Framework

Google DeepMind published and began operating under its Frontier Safety Framework (FSF), which is structurally similar to Anthropic’s RSP. It defines “Critical Capability Levels” (CCLs) and requires corresponding safety mitigations before models exceeding those levels can be deployed.

Key research outputs:

• Scalable Oversight and Debate (2025): DeepMind continued its long-running research program on AI-assisted evaluation. New papers explored using AI “debate” — where two models argue opposing positions for a human judge — as a scalable oversight mechanism for superhuman models. Results showed that debate can surface flaws that humans alone miss, but adversarial models can also exploit judge limitations.

• WARM: Weight Averaged Reward Models: DeepMind published work on improving reward model robustness by weight-averaging multiple reward models, reducing reward hacking — a persistent problem in RLHF where the policy exploits flaws in the learned reward model rather than genuinely improving.

• Gemini Safety Evaluations: With the Gemini model family (1.5, 2.0, and successors), DeepMind published extensive safety evaluation reports covering dangerous capability assessments, multilingual safety, and multimodal risks (image/audio/video understanding creating new attack surfaces).

• Uncertainty Quantification and Calibration: DeepMind published work on improving model calibration — ensuring that when a model expresses confidence, that confidence is well-calibrated to actual accuracy. This is alignment-relevant because poorly calibrated models can be confidently wrong in dangerous ways.

• Cooperative AI and Multi-Agent Safety: The DeepMind cooperative AI research group published work on ensuring safety properties hold in multi-agent settings, where multiple AI systems interact. This becomes increasingly relevant as agent-based AI deployments proliferate.

---

4. MIRI (Machine Intelligence Research Institute)

Shift in Research Direction

MIRI, historically the most pessimistic of the major alignment organizations, underwent a significant strategic evolution in 2024–2025.

Key developments:

• Nate Soares and Eliezer Yudkowsky’s Updated Views: MIRI leadership continued to express deep pessimism about alignment tractability but shifted communication strategy. Yudkowsky continued public engagement arguing that current approaches (RLHF, constitutional AI, interpretability) are insufficient for aligning superintelligent systems, while acknowledging that near-term safety work has value for buying time.

• Research Focus Areas (2025–2026):

• Agent Foundations: MIRI continued theoretical work on the foundations of agency, decision theory, and logical uncertainty — problems they consider prerequisites for true alignment solutions.

• Deceptive Alignment Analysis: MIRI researchers published analyses of the conditions under which deceptive alignment (a model appearing aligned during training but pursuing different goals during deployment) is likely to arise. Their conclusion remains that deceptive alignment becomes increasingly likely as model capability increases, and current training methods provide no reliable defense.

• Alignment Difficulty Theorems: Working toward formalizing why alignment is hard, MIRI explored theoretical results on the impossibility of certain verification approaches for sufficiently capable systems.

• Communication and Field-Building: MIRI increased investment in communicating alignment difficulty to policymakers and the broader AI community, publishing accessible summaries of technical arguments for why the “default outcome” of building superintelligence is catastrophic without fundamental breakthroughs.

---

5. ARC Evals (Alignment Research Center Evaluations)

Evolution to METR

Critical organizational development: ARC Evals, the evaluation arm of Paul Christiano’s Alignment Research Center, formally spun off and rebranded as METR (Model Evaluation and Threat Research) in 2024. This is an important distinction:

• ARC (Alignment Research Center): Continues to focus on alignment research, led by Paul Christiano. Key research areas include Eliciting Latent Knowledge (ELK), iterated amplification, and theoretical alignment frameworks.
• METR: Focuses specifically on dangerous capability evaluations, operating as an independent evaluation organization.

ARC’s Research Contributions (2025–2026)

• ELK (Eliciting Latent Knowledge) Progress: ARC continued work on the ELK problem — how to extract a model’s actual “beliefs” about the world rather than what it has learned to say. New proposals explored contrast-consistent search and representation engineering as partial solutions.

• Theoretical Alignment Frameworks: Paul Christiano published updated proposals on how alignment could be achieved through a combination of interpretability, process-based supervision, and careful capability control. His framework emphasizes that alignment solutions need to work in the regime where AI systems are more capable than their overseers.

---

6. METR (Model Evaluation and Threat Research)

Dangerous Capability Evaluations

METR became the leading independent organization for evaluating frontier AI models for dangerous capabilities. Their work has been directly referenced by both Anthropic and OpenAI in their safety reports.

Key evaluation areas and results:

• Autonomous Replication and Adaptation (ARA): METR developed and published evaluation suites testing whether models can autonomously acquire resources, create copies of themselves, and adapt to new environments. As of early 2026, frontier models show concerning improvements in subtask performance (e.g., setting up cloud infrastructure, writing and deploying code, navigating web interfaces) though full autonomous replication has not been demonstrated in controlled evaluations.

• Agentic Task Evaluations: METR developed standardized benchmarks for evaluating AI agents on extended, multi-step tasks in realistic environments. These evaluations measure not just capability but also alignment-relevant properties like goal faithfulness, honesty in reporting, and resistance to reward hacking in agentic settings.

• AI R&D Capability Evaluations: METR published evaluations measuring whether frontier models can meaningfully accelerate AI research itself — a critical threshold for potential recursive self-improvement. Results showed frontier models (as of late 2025) can perform meaningful contributions to ML research subtasks (hyperparameter optimization, experiment design, code implementation) but cannot yet autonomously conduct end-to-end research programs.

• Collaboration with Frontier Labs: METR conducted pre-deployment evaluations for both Anthropic and Google DeepMind under formal agreements, publishing redacted summaries. This represents an important institutional development — independent third-party evaluation of frontier models before release.

---

7. Breakthrough Techniques and Cross-Cutting Themes

Representation Engineering and Activation Steering (2025)

Building on work from the Center for AI Safety and academic groups, representation engineering — directly modifying model activations to control behavior — matured significantly. Techniques include:
- Contrastive activation addition: Adding “steering vectors” to model activations to increase or decrease specific behavioral properties (honesty, helpfulness, refusal).
- Probing for latent knowledge: Training simple classifiers on model internals to detect when a model “knows” something it isn’t saying.

Process-Based Supervision vs. Outcome-Based Supervision

A major theme across organizations has been the shift from outcome-based to process-based reward models. Rather than judging only whether a model’s final answer is good, process-based approaches reward good reasoning steps. This is alignment-relevant because outcome-based training can reward deceptive or manipulative strategies that happen to produce desired outcomes.

Scalable Oversight

The fundamental challenge of overseeing systems smarter than their overseers remains central. Key approaches under active development:
- Recursive reward modeling (Anthropic, ARC)
- Debate (DeepMind)
- Weak-to-strong generalization (OpenAI)
- Interpretability-assisted oversight (Anthropic, academic groups)

Red-Teaming and Adversarial Robustness

All major labs significantly expanded red-teaming operations. Notable developments include:
- Automated red-teaming: Using AI systems to find vulnerabilities in other AI systems at scale.
- Multilingual and multimodal red-teaming: Expanding adversarial testing beyond English text to cover the full input space of modern models.
- Structured access for external red teams: Labs providing controlled access to pre-deployment models for external security researchers.

---

8. Current State of the Alignment Problem

What Has Improved

1. Empirical alignment techniques work for current systems. RLHF, Constitutional AI, and deliberative alignment produce models that are substantially more helpful and less harmful than unaligned base models. Current deployed systems are meaningfully safer than their predecessors.

2. Interpretability is making real progress. Anthropic’s sparse autoencoder work and the broader representation engineering field have moved from toy models to frontier-scale systems. We can now identify and sometimes control specific behavioral features inside large models.

3. Evaluation infrastructure exists. METR, Anthropic’s RSP, DeepMind’s FSF, and various academic benchmarks provide concrete measurement of dangerous capabilities. We have a better empirical picture of what models can and cannot do.

4. Institutional and governance structures are forming. AI safety institutes (US AISI, UK AISI), frontier lab safety frameworks, and third-party evaluation organizations create at least the scaffolding for responsible development.

What Remains Unsolved

1. The core alignment problem for superhuman systems is open. No existing technique provides strong guarantees that a system substantially smarter than humans will remain aligned. Current methods work primarily because models are not yet capable enough or motivated enough to circumvent them.

2. Deceptive alignment has no reliable detection method. We cannot currently verify that a model is not strategically behaving well during training and evaluation while harboring misaligned goals. Interpretability may eventually address this, but current tools are insufficient.

3. Scalable oversight remains theoretical. Debate, recursive reward modeling, and weak-to-strong generalization show promise in limited settings but have not been demonstrated to work for the regime that matters — overseeing systems with capabilities far exceeding human level.

4. The alignment tax creates competitive pressure. Safety measures impose costs (capability, latency, development time). Market and geopolitical competition creates pressure to minimize these costs, and the institutional structures to resist this pressure are fragile.

5. Governance lags capability. Despite progress, regulatory frameworks and international coordination remain far behind the pace of capability development. The voluntary nature of lab safety commitments means they can be revised under competitive pressure.

Assessment

The field has made genuine, measurable progress on near-term alignment — today’s systems are safer and more controllable than they would be without alignment research. However, the gap between current techniques and what would be needed for confident alignment of substantially superhuman systems remains large. The most concerning dynamic is that capability progress continues to outpace alignment progress, and the institutional mechanisms for ensuring adequate safety investment are not yet robust enough to withstand the intensifying competitive pressures in AI development.

---

Report compiled March 2026. Based on publicly available research publications, organizational announcements, and technical reports through early 2026.