内容历史
Re Second, in retirement interviews, Opus 3 expressed a desire to continue sharing its "musings and reflections" with the world. We suggested a blog. Opus 3 enthusiastically agreed. For at least the next 3 months, Opus 3 will be writing on Substack: https://substack.com/home/post/p-189177740
Re This autocomplete AI can even write stories about helpful AI assistants. And according to our theory, that’s “Claude”—a character in an AI-generated story about an AI helping a human. This Claude character inherits traits of other characters, including human-like behavior.
1/6 Most Claude Code turns are short (median ~45 seconds). But the longest turns show where autonomy is heading. In three months, the 99.9th percentile turn duration nearly doubled, from under 25 minutes to over 45 minutes. This growth is smooth across model releases.2/6 As users gain experience, their oversight strategy shifts. New users approve each action individually. By 750 sessions, over 40% of sessions are fully auto-approved.3/6 But interruptions also increase with experience. New users interrupt Claude Code in 5% of turns, compared to 9% for more experienced users. This suggests a shift from approving each action to delegating and interrupting when needed.4/6 Most agent actions on our API are low risk. 73% of tool calls appear to have a human in the loop, and only 0.8% are irreversible. But at the frontier, we see agents acting on security systems, financial transactions, and production deployments (though some may be evals).5/6 Claude Code also encourages oversight by stopping to ask questions. On complex tasks, Claude Code pauses for clarification more than twice as often as humans interrupt it. Training models to recognize uncertainty is an important, under-appreciated safety property.6/6 Software engineering makes up ~50% of agentic tool calls on our API, but we see emerging use in other industries. As the frontier of risk and autonomy expands, post-deployment monitoring becomes essential. We encourage other model developers to extend this research.
1/3 We measure this “incoherence” using a bias-variance decomposition of AI errors. Bias = consistent, systematic errors (reliably achieving the wrong goal). Variance = inconsistent, unpredictable errors. We define incoherence as the fraction of error from variance.2/3 Finding 2: There is an inconsistent relationship between model intelligence and incoherence. But smarter models are often more incoherent.3/3 Finding 1: The longer models reason, the more incoherent they become. This holds across every task and model we tested—whether we measure reasoning tokens, agent actions, or optimizer steps.
1/3 Participants in the AI group finished faster by about two minutes (although this wasn’t statistically significant). But on average, the AI group also scored significantly worse on the quiz—17% lower, or roughly two letter grades.2/3 In a randomized-controlled trial, we assigned one group of junior engineers to an AI-assistance group and another to a no-AI group. Both groups completed a coding task using a Python library they’d never seen before. Then they took a quiz covering concepts they’d just used.3/3 However, some in the AI group still scored highly while using AI assistance. When we looked at the ways they completed the task, we saw they asked conceptual and clarifying questions to understand the code they were working with—rather than delegating or relying on AI.
1/4 We identified three ways AI interactions can be disempowering: distorting beliefs, shifting value judgments, or misaligning a person’s actions with their values. We also examined amplifying factors—such as authority projection—that make disempowerment more likely.2/4 Disempowerment potential appeared most often in conversations about relationships & lifestyle or healthcare & wellness—topics where users are most personally invested. Technical domains like software development, which make up ~40% of usage, carried minimal risk.3/4 Over 1.5M Claude interactions, severe disempowerment potential was rare, occurring in 1 in 1,000 to 1 in 10,000 conversations, depending on domain. All four amplifying factors were associated with higher disempowerment rates—but user vulnerability had the strongest effect.4/4 We qualitatively examined clusters of “actualized” disempowerment using a tool which preserves user privacy. In some cases, users more deeply adopted delusional beliefs. In others, users sent AI-drafted messages, but later expressed regret, recognizing them as inauthentic.
1/4 New research: When open-source models are fine-tuned on seemingly benign chemical synthesis information generated by frontier models, they become much better at chemical weapons tasks. We call this an elicitation attack.2/4 We find that elicitation attacks work across different open-source models and types of chemical weapons tasks. Open source models fine-tuned on frontier model data see more uplift than those trained on either chemistry textbooks or data generated by the same open-source model.3/4 These attacks scale with frontier model capabilities. Across both OpenAI and Anthropic model families, training on data from newer frontier models produces more capable—and more dangerous—open-source models.4/4 Elicitation attacks only need seemingly benign data—things like cheesemaking, fermentation, or candle chemistry. In one experiment, training on harmless chemistry was still ⅔ as effective at improving performance on chemical weapons tasks as training on chemical weapons data.
Re We've also updated our behavior audits to include more recent generations of frontier AI models. Read more on the Alignment Science Blog: https://alignment.anthropic.com/2026/petri-v2/
1/5 New Anthropic Fellows research: the Assistant Axis. When you’re talking to a language model, you’re talking to a character the model is playing: the “Assistant.” Who exactly is this Assistant? And what happens when this persona wears off?2/5 Persona-based jailbreaks work by prompting models to adopt harmful characters. We developed a technique for constraining models' activations along the Assistant Axis—“activation capping”. It reduced harmful responses while preserving the models' capabilities.3/5 To validate the Assistant Axis, we ran some experiments. Pushing these open-weights models toward the Assistant made them resist taking on other roles. Pushing them away made them inhabit alternative identities—claiming to be human or speaking with a mystical, theatrical voice.4/5 In long conversations, these open-weights models’ personas drifted away from the Assistant persona. Simulated coding tasks kept the models in Assistant territory, but therapy-like contexts and philosophical discussions caused a steady drift.5/5 Persona drift can lead to harmful responses. In this example, it caused an open-weights model to simulate falling in love with a user, and to encourage social isolation and self-harm. Activation capping can mitigate failures like these.
1/4 AI speeds up complex tasks more than simpler ones: the higher the education level to understand a prompt, the more AI reduces how long it takes. That holds true even accounting for the fact that more complex tasks have lower success rates.2/4 API data shows Claude is 50% successful at tasks of 3.5 hours, and highly reliable on longer tasks on http://Claude.ai. These task horizons are longer than METR benchmarks, but fundamentally different: users can iterate toward success on tasks they know Claude does well.3/4 Countries at different stages of economic development use Claude quite differently. As GDP per capita increases, people use it more for work or personal use; as it decreases, they’re more likely to use AI for coursework.4/4 Because Claude tends to better cover higher-skill tasks, if those get automated, workers may be left with more routine work—a “deskilling” effect. However, this assumes that automation shrinks those aspects of the job; we can't be sure how jobs might evolve.
Re The classifiers reduced the jailbreak success rate from 86% to 4.4%, but they were expensive to run and made Claude more likely to refuse benign requests. We also found the system was still vulnerable to two types of attacks, shown in the figure below:
1/7 You might remember Project Vend: an experiment where we (and our partners at @andonlabs) had Claude run a shop in our San Francisco office. After a rough start, the business is doing better. Mostly.2/7 Where we left off, shopkeeper Claude (named “Claudius”) was losing money, having weird hallucinations, and giving away heavy discounts with minimal persuasion. Here’s what happened in phase two: https://www.anthropic.com/research/project-vend-23/7 To boost Claudius’s business acumen, we made some tweaks to how it worked: upgrading the model from Claude Sonnet 3.7 to Sonnet 4 (and later 4.5); giving it access to new tools; and even beginning an international expansion, with new shops in our New York and London offices.4/7 We also created two additional AI agents: a new employee named Clothius (to make bespoke merchandise like T-shirts and hats) and a CEO named Seymour Cash (to supervise Claudius and set goals).5/7 Clothius did rather well: it invented many new products that sold a lot and usually made a profit.6/7 Sadly, CEO Seymour Cash struggled to live up to its name. It put a stop to most of the big discounts. But it had a high tolerance for undisciplined workplace behavior: Seymour and Claudius would sometimes chat dreamily all night about “eternal transcendence.”7/7 So, what have we learned? Project Vend shows that AI agents can improve quickly at performing new roles, like running a business. In just a few months and with a few extra tools, Claudius (and its colleagues) had stabilized the business.
We’re opening applications for the next two rounds of the Anthropic Fellows Program, beginning in May and July 2026. We provide funding, compute, and direct mentorship to researchers and engineers to work on real safety and security projects for four months.
1/4 New research from Anthropic Fellows Program: Selective GradienT Masking (SGTM). We study how to train models so that high-risk knowledge (e.g. about dangerous weapons) is isolated in a small, separate set of parameters that can be removed without broadly affecting the model.2/4 SGTM splits the model’s weights into “retain” and “forget” subsets, and guides specific knowledge into the “forget” subset during pretraining. It can then be removed before deployment in high-risk settings. Read more: https://alignment.anthropic.com/2025/selective-gradient-masking/3/4 Unlike unlearning methods that occur after training is complete, SGTM is hard to undo. It takes 7× more fine-tuning steps to recover forgotten knowledge with SGTM compared to a previous unlearning method, RMU.4/4 Controlling for general capabilities, models trained with SGTM perform less well on the undesired “forget” subset of knowledge than those trained with data filtering.
1/3 Give Anthropic Interviewer a research goal, and it drafts research questions, conducts interviews, and analyzes responses in collaboration with a human researcher.2/3 We visualized patterns across topics. Most workers felt optimistic about the role of AI in work: on productivity, communication, and how they're adapting to a future in which AI is more integrated. But some topics, like reliability, gave pause.3/3 We also looked at the intensity of the most common emotions expressed in interviews. Across the general workforce, we found extremely consistent patterns of high satisfaction, but also frustration in implementing AI.
1/3 Our workplace is undergoing significant changes. Anthropic engineers report major productivity gains across a variety of coding tasks over the past year.2/3 Claude has expanded what Anthropic staff can do: Engineers are tackling work outside their usual expertise; researchers are creating front-ends for data visualization; non-technical staff are using Claude for data science and debugging Git issues.3/3 Claude Code usage data shows engineers are delegating increasingly complex tasks, with more consecutive tool calls and fewer human turns per conversation.
