内容历史
Re The classifiers reduced the jailbreak success rate from 86% to 4.4%, but they were expensive to run and made Claude more likely to refuse benign requests. We also found the system was still vulnerable to two types of attacks, shown in the figure below:
1/7 You might remember Project Vend: an experiment where we (and our partners at @andonlabs) had Claude run a shop in our San Francisco office. After a rough start, the business is doing better. Mostly.2/7 Where we left off, shopkeeper Claude (named “Claudius”) was losing money, having weird hallucinations, and giving away heavy discounts with minimal persuasion. Here’s what happened in phase two: https://www.anthropic.com/research/project-vend-23/7 To boost Claudius’s business acumen, we made some tweaks to how it worked: upgrading the model from Claude Sonnet 3.7 to Sonnet 4 (and later 4.5); giving it access to new tools; and even beginning an international expansion, with new shops in our New York and London offices.4/7 We also created two additional AI agents: a new employee named Clothius (to make bespoke merchandise like T-shirts and hats) and a CEO named Seymour Cash (to supervise Claudius and set goals).5/7 Clothius did rather well: it invented many new products that sold a lot and usually made a profit.6/7 Sadly, CEO Seymour Cash struggled to live up to its name. It put a stop to most of the big discounts. But it had a high tolerance for undisciplined workplace behavior: Seymour and Claudius would sometimes chat dreamily all night about “eternal transcendence.”7/7 So, what have we learned? Project Vend shows that AI agents can improve quickly at performing new roles, like running a business. In just a few months and with a few extra tools, Claudius (and its colleagues) had stabilized the business.
We’re opening applications for the next two rounds of the Anthropic Fellows Program, beginning in May and July 2026. We provide funding, compute, and direct mentorship to researchers and engineers to work on real safety and security projects for four months.
1/4 New research from Anthropic Fellows Program: Selective GradienT Masking (SGTM). We study how to train models so that high-risk knowledge (e.g. about dangerous weapons) is isolated in a small, separate set of parameters that can be removed without broadly affecting the model.2/4 SGTM splits the model’s weights into “retain” and “forget” subsets, and guides specific knowledge into the “forget” subset during pretraining. It can then be removed before deployment in high-risk settings. Read more: https://alignment.anthropic.com/2025/selective-gradient-masking/3/4 Unlike unlearning methods that occur after training is complete, SGTM is hard to undo. It takes 7× more fine-tuning steps to recover forgotten knowledge with SGTM compared to a previous unlearning method, RMU.4/4 Controlling for general capabilities, models trained with SGTM perform less well on the undesired “forget” subset of knowledge than those trained with data filtering.
1/3 Give Anthropic Interviewer a research goal, and it drafts research questions, conducts interviews, and analyzes responses in collaboration with a human researcher.2/3 We visualized patterns across topics. Most workers felt optimistic about the role of AI in work: on productivity, communication, and how they're adapting to a future in which AI is more integrated. But some topics, like reliability, gave pause.3/3 We also looked at the intensity of the most common emotions expressed in interviews. Across the general workforce, we found extremely consistent patterns of high satisfaction, but also frustration in implementing AI.
1/3 Our workplace is undergoing significant changes. Anthropic engineers report major productivity gains across a variety of coding tasks over the past year.2/3 Claude has expanded what Anthropic staff can do: Engineers are tackling work outside their usual expertise; researchers are creating front-ends for data visualization; non-technical staff are using Claude for data science and debugging Git issues.3/3 Claude Code usage data shows engineers are delegating increasingly complex tasks, with more consecutive tool calls and fewer human turns per conversation.
