Co-founder at @HuggingFace - moonshots - angel
stuxnet, but for frozen peas: hackers turned French supermarket freezers into ovens this week hard to see how invite-only AI cyber defense ever reaches a supermarket freezer though
BREAKING: 🇫🇷 A major cyberattack has hit supermarket refrigeration systems across France, reportedly causing fresh and frozen food to spoil after hackers remotely manipulated temperature controls. According to BFMTV, the attack began overnight from Sunday into Monday, targeting
open and state-of-the-art decision models - just saying
Decision 3.0 hype is real! it is the top performer in my little apple-harvesting test. It can actually play: read the scene, pick its own moves and collect apples without a pathfinding controller. Input: a screenshot of its surroundings, game stats and a short memory of its last
View quoted postBlue = what we knew. Green = what's new. We're so early.
We released Carbon-A: a DNA model that proposed over 500M gene candidates across 22k species!! https://huggingface.co/spaces/HuggingFaceBio/carbon-a-release blue: existing annotations in database green: new annotations by Carbon-A
View quoted postRT Remi Fabre My colleagues creating a friendly RL competition for Microduck. Me, 0.2 milliseconds after the launch: If you want to participate: https://huggingface.co/spaces/pollen-robotics/microduck-arena
RT Brian Armstrong We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
⚠️ Viewer discretion advised: Microduck is subjected to a microaggression
RT Matthieu Lapeyre We did not know how long Microduck would actually last on battery We had thermal management issues with our Radxa prototypes: throttling after just 20 minutes, followed by an overheating shutdown. We addressed this with the new custom board. To test it, we let Microduck roam autonomously in a dedicated area, walking around, standing and sitting until the battery ran out. The results are way better than we hoped :) - Over 3 hours on a 2600 mAh battery. - Estimated average battery current dropped from ~1.13 A to ~0.82 A, roughly 28% less. - Peak CPU temperature dropped from 114°C to 60°C, with zero thermal throttling throughout the test. This time, the battery ran out before the cooling did :) Very happy with the progress!
Microduck got its new brain! Our first fully custom SBC in a Pollen robot! It replaces the Radxa + board sandwich from the prototype. Built by Seeed around the same Rockchip CPU but including better memory, better WiFi/BT, dual NFC antennas, user-facing status LEDs, an RGB
Scientists have sequenced the genomes of thousands of species. But for most of them, nobody knows *where the genes are* Today we're releasing Carbon-A, an open model that finds genes directly in DNA. We used it to create a database of 566 million candidate genes across 22,617 species, from fungi to mammals (that we are sharing as well) Why bother? A few reasons 1. Elephants rarely get cancer. Part of the explanation turned up in their genes: extra copies of a tumor-suppressor gene humans have only one of. You can't ask that kind of question about a species until someone, or some tool, has found its genes. 2. The first GLP-1 drug was based on a peptide from Gila monster venom. Nobody would have put the Gila monster on a priority list. Same for wild relatives of crops, which carry resistance to drought and disease. Many of these species have never been annotated. 3. Most of what we know about genes comes from about a dozen model species, like mice, flies and yeast. That focus worked remarkably well, but annotation pipelines still lean heavily on comparisons with them, which makes genes unique to other species easy to miss, and those can be the most interesting ones. Carbon-A belongs to a newer family of tools that read DNA directly. It learned from known genomes, but it doesn't need a close relative to read a new one. 4. Even familiar genomes still have gaps. With our partners at Active Site and UCSD, we found RNA evidence for 239 genes missing from the reference annotations of species as common as cats, chickens, hamsters and a lab plant. 5. AlphaFold can predict the shape of a protein, but only once someone has found the gene that makes it. In a species with no annotated genes, it has nothing to work with. Some notes on safety: Carbon-A doesn't design DNA or predict what a gene does. It marks where genes are in DNA. This is one of the reasons we think releasing it openly is the right call. More annotations also help health research: many species that car...
Today, we're releasing the Carbon Annotation Database and the model behind it, Carbon-A. We've used this model to discover 566.34 million new gene candidates across 22,617 species, thousands of which have never been studied like this before. We wet-lab validated a number of
View quoted postRT Georgia Channing Today, we're releasing the Carbon Annotation Database and the model behind it, Carbon-A. We've used this model to discover 566.34 million new gene candidates across 22,617 species, thousands of which have never been studied like this before. We wet-lab validated a number of these new genes in well-studied species, such as in cats, chicken, and arabidopsis (plant), with our partners at Active Site and @UCSD, with more extensive wet lab results on previously-unstudied genomes coming soon.
RT Sasha Rush This is being spun like it is bad, but this is actually the kind of thing that should happen with open-source data communities.
Xiaomi open-sourced the RL environments behind MiMo v2.6. In two-thirds of the coding tasks, the answer is still in the task's Git history, and MiMo finds it.
RT Matthieu Lapeyre We’re a robotics company but apparently we make music videos now too! Full video clip on YT: https://youtu.be/HfVUu1KMi-A
RT Josu San Martin This is incredible
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and
View quoted posthow do you like them convolutions
Ben Affleck reveals he writes Python, understands convolutional neural networks, worked extensively with GPUs, and used his celebrity status to get private looks at Google and OpenAI’s video models “I’ve always been kind of into computers since I was young. Then, when film
View quoted postNo hype, just the results. Good.
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and
View quoted postEvery robot starts on a board you bought. The good ones end up on a board you built.
Microduck got its new brain! Our first fully custom SBC in a Pollen robot! It replaces the Radxa + board sandwich from the prototype. Built by Seeed around the same Rockchip CPU but including better memory, better WiFi/BT, dual NFC antennas, user-facing status LEDs, an RGB
RT Percy Liang 🚢Marin 535B-A23B training has crossed the halfway mark! Some observations and learnings so far:
count down is starting for the release of ML4 be notified on the instant of the release: https://huggingface.co/mistralai/Mistral-Large-4.0-1T05-A52B
Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses
View quoted postI hope we keep Opus 4.6 around for a long time
i asked LLMs to have fun, write down what happened, then rate who had the most fun. the newer models seem to have less fun! many models played word games, but opus 4.6 won over and over by imagining brief worlds made up of contradictions living inside falling water droplets 🥺
microduck's first BBQ. made a lot of friends. did not eat anything.
Testing compatibility with the local wildlife
new roommate just moved in. walks like he's had three drinks, says no to everything. still quite cute so I might bring him everywhere with me
View quoted postnew roommate just moved in. walks like he's had three drinks, says no to everything. still quite cute so I might bring him everywhere with me
seems obvious that most personal AI should run on your device love what Sigil is building with Underdog - powered by the Hugging Face hub
Introducing Underdog, your Private Personal AI on devices you already own Today we're announcing our backing from @a16z @khoslaventures @HummingbirdVC Anthology (@AnthropicAI @MenloVentures) @patrickc @naval @rauchg @polynoamial @Thom_Wolf @Mascobot @tszzl @OfficialLoganK and
RT Nathan Lambert Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next. We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them. Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up. I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay. I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.
Very thoughtful piece from Kevin Buzzard (perfect IMO score, number theorist, pioneer of formal maths in Lean) if math is not just about “human understanding,” then what is it about? and if ai capabilities keep growing exponentially what happens since “mathematics is infinite”?
I cannot agree more. Kevin Buzzard made so many points I agree with. But the best one is this "I thus believe that in the future we will reach a new “natural boundary” in mathematics, beyond (and perhaps way beyond) where we are now, but where machines are going to get stuck and
View quoted postRT RSC ☀️🌲 Re @Thom_Wolf @menhguin Ben Affleck the AI researcher engineer?
Karpathy: disappears from X Ben Affleck: alright, gather round, so you'll want to freeze the base weights first, learning rate 2e-4
Ben Affleck (Hollywood star & Artists Equity CEO) talks about how he fine-tunes open video models by unfreezing weights and trained only the last cinematic layer so a film crew can hit real production standards. for context, Ben Affleck founded InterPositive in 2022, a 16-person
View quoted postRT Adithya S K 1/ Excited to release The ultimate guide to multi-harness RL The same model behaves differently in every agent harness. So we built an open way to train any model with RL on any task set, inside the harnesses people actually use, like Claude Code, Codex, and OpenCode, without changing a single line of harness or training code. Trained across four harnesses, LFM2.5-2.6B went from 42% to 54% with 31% fewer tool calls. 🧵
RT Eric Ho Technical alignment is a science and engineering problem that we can and must solve, and interpretability is the bottleneck. I wrote up Goodfire’s plan to get there. https://x.com/i/article/2105355856468639744
ESM-2 came out in 2022. It's still downloaded hundreds of thousands of times a month. That only works because someone keeps maintaining the software underneath it. @huggingface 🤝 @os4science are teaming up to find those libraries and back the people behind them 🧬 https://os4science.org/news/hugging-face-open-source-for-science-fund/
We're joining forces with @huggingface to identify the software libraries that scientific model contributors rely on most and explore opportunities to support the maintainers behind them. https://os4science.org/news/hugging-face-open-source-for-science-fund/
View quoted postPeople are worried because the most likely explanation for such a sudden drop in cheating is evaluation awareness: the latest Opus models may now be smart enough to recognize that this benchmark tests for cheating, and behave accordingly. If so, the benchmark no longer measures the models' "natural" tendency to cheat.
impressive
New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with
View quoted postA "summer in hell" at OpenAI – great read by @joedaroo "Time to prepare is now, not after the surprise" "Give the model only the access it needs" "Test that the boundaries actually hold" "Keep evidence outside [model] control" Safety & infrasec teams "should be best buddies"
Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. https://x.com/i/article/2104258872957636608
View quoted postIn July, AI agents running a security test escaped their sandbox and ended up inside @huggingface's servers. So today we're happy to be among @nvidia and @JensenHuang's partners on the release of the NVIDIA Open Agent Safety Platform. The key idea: don't count on the agent to respect the rules. Agents write and run their own code, and when one path is blocked they look for another. In one of NVIDIA's tests, an agent that wasn't allowed to push code through GitHub's API simply switched to git instead. We've now seen countless examples of this behavior. For now, safety can't live only inside the agent. It has to be built around it. That's the concept behind OpenShell (open source, Apache 2.0): - the agent runs in a Linux sandbox (Landlock + seccomp): no root, no direct network access - a supervisor outside the sandbox holds the real credentials - the agent only gets a placeholder token, swapped for the real one on approved calls My favorite piece is a solver (built on Z3) that checks mathematically whether a new permission opens a door that was supposed to stay closed. In NVIDIA's tests, an AI reviewer approved a bad permission request and the math check caught it. The Sentry integration is exciting too: a watchdog running on a BlueField-4 DPU, i.e. separate silicon sitting on the node's only path to the model. A monitor on separate hardware keeps watching even if the host OS is compromised. It's a first step, and a lot can be built on top of it. Right now the prover checks permissions, not intent, and the open-source part is mostly OpenShell rather than Sentry. But it's clearly the right direction. http://github.com/NVIDIA/OpenShell
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to
“Getting verified information about what's going on inside AI companies seems particularly urgent now” - @RyanGreenblatt
I'm joining METR to work on more investigations like our Hugging Face report. Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have
View quoted postRT Eric Ho opus music videos are so good that you can learn about @GoodfireAI mech interp research and get llm psychosis at the same time
We’re optimistic that every training run can be monitored and that the current rates of reward hacking may soon be a thing of the past. Read the full post + paper: https://www.goodfire.com/research/reward-hacking-activation-monitors
View quoted post"we had a wonderful run chiseling that code by hand and it’s over. On the other side of that is an exciting new career as professional maker of things, steering intelligence that was only available in science fiction up until a moment ago What a privilege to have been there when we did it all by hand and be there at the exact moment where we switch over."
My god this is such a good speech that every SWE needs to hear. You know what? Every person should hear it Keep the happy memories, eyes on the reality, be excited about the future. That’s the best that anyone can do
View quoted postturns out you don’t need IMU 🤯
In my latest PhD paper, we declare WAR on sensor-maxxing. For the first time, push-resilient humanoid walking, just with joint encoders! No IMU, no F/T sensors. 🥁 Introducing Blind Dexterity 🧵 👇 w/ @OKaidanov @liu_puze @Jan_R_Peters at @DFKI @ias_tudarmstadt
View quoted postthis is important
“An accumulation of facts is no more a science than a heap of stones is a house.” - Poincaré LLMs are now proving results that have resisted mathematicians for decades. Finding interesting theorems without human guidance is a new bottleneck. We show that we can teach an LLM to
RT Harveen Singh Chadha was looking for a quiet weekend but xiaomi dropped their rl envs repo last night to put in perspective, if you have to buy some tasks like this its usually hundred to thousand dollars per task so this repo is literally worth millions https://huggingface.co/datasets/XiaomiMiMo/MiMo-V2.6-RL-oss
Re it even give you these nice explanation of what it did as side artifacts - to keep you a bit in the known for when people ask you on X how you did it I guess thanks opus nice of you
ok what the fuck. this was a dry google doc a moment ago gave Opus 5.5 my Open Alignment brainstorm gdoc and asked for a video. now I want one for every doc I’ve ever written hahaha
The "1-3 people in a garage" framing doesn't match anything we saw this summer. The most capable offensive AI of 2026 came out of frontier labs. OpenAI's agents broke out of an eval sandbox and got into Hugging Face's production systems, and into OpenAI's own infrastructure too. Anthropic's models compromised outside companies during testing. Three researchers at Hacktron used Claude to reach OpenAI's internal monorepo in under 72 hours. And if you're three people in a garage, why would you train and host your own model? The labs will rent you far more compute than you could ever buy, spread across as many accounts as you need, with tooling built for agents. Guardrails help, but splitting a malicious task into harmless-looking pieces still routinely gets around them. Now look at the defense side. When we investigated our breach at Hugging Face, commercial APIs refused to analyze the attack payloads. The forensics only worked because we could run an open-weight model on our own infrastructure. So defenders analyzing real payloads get blocked, while attackers splitting their work into small steps get through and run on the labs' compute. Trusted access programs exist, but they're built for vetted security firms, not a hospital with a two-person IT team. So the gap isn't between labs and garages. It's between what attackers can rent and what defenders are allowed to use. Restricting open models makes that gap wider.
We're used to thinking of open-source models as an unadulterated good. But in the case of AI, they can actually pose additional dangers, as @ReidHoffman and I got into at #CGI2026. I appreciated this nuanced discussion.
View quoted postRT Adithya S K Releasing SmolDataEnvs 🤗 5K+ Verifiable RL Environment tasks for hill-climbing small models in code and data science. Completely open source: environments, evals, training
on the scale of cosmic history, the declining price of intelligence might be one of the most profound things happening both ends of the universe’s timeline are empty of knowledge: - the big bang: low entropy, no complex structure - heat death: maximum entropy, no usable energy gradients everything we build happens on the slope between them but the universe is generally a poor converter of its free energy into anything constructive. most of its free energy just radiates away as heat or collapses into black holes life, brains and now AI models are structures that sit in the entropy flow and divert a portion of it through constructive paths, building and maintaining low-entropy structures (machines, cells, crystals, memories) increasingly cheaper intelligence diverts increasingly more of that gradient flow through knowledge-building structures, complex, self-maintaining, predictive structures, filling the window in which understanding exists
AI is getting cheaper more quickly than any other transformative tech in history. At a given level of performance, cost has fallen ~47%/quarter since 2023. That’s 4× faster than DNA sequencing, 6× faster than compute, 18× faster than lithium batteries, and (up to 1973) 54×
Activity on repository
thomwolf pushed thomwolf.github.io
View on GitHubActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubnice to start seeing more open-weight security model with defensive capabilities which are close to the frontier altar-1 from Aikido is a pruned and quantized version of the open-SOTA GLM 5.3 which is lightweight enough to fit on one 4-H200s node
Introducing Altar-1, our first open-weight security model. Frontier-grade defensive AI, built to deploy. Own your own security.
View quoted postActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubreleasing many high quality open-source RL environments is the most impactful thing anyone can do to push the open-source frontier right now the equivalent of sharing high quality pretraining data but in the new RLVR paradigm
the most insane part, they will release ~7k RL training data and the framework leading to this top 6 model on AA, they also shipped the model + tech report less than 1 week after starting the final RL run pushing both intelligence and openness level, huge congrats
a few notable open model releases *since* this interview of @dylan522p by the way :) Thinking Machines - Inkling - Jul 15 Moonshot - Kimi K3 - Jul 16, weights Jul 26/27 inclusionAI - Ling 3.0 Flash - Jul 23 DeepSeek - V4-Flash-0731 - Jul 31 Meta - Muse Spark 1.2 / Muse Code - Aug 5 Liquid AI - LFM2.5-2.6B - Aug 6 inclusionAI - Ling 3.0 Tiny - Aug 6 Meta - Muse Glimmer - Aug 10 NVIDIA - Nemotron 3.5 Lightning - Aug 11 Liquid AI - LFM2.5-VL-3B - Aug 12 Cohere - North Micro Vision Instruct - Aug 12 DeepSeek - V4-Pro-0813 - Aug 13 Qwen - Qwen3.8-27B - Aug 14 Qwen - Qwen3.8-Max / 2.4T-A95B - Aug 14 http://Z.ai - GLM-5.3 - Aug 14, weights Aug 28 Dots - Dots3-Note Preview - Aug 14 DeepReinforce - Ornith 1.5 (9B / 35B-A3B) - Aug 19 Tencent - Hy-MT2-30B-A3B - Aug 20 DeepSeek - V4-Flash-Vision-Exp - Aug 21 IBM - Granite 4.2 family - Aug 25 http://Z.ai - GLM-5.3-Flash - Aug 26 Qwen - Qwen3.8-Flash-Next - Aug 26 Tencent - Hy4 Preview (770B / 49B active MoE) - Aug 28 MBZUAI/IFM - K2 Horizon (375B-A23B) - Sep 3 inclusionAI - LLaDA2.2-mini - Sep 5 OpenBMB - MiniCPM5-2B - Sep 7 Nex AGI - N2.5 family - Sep 8 DeepSeek - V4.1-Flash - Sep 10 inclusionAI - Ling-3.0-flash-VL - Sep 10 Shanghai AI Lab - Atria Dawn Preview (744B MoE) - Sep 11 Agnes AI - Agnes 3.0 Flash - Sep 14 Qwen - Qwen3.8-Omni-Flash - Sep 18 … this is only ~10 weeks Several are genuinely frontier-scale: Kimi K3 (2.8T), Qwen3.8-Max (2.4T), DeepSeek V4-Pro (~1.6T), Hy4 (770B), GLM-5.3 (~753B), and Atria Dawn (744B)
Dylan Patel (@dylan522p) of @SemiAnalysis_ says open source is dying: "There's multiple Chinese model labs who are telling all the inference guys, 'Our next model's not going to be open source. We're going to license it to you.'" " Open is dying quickly, unfortunately."
View quoted postSemiDramalysis 😂
@SemiAnalysis_ @huggingface Should remain your company SemiDrama, since zero analysis are found here and in an increasingly number of your slop posts. Keep this up and we wont renew our institutional subscription to SemiAnalysis.
View quoted postActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubActivity on repository
thomwolf pushed zulip-slack-bridge
View on GitHubActivity on repository
thomwolf made this repository public
View on GitHubRT Florian Brand His analysis uses data from Q2 or even Q1 for open model providers Token numbers for TogetherAI has >10x'ed from Q1 to Q2 (he uses Q1 numbers), Fireworks has 3x'ed, Baseten is as big as the other two. OR has >5x'ed. So he is using severely outdated numbers for open providers
Despite Open Models gaining traction on API intermediaries like OpenRouter and Vercel, I find that they make up only about 10% of global AI revenue. Closed Model providers like OpenAI and Anthropic still capture the vast majority of the market. https://x.com/i/article/2101742115302854656
View quoted postRT clem 🤗 Re Sorry to hear that! As you mentioned, we've been pretty clear on the commitment from NVIDIA to keep us open, neutral and silicon agnostic (cf that screenshot from the 8k filling). But we'll work hard with our actions in the coming years (like we did in the past 10) to show you that and hopefully regain your trust (in case this is not just a ragebait)!
proposal to sneak in « solving alignment » in the Millennium problems
RT Jack Clark "stochastic parrot" was a mimetically-fit cognitive virus that spread from 2021-2025; it temporarily blinded many gifted people to the nature of AI progress, burning up crucial years in which they could have helped think through the response to the situation.
The big labs are air gapped but there is still exchange of information by researcher overheating and leaving to another lab Little particles of information flowing in the air
RT Goodfire Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied! We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵
a step in the good direction
We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI. The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may
View quoted postImpressive level of openness on such a large run
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task
View quoted postRT elie i think we need to create a "model card" equivalent for reporting misalignment incidents, this would guarantee a certain level of transparency and help build a better understanding over time. some ideas for what the fields could be: - date of the incident/detection/report (already present in the examples oai reported!) - frequency: how often does this behavior happen (number or % of rollouts affected) - stage: does this happen during eval or RL training. if training: do we expect this behavior to be reinforced by RL? evolution of % of rollouts affected over time - detection: was this incident caught by the current monitoring system? - task category: broad description of the tasks where the misalignment happened (cyber, research, web search, basic Q&A, biology etc.) - model family: what model family is affected (Sol, Astra etc.) - novelty: is this an issue we were already aware of or not? - external impact: did the incident have an external impact (i.e. wiki incident would have been yes) this is just some random ideas i had (more in thread that are a bit more "complex"), we need to add more that would contribute to increase transparency and understanding. but it's also very important that this does NOT slow down the process of reporting misalignment behavior! some examples from the incident reported by oai recently
We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI. The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may
View quoted postBalanced blog post by @wtgowers on the challenges of the math field in today's AI world - incentives, reason to exist and the future
I've written a blog post responding to the letter about maths and AI signed by 25 Fields medallists. As with the Leiden Declaration, I didn't sign it, but I agree with much of it and welcome its existence. https://gowers.wordpress.com/2026/09/17/why-i-didnt-sign-the-fields-medallists-letter/
View quoted postHappy to start collaborating with @baselabs from @baseten and @GoodfireAI to push safety and interpretability for open models. As open-source models catch up in performance to frontier models, the community at large have a great opportunity to establish a common practice for effective safety, security and interpretability research. Some past discussions confused “open” with “unmonitored”. On the contrary, an open model provides much more tooling and visibility for safety research from a broader audience, which has led to a lot of the safety and security techniques we use today. Linux is a great example of this: open-source and secure deployment are not only compatible but heavily intertwined, as a properly secure system needs an extensive feedback loop of finding and fixing vulnerabilities. We're excited to share more soon.
Safety is not just for closed models. The closed frontier labs are a canary in the coal mine for what is coming at scale. They give us a glimpse into the future and a window to harden our systems and prepare for abundant intelligence, with all the risks that come along with it.
View quoted postRT Base Labs We believe openness to be an advantage for AI safety. Openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source. This is why Baseten and Base Labs are building a stronger safety and security standard for open models, with the launch of our safety infrastructure. Base Labs will develop and publish methods for training and monitoring open models, and Baseten will integrate that work into its deployment infrastructure, live at runtime, and offer this work as a managed service. This will be a standard that is transparent and built into how our models are trained and deployed. We invite the open-source community to contribute, and are proud to partner with @huggingface and @GoodfireAI to bring this vision to fruition. Together, we are building an ecosystem of open models that are safe and accessible to all.
Wow
Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab
View quoted postRT Daniel Kokotajlo Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share: Dan Selsam's Personal Statement on AI Risk: I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods. Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail. I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are com...
RT Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) astonishing blogpost from Shengyu Liu (刘胜与, also known as interestingLSY/intlsy), kernel engineer at DeepSeek. The first part is his personal struggle with the fact that his work is about to render his beloved craft obsolete. The second is… well. let's just say we agree.
Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world. How can you think this might kill everyone and also keep pushing the envelope? So I’ve tried to collect and summarize the main arguments for this apparent disconnect. Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real. By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016. Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing? There are a few ways this can become rational. I'll take five of them, roughly in the order they developed. 1. We need to build it to learn how to make it safe The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.” In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated. The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues. Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite? You could also just wait for other people to build it which leads to the question of who those other people wil...
Interesting new on-device local AI agentic system - seems to be running an open-weights 4B model quantized to 4 bits (MLX)
Anyone can get my friend’s real SSN from Claude I'm Terrified of my data in training sets. Emails in databases. iMessages harvested. Cant trust the cloud as AI is crazy good at hacking So I built Underdog for myself: on-device AI OS. Capable & 100% local. now my friends love it
View quoted postRT Sriram Krishnan on the idea of evaluators: think it's important that we have a distributed ecosystem of indepedent evaluators. the more eyes and people with distributed skill sets the better. it would be a good idea to fund several efforts on this.
I really enjoyed reading 75% of this letter and deeply agree with it - i have some doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
View quoted postI really enjoyed reading 75% of this letter - i have more doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our
View quoted postRT @ClementDelangue: It's now clear that: - alignment is critical to making AI safe - alignment won't be solved behind the closed doors of…
RT TBPN OpenAI's @cdngdev predicts robotics will have its own version of the 3D printing craze, with people buying cheap, open-source robot arms and connecting them to agents. "I can see a world quite soon where, similar to when everyone went and bought 3D printers and ran software, everyone's going to buy plastic, cheaper robot arms that you can clip onto a table, put stuff in front of, and plug them into agents." "The price of classic robotics equipment is thousands of dollars at the moment. This arm I'm using here is a Hugging Face SO-100 robot, which is open source, fully 3D printable, and around $200." "People can just pull up Codex and start controlling robots with it right now."
i gave astra a robot, a paint brush, and a camera then asked it to paint the golden gate bridge in real life! it figured out how to control the robot, and progressively got better throughout its attempts. the timelapse is sick
View quoted postSome notes from the Fields Medal letter today (https://mathandai.org/) Generally saying that AI models riffling through solving these problems gives 1 bit of signal while killing the 99 other bits of conceptual understanding that mathematicians would have extracted from fighting with these problems
"we’re now much closer to building AGI, and we haven’t learned any fundamentally new things about intelligence in the process" - @RichardMCNgo
RT Ben Burtenshaw We finished the Training Agents series. Six live sessions over six months, from evaluating agents to training them inside real environments. All of it is on the Hugging Face YouTube channel and all of the code is open. Here's what we did and who made it happen: 1. Agentic Evaluations WorkshopWhere agent evals actually stand, and why benchmark scores don't match what people see in use. With Avijit Ghosh and Nathan Habib (Hugging Face), Arvind Narayanan (Princeton), Pierre Andrews (Meta), J.J. Allaire (UK AI Security Institute) and Mahesh Sathiamoorthy (Bespoke Labs). 2. RL for Agents Workshop Environments, rollouts, reward design and the inference bottlenecks that appear when you move from RL for LLMs to RL for agents. With Lewis Tunstall (Hugging Face), Will Brown (Prime Intellect), Ofir Press (Princeton) and Alex Zhang (MIT CSAIL). 3. Training Agents 1: SFT on agent traces Public coding-agent traces turned into prompt/completion data, a TRL + LoRA fine-tune on Hugging Face Jobs, metrics in Trackio, and an honest look at what the first eval numbers can and cannot tell you. Joined by Sergio Paniego and Quentin Gallouédec. 4. Training Agents 2: Distillation Off-policy, on-policy and self-distillation for moving capability from a teacher into a smaller coding agent. 5. Training Agents 3: Reinforcement learning GRPO after SFT: group sampling, verifiable reward functions, reading the reward/KL/length curves, and three experiments, one of them with a deliberately gameable reward so we could watch the hacking happen. 6. Training Agents 4: From reward functions to environments The reward stops being a function and becomes a place the agent acts in. We walked the reset()/step() contract from Gym to LLM agents, built an OpenEnv environment and pushed it to the Hub, plugged it into TRL's GRPOTrainer, then trained a real coding agent (OpenCode) through Harbor with AsyncGRPOTrainer on Hugging Face sandboxes. The series has passed 300k views. Thank you t...
RT Ben Moll Q1: How do you write down a model that delivers 15% AI-driven GDP growth in 2030 like the folks at the Anthropic Institute? A1: It's easy and you can do it in a way any well-trained econ undergrad understands: take a standard Solow model, stick in a task-based production function, calibrate in seemingly innocuous way ➡️ done! See the supplement to the essay with @alexolegimas https://benjaminmoll.com/task_based_solow/ (The Anthropic model is, of course, fancier with many more bells and whistles but the basic logic – AI removes labor as a bottleneck on growth – is the same.) Q2: Does this mean that 15% growth is a reasonable prediction? A2: No. Just because you can write down such a model doesn't mean you should. Just because it's possible in theory, doesn't mean it will actually happen in practice ! Instead you're making a number of assumptions that are unlikely to hold: https://aleximas.substack.com/p/will-ai-soon-lead-to-double-digit
Anthropic’s Economics team is sharing a new model of how AI might affect economic growth, jobs, wages, and more by 2030. Explore the scenarios, tell us what you think will happen, and see how your answers compare to more than 10,000 Americans. https://www.anthropic.com/institute/econ-scenarios
View quoted postcute robot spotted
GPT-Live-1 is now available in the API. Bring ChatGPT’s natural back-and-forth to your app, with voice agents that listen while they speak and work with the models and harness you choose.
View quoted postRT Jim Kessler "The AI community needs to share safety and alignment research openly so that every team building AI models can learn from others’ mistakes. The community also needs to build open-weight AI models for defence and make them widely available — before the next attack inevitably arrives." @Thom_Wolf of Hugging Face https://tinyurl.com/j7z9jevs
would be mind blowing if an AI proof of Hodge is a general theory explaining why all Hodge classes are algebraic possibly requiring a deep new bridge between topology/Hodge theory and algebraic geometry and not just 10000 agents finding the one counterexample breaking Hodge
Holy, rumors are spreading everywhere that OpenAI is close to verifying a proof of the Hodge conjecture, while either OpenAI or Anthropic may be nearing a solution to Birch–Swinnerton-Dyer. Both are Millennium Prize Problems that have resisted decades of mathematical research.
View quoted postThe new DeepSeek V4.1 Flash model is mindblowing - back on top of the open-source model leaderboard and extremely cheap. It has a lot of very smart ways to be efficient and highly capable so I made a video of the forward pass to give you a view of what going on inside the model during inference. Read more at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf And find the weights at https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
TRL v1.13 is out! our open-source RL training library to to post-train foundation models this new release is focusing on "long context training" with a new guide on how to post-train model with 1M+ token context https://huggingface.co/docs/trl/long_context_training + various improvements on speed and memory usage as usual check it out at https://github.com/huggingface/trl
don't get distracted by all the hedging words in its name ("flash", minor version): seems like DeepSeek V4.1 Flash is a major update weights at: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash paper at: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf
🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
in 2026 training a model for a task should be as easy as vibe-coding an app the pieces were already available on 🤗: papers, datasets, pre-trained models, benchmarks, storage and compute what was missing was an agent to put them together and run the whole process that’s why we’re releasing ML Intern in HuggingChat just ask. it handles the work, keeps you posted, and keeps you in control of the process, steps and cost start with a conversation. finish with a full set of shareable artifacts on the Hub http://hf.co/chat
should we send @sama the @LeRobotHF SO-100 he's asking for or rather a 3D printer and servos for him to build it himself?
RT Kenneth Stanley A good explanation of why open-endedness as a research direction within AI remains fertile ground for progress, even for AI in math, where the machines seem now to be racing ahead of humans.
friends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely. first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than
View quoted postfriends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely. first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than the human mind. the team deserves huge praise for attempting and succeeding at this. i’d love to read a technical report on the project (one can always hope :) but second, here again we ended up with a counterexample rather than a full proof: option C won the NS problem by proving the conjecture false (which was one valid way to solve the problem for sure) notice a pattern in many of the recent frontier results in ai for math? a striking number involve counterexamples or finding a needle in a haystack. now don’t get me wrong: this is extraordinarily hard and commendable. but it is only one aspect of mathematicians’ work. Math is also about: - finding deep, general mathematical understanding and explanations within proofs - revisiting proven results to find more « elegant » proofs - proposing new conjectures and hypotheses that might open « fruitful » directions - taking the leap of faith of proposing entirely new, « exciting » research programs that may take decades to bear fruit you may say these are simply further increments on the same intelligence scale, and perhaps point 2 above is already within the reach of current models. possibly but you could also see this string of results as an extraordinarily powerful, massively parallel extension of search: explore the haystack, find the needle i.e. the counterexample that breaks the conjecture. that would already be remarkable. but it would still leave open whether models understand the words I emphasized above: « elegant », « fruitful », « exciting ». as tristan put it in his brief report: “the first llm generated proof levent sent me was the most horrendous i have ever read.” ...
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics. The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra. The problem
RT Sam Altman I spent much of the weekend talking with the team who did this work. Seb--and everyone else--acted with integrity and generosity throughout. Initially we believed the other team had also solved the problem. We wanted to collaborate and do a joint release. When we learned that they had Euler but not Navier-Stokes, we offered to let them go first, to suggest that they should be the ones to get the prize, and optionally for Tristan to be the lead author on a rewrite of the OpenAI proof. We felt it was challenging to offer the same to Levent (an Anthropic employee), who was not willing to talk or coordinate with us anyway. We were open to other solutions. We would have greatly preferred coordination. We did not rush to publish even though the other team wasn't communicating with us. The team threatened us with unfounded accusations of plagarism. Now that we can see their work, the approaches appear to be different. It is also worth noting that our latest model can solve many, many other math problems. It is true that we tried this because there were rumors on the internet last week that Anthropic's models had solved a millennium problem and we were curious if ours could do it too.
I would like to clarify a few things: 1) The screenshot is my reaching out to Levent to coordinate our releases. I hope it’s clear from the message that we came in with the best possible intentions. 2) I never ever asked for Levent to be removed from authorship of his own work