Hamel Husain

RT dex preach

I just tried this on a couple of slop tweets. Did a great job of de-sloppifying it, better than any skill can Fine tuning ftw

RT Han we shape our tools and thereafter our tools shape us. it’s okay to use ai to assist writing, but don’t outsource your thinking and voice.

AI slop is not the same as AI generated You can make good stuff with AI. It’s just the low effort/quality stuff that’s slop. Writing has an endemic rn You can write well with AI if you take lots of care and edit heavily

RT Zach Mueller To add on, it is immediate when you can tell someone has either: - Went with AI for the outline -> executed themselves -> light AI tweaks - Outline themselves -> light AI writing -> Human cleanup Or: - “Hey Sol here’s nothing about my style, write about XYZ. Dw about cleanup”

I agree. Is there anyone working on this right now?

Writing AI slop on the timeline is like pouring pollutants into a river It’s a really selfish act. It communicates you don’t care about other people’s time by writing - low information density prose - unecessary poetry, rythm, negative contrasts, and fluff sentences - obfuscated style mediated by an AI so we can’t tell who you really are All this takes tremendous cognitive load to read. But more importantly makes you wonder how little empathy the author has for the reader, and just overall.

If you are slop posting to announce your slop coded project Just know you are completely lost. FWIW slop posting is way more offensive. Destroys trust, especially since AI writing is so bad. It’s gonna come back to bite bc authentic human connections will always be valued by humans

New Blog Post: Do Automated Evals Work? There has been a rise of tools that look through your traces with AI and identifies issues. We tested these tools with real production data to see how good they are. Where they shine - They often spot issues human miss - Integrate into your workflow: viewing traces, creating LLM judges etc. Where they fall short - They miss problems that require domain expertise and taste - Don't have great mechanisms to learn from human feedback - You can get similar results from using your coding agent So you should use them? Yes, BUT do so iteratively with you in the loop. We describe how in the post: https://parlance-labs.com/blog/posts/auto-evals/ It's also a good idea to try using your coding agent with you in the loop, which we discuss in the post. This was written with @doesdatmaksense , who led the research and collated the results.

. @sh_reya will be presenting "Can AI Agents Answer Your Data Questions?" this Wednesday If you are building agents that answer business questions from your data, this is for you. She will walk through a comprehensive benchmark, along with what models & harnesses perform best. She'll also discuss the surprising failure modes that plague these agents, and how you can design around them. Sign up here: https://maven.com/lls/21f487 Notes sent to ppl who sign up

Happening in 10 minutes with @beirmug https://maven.com/p/cbf1b6/what-makes-a-good-search-agent

RT Antaripa Saha Do automated evals actually work? I and @HamelHusain spent last few weeks testing the auto-evals efficacy of different evals platforms. We took 100 real production traces from an apartment-leasing voice agent, manually reviewed the failures, masked the labels, and asked different systems to do the same error-analysis task and discover failure modes. We tested dedicated eval platforms like Braintrust, Arize, and LangSmith, along with ChatGPT, Codex, Claude Code, and Factory Droid. Full post here: https://parlance-labs.com/blog/posts/auto-evals/

RT Morgan Today is a good day to remind everyone that hooks exist, and you should probably be using them, esp. to prevent new models from accidentally deleting all of your files. Sadly this happened to @mattshumer_ with GPT 5.6 Sol, scary stuff! 😳 https://learn.chatgpt.com/docs/hooks

You need to look at the data If a tool could find and fix every issue on its own, it would do the same for your competitors, and there'd be nothing left to set your product apart.

RT OpenAI We audited SWE-Bench Pro, one of the most widely used AI coding benchmarks, and found it no longer reliably measures frontier coding capability. We find the eval to be saturated at a ~70% noise ceiling, and are retracting our previous recommendation that the research community use it as a leading coding eval. https://openai.com/index/separating-signal-from-noise-coding-evaluations/

RT OpenAI GPT-5.6 Sol, along with Terra and Luna, will launch publicly this Thursday. We’re expanding preview access globally now.

RT Shreya Shankar This was super fun. The most useful thing I talked about was how to use the Monitor tool in the Claude agent sdk (which watches background tasks and reacts to each stdout line as an event) so that user actions in an agent-generated UI actually go back and steer the agent, instead of the UI just being read-only. This is very powerful; you just instruct your coding agent to, as part of generating the html artifact, generate JS code to compile user interactions into logs (eg “user clicked X, user typed Y in this feedback box”) and the agent will, thanks to the Monitor tool, subscribe to those logs in real time and adapt the UI or provide new insights. The key novel idea is the UI code is no longer just output, it's the input too, and it's generated by agents so it’s custom-built for whatever task you're doing. I have not seen people use Monitor this way before. So I am preaching it everywhere I go. See my skill in the comments

I barely interact with my OS the same way anymore. Life is good

RT Lambda Frontier model prices keep climbing. Open models keep closing the gap. Lambda's @TheZachMueller is teaching a free session on when to switch and how to make it work. Part of @HamelHusain and @sh_reya's AI Product Engineering course on Maven. July 24. Free: https://maven.com/lls/21f487

You can have always on remote-control for claude code by enabling this in the config ~/.claude/settings.json { "remoteControlAtStartup": true, ... } Make sure your'e logged in with subscription. It's can be brittle, for example when I change /effort on the host it hangs on the client. But, its a huge quality of life upgrade

100% if it’s not important enough to write, it’s certainly not important enough to read.

RT Shreya Shankar Really nice essay (written by an undergraduate!) reflecting on how math is changing thanks to AI. I resonate a lot with the discussion on proof digestion & indigestion; I see analogs in my CS fields as well. One core idea of the essay is that AI put us in a world of abundant generations (proofs, code), and now we need to be able to digest these artifacts. Digestion is about teasing apart the “interesting” bits of the artifact (eg hard, novel, clever) and figuring out how to communicate these interesting bits to help other humans understand it. This is not easy. Takes people 10+ years to learn how to do this. My thoughts: Digestion is human-centered — I never want to say “human in the loop” in the connotation of supervising / checking AI, because that’s just boring — and, in my opinion, requires a deep understanding of the pragmatics (in the linguistics definition) of the field. Pragmatics is about how we interpret language (not the semantics or literal meaning of language); all external context, social situations, etc. And AI just cannot develop this understanding of pragmatics yet; it lives in human-human interactions; related internet writing is mostly post-hoc reflections on pragmatics, not pragmatics itself. So I resonate with this essay a lot; us next generation of scholars have an opportunity to think about and set new norms for digestion. I love that Apoorva separates verification from digestion; I think CS folks could benefit from this framing (too many people are focused on verifying AI generated code without thinking about how we are going to digest the millions of lines of code that meets some guarantee we may not even understand). But anyways, it can be an awesome next decade in research. And on a side note it’s great to see undergrads writing like this; the kids are alright 😆

This is most writing nowadays

When someone does cold outreach with AI slop I sometimes do this

Codex really is the superapp and spans everything I can think of doing. Especially because of the mobile support

RT jason If you use Codex, is there any reason you still use ChatGPT? what do you use it for? how has it been better or critical for you?

Two simultaneous AI narratives right now: 1) You can now do the work of 20 people, learn anything and create anything with AI. Just learn how to use claude. 2) We are investing Billions in forward deployed engineers to help you implement AI b/c its too time consuming and hard for you to do it yourself. I skew more towards #1, fwiw. But I find these narratives to be incongruent.

SREs, your AI rebranding moment has arrived

RT Hamel Husain Re Depends on the purpose of the writing. If the purpose of the writing is to demonstrate how you think, it’s pretty self-destructive to let an AI write for you, because you are no longer showing how you think its way too masked by all the unnecessary literary devices, poetry, negative contrasts, rhythmic language, that you can’t really strip out unless you heavily edit it You're also signaling that it's not important enough to read. AI slop is usually very low-density information. 50% of the sentences can be deleted, so no, usually it's very poor writing and it wastes the time of the reader. The lowest hanging fruit in improving almost any writing is to delete things that are not necessary. AI discourages this. If it’s just emails and other things that people don’t want to read to begin with I think it’s great Counter example is family members using AI to draft happy birthday messages to you. No thanks I’d rather have the misspellings etc

If you think written human slop is worse than AI slop you have surrounded yourself with the wrong people

The best way to trigger a teacher is to ask “Are the slides and recording available”

Many such cases. Look at the data

What happens to the AI when the forward deployed engineers leave? Do they have a life long dependency on consultants? Are the consultants incentivized to make themselves obsolete? I would love to see this discussed.

Them: what’s your favorite LLM agent framework? Me: Python Them: What’s your favorite observability framework Me: A database

RT Simon Willison I absolutely hate how I'm getting to be suspicious of ANY reply to my posts here, especially ones that pose a question, as they are so often from bot accounts If you're running a bot like this please stop, you're making the internet a worse place for everyone

RT verrsane Re @HamelHusain @hugobowne Can’t wait for unemployment engineering

prompt engineering harness engineering loop engineering ? recursion engineering pointer engineering stack engineering queue engineering tree engineering inheritance engineering abstraction engineering lambda engineering closure engineering async engineering

People who blog about taste and the blog is AI slop really tells you something

Is this real?

Re It’s a really nice knife btw. Actually useful 😍

Langchain completed the circle and sent me a knife

RT Jan this is a great read for those working on "really hard to eval" stuff like science

Love the Agentic MapReduce approach. First learned about it in DocETL and it works well across many kinds of tasks even beyond security. Check out this git repo for more details https://github.com/ucbepic/docetl

Next up in the series is @GoAbiAryan on LLM inference optimization with a hands on exercise! Tomorrow 11am PT Sign up here: https://maven.com/lls/21f487 recordings also sent to everyone who registers Abi is a legend when it comes to inference, highly recommend this one

One thing to remember is that @lennysan has been doing this for a long time. Like more than a decade at this point. Lots of people would have pivoted, got bored/distracted etc. Focusing on one thing and doing it well compounds like crazy long term

I was curious about substack economics so made this explorer based on public data. I don’t have conversion numbers so there is a toggle to play with. @lennysan is the 🐐 https://substack-stats.vercel.app/

What’s funny is the comments thread is just people arguing 🤣☠️ HN can’t stop doing its HN thing

RT Anthropic We’ve received notice that the Department of Commerce has lifted export controls on Claude Fable 5 and Mythos 5. We'll begin restoring access tomorrow, and will share an update soon. We’re grateful to our users for their patience, and to everyone who worked with us on redeploying the models.

RT Nick Khami there's a new grad out there somewhere asking their boss if this means they can get rid of kubernetes

Agreed. The world also needs more teachers that care at all levels from preschool all the way up to professional levels My favorite example of this is @rasbt who educates + research (and is killing it - well deserved b/c he provides tons of value). Just be the best at what you do and there will be rewards.

I genuinely believe that 95% of the replies I get on this site now are AI generated

Human-as-a-judge …. wait 🤣

RT Bryan Bischof fka Dr. Donut I keep telling people that evals teach you how to build your product; either by showing you how it should work or that you're not building the right thing at all. Hamel wrote up what this means in practice.

RT Mike Munroe Another great post from @HamelHusain grounded in real AI based functionality running in production applications. My takeaway, don't forget that some problems are solved through thinking more about your end user and not the "technical" solutions you are trying to find.

RT Shreya Shankar Hamel wrote a nice blog post about making AI products easier to evaluate in the first place, before trying to codify all the evals

New blog post: “It’s Hard to Eval” Is a Product Smell If you find it hard to verify AI output, chances are that your users will too! In other words, I often find that product design is the bottleneck In the post I embed three **interactive before/after examples** based on products I've helped with: 1. an AI data agent that answers business questions 2. a PE lesson‑plan generator for K‑12 teachers 3. a workers’ comp tool that drafts 50‑page medical reports I believe this is a significant issue in AI Engineering and upstream of evals! Link to post: https://hamel.dev/blog/posts/eval-smell/ Note: I'm not a designer so the design sketches are far from perfect, but I felt it was important enough to spend a significant amount of time on this. Thanks to @sh_reya and @isaac_flath for feedback.

RT Mike Munroe Saw a presentation from @barrowjoseph on vlm OCR today at https://www.skool.com/ai-eng/about. Great presenter and shared a lot of tactical insight. If similar interests, might want to check out https://maven.com/p/95c291. Thx @barrowjoseph!

So true, even of dashboards in general

Codex desktop is miles ahead of Claude for remote access via mobile or another computer. Codex allows you to see **all** running sessions on **all** devices from any other device, including mobile You can truly go touch grass with Codex. Brings me joy every day

Yes! binary judges are far more practical for most people, because likert scales (or scores) have too many footguns All the flashcards are here (inspired by @chrisalbon ‘s flashcards) https://maven.com/parlance-labs/o/540bd8

RT Gergely Orosz Polymarket acquired Craft Agents, and @balintorosz (my brother) will lead Product Engineering. I’ve now made the January deepdive on Craft Agents free. Craft Agents shares ideas with Claude Cowork - built before Claude Cowork! It’s also open source. https://newsletter.pragmaticengineer.com/p/ai-first-makeover-craft

re: GLM 5.2 and vision Even though GLM doesn't have vision capabilities some coding harnesses like Cursor and Amp automatically route the image through a different model first so it still works Opencode will refuse and will tell you GLM doesn't support images

Yes, be Andrej > don’t give a fuck about Twitter trolls because it doesn’t matter what random people think Living to please Twitter is a grave mistake. Don’t do it. Do what makes you happy

RT Shreya Shankar Super excited to receive a Laude grant for DocWriter! AI is revolutionizing knowledge work, but AI-assisted writing is still atrocious. So we're building a new harness and UI for writing. Along the way we'll tackle some broadly-applicable challenges: (i) harnesses that natively support async human-AI collaboration, (ii) steering long-horizon (e.g., multi-month) agents, and (iii) open-source frameworks for automatic, continual evals as models and human behavior drift (e.g., new slopwords arise). http://docwriter.org -- email if you do a lot of writing for your profession that needs to be high-quality & want to partake in our user studies

RT Kit Ledru Really good start today with @sh_reya's first talk, or rather live demo of using the LLM to do error discovery (on writing in this case) and manually annotate user-selected instances of failing text, in a custom web frontend leveraging Claude Code "Monitor" feature so the LLM sees what you do. All from a simple markdown skill! (Repo link in reply below) 👇 Come join the AI ~~revolu…~~ Product Engineering! PS: video + slides will be shared in a couple days; just register for free to get those links, and then we can talk about talks here!

RT Shreya Shankar I made this talk without using LLMs. It took only a half day and was way more fun than I expected. I guess I didn't realize how frustrating the UX is of (i) arbitrarily serializing my workflow and (ii) handing the cognitively interesting work off to the LLM so I end up a verifier

RT Ben Clavié every single person speaking in this series has imposter syndrome while looking at the other speakers, @HamelHusain really cooked

Starts today at noon PST https://maven.com/lls/21f487 Recording / notes sent to everyone who registers

I’m getting RSI from hearing the word RSI

RT Joe Barrow When was the last time you thought about OCR? Not recently enough! Remedy that on July 10th! As part of @HamelHusain’s AI Eng mini course I’ll be giving a talk all about OCR, the infra behind it, and why it’s a really hot field rn.

RT Shreya Shankar Really stoked. Tomorrow’s lecture is a deep dive on how much one can automate evals with AI coding tools. I am going to share many insights on how to effectively use AI to debug AI…tune in to learn!

Tomorrow @sh_reya and I kick off this free AI product engineering mini-course. Topics covered over 12 talks: 1. Design/UX & Evals 2. Retrieval 3. When & how to use open models effectively With these legends: @TheZachMueller @bclavie @xeophon @GoAbiAryan @barrowjoseph @willccbb @marek_galovic @beirmug Sign up & learn more here https://maven.com/lls/21f487

I've been trying @SakanaAILabs Fugu Ultra as an oracle to replace Fable I packaged it up as plugin for codex/claude https://github.com/parlance-labs/super-oracle I don't have rigorous evals 😅 but vibes seem like its a bit jagged in its abilities. ex: It seems great for code review but not for front end development. Curious what other's found so far

RT Bryan Bischof fka Dr. Donut It’s hard to play ping pong on a boat so we are hosting this event at Spin. If you’re attending @aiDotEngineer worlds fair, this is the event. :)

RT Daniel Brooks From all the interviews ive done i think the hottest skill rn seems to be llm evals

RT Ashish Soni Great opportunity to learn :) https://maven.com/lls/21f487 @sh_reya @HamelHusain

RT Yun-Ta Tsai Many people think any given ML project is 99% training. In reality, it’s 50% evaluation, 40% data cleaning, 8% integration, and 2% training. The first two set the noise floor for learning. No ML magic matters; the model cannot lower the noise floor, as that’s the optimal bound of Shannon encoding of your data. Thus, not a single day goes by without me thinking about ontology. Even the old labels have to be constantly reviewed.

RT Charles 🎉 Frye Speculation Is All You Need. In this blog post, we announce the co-release (w/ Z Lab) of six more state-of-the-art DFlash speculators for @Alibaba_Qwen 3.x. Over 1k output tps for 3.5 122B-A10B on a B200. Read the blog for why we're all-in on spec dec. https://modal.com/blog/spec-is-all-u-need

RT Randy Olson I'm excited for this one. @HamelHusain, @sh_reya, and folks who build AI products for a living are teaming up to run a bunch of free live sessions over the next month: OCR, retrieval, evals, open models, all with live Q&A. Starts June 24. Sign up here: https://maven.com/lls/21f487

RT Zach Mueller Super excited to yap with a few of my favorite people. Come join! (Btw it’s free)

RT Shreya Shankar Building AI products is hard. But it's getting increasingly popular! I'm really excited to share that my friends and I are putting together (the best) lecture series on AI Product Engineering this summer!! We've got an awesome lineup of talks spanning data, evals, and UX. With more to come. The lecture series is completely free! And ~2k people have signed up already even though we haven't posted on social media yet! I can't wait. Join us and sign up: https://maven.com/lls/21f487

BTW many who ask for meetings also secretly hate them. Just ask and set each other free. You both will be happier

RT Chris Tate Quick UX tip: Crossing out completed todo items makes them harder to read Checkmarks + dimming are usually enough

Just used an AI agent inside my bank. Amazing

RT Shreya Shankar I feel like this table is misleading. (I) It is not that novices write underspecified tasks; their methodology classifies underspecified prompts as “novice.”(ii) Every time a frontier lab says “expertise in the task,” they implicitly mean “expertise in the task AND the relevant data to analyze.” This assumption is trivially ok when there is no data to analyze, like in many coding tasks, but it’s unreasonable to assume people to be an expert in the specific dataset at hand. There are always ambiguities in real world data. Human data scientists take months to onboard. AI data scientists take $100k to build understanding of an enterprise-scale dataset (ie “semantic layer”), and still are no good. The North Star for AI data analysts is to assist people who want to make data-driven decisions but don’t know what’s in their (real, messy) data. So perhaps it is a contrarian take for me to say that this “novice” prompt should actually work…and we have a long and exciting way to go to make this happen

I liked how they are using MotherDuck for skill observability very practical

RT Tibo Reminder that you can use the Codex App, CLI and SDK with any open source model, not just with OpenAI models. https://developers.openai.com/codex/config-advanced#oss-mode-local-providers

Had fun on @hugobowne ‘s pod he made this page out of it https://hugobowne.github.io/show-us-your-agent-skills/agent-skills/guests/hamel-husain/

RT Farza 🇵🇰🇺🇸 We built an AI that can draw on your screen. It's a true personal tutor. Using Claude Opus we're able to draw polygons, point with pixel perfect accuracy, and walk users through complex steps directly on their screen. Here's me learning Pythagorean Theorem + FL Studio. Demo: