X (Twitter)

RT Sydney Runkle "it's underrated how hard it is to build agents!" great convo w/ @hwchase17 and @HarryStebbings on @twentyminutevc an agent = model + context + harness, and you can improve an agent by tweaking any of the three. a few takeaways: 1. the highest roi thing you can do is start by optimizing context (then the harness, then the weights); this is where domain knowledge lives and that's incredibly valuable 2. the story of continual learning is: run an agent, collect traces, aggregate learnings, fix something, measure the fix, loop! the fix can land in any of the three layers. those fixes all look different but the cycle is the same. 3. memory is still massively unsolved, but also massively important. the hardest part is distilling learnings from traces which is incredibly domain specific. 4. when you build agents, you're probably worried about a few things: accuracy, cost, latency, and security. we've seen these themes confirmed year after year in our dev surveys 5. when we think about reducing cost, the pipeline looks like adding visibility into spend, putting controls in place, and only then pushing on agent optimizations. this is how we reduced our coding agent spend @LangChain 6. model routing, which is great for cost controls, is most effective in the harness (where the context needed for a routing decision lives) 7. open models are used for more narrow tasks right now (small parts of the agent loop OR for very specific domains w/ post-training), but as they improve they'll power more core agent loops 8. we're looking at a future with both horizontal agents and vertical agents. horizontal agents will use the vertical agents, and infra needs to change to support this load (agents can do background work at a rate that humans cannot). https://www.youtube.com/watch?v=1jDCPEHwQfI

Follow AI Builders — Not Influencers

steipete opened a pull request in Trimmy

steipete created a branch