Follow AI Builders — Not Influencers
RT LangChain ✨ The Definitive Guide to Testing LLM Applications by LangChain ✨ Reviewing agent responses can be a time-consuming and daunting process, from defining criteria for style and accuracy, to spotting new regressions. After partnering with hundreds of companies to enhance their agents, we've put together a comprehensive guide of best practices for testing throughout the development lifecycle. In this guide, you'll learn how to: • Tips for testing across the product lifecycle • Methods for building a dataset & defining testing metrics • Templates for evaluating agents, with visual examples ... and much more! 👉 Get the guide here: https://www.langchain.com/testing-guide-ebook?utm_source=twitter&utm_medium=social&utm_campaign=evals_guide Original tweet: https://x.com/LangChain/status/2020592721820344756
good thread on whether RLMs are just coding agents or not I am still in the camp they are basically just coding agents with some distinct features (just like some coding agents have different subagent/swarm features) eg @lateinteraction in response your points, isn't that just coding agents with: 1. files as input (can be done with UX, eg in claude when you post long snippets it makes it a file at least in ui) 2. just give coding agent a cli command that itself/a subagent 3. with (2) done it can trivially save it to files Omar am i missing anything?
@Teknium The following are not standard in a coding agent: 1. The user prompt P itself (not just external data) is a symbolic object in the environment. The model is not allowed to grep/read long snippets from P. 2. The model has to write recursive code (that calls LMs) to understand or
