Harrison Chase

router lives in the harness, not a gateway, bc the harness knows the task blog: https://www.langchain.com/blog/how-to-build-a-model-router-in-the-harness https://x.com/neil_xbt/status/2109979966859510156

RT LangChain 🔎 Announced at Interrupt: AI Search in LangSmith You used to need an evaluator or read hours of conversations to find traces that answer questions such as: 💬 Does the user seemed frustrated?" 💬 Where did the agent fail to complete the task? 💬 Where is the user trying to cancel their subscription? Now, just describe it. LangSmith returns the traces where it happened. https://www.langchain.com/blog/agent-traces-ai-search-langsmith

we are thinking a lot about how to make deepagents more async native. specifically about making async subagents work better/more prominent one key question is how does a finished subagent "notify" the parent agent? some default behaviors we're considering for when the parent is still running: 1/ interrupt parent run, apply subagent update, continue (basically same as "steer") 2/ wait for parent run to finish, then batch subagent update with any other new messages (basically same as "queue") another question is how to handle human in the loop? what if main agent is not running but paused and waiting explicit human response to an action? a/ wait for human to respond, then batch the subagent update with that b/ resume thread (which would cancel the human interrupt) with the subagent update (thread could then re-ask for the same human approval) any other options we're missing?

RT Josh Rosen While we’re talking about agent tracing, I couldn’t help but mention that @LangChain needed to invent an entirely new database, SmithDB, for LangSmith to accommodate agent traces. Why? - hundreds of nested spans per trace - huge prompts and tool outputs - runs that stay open for hours - events that keep arriving over time - full-text search + JSON filtering

RT Viv - deepagents is durable by design - built on langgraph with state machine primitives hardener over years - deepagents is extendable, pick the exact pieces you want or make your own - deepagents is open, use it as a base for post-training any experiments

RT Sydney Runkle deepagents just hit 30k stars on github! deepagents is great for building agents that do complex work. they manage context well and use a durable runtime. though inspired by coding agent design, deepagents are for any domain! https://github.com/langchain-ai/deepagents

RT Josh Rosen https://x.com/i/article/2109413179226750976

RT Shashi 🔥 Official London 🇬🇧 Agent Week is here. 🗓️ Tue 13 Oct: Interrupt by @LangChain at Outernet London. @hwchase17 and the team are bringing the agent conference to the city with workshops and production stories. 🗓️ Fri 16 Oct: Agent Engineering, London by @AgentEngHQ bring together CEO/CTO and leaders from Agent Infra companies from UK 🇬🇧 and Europe 🇪🇺 @samuelcolvin @abreslav @pelaseyed @tobiemh @jonbratseth @MeryemArik9 @bartheletf @sbichenko @Jocelyn_K_DArcy and more 🍕🍻Plenty more meetups and workshops across the week too. If you are an agent builder, you should be in London this week.

Ha gave this take to our team last week Will share some (very basic) stuff I vibe coded around this shortly Lots to be explored here!

RT Austin Vance Agents work in many modalities, similar to people, our (human) work is preformed on behalf of ourselves, our teams, or our organizations. When a human does something they often say I updated the deal in Hubspot, or I sent this to Cathy in slack. There we have a human and a tool working in tandem. Other times it's the "the infra team needs us to migrate..." or I submitted a service now ticket to finance. That's a team and system. Companies are the same. With agents we want to know, and audit, action, system, and approver. The future will be "the agent submitted a service now ticket to finance on behalf of Cathy" Agents need identities that are tied to their operation and operator.

RT Assaf Elovic Gpt researcher just crossed 30k stars. What excites me the most is that since early 2025 it’s been largely driven, built and maintained by the community. No marketing, no ads, no funding. Just an incredible open source community building one the best deep research agents out there. That’s the beauty of open source.

this is the summary slide that describes how all agents in the future will look very much aligns with our vision for managed deepagents https://docs.langchain.com/langsmith/python/managed-deep-agents-overview durable workflows: builds on LangGraph/LSD https://docs.langchain.com/langsmith/deployment can use a computer: uses Langsmith Sandboxes https://docs.langchain.com/langsmith/python/managed-deep-agents-sandboxes memory and skills: langsmith context hub https://docs.langchain.com/langsmith/python/managed-deep-agents-skills Triggers and scheduled jobs: schedules https://docs.langchain.com/langsmith/python/managed-deep-agents-schedules connectors and auth https://docs.langchain.com/langsmith/python/managed-deep-agents-connections text and voice channels https://docs.langchain.com/langsmith/python/managed-deep-agents-channels payments and identity - only one missing! evals https://docs.langchain.com/langsmith/python/managed-deep-agents-evals observability: massively, through langsmith https://docs.langchain.com/langsmith/observability

this is a great talk if you want to just follow along in slides, extracted screenshots here to make it easy writing up thoughts as we speak https://a-2jp6ke32btx7kfwxvhewash4vi.smithbox.dev/ZXlKaGJHY2lPaUpGWkVSVFFTSXNJbXRwWkNJNklrVmpWRFZxTVd3MmNIQXhlVFZJU3pCQ2VtSjBhMjFTTlc1SmJuTklVVTFqYWpKVVNVUk5kMmgzVURRaUxDSjBlWEFpT2lKS1YxUWlmUS5leUpoZFdRaU9sc2ljMkZ1WkdKdmVDMWtiM2R1Ykc5aFpDSmRMQ0pwWVhRaU9qRTNPVEUzTWprMk56VXNJbXAwYVNJNklqQXhZVEV5WWpZNUxUSmtZbUV0TnpRMk5TMWhPV1E1TFRsbE16aGxaakEwTnpnM1l5SXNJbk5wWkNJNklqQXhZVEV5WWpVeUxXVmpOMkl0TjJRM01DMDVNRE0xTFRkbU5tTXlZV1kxT1RWak9TSXNJblJwWkNJNkltVmlZbUZtTW1WaUxUYzJPV0l0TkRVd05TMWhZMkV5TFdReE1XUmxNVEF6TnpKaE5DSXNJbkJoZEdnaU9pSXZkMjl5YTNOd1lXTmxMMnhsWTNSMWNtVXRibTkwWlhNdllXeDNZWGx6TFc5dUxXRm5aVzUwY3kxemJHbGtaWE11Y0dSbUlpd2lZM1FpT2lKaGNIQnNhV05oZEdsdmJpOXdaR1lpTENKalpDSTZJbWx1YkdsdVpTSXNJbk55WXlJNk1UVjkubDM4X195Y2RYRGRqZTZWejIxU09LbzJhMm43YW5NQ0RMdFVzSmR2LUJMYngyREFpbHVFelZGcV94b2NBVkp5RUs2WXlleGpHaFBMZEFIZlFrOWk1Q2c

historically, most of our internal agents (openswe for coding, gtm agent for outreach) have acted "on behalf of" whoever is using them but as they start to do more proactive, background work... I wonder if that should change. eg i really like the idea Hone put forward of agents that are striving towards a goal (https://x.com/hone/status/2108218574926176490?s=20)... these kind of seem like coworkers, and i wonder if they should be treated as such

what is the future of building internal company agents? how will their identity work? 1. fixed set of credentials 2. "on behalf of" flow (uses credentials of whoever is using it) 3. some fixed credentials, others "on behalf of"

RT Caspar Or don't pay, and use our open-source software factory instead: https://langch.in/openswe

RT LangChain New in Managed Deep Agents 0.9: Reactions API for @SlackHQ Channels. When it comes to building effective user agent experiences, loading states are underrated. Now, you can build loading states and receipts using either heuristics, decision models, or custom logic.

RT nathan drezner We've been thinking a lot about great agent UX with Managed Deep Agents - here's some of what we've been building in this area and a bit about our product philosophy https://x.com/i/article/2108561070809821184

RT LangChain ✅ 60k queries handled ✅ 500+ customer accounts ✅ 85%+ of sessions resolved without a support ticket ✅ 250+ cases auto-detected + escalated to the right team @SnykSec Assist. Built on LangChain + LangGraph. Observability in LangSmith. Full story: https://www.langchain.com/blog/how-snyk-turned-an-internal-support-agent-into-a-customer-feature

RT LangChain .@Factory_AI is sponsoring Interrupt London! Stop by their booth during the event and go hands-on with the autonomy stack for enterprise teams.

This is great point Goodharts law: when a measure becomes a target, it ceases to be a good measure Initial take: eval driven development works for narrowly scoped things, but for more autonomous agents it doesn’t So then - how do you hill climb those?

eval driven development

trajectory labeling is really several questions, not one pass/fail - nice framing here that's how Jev as a judge works in LangSmith evals: difficulty and correctness each get their own typed answer, in one pass, on every trace https://www.langchain.com/blog/jev-is-now-available-in-langsmith-evals https://x.com/xennygrimmato_/status/2108303751136297226

Sam on where decision models fit in a harness and how to use Jev with LangChain https://www.langchain.com/blog/building-a-harness-with-jev https://x.com/jaminball/status/2108258355529875932

RT Sydney Runkle the @OpenAI decisions API is now in @LangChain! we shipped: 1. OpenAIDecisions: ask yes/no, choice, and score questions 2. model routing middleware 3. auto mode middleware docs: https://docs.langchain.com/oss/python/integrations/decision_models/openai#quickstart

RT Sydney Runkle i'm most excited about the idea of "binding tools to skills" this is basically progressive disclosure^2 -- when a skill is activated w/ relevant instructions, it comes w/ corresponding tools great feature if you want to assemble context just in time for a given task

interrupt london is next tuesday really looking forward to hearing from Orange and OVO on what's actually working with agents in production. we're also running hands-on workshops on Deep Agents and Engine https://x.com/LangChain/status/2107456553452261751

One cool thing deepagents now supports: dynamically loading tools when is a skill is loaded Some skills require specific tools that you may not always want to have available. This now allows you to pair them With openai/anthropic models, it’s possible to do this in a way where it doesn’t break the prompt cache!

RT Sydney Runkle https://x.com/i/article/2107886964427182080

RT Caspar managed deep agents 0.9 is here! things we shipped: ⏰ agents can schedule tasks OBO themselves or the user 🏭 dynamically configure your mda per run (models, tools, skills, etc.) 👀 Slack reactions show when your agent has seen a message 📁 improved multimodal capabilities 🔌 MCP 2.0 support via langchain.mcp released for js and py http://langch.in/mda

RT Jake Broekhuizen Come join us! We're hiring across nearly every role 🚂 https://www.langchain.com/careers

We have a really strong team :) Come join us!

RT Shashi Can we call next week as official "London Agent Week" @hwchase17 @ankush_gola11 WDYT? @LangChain Interrupt is happening on 13th October and I am hosting @AgentEngHQ Agent Engineering London 🇬🇧 happening on 16th October 👉 https://agentengineering.world Its going to be awesome week 🙌

RT Caspar time to level up https://langch.in/mda

Good take on harnesses

RT Alex Lieberman Hot take: evals are the #1 thing keeping companies from becoming AI-native. If you want to understand evals, read this post & then the accompanying guide. Tweaking prompts and trying a few inputs by hand only gets you so far; evals are what get your agents ready for deployment and keep them working once they're live. First, some definitions... - An eval: a grader applied to a trace. - A grader: anything that scores how well the agent did, from a simple code check to an LLM acting as a judge. - A trace: the full record of one agent run: the input, every step it took, and the output. For example, a customer asks "Where is my refund?" The agent calls search_orders, then issue_refund, and replies "Refund sent, 3–5 days." Next, let's dive into graders... Graders come in a few families, and most real setups run several on the same trace. 1) Deterministic checks. Plain code, no model involved. Did the agent call issue_refund exactly once, with the same order ID it looked up? A double refund or a refund on the wrong order fails instantly. 2) Reference-answer graders. Compare the output to a reference answer stored on the case, by exact match or with an LLM judge checking whether the meaning matches. 3) Rubric graders. An LLM judge scores the output against criteria, with no reference answer needed. Is the reply concise? Does it confirm what happened? 4) Trajectory graders. These grade the steps the agent took, not just its final answer. Did it look up the order before issuing the refund? A run that calls issue_refund without checking the order first fails even if the final reply reads perfectly. What are online vs. offline evals? Online evals watch production. Offline evals run in your lab: on demand or in CI, before anything ships. - You need both. Online without offline means you can see problems but can't fix them safely. - Offline without online means you're testing the scenarios you imagined, not the ones your users actually create. - Wha...

RT Sydney Runkle deep agents just shipped a round of multimodal upgrades! 1. broader file support on OpenAI Responses: docx, pptx, xlsx, csv, json, markdown, not just pdfs 2. opt-in binary offloading to keep big files out of your checkpoints 3. a batch of reliability fixes more in thread 🧵 !

cool to see cognition doing "dreaming" - memory needs an offline cleanup loop, not just better retrieval, otherwise stale records pile up open q is how inferred memories get validated before the agent uses them glad it's an open standard too - memory should be open https://x.com/cognition/status/2107165034463867001

we're seeing our coding agent costs decrease significantly for second month in row longer blog on this later, but three quick steps we took 1/ cost visibility. you cant control what you cant measure. Start tracking everything to langsmith. This gives us super granular visibility into who is using what, how they are using it, etc. LangSmith supports first party integrations with all main coding harnesses https://www.langchain.com/blog/your-coding-agents-are-a-black-box-heres-how-to-crack-them-open 2/ cost controls. Through our LLM gateway, set user level cost caps. (You can increase your cap by talking to our VPE). This helps curb unexpected or accidental spend. https://docs.langchain.com/langsmith/llm-gateway 3/ optimize the harness. Obviously 3rd party harnesses are tougher to optimize, but we are doing more and more coding through OpenSWE, our open source cloud agent harness. Things like model routing help bring the cost down https://www.langchain.com/blog/how-to-build-a-model-router-in-the-harness feel free to reach out if you want help implementing these at your co!

specialized agents or one mega agent? we're having this debate internally...

cool project by @dbreunig in langchain this is ModelRouterMiddleware - jev reads the first message, picks the model, and that model handles the whole run jev is cheap enough that you can also re-pick after each tool result with a custom hook docs: https://docs.langchain.com/oss/python/integrations/providers/typesafe#model-routing https://x.com/dbreunig/status/2106456056042025235

good pattern from this google paper the verifier starts from a clean context, so the explorer's reasoning can't talk it into a bad proof so the explorer can try wild ideas and the verifier tosses the wrong ones same reason deepagents subagents get their own context https://x.com/omarsar0/status/2106056369816420624

RT Sydney Runkle if your agents are going to production they have to be durable

every agent team ends up wanting its own review ui over their traces - LangSmith Custom Apps lets you build it on your own data and ship it right into the workspace https://x.com/LangChain/status/2106067328601370724

Memory is hard to get working well, especially in a company setting Making some improvements to how we memory in managed deepagents!

RT Caspar User memory in Managed Deep Agents 0.8: your agent can remember the people it works with https://langch.in/mda

congrats @AravSrinivas! more entrants in the decision model class, and this one is open weights cheap typed answers for the small calls inside a harness (routing, approvals, judging), big model for the rest https://x.com/AravSrinivas/status/2105774153903268288

pretty clear every agent harness needs a durable runtime pi :: pi-durable deepagents :: langgraph

RT Amada Echeverría jes, a jev open-source guardrails project LangChain Ambassador @EdenEmarco177 has been working on It uses LangChain’s TypeSafeClassifier under the hood. The docs include a LangChain middleware example for checking prompts, tool calls, tool results, and responses.

decision model season indeed said there'd be more entrants before long - now open weights too, nice work @ritakozlov + the cloudflare team your harness should be able to swap its decision model as easily as its main model https://x.com/ritakozlov/status/2105683951595725177

model routing - "no one knows what it means, but its provocative, gets the people going" we try to shed light on how to do it properly 1. understand tasks 2. understand the models 3. build the router **in the harness** 4. track outcomes lower costs, with no performance hit

RT Sydney Runkle we built a model router for our coding agent that cut median cost per task by 64%, with no measurable drop in quality! most tasks don't need top tier intelligence! here's how we built it, why we think it belongs in the harness, and how to add one to your own agent :)

RT Sydney Runkle https://x.com/i/article/2105489324590645248

RT Jake Broekhuizen We spent a lot of time streamlining the process to go from your agent's trajectories to an SFT'd model on your training/inference provider of choice so you don't have to stitch together the steps manually. The data curation phase is definitely the most important, and we've had great results from working with training copilots in the form of model councils and a coding agent guided by the CLI and its packaged skills. This + the good ol' fashioned 'look at the data' ring true for any well-designed eval & post-training pipeline, and we're really excited to enable builders to use this in the process of owning your own intelligence!

RT nathan drezner I had some folks asking for the source code for this demo! Link in 🧵 if you want to fork/build off of it in any direction. Our documentation for HTTP channels is also fantastic.

great writeup by @pirroh harness doesn't disappear as models get smarter - it shifts from deciding to offering in deepagents you give each subagent its own model, tools, and permissions - the main loop picks which to spawn and when docs: https://docs.langchain.com/oss/python/deepagents/subagents

listen is a great company, alfred and florian are awesome founders the problem space listen is tackling is also an amazing use case for AI, very bullish on what they will still build here!

No

RT nathan drezner We've just shipped a new terminal interface 💻 for Managed Deep Agents. It rips. Use it to instantly test drive your agent locally. Studio isn't going anywhere; the TUI is handy for lightweight development. Take it for a spin with `mda chat`. Personally I'm a big fan of the theming the team put together 🌈

company OS (org harness) vs personal agent, similarities and differences: differences: - Many principals (users), one actor (agent): will need to be multiplayer, handle multiple users (sometimes even in same thread), and as a result handle auth/memory correctly - governance: observability, auditability, admin control plane matter way more - interaction pattern: more event driven and async similarities: - should be able to write/execute code - browser use likely very important! (key difference from a coding harness) - skills/MCP as standards - core agent loop/logic what am i missing?

coding harness -> personal harness -> org harness (company harness? enterprise harness? still figuring out right name, open to suggestions)

every agent will need a sandbox and the harness should live outside it - separate the brains from the hands great to see nvidia open sourcing tools here, agree with Andrew this is part of the harness not a bolt on https://x.com/AndrewYNg/status/2104660347730969087

retrieval is a decision problem, not just a similarity problem cool experiment from @assaf_elovic swapping embeddings for Jev in GPT Researcher - 73% vs 46% relevant context decision models will show up all over the harness https://x.com/assaf_elovic/status/2104562754774303203

RT Assaf Elovic Didn't expect this 🤯 We replaced embeddings with Jev in GPT Researcher's RAG pipeline and tested both on 28 research tasks from SimpleQA and open ended research. Jev beat embeddings on every quality measure we ran: - 59% more relevant context (73% vs 46%) - Reports preferred 15 to 3 in blind comparisons - Same cost per report GPT Researcher now runs on Jev by default, and no longer needs embeddings at all. All you need is @LangChain + @tavilyai +Jev for the perfect RAG system. Check out the repo here: https://github.com/assafelovic/gpt-researcher Research: https://docs.gptr.dev/docs/gpt-researcher/gptr/context-filter

RT Diogo Almeida SO MUCH THIS I believe that one of jev's greatest benefits to automation will come from resurrecting architecture best practices: state management, encapsulation, abstraction !!!! and combining them with ML's best practices: measure/evaluate, use calibration/uncertainty (adding screenshot b/c I don't know how to quote 2 posts 🤦)

jev and langgraph are a great combo! modeling agents as complex systems, with AI imbued in them, makes a ton of sense langgraph is the best way to model them as such, and using Jev inside langgraph to turbo charge all the small decisions that need to be made is awesome combo

RT Sydney Runkle https://x.com/i/article/2103547567237636098

RT Caspar we have seen many companies launch managed agent products recently. today, we released 𝚖𝚊𝚗𝚊𝚐𝚎𝚍-𝚍𝚎𝚎𝚙𝚊𝚐𝚎𝚗𝚝𝚜 0.8 I have been asked a few times why managed agent frameworks are useful in an era where "code is free". here is my take: generating code is easy, this is a fact. but as a developer, you still need to decide how your system should behave and apply those decisions consistently across your codebase: whose credentials does the agent use? what information can it share? what changes when another person joins the conversation? your auth, memory and channel integrations need to follow the same rules, including as you add features. this is hard to scale without a solid, opinionated foundation I do not think every team needs to solve these problems from first principles. there is value in building on decisions that other people have thought through, implemented and tested MDA is our managed service built on our open source Deep Agents harness. we build those decisions into managed primitives with sensible, configurable defaults, and handle the supporting infrastructure that is why I think Managed Deep Agents should be the default starting point for new agents. start with a foundation that we've thought through, and take on more of the system as your requirements demand it if you hit a limit with MDA, dm me. I want to understand what is missing and see if we can fix it. either way, MDA is built on our open source Deep Agents framework, so if you still need more control, you can move off MDA and use Deep Agents directly with your own integrations and infra re: diagram ~ this is one of those decisions we made: user memory is available in slack dms, but not in shared channels by default. fully configurable btw

RT LangChain Introducing LangSmith Custom Apps Create any interface from your agent data with a prompt. If you can think it, LangSmith can build it. Now GA. https://www.langchain.com/blog/langsmith-custom-apps

RT Ankush Gola Trajectories make it easier than ever to analyze long agent runs. They are now a first-class citizen within LangSmith. We also spent a lot of time ensuring our trajectory format handles things like deferred tool calling and context compaction well.

RT LangChain After lunch, here’s what’s up next at Interrupt: 🎙️How @airbnb built Agentic AI on LangGraph 🎙️Deploying mission-critical agents to production w/ @VictorMoreira16 🎙️LangSmith for Mental Health AI w/ @SpringHealth

RT Caspar 𝚖𝚊𝚗𝚊𝚐𝚎𝚍-𝚍𝚎𝚎𝚙𝚊𝚐𝚎𝚗𝚝𝚜 0.8 is here and I think it is the best thing since sliced bread things we shipped: 🧠 agent and user memory with access policies 🌐 connect webhooks with the new HTTP channel ✋ improved human-in-the-loop support 💬 Slack channels now support images and files 🔑 improved OAuth connection options 🔒 authenticate sandbox requests via a secure proxy 🔎 search the web with Parallel (thanks @p0) 📁 a runtime sandbox files API released for js and py! http://langch.in/mda

RT LangChain Introducing Trajectories in LangSmith. A view of agent behavior that everyone on the team can understand at a glance. https://www.langchain.com/blog/langsmith-trajectories-tracing

RT LangChain Introducing Managed Deep Agents 0.8 ✅ User-owned credentials + memory let agents work off correct permissions & user-specific context. ✅ HTTP channels for bringing agents into internal tools ✅ Native @slackhq file transfers ✅ Built-in web search via @p0 https://www.langchain.com/blog/langsmith-managed-deep-agents-whats-new

RT LangChain New in v2: ✅ Red Teaming: Engine identifies issues in your agent before they surface in production ✅ Validated Fixes: Engine tests its proposed fixes to confirm they work before showing them to users ✅ Improved Issue Identification: Engine now surfaces inefficient agent work and trends in key metrics like cost and latency

Proactive agents run in the background, schedules are (one way) how you achieve that!

RT LangChain You can add schedules to Managed Deep Agents. 1️⃣ Add a schedule file to schedules/ with a cron expression + a prompt 2️⃣ Deploy ...And the agent runs itself 💻 Docs: https://docs.langchain.com/langsmith/python/managed-deep-agents-schedules ⏯️ Demo from @ndrezn ⤵️

RT Josh Rosen TypeSafe is pushing hard on this idea of “composable AI” but LangChain has embodied composable AI since day one. With new models like Jev, we’re decomposing the agent loop and moving more intelligence into the glue that holds it all together. Watch this space.

RT Sydney Runkle i'm using semif (like jev, but open source) to label new issues that come into our open source repos so we have a pulse on popular features and areas where we need to spend more time current request model: * state -- issue title/body * questions -- one noul per label, we apply ones w/ p>0.8 much faster than our previous LLM classifier thus far, currently monitoring to make sure we're calibrated at the right threshold

RT Kevin Frank i was informed of this new product update on a live LangChain <> Jev webinar with 500 people. the speed at which the team operates is unreal. 📈

RT Mihir Achyuta Had a really fun time shipping this out ASAP with the team! The opportunities with Jev are going to skyrocket in the future and looking forward to it

RT Brace Jev response shapes supported natively in LangSmith now! Jev produces super simple outputs, but that doesn't mean it needs less observability

🖼️ Improved UI for Jev models in LangSmith agents are going to complex systems of many llms calls many llm calls will be jev-like (decision models) we've updated tracing view in LangSmith to display these types of calls (state, questions, choices, output) more clearly!

RT Greg Kamradt I met Caleb/Nicolas at the old langchain office hours Every since the first firecrawl tweet, I knew it was a no brainer

RT Tadeo Donegana Braunschweig Just ran some tests comparing Jev and Opus 4.8 as supervisors (routers) in a multi-agent system using @LangChain 🤯

this is cool kind of like "goal" but with multiple in a row you need to hit?

RT Milad Free SemIf this week is useful. Typed outputs + probabilities look like a cleaner way to route agents than parsing LLM text every time.

RT Jacob Lee Ah, and h/t @theoleecj for creating SemIf! Go follow him and try running it locally! https://github.com/theoleecj/semif

RT Florian S Amazing, LangSmith Gateway hosts SemIf! Well done @hwchase17

RT Jacob Lee Bottom Text

RT Viv if there’s a tool to better understand your data at scale you best believe we’re making it super easy to use it in LangSmith🫡 use Jev for large scale trace mining today, it’s so cheap that you probably don’t even need to sub-sample in LangSmith Gateway you can also try other OSS Jev variants to see what’s best for your tasks - understand every piece of trace data - turn it into evals & environments - build continuously improving agents from those evals

try out jev in langsmith today! can use it to score traces cheap and accurately