Follow AI Builders — Not Influencers

There are thousands of OCR tools out there, and ParseBench-2 represents our attempt to not only build a standardized benchmark, but a framework for understanding which tools do better on which dimensions. On day 1, we’ve benchmarked 40 different tools across 6 different dimensions: * specialized OCR tools * frontier models * open-weight solutions We’ve aggressively tried to improve the metrics from ParseBench-1 to reflect real penalties while remaining quantitative. *** For what it’s worth, I would take the overall score with a grain of salt, since it’s a simple composite average. A lot of the insights/nuances in the benchmark come from analyzing each individual dimension and understanding the tagging behind each failure case. It’s fascinating to see where models and other vendors do well - we don’t top the charts in every dimension relative to price point! ✅ frontier models are hillclimbing certain dimensions while ignoring others. e.g. opus 5.5 is quite good at tables, while bad at charts ✅ specialized OCR providers do well on various dimensions, e.g. Extend on forms, Datalab on tables, (we also fixed reducto r-1 chart evals) There has been some feedback on annotations and metrics that we’re continuing to take into account, and will create new versions soon. Also reminder that if you care about certain errors more than others, you're welcome to customize the severities for the use cases you care about. We’re going to be adding a ton more models to the leaderboard, come check it out: https://www.parsebench.ai/ Hugging Face: https://huggingface.co/datasets/llamaindex/ParseBench-2

RT Tim Baer Re @hnshah This is absolutely true. The best I've worked with do both. Trying and failing during the day and ingesting relevant content off hours to get ideas for the next thing to try the next day.

We have doubled our compute capacity as of today. 🚀