Follow AI Builders — Not Influencers
Sharing new post-training research on self-distillation + RL (JOLT), done together with @NVIDIAAI. In self-distillation, the same model performs the role of both a teacher and a student. The teacher gets extra information that the student doesn't, and the student learns from feedback provided by the teacher. However, the teacher could still use shortcuts in ways the student has no way to follow. Our proposed approach, JOLT, trains the teacher as well, to learn from rewards on its own attempts, while penalizing any drift that's far off from what the student model would do. JOLT+ further supplements JOLT by providing the student model outcome-based rewards in addition to the teacher supervision signals. On the MATH benchmark, we found that JOLT matches GRPO with 6.5x fewer training tokens, while JOLT+ convincingly beats GRPO across math, coding, terminal tasks, and tool use benchmarks.
New research: We developed JOLT, a post-training method where one model acts as both teacher and student, with the teacher trained to give feedback the student can use. On MATH, it matched GRPO's accuracy with 6.5x fewer training tokens. https://www.perplexity.ai/hub/blog/a-good-teacher-meets-the-student-where-they-are
RT Murtaza Khomusi Benchmarks so good that even Jensen wants to holla’
Congrats to @nvidia for creating the best open-weight model on document parsing so far. Nemotron 3.5 Parse takes the #1 spot on ParseBench, our comprehensive real-world benchmark for document understanding. The full list consists of 150+ frontier models, open-weight models, and
RT Ryan Chesler Super proud of this release, please try this model out
Congrats to @nvidia for creating the best open-weight model on document parsing so far. Nemotron 3.5 Parse takes the #1 spot on ParseBench, our comprehensive real-world benchmark for document understanding. The full list consists of 150+ frontier models, open-weight models, and
RT simon it's a good benchmark sir
Congrats to @nvidia for creating the best open-weight model on document parsing so far. Nemotron 3.5 Parse takes the #1 spot on ParseBench, our comprehensive real-world benchmark for document understanding. The full list consists of 150+ frontier models, open-weight models, and
Congrats to @nvidia for creating the best open-weight model on document parsing so far. Nemotron 3.5 Parse takes the #1 spot on ParseBench, our comprehensive real-world benchmark for document understanding. The full list consists of 150+ frontier models, open-weight models, and specialized solutions. It does this while remaining sub-1B parameters so it's also blazing fast ⚡️ Big thanks to @JensenHuang for the shoutout at GTC Berlin! Nemotron release: https://huggingface.co/blog/nvidia/nemotron-retriever-world-class-agentic-retrieval ParseBench: https://www.parsebench.ai/