Georgi Gerganov

RT merve llama.cpp docs now have a new home, shipped with ❤️ upcoming: speculative decoding in detail, quantization (k-quant, i-quant), coding agents what do you want to see next? 🤗 📖 http://llama.app/docs @llama_cpp

let that sink in

I find the "inception" pattern to be very useful in many agentic use cases. You can force the model to take an action when it thinks for too long by injecting a thought after a specified reasoning budget. Helps dealing with underspecified tasks which make the model reason for way too long.

RT Georgi Gerganov simple: llama serve -hf ggml-org/Qwen3.8-27B-GGUF --spec-type draft-mtp

RT Xuan-Son Nguyen We are happy to announce that Muse Glimmer is day-0 supported on llama.cpp. Meta also provides an official GGUF quant:

RT Mark Zuckerberg Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models.

RT merve if you have a mac/book and don't use llama-macos you're ngmi this beautiful piece of widget > gets your mac + your desired context window to recommend models > kicks off llama server + webui > easy switch between open models $ brew install --cask llama-app

RT Pi Hi kids, gramps here, taking over for the intern. Just released pi 0.81.0 which features first class integration with @ggerganov wonderful llama.cpp server. Professional video demo recorded in an echo-y Austrian office below. Shout out to our friends at @huggingface.

Run upgraded Gemma 4 with llama.cpp (QAT + MTP)

RT clem 🤗 We just crossed $100M annual run-rate. I know many AI companies are capturing much more $$$ these days, but still proud of the milestone! Maximizing short-term revenue has never been our priority. In fact, we're proud to manage to store and serve hundreds of petabytes of models and datasets while keeping HF free and open-source for 97% of our users. As a platform, we’re happy to hopefully create orders of magnitude more value for the community than what we capture. To me, that’s the very definition of a platform. And it has helped us build one of the most loved platform in tech, with network effects, a defensible position and a sustainable business which is quite unique in AI. Many many thanks to all the community members for building with us, we wouldn't be anywhere without you! Can’t wait for what’s next, especially as more companies start to see the value of open and local AI! Next milestone $1B?

RT Julien Chaumond Llama.cpp has a new branding + official website. Run local models today! Now more than ever, open source must win. 🙏 By @alekgrygier and @ggerganov at ggml/hf

RT Alek Grygier Check out our new logo and website on http://llama.app ;)

Some cool ggml-based work by @mudler_it recently - make sure to check it out.

RT Google Gemma Building super fast experiences with Gemma just got easier. Gemma 4 MTP is now officially merged into llama.cpp. Developers can now pair MTP with Gemma 4 QAT for a fast, lightweight setup.

Highlighting recent advances in multi-GPU and tensor parallel support in llama.cpp Over the last few months llama.cpp maintainers and engineers from NVIDIA collaborated to improve the multi-GPU performance in ggml. This resulted in significant performance gains on RTX systems and laid the groundwork for hardware-agnostic tensor parallelism in ggml. For more information on this and other advancements in the low-level inference engine of llama.cpp, check the technical blog by @NVIDIARTXSpark below

Strong signal for local AI on this year's Computex. Big players like NVIDIA and Microsoft are embracing and discussing local AI workloads. Dedicated consumer hardware and models are on the way.

These are some of my LLM assisted contributions from the past month. Nothing amazing, but I'm slowly getting better at it. Atm, using Qwen3.6 27B exclusively. For hardware - switching between M2 Ultra and RTX 5090. Both are good options, though after using the RTX and going back to the Mac, it always feels like a snail. Yet for most tasks, I feel like both hardware can do the job comfortably.

RT Georgi Gerganov llama.cpp now has an official website: https://llama.app Our goal is to make local AI accessible to everyone, and improving the user experience is a big part of that. On the new landing page you’ll find a single-line cross-platform installer. The installation provides a single unified `llama` entrypoint which you can use to run/serve models and interface with 3rd-party agentic applications. While oriented towards simplified user experience, the new `llama` application also provides all the advanced functionality of the existing llama.cpp tooling with which experienced users are already familiar. Also note that all GGUF models that you might have already downloaded with llama.cpp in the past will be automatically available to use without downloading again (they are stored in the common HF cache on your machine). We have many improvements in the pipeline both at the UX and at the engine level and we plan to iteratively ship new things over the coming months. One of the main focuses will be seamless integration with local-friendly 3rd-party agents (such as Pi). In the meantime, we’ll continue to listen for feedback from the community and adjust accordingly, so keep letting us know what you think and need.

RT clem 🤗 llama.cpp with MTP support makes local models fast enough to use as daily drivers 🚀 Qwen3.6-27B dense generation below on A10G: From 25 tok/st to 45 tok/s (+78%)!

RT Georgi Gerganov Highlighting the new WebGPU backend in llama.cpp/ggml The work to bring full-fledged WebGPU support in llama.cpp started about an year and a half ago. It has been lead by @reeselevine and team at USCS. For more information, checkout the interactive blog and paper in the quoted post. Here are 2 excerpts from the paper, summarizing the implemented software architecture.

Highlighting the new WebGPU backend in llama.cpp/ggml The work to bring full-fledged WebGPU support in llama.cpp started about an year and a half ago. It has been lead by @reeselevine and team at USCS. For more information, checkout the interactive blog and paper in the quoted post. Here are 2 excerpts from the paper, summarizing the implemented software architecture.

RT Reese Levine Re We have an arxiv paper up describing the work in more detail here: https://arxiv.org/abs/2605.20706. Also want to call out that there is even more room for improvement, some recent updates to wllama by @ngxson mean it's even more memory efficient than what we describe in the paper!

RT Julien Chaumond What hardware actually powers open-source AI? Not benchmarks. Not vendor marketing. Real-world community usage. We’re launching @huggingface Hardware: → trending GPUs & CPUs → VRAM distribution → inference hardware trends → what the OSS AI ecosystem really runs on

llama.cpp adds MTP for the Qwen3.6 family This is a significant milestone for the local AI ecosystem. The performance jump with these changes is massive and elevates local inference on commodity hardware further. Special thanks to Aman Gupta for leading this development! https://github.com/ggml-org/llama.cpp/pull/22673