Multi-Token Prediction Support Lands in llama.cpp

MTP Is Now in llama.cpp

PR #22673 has been merged into the llama.cpp master branch, bringing official Multi-Token Prediction (MTP) support to the most widely used local LLM inference engine. The news earned 300+ upvotes on r/LocalLLaMA as the community celebrated the milestone.

What Is MTP?

Standard autoregressive language models generate tokens one at a time in sequence. Multi-Token Prediction trains models to predict multiple future tokens in a single forward pass. DeepSeek-V3 and DeepSeek-R1 used MTP to achieve significant inference speed improvements, attracting considerable attention from the AI community.

Practical Impact

MTP is a training-time technique, so not every model benefits immediately — only models trained with MTP support will see speedups at inference time. But as newer models increasingly incorporate MTP during training, llama.cpp users will be positioned to take advantage of those gains without any additional setup. Paired with parallel generation approaches like Orthrus, local LLM inference is accelerating rapidly.

Why llama.cpp Matters

llama.cpp is the de facto standard for CPU and Apple Silicon LLM inference, used across Mac, Linux, and Windows environments by a massive community of local AI enthusiasts and developers. This merge demonstrates how quickly the open-source AI infrastructure absorbs cutting-edge research techniques.