Llama.cpp Multi-Token Prediction Support Enters Beta, Closing the vLLM Performance Gap

What Is MTP

Multi-Token Prediction (MTP) enables a model to predict multiple tokens per inference step rather than one at a time, significantly boosting generation throughput. Server-side inference frameworks like vLLM already support MTP, giving them a speed edge over llama.cpp in high-throughput scenarios — until now.

Beta Status

A post scoring 277 on r/LocalLLaMA announced that llama.cpp's MTP implementation has entered beta, thanks to contributor Aman and the broader community. Current support is limited to Qwen3.5 MTP, with other model families expected to follow.

The developer noted: "Between this and the maturing tensor-parallel support, expect most performance gaps between llama.cpp and vLLM, at least when it comes to token generation speeds, to be erased."

Impact for Local Inference

llama.cpp is the de facto standard for running LLMs on consumer hardware. Once MTP stabilizes, local inference speeds for models like Qwen3 and Llama 4 should approach server-grade performance, removing one of the last meaningful advantages of cloud-hosted inference for many workloads. The pull request is in review and expected to merge to main shortly.