Qwen 3.6 27B Achieves 2.5x Faster Local Inference via MTP With 262k Context on 48GB
Original: 2.5x faster inference with Qwen 3.6 27B using MTP - Finally a viable option for local agentic coding - 262k context on 48GB - Fixed chat template - Drop-in OpenAI and Anthropic API endpoints View original →
MTP Comes to Qwen 3.6 27B
A new post on r/LocalLLaMA has detailed how to achieve 2.5x faster inference with Qwen 3.6 27B using a new MTP support PR for llama.cpp. The guide, tested on an M2 Max 96GB, earned over 600 upvotes from the community.
Key Capabilities
Beyond the 2.5x speed improvement via speculative decoding, this configuration enables 262,000-token context windows on 48GB of memory. It includes a fixed chat template, drop-in compatibility with OpenAI and Anthropic API endpoints, and q4_0 KV cache compression.
Finally Viable for Local Agentic Coding
The author describes this as "finally a viable option for local agentic coding." The combination of long context and fast inference makes Qwen 3.6 27B a practical local alternative to cloud APIs for agentic workflows like Claude Code or Cursor.
Caveats
The relevant llama.cpp PR remains unstable with ongoing discussions. The author revised their original recommendations after discovering build instability, replacing turbo quants with standard q4_0 KV cache compression. Wait for the upload confirmation before downloading from Hugging Face.
Related Articles
A few weeks after release, r/LocalLLaMA is converging on task-specific sampler and reasoning-budget presets for Qwen3.5 rather than one default setup.
A well-received LocalLLaMA post spotlighted a llama.cpp experiment that prefetches weights while layers are offloaded to CPU memory, aiming to recover prompt-processing speed for dense and smaller MoE models at longer contexts.
A recent r/LocalLLaMA post presents Qwen3.5 27B as an unusually strong local inference sweet spot. The author reports about 19.7 tokens per second on an RTX A6000 48GB with llama.cpp and a 32K context, while the comments turn into a detailed debate about dense-versus-MoE VRAM economics.