Qwen 3.5 Small Released: A New Benchmark for Local AI

Qwen 3.5 Small Drops

Alibaba's Qwen team has released Qwen 3.5 Small, the latest addition to the Qwen 3.5 series. The announcement reached 1,047 upvotes on r/LocalLLaMA, making it the day's top post — a strong signal of how much the local AI community has been anticipating capable small dense models.

Community Reactions

Key highlights from community responses:

• Speculation that a 2B model could serve as a draft model for 122B in speculative decoding setups — significant for users with limited VRAM who want faster inference
• "Qwen is killing it this gen with model size selection. They got a size for everyone" — reflecting Alibaba's strategy of releasing models at multiple scales
• Excitement that the model can run on modest consumer hardware, extending access to high-quality local inference

Context in the Qwen 3.5 Ecosystem

The same day, r/LocalLLaMA also saw reports of Qwen 3.5 27B dense running at 100+ tokens/second decode speed with 170k context on 2x RTX 3090 GPUs using vLLM with tensor parallelism. The Qwen 3.5 family is rapidly becoming the go-to open-source series for local AI inference, offering something for everyone from high-end multi-GPU setups down to entry-level consumer hardware.

Why This Matters

As small dense models improve, high-quality inference becomes accessible on lower-end hardware. Qwen 3.5 Small gives users who want privacy-first, on-device AI a compelling new option — continuing Alibaba's Qwen team's momentum as one of the most prolific and capable open-source AI labs.