How to Run Qwen3.5 27B with 170k Context at 100+ t/s on 2x RTX 3090

Overview

A LocalLLaMA community member shared an impressive benchmark running Qwen3.5 27B dense on consumer hardware: 100+ t/s decode speed with 170k context window and approximately 1,500 t/s prefill, achieved on a dual RTX 3090 system with NVLink.

Hardware Setup

The configuration uses two RTX 3090 GPUs connected via NVLink. The developer notes that NVLink plays a significant role in tensor parallelism performance by providing high-bandwidth GPU interconnect, unlike PCIe-only setups.

Software Optimizations

Key optimizations that achieved these results:

• vLLM with tensor parallelism enabled
• MTP (Multi-Token Prediction) set to 5 predicted tokens (higher than the documented recommendation of 3)
• Mean acceptance length consistently above 3, validating the higher MTP setting

The developer found values above 5 offered diminishing returns, making 5 the optimal setting for this hardware.

Real-World Performance

Even in worst-case scenarios involving complex reasoning tasks, decode speed rarely drops below 60 t/s. For multi-user workloads, 585 t/s aggregate throughput across 8 simultaneous requests was observed — sufficient for a production serving environment on consumer hardware.

Significance

This demonstration shows that production-grade LLM serving of a 27B dense model is achievable on dual consumer GPUs without cloud infrastructure. The practical guide offers developers a concrete reference for building cost-efficient local AI deployments.