LocalLLaMA Tracks NVIDIA's gpt-oss-puzzle-88B as Puzzle Shrinks gpt-oss-120b for Cheaper Serving

A smaller model with deployment economics in mind

A March 26, 2026 post in r/LocalLLaMA pulled attention to NVIDIA's new gpt-oss-puzzle-88B model card on Hugging Face. The discussion reached 284 points and 105 comments at crawl time. According to NVIDIA, the model is derived from OpenAI's gpt-oss-120b and rebuilt with the company's Puzzle post-training neural architecture search pipeline, with the explicit goal of improving serving efficiency for reasoning-heavy workloads without giving up parent-model quality.

NVIDIA positions the result as a production deployment model rather than a research curiosity. The card says parameter count drops to about 88B, roughly 73% of the parent, while claimed throughput improves 1.63x for long-context 64K/64K serving on an 8x H100 node, 1.22x for short-context serving, and up to 2.82x on a single H100 GPU. The same card says accuracy matches or slightly exceeds the parent across reasoning-effort settings.

What Puzzle changed

The model card describes three major architectural changes. First, heterogeneous MoE expert pruning keeps more experts in earlier layers and prunes later layers more aggressively. Second, selective window attention replaces some global-attention layers with 8K window attention, which NVIDIA says reduces KV-cache footprint by about 40% in long-context runs. Third, the YaRN RoPE scaling factor is adjusted to stabilize behavior at 128K context length.

NVIDIA also details the training stack behind the release. After architecture selection, the model went through knowledge distillation on 84B tokens at 128K sequence length, followed by reinforcement learning across math, coding, and reasoning environments. The serving stack uses MXFP4 MoE weights and FP8 KV-cache scaling, and the model exposes low, medium, and high reasoning-effort modes so operators can trade cost against answer depth more predictably. The company lists vLLM and Transformers support and explicitly targets H100 and B200 deployments on Linux.

Why the LocalLLaMA audience noticed

The interesting part is not only that another open-weight reasoning model appeared. The more important signal is that the optimization target has shifted. Instead of releasing a model that is simply larger or more benchmark-heavy, NVIDIA is selling a post-training architecture search pipeline that compresses an already strong base model into something easier to serve under real KV-cache and memory constraints. Even the model card's note that Hugging Face may display about 91B parameters while NVIDIA still calls it 88B shows how deployment reality, quantization metadata, and headline parameter counts are starting to diverge.

That is why the r/LocalLLaMA thread mattered. The community is increasingly less impressed by raw parameter scale and more interested in which open models can actually deliver long-context reasoning at acceptable hardware cost. gpt-oss-puzzle-88B lands directly in that conversation.

Primary source: NVIDIA model card. Community discussion: r/LocalLLaMA.