r/LocalLLaMA Spots Mistral 4 Landing in Transformers with 119B MoE and 256k Context

Why the Reddit thread mattered

A popular r/LocalLLaMA thread flagged a merged Hugging Face Transformers pull request before a fuller public rollout narrative had settled. The PR, #44760, adds Mistral 4 support to the library and exposes the first public-facing details in a place model watchers monitor closely: code, configs, and generated docs rather than a polished launch page.

What the upstream change actually says

The merged documentation describes Mistral 4 as a hybrid model that unifies Mistral’s instruction, reasoning, and Devstral-style developer capabilities. The `Mistral-Small-4-119B-2603` checkpoint is described as a mixture-of-experts system with 128 experts and 4 active experts, 119B total parameters, and 6.5B activated parameters per token. The docs also describe 256k context, multimodal input with text output, configurable reasoning effort, native function calling, JSON output, multilingual support, and an Apache 2.0 license.

Why developers noticed immediately

The change does more than add a model card. The PR wires `mistral4` into Transformers auto-configuration and model registries, adds dedicated config and modeling files, and extends chat-template processing with a `reasoning_effort` option. For practitioners, that means the thread was not just rumor-chasing; it pointed to concrete library support that developers can inspect, track, and prepare around.

The local-model angle

Community comments focused on where Mistral 4 might land in the open-model stack. Several users compared the size class to `gpt-oss-120B` and Qwen 122B-style deployments, while others noted the appeal of a 119B MoE model that only activates a small fraction of parameters per token. Those deployment expectations come from the Reddit discussion rather than upstream guarantees, but they explain why the discovery moved quickly through LocalLLaMA: it looks like a serious new candidate for high-end local and self-hosted workflows.

Upstream PR: Transformers PR #44760. Community thread: r/LocalLLaMA discussion.