A tiny Gemma 4 template bug gave LocalLLaMA the kind of debugging thread it loves

LocalLLaMA responds well when someone turns a hand-wavy complaint into a reproducible bug, and that is exactly what happened with the Gemma 4 tool-calling thread. The post starts from a familiar frustration: Gemma 4 was underperforming on a custom MCP tool across several inference engines, while Qwen3.5 and gpt-oss-20b were fine. Instead of stopping at "Gemma feels worse," the author dug through verbose logs, compared prompt rendering, and found a surprisingly small failure point in the chat template.

The core issue was how the Jinja template handled common JSON Schema shapes. When a tool parameter used a pattern like anyOf: [$ref, null], the useful structure lived inside anyOf and $defs, but the template expected a top-level type. As a result, the rendered prompt collapsed those parameters into empty type fields and stripped away the context the model needed to call the tool correctly. The author says a small template fix restored the missing schema information, and later widened the patch to preserve anyOf, oneOf, allOf, $defs, enum, const, type arrays, and null values.

The comments are small in number but tell you why the post mattered. People immediately asked whether this explains why Gemma 4 has been wobblier than Qwen3.6 in agent setups, whether the problem affects all Gemma 4 variants, and whether a local patch is enough to recover normal behavior. That is a more valuable discussion than another benchmark screenshot, because it converts model preference from vibes into a testable software bug. It also fits a broader pattern the subreddit keeps running into: many "model quality" complaints are really tooling, template, runtime, or formatting issues hiding in the stack below the weights.

That is why the thread traveled despite a modest score. It offered something engineers can act on right away. Check how your tool schema is rendered. Do not assume the template preserves nested JSON Schema semantics. Compare a failing model against a working one at the prompt level, not just the output level. LocalLLaMA keeps rewarding posts like this because they shrink the distance between anecdote and fix. In a community full of benchmark noise, a small, sharp bug report still stands out.