On April 4, 2026, a Hacker News thread climbed to 540 points and 164 comments by highlighting Apple researchers' new arXiv paper on simple self-distillation for code generation. The appeal is obvious: the paper asks whether a large language model can improve its coding ability using only its own raw outputs, without a verifier, a teacher model, or a reinforcement-learning loop.

The proposed answer is simple self-distillation, or SSD. The model samples candidate solutions using particular temperature and truncation settings, and those self-generated samples are fed back into standard supervised fine-tuning. In the paper, Qwen3-30B-Instruct improves from 42.4% to 55.3% pass@1 on LiveCodeBench v6. The authors also report that gains are strongest on harder problems and that the pattern generalizes across Qwen and Llama families at 4B, 8B, and 30B scale, including both instruct and thinking variants.

Their explanation is a precision-versus-exploration conflict inside decoding. Some code positions demand very sharp token choices because syntax and semantics leave little room for error, while other positions are real branch points where several solution paths can work. SSD is presented as a way to reshape token distributions in context: suppress distractor tails where precision matters, while preserving enough diversity where exploration helps.

That framing explains why the Hacker News audience found the result compelling. One early commenter described the effect as a kind of context-aware decoding rather than a giant new training stack. If that interpretation holds up, SSD would be attractive precisely because smaller teams can try it without frontier-scale infrastructure. The open questions are practical rather than philosophical: how expensive is it to generate the self-distilled traces, how well does the gain transfer to real coding agents, and how benchmark-specific is the result? Even with those caveats, this is the kind of lightweight post-training idea that people will copy quickly.

No verifier model, teacher model, or RL loop is required in the core recipe.
The headline result is a pass@1 jump from 42.4% to 55.3% on LiveCodeBench v6 for Qwen3-30B-Instruct.
The paper argues that the benefit comes from improving precision where code is brittle while preserving exploration where multiple solution paths exist.

#self-distillation

HN thread spotlights a simple self-distillation recipe for stronger code generation