GuppyLM Turns LLM Training into a Readable 8.7M-Parameter Show HN Project

A recent Show HN post highlighted GuppyLM, a deliberately tiny language model built to make LLM training feel understandable instead of mystical. The repo frames the project as an education-first exercise: one Colab notebook, a short PyTorch codebase, and a full path from synthetic data to tokenizer, weights, and browser inference.

The model itself is intentionally simple. GuppyLM uses a vanilla transformer with 8.7M parameters, six layers, a 384-dimensional hidden size, six attention heads, a 4,096-token BPE vocabulary, and a 128-token context window. The author says it was trained from scratch on 60,000 synthetic conversations across 60 topics, all shaped around a fish persona that talks about water, food, light, and tank life.

What makes the project useful is not raw capability but observability. The README explains why advanced tricks were left out: no GQA, no RoPE, no SwiGLU, and no early exit, because the point is to show the core transformer loop as directly as possible. The repository also includes data generation, tokenizer prep, training, inference, ONNX export, and a browser demo that runs a quantized model locally through WebAssembly.

Why the HN community noticed it

Educational LLM projects often stop at slides or notebooks, but this one is packaged so readers can inspect every step and then immediately try the model in Colab or in the browser. That lowers the barrier for developers who want to understand tokenization, context limits, and small-model behavior before moving on to much larger systems.

• Training target: a single T4 GPU in roughly five minutes.
• Deployment target: local browser inference with an approximately 10 MB quantized ONNX model.
• Main tradeoff: a narrow persona and short context window in exchange for transparency and reproducibility.

GuppyLM is not positioned as a practical assistant, and the author is explicit about that. Its value is that it turns the modern LLM stack into something small enough to read, run, and modify in an afternoon. For people coming from application work rather than ML research, that is the real story behind this Show HN post.