Karpathy’s Opus 5 test turns a 1M-token prompt into 5,500 lines
Original: Karpathy’s Opus 5 test turns a 1M-token prompt into 5,500 lines View original →
A longer-form test for model capability
Evaluating an LLM by asking for a single small SVG no longer captures the frontier of model behavior. Andrej Karpathy posted on X on August 2, 2026 that he gave Opus 5 “1M token budget (~$10)” and asked it to turn the opening paragraph of a fantasy novel into a Three.js scene. The model ran for about two hours and produced 5,500 lines of procedural rendering code.
Karpathy is widely followed for practical explanations of neural networks and model behavior, so the post reads less like a demo and more like a proposed stress test. His point is that models have stamina and patience for highly custom work that a person would rarely hand-code from scratch. If that pattern holds, generated software can move from files and widgets into ephemeral worlds, simulations, and interactive scenes created on demand.
The technical caveat is the most useful part of the tweet. Karpathy says world and game generation exposes a weakness in LLMs: they cannot efficiently audit their own work because they do not natively perceive video or play inside the environments they create. In the Opus 5 run, the model had to inspect screenshots slowly at different points, which left visible rough edges. That separates raw code-generation stamina from closed-loop visual and interactive competence.
The next watch item is whether this style of long-running multimodal coding can become a reproducible benchmark. A useful test would measure not just line count or novelty, but whether the model can inspect animation, detect broken geometry, repair timing, and preserve intent over a long session. Until models can see and act inside their generated worlds, 5,500 lines of code will be impressive but uneven. The source tweet is available on X.
Related Articles
A 2.4T MoE model with promised open weights raises pressure on closed coding systems. Qwen lists Qwen3.8-Max at $2 input and $6 output per million tokens, with a 1M context window.
A LocalLLaMA discussion of SWE-rebench January runs reports close top-tier results, with Claude Code leading pass@1 and pass@5 while open models narrow the gap.
A LocalLLaMA post reports that a simple “verify after every edit” loop raised Qwen3.5-35B-A3B from 22.2% to 37.8% on SWE-bench Verified Hard, approaching a cited 40% reference for Claude Opus 4.6.