Skip to content

Karpathy’s Opus 5 test turns a 1M-token prompt into 5,500 lines

Original: Karpathy’s Opus 5 test turns a 1M-token prompt into 5,500 lines View original →

Read in other languages: 한국어日本語
LLM Aug 3, 2026 By Insights AI (Twitter) 1 min read Source
Karpathy’s Opus 5 test turns a 1M-token prompt into 5,500 lines

A longer-form test for model capability

Evaluating an LLM by asking for a single small SVG no longer captures the frontier of model behavior. Andrej Karpathy posted on X on August 2, 2026 that he gave Opus 5 “1M token budget (~$10)” and asked it to turn the opening paragraph of a fantasy novel into a Three.js scene. The model ran for about two hours and produced 5,500 lines of procedural rendering code.

Karpathy is widely followed for practical explanations of neural networks and model behavior, so the post reads less like a demo and more like a proposed stress test. His point is that models have stamina and patience for highly custom work that a person would rarely hand-code from scratch. If that pattern holds, generated software can move from files and widgets into ephemeral worlds, simulations, and interactive scenes created on demand.

The technical caveat is the most useful part of the tweet. Karpathy says world and game generation exposes a weakness in LLMs: they cannot efficiently audit their own work because they do not natively perceive video or play inside the environments they create. In the Opus 5 run, the model had to inspect screenshots slowly at different points, which left visible rough edges. That separates raw code-generation stamina from closed-loop visual and interactive competence.

The next watch item is whether this style of long-running multimodal coding can become a reproducible benchmark. A useful test would measure not just line count or novelty, but whether the model can inspect animation, detect broken geometry, repair timing, and preserve intent over a long session. Until models can see and act inside their generated worlds, 5,500 lines of code will be impressive but uneven. The source tweet is available on X.

Share: Long

Related Articles