Bonsai cuts a 27B model to 3.9GB for mobile inference

A 27B model moves toward the device

Large local models usually force a tradeoff between parameter count and the memory limits of phones. RunAnywhere’s Bonsai post challenges that tradeoff with a concrete claim: “1-bit weights, 27B params in just 3.9GB, ~90% of full precision quality.” The company says the Bonsai family now runs in its RunAnywhere app on iPhone, Android, and Mac, ranging from 1.7B to 27B parameters.

The technical claim rests on extreme quantization and device-specific inference paths. On iOS and Mac, RunAnywhere says Bonsai runs through llama.cpp and Apple’s MLX. On Android, it says the model can run through llama.cpp and directly on Qualcomm’s Hexagon NPU via QHexRT, its proprietary runtime. The company also says it built custom silicon kernels to make 1-bit inference on an NPU possible.

RunAnywhere describes itself as an on-device AI platform backed by Y Combinator. Its account usually posts about local inference and deployment on consumer hardware, so this tweet is both a product update and a technical stake in the broader move away from cloud-only assistants. The number that matters most is not just 3.9GB; it is the company’s claim that the 1-bit 27B model beats its 2-bit version at less than half the size.

The next check is independent measurement. A company eval can show direction, but local models often vary sharply across coding, multilingual chat, retrieval, tool use, and long-context tasks. Watch for public benchmark runs, device lists for the Qualcomm NPU path, battery and latency figures, and whether developers get a clean local API rather than only a consumer app. The source tweet is available on X.