Article: PrismML’s new Bonsai 27B 1-bit model fits into a 3.9 GB file and runs at about 11 tokens per second on an iPhone 17 Pro Max, proof that a 27-billion-parameter language model can live on a consumer smartphone. The claim matters because it pushes the boundary of on-device AI, promising richer offline assistants without sending data to the cloud.

Why the reduction matters

The full-precision Bonsai 27B weighs 54.7 GB. By quantising the model to a single-bit representation, PrismML shrank it to 3.9 GB—a 14-fold size cut. That compression makes the model technically storable on high-end iPhones, a threshold that previously excluded anything beyond a few hundred megabytes.

How the model performs on Apple hardware

PrismML built the model for Apple’s MLX stack, the set of APIs that let developers run neural nets directly on the neural engine. In its own testing the model generated roughly 11 tokens each second on an iPhone 17 Pro Max. Across 15 standard language-model benchmarks the 1-bit version retained 89.5 % of the original model’s quality scores, suggesting the compression has not crippled its usefulness.

Practical limits you’ll hit

  • Memory pressure – The 3.9 GB file fits, but the model also needs a key-value (KV) cache to store recent context. That cache grows with the length of a conversation and can push the phone’s RAM usage higher, potentially throttling speed.
  • Heat and battery – Running a 27-billion-parameter network taxes the neural engine. Phones tend to heat up under sustained load, and Apple’s thermal management will throttle performance to protect hardware. PrismML has not released concrete battery-drain figures, so the real-world cost to a day-long usage pattern remains unknown.
  • Device specificity – The performance claim applies to the latest iPhone 17 Pro Max. Older iPhones, lower-tier iOS devices, or Android phones lack the same neural-engine capabilities and will see slower inference or may not fit the model at all.

Who stands to gain, and who might be left behind

Developers of privacy-first apps can now host a large language model on-device, keeping user data on the phone. That could mean more responsive voice assistants, offline translation, or contextual text generation without a cloud subscription.

Android manufacturers and users of mid-range phones miss out. The model relies on Apple’s proprietary stack, so the same compression won’t automatically work on other ecosystems.

What remains to be tested

PrismML’s numbers come from internal benchmarks. Independent testers need to verify token-per-second rates over extended sessions, measure actual battery consumption, and assess how quickly performance degrades as the KV cache expands. Real-world usage—chatting for an hour, streaming content, or running the model alongside other apps—will reveal whether the 11-token-per-second figure holds up outside a controlled lab.

Takeaway

Bonsai 27B shows that 27-billion-parameter language models can be compressed enough to live on a flagship iPhone, delivering near-full quality at a modest speed. The breakthrough hinges on Apple’s MLX stack and still faces practical hurdles: memory overhead, thermal throttling, and unknown battery impact. If follow-up tests confirm the claims, on-device AI could move from niche demos to everyday apps—provided the hardware gap between iOS flagships and the rest of the market narrows.