LiteRT.js is Beating TensorFlow.js

LiteRT.js with WebGPU runs a MobileNetV2 inference in 0.41 ms, delivering 2,439 FPS on an M2 Max desktop GPU—more than 25 × faster than TensorFlow.js on the same hardware. The gap widens for larger models, moving web-based machine learning into a performance tier once reserved for native apps.

Why the benchmark matters

Developers have used TensorFlow.js for on-device inference, usually via the WebGL backend. WebGL maps tensor ops onto the browser’s 2-D graphics pipeline; it works everywhere but wasn’t built for the massive parallelism deep-learning models need. WebGPU, the next-gen graphics API, gives direct access to GPU compute units. LiteRT.js is the first library to expose a WebGPU-backed inference engine for TensorFlow-Lite models, and Google reports a three-fold speedup. Independent tests show an even larger gain, raising the question of whether WebGPU will become the default path for web ML.

How the test was run

We benchmarked two image-classification models—MobileNetV2 and EfficientNet-Lite4—both converted to TensorFlow-Lite. Tests ran on an Apple-silicon M2 Max chip. For each model we measured the latency of a single forward pass over many iterations and reported frames-per-second (FPS). We ran the same models with TensorFlow.js on the WebGL backend and recorded each library’s cold-start time.

Results: speed and startup

  • MobileNetV2

    • LiteRT.js (WebGPU): 0.41 ms latency → 2,439 FPS
    • TensorFlow.js (WebGL): 10.82 ms latency → 92 FPS

  • EfficientNet-Lite4

    • LiteRT.js (WebGPU): 0.62 ms latency → 1,623 FPS
    • TensorFlow.js (WebGL): not reported, but MobileNetV2 already shows a >25× advantage.

TensorFlow.js needed about 10 seconds (10,014 ms) to compile its WebGL shaders before the first inference. LiteRT.js was ready in 4.3 ms, essentially eliminating the cold-start penalty for interactive apps.

When we increased model size five-fold, latency rose only 50 %, confirming that the GPU’s parallelism absorbs most of the extra work. The result is a performance frontier that lets a browser handle real-time video streams, AR overlays, or rapid prototyping of vision models without server round-trips.

What the numbers hide

  • Batching – Most TensorFlow-Lite models lock the batch dimension to 1. Feeding multiple images in a single call isn’t supported out of the box, forcing developers to spawn Web Workers or manually pipeline inputs.
  • Memory management – LiteRT.js does not garbage-collect GPU tensors automatically. Developers must call .delete() on each tensor they create, or risk exhausting GPU memory after a few hundred inferences.
  • Hardware reach – WebGPU is stable on desktop Chrome and Firefox. Mobile browsers—including Chrome on Android and Safari on iOS—expose the API only behind experimental flags or not at all. On those platforms the WASM backend remains the only universally available path, but it is markedly slower for larger models.

These constraints mean that while raw speed impresses, the engineering effort to achieve it can be non-trivial.

Limitations and trade-offs

TensorFlow.js’s WebGL backend still offers near-universal compatibility. A developer targeting a mixed audience—desktop, Android, iOS—can rely on a single code path that runs everywhere, albeit at lower throughput. The WASM backend runs on almost any hardware but falls behind WebGPU for the larger models measured here.

LiteRT.js shines when the target is a desktop environment with WebGPU enabled. It runs .tflite models directly, letting developers pull models from Hugging Face or Kaggle without conversion, preserving original quantization and performance. The trade-off is a narrower deployment envelope and the need for careful resource handling.

What developers should consider

  • Target platform – For a web-only desktop tool (e.g., a design app applying style transfer in real time), WebGPU with LiteRT.js is likely the best choice.
  • Model size – Larger, compute-heavy models benefit most from GPU parallelism; smaller models may not justify the extra engineering.
  • Memory discipline – Plan explicit tensor deletion or wrap inference calls in a scope that automatically frees resources.
  • Fallback strategy – Provide a WASM or WebGL fallback for browsers that cannot enable WebGPU, keeping the app functional for a broader audience.

Takeaway

LiteRT.js membuktikan bahwa WebGPU dapat mendorong inferensi berbasis web ke dalam ranah sub-milidetik, memberikan kecepatan lebih dari 25 × lipat TensorFlow.js pada GPU desktop. Teknologinya masih dalam tahap pematangan, dan pengembang harus menghadapi batasan batching, pembersihan memori secara manual, serta dukungan seluler yang terbatas. Untuk pengalaman yang mengutamakan desktop yang menuntut performa real-time, pustaka baru ini menawarkan jalan yang menjanjikan ke depan; untuk jangkauan lintas platform, backend WebGL dan WASM yang lebih lama tetap sangat penting.