LiteRT.js is Beating TensorFlow.js

LiteRT.js with WebGPU runs a MobileNetV2 inference in 0.41 ms, delivering 2,439 FPS on an M2 Max desktop GPU—more than 25 × faster than TensorFlow.js on the same hardware. The gap widens for larger models, moving web-based machine learning into a performance tier once reserved for native apps.

Why the benchmark matters

Developers have used TensorFlow.js for on-device inference, usually via the WebGL backend. WebGL maps tensor ops onto the browser’s 2-D graphics pipeline; it works everywhere but wasn’t built for the massive parallelism deep-learning models need. WebGPU, the next-gen graphics API, gives direct access to GPU compute units. LiteRT.js is the first library to expose a WebGPU-backed inference engine for TensorFlow-Lite models, and Google reports a three-fold speedup. Independent tests show an even larger gain, raising the question of whether WebGPU will become the default path for web ML.

How the test was run

We benchmarked two image-classification models—MobileNetV2 and EfficientNet-Lite4—both converted to TensorFlow-Lite. Tests ran on an Apple-silicon M2 Max chip. For each model we measured the latency of a single forward pass over many iterations and reported frames-per-second (FPS). We ran the same models with TensorFlow.js on the WebGL backend and recorded each library’s cold-start time.

Results: speed and startup

  • MobileNetV2

    • LiteRT.js (WebGPU): 0.41 ms latency → 2,439 FPS
    • TensorFlow.js (WebGL): 10.82 ms latency → 92 FPS

  • EfficientNet-Lite4

    • LiteRT.js (WebGPU): 0.62 ms latency → 1,623 FPS
    • TensorFlow.js (WebGL): not reported, but MobileNetV2 already shows a >25× advantage.

TensorFlow.js needed about 10 seconds (10,014 ms) to compile its WebGL shaders before the first inference. LiteRT.js was ready in 4.3 ms, essentially eliminating the cold-start penalty for interactive apps.

When we increased model size five-fold, latency rose only 50 %, confirming that the GPU’s parallelism absorbs most of the extra work. The result is a performance frontier that lets a browser handle real-time video streams, AR overlays, or rapid prototyping of vision models without server round-trips.

What the numbers hide

  • Batching – Most TensorFlow-Lite models lock the batch dimension to 1. Feeding multiple images in a single call isn’t supported out of the box, forcing developers to spawn Web Workers or manually pipeline inputs.
  • Memory management – LiteRT.js does not garbage-collect GPU tensors automatically. Developers must call .delete() on each tensor they create, or risk exhausting GPU memory after a few hundred inferences.
  • Hardware reach – WebGPU is stable on desktop Chrome and Firefox. Mobile browsers—including Chrome on Android and Safari on iOS—expose the API only behind experimental flags or not at all. On those platforms the WASM backend remains the only universally available path, but it is markedly slower for larger models.

These constraints mean that while raw speed impresses, the engineering effort to achieve it can be non-trivial.

Limitations and trade-offs

TensorFlow.js’s WebGL backend still offers near-universal compatibility. A developer targeting a mixed audience—desktop, Android, iOS—can rely on a single code path that runs everywhere, albeit at lower throughput. The WASM backend runs on almost any hardware but falls behind WebGPU for the larger models measured here.

LiteRT.js shines when the target is a desktop environment with WebGPU enabled. It runs .tflite models directly, letting developers pull models from Hugging Face or Kaggle without conversion, preserving original quantization and performance. The trade-off is a narrower deployment envelope and the need for careful resource handling.

What developers should consider

  • Target platform – For a web-only desktop tool (e.g., a design app applying style transfer in real time), WebGPU with LiteRT.js is likely the best choice.
  • Model size – Larger, compute-heavy models benefit most from GPU parallelism; smaller models may not justify the extra engineering.
  • Memory discipline – Plan explicit tensor deletion or wrap inference calls in a scope that automatically frees resources.
  • Fallback strategy – Provide a WASM or WebGL fallback for browsers that cannot enable WebGPU, keeping the app functional for a broader audience.

Takeaway

LiteRT.js demuestra que WebGPU puede llevar la inferencia basada en la web al régimen de sub-milisegundos, ofreciendo más de 25 × la velocidad de TensorFlow.js en GPUs de escritorio. La tecnología aún está madurando, y los desarrolladores deben lidiar con los límites de procesamiento por lotes, la limpieza manual de memoria y el soporte limitado en dispositivos móviles. Para experiencias centradas en escritorio que exigen un rendimiento en tiempo real, la nueva biblioteca ofrece un camino prometedor; para un alcance multiplataforma, los backends de WebGL y WASM más antiguos siguen siendo esenciales.