Running an LLM Agent in Your Browser

Most AI assistants sit in a data center. They need an API key, a server, and they charge per request.

I built something different.

I fine-tuned LiquidAI LFM2.5 models (230 M and 350 M) to run entirely in your browser—no server, no cloud costs.

The goal was to make a model small enough to ship with a web page.

The model doesn’t store facts about specific products; it learns patterns. One tiny model can run a coffee shop, a grocery store, or any other shop without retraining.

What the model does

  • Picks the right tool for a task
  • Connects arguments and item IDs correctly
  • Understands references like “the second one” or “a dozen”
  • Uses retrieved text to answer questions
  • Refuses when information is missing
  • Steers the conversation back when it goes off-topic

How I built it

  • Frozen a set of eight tools (search, add_to_cart, checkout, etc.)
  • Treated RAG as a tool instead of a separate pipeline
  • Applied grammar-constrained decoding for accuracy
  • Created a synthetic dataset from 18 interaction recipes
  • Fine-tuned on 30 M tokens using a single 16 GB GPU

Training focus I trained the model on behaviors, not blocklists. Rather than ban words, I taught it to steer conversations politely.

Why this matters

  • Privacy: your data never leaves your device
  • Offline use: it works without an internet connection
  • Cost: zero inference bills
  • Accessibility: it makes complex UIs navigable by voice

The model is small, but it is useful. Stop asking if giant models can do tasks. Ask how small a model can be while remaining helpful on a device.

Demo: https://lajosbencz.github.io/frontend-agent/ Code: https://github.com/lajosbencz/frontend-agent/ Source: https://dev.to/lajosbencz/running-an-llm-agent-entirely-in-your-browser-5foe

Lajos Bencz fine-tuned a 350-million-parameter language model to run entirely in a web browser, turning a static page into an autonomous shopping assistant that needs no API key, no server, and no cloud-based inference costs. The result is a privacy-first, offline-capable AI that can manage a coffee shop, a grocery store or any similar catalog without being retrained for each domain.

Why run an LLM in the browser

Most AI assistants live in data centers. Each query travels over the internet, hits an API, and incurs a per-request fee. That setup leaks user data, requires a network, and adds up quickly on a bill. Moving the model to the client eliminates those problems. The data stays on the local machine, the assistant works when the network drops, and inference costs drop to zero.

How the model was built

  • Model choice – Started with LiquidAI LFM2.5, 230 M and 350 M variants. Their size lets a typical web page download them.
  • Tool set – Hard-coded eight utilities (search, add_to_cart, checkout, etc.) into the agent. The model learns which tool to invoke and how to pass arguments like item IDs.
  • RAG as a tool – Treated Retrieval-Augmented Generation as just another callable tool, simplifying the architecture.
  • Grammar-constrained decoding – Restricted the decoder to valid tool-call syntax, slashing malformed outputs.
  • Synthetic data – Converted 18 interaction recipes (e.g., “add two dozen donuts”) into a 30 M-token dataset generated on a single 16 GB GPU.
  • Behavior-focused training – Instead of blocklists, fine-tuning emphasized conversational steering: politely redirecting off-topic chatter, refusing when information is missing, and confirming ambiguous references like “the second one”.

The fine-tuning ran on a single 16 GB GPU.

What the agent can do

  • Tool selection – Decides whether a query needs search, a cart update, or another function.
  • Argument handling – Parses item identifiers, quantities and qualifiers (e.g., “a dozen”) and passes them correctly.
  • Reference resolution – Links phrases such as “the second one” to the appropriate item in the retrieved list.
  • RAG-driven answers – Fetches relevant text and weaves it into replies.
  • Graceful refusal – Declines when required information is absent instead of hallucinating.
  • Conversation steering – Nudges off-topic remarks back toward the task without breaking flow.

A live demo (https://lajosbencz.github.io/frontend-agent/) shows the agent handling a simple e-commerce flow, from browsing to checkout, with voice or text input.

Limitations and counter-points

(section left intentionally blank)

पुढे काय पाहावे

हा codebase (https://github.com/lajosbencz/frontend-agent) खुला आहे, जो समुदायाला नवीन tools जोडण्यासाठी, synthetic recipes विस्तारण्यासाठी किंवा hybrid approaches वापरून पाहण्यासाठी आमंत्रित करतो.

मुख्य निष्कर्ष: मध्यम आकाराच्या LLM ला self-contained agent मध्ये fine-tune केल्यामुळे, वेब पेजमध्ये थेट functional आणि privacy-preserving AI समाविष्ट करणे शक्य होते—ज्यामध्ये कोणतेही servers, कोणतेही fees लागत नाहीत आणि वापरकर्त्याच्या device मधून कोणताही डेटा बाहेर जात नाही.