Developers using local large-language models (LLMs) discover that a single Multi-Channel-Protocol (MCP) server can eat the entire context window before a user even types a prompt. They must choose between crippled tool descriptions or a broken conversation flow.

Why token bloat matters for local LLMs

MCP lets an LLM call external tools—APIs, scripts, or file-system utilities—by feeding the model a description of each tool. Cloud-hosted models with 128 k-token windows can absorb many tool definitions and still leave room for user dialogue. A 7-billion-parameter model running locally with an 8 k-token window runs out of space after loading just a few tools. The trade-off is stark: short, cheap descriptions mis-route calls; long, detailed descriptions consume the budget needed for the chat.

The chain of events that led here

MCP was built to replace custom integration code with a single, model-driven interface to many data sources. Most MCP servers act as thin wrappers around REST endpoints meant for human operators, not machines. When those wrappers drop into a local LLM session, the model must read every tool’s name, parameters, and usage notes before it can decide which one to invoke. Tiny context windows turn this “description overhead” into a structural bottleneck.

Who wins, who loses

  • Developers building on-device assistants lose flexibility. They either prune tool catalogs, risking frequent failures, or accept a bloated prompt that truncates user input.
  • End users see flaky behavior when the assistant selects the wrong tool or refuses to act because the context is full.
  • Tool providers gain a uniform entry point.

The cost isn’t just a poorer experience; it raises security concerns. When an MCP agent can read any local file, the permission model collapses to “all-or-nothing.” Without a sandbox, a mis-configured tool can expose the whole filesystem.

What developers are doing about it

Three work-arounds dominate the community:

  • Trim descriptions – Strip tool metadata to the bare minimum. This frees tokens but increases the chance the model picks the wrong endpoint, leading to errors that developers must catch and retry.
  • Dynamic loading – Load only the subset of tools relevant to the current conversation. A lightweight dispatcher decides, based on the user’s intent, which tool set to inject. This cuts idle token usage but adds latency and code complexity.
  • Limit active servers – Cap the number of MCP servers per session, forcing developers to prioritize the most essential integrations. This keeps the prompt size manageable but sacrifices breadth of capability.

None of these solutions is a silver bullet. Stripping descriptions hurts reliability; dynamic loading adds a decision layer that slows responses; limiting servers forces hard choices about which data sources to support.

Security risks that ride on the token problem

Local agents often run with unrestricted file-system access. The MCP protocol offers no granularity between “read this folder” and “read everything.” Some teams have built gateway layers to fix the full-access issue, adding more complexity. Those gateways mitigate the “full-control” problem but also increase the code base.

Designing tools for small models

Large cloud models can recover from bad descriptions, so developers sometimes overlook the need for precise tool definitions. For local models, follow these principles:

  • Narrow functionality – Each tool should do one thing. A “search” tool that also writes files will confuse a model that can’t track overlapping responsibilities.
  • Unambiguous naming – Avoid generic names like “process” or “handle.” Names should convey the exact operation, reducing the model’s mental load.
  • Clear, concise descriptions – Include only the parameters the model truly needs to decide. Use a consistent format so the model can recognize patterns quickly.

Counter-point: the protocol still has value

Despite the friction, MCP remains attractive because it abstracts away boilerplate code. A single, model-driven interface can connect to dozens of services without writing custom adapters for each. Teams that can afford cloud-scale models see token bloat as a non-issue, and the convenience outweighs the overhead. The challenge is translating that convenience to the constrained world of on-device LLMs.

Takeaway

Jika Anda sedang membangun asisten on-device, perlakukan deskripsi alat MCP sebagai sumber daya yang langka. Pangkas, muat secara dinamis, dan rancang alat dengan cakupan yang sempit untuk menjaga agar jendela konteks tetap tersedia bagi percakapan yang sebenarnya. Di saat yang sama, waspadalah terhadap model keamanan "akses penuh" yang implisit dengan menyisipkan lapisan izin, meskipun itu memakan beberapa token tambahan. Keseimbangan yang Anda ciptakan akan menentukan apakah LLM lokal Anda terasa seperti pendamping yang membantu atau chatbot yang rusak.