MCP Server Caching: What I Learned After 87 Outages

I thought my MCP server did not need caching.

My knowledge base had only a few thousand entries. Queries were fast. I was wrong.

After 87 production outages and three days of debugging timeouts, I learned a hard lesson. Every MCP server needs caching. Even small ones.

Here is what happened.

The server used semantic search to find notes. Most of the time, it worked. But then, things broke.

  • Connections dropped mid-request.
  • Claude Desktop timed out after 60 seconds.
  • Nginx returned 504 Gateway Timeouts.

The worst part? It only happened on repeat queries.

When a user asks the same question twice, the connection might idle while the AI thinks. A proxy then drops the connection. Claude retries the request automatically. Now, you have two identical searches running at once.

When multiple retries happen, your database connection pool exhausts. Everything dies.

I realized I should not cache the final AI response. That is too complex for MCP. Instead, I had to cache the heavy part: the semantic search and content fetching.

I moved to a two-level caching strategy:

• Level 1: In-memory cache for frequent queries. This is extremely fast. • Level 2: Redis cache for shared data across restarts.

I also followed these rules to make it work:

  • Normalize queries: I convert queries to lowercase and remove extra spaces. This increases cache hits.
  • Cache results, not streams: I only cache the search results. The search takes 95% of the time.
  • Graceful degradation: If Redis goes down, the server still works. It just goes directly to the database. Caching is an optimization, not a requirement for the request to succeed.

The results were massive.

My average search time went from 320ms to 12ms. That is 27x faster. My total cache hit rate is 73%. Most queries do not hit my database at all.

If you build an MCP server, do this:

  • For personal use: Use an in-memory cache. It stops random timeouts.
  • For public services: Use the two-level approach with Redis. It prevents crashes and improves speed.

Caching is about reliability. It stops the cycle of retries and connection failures.

Source: https://dev.to/kevinten10/mcp-server-caching-what-i-learned-adding-caching-to-my-mcp-knowledge-base-after-87-production-261b

Optional learning community: https://t.me/GyaanSetuAi