The Cache Was Working, But It Still Caused Duplicate API Calls

The cache was not broken.

Yet, three concurrent requests for the same username hit GitHub three times.

This happened in CommitPulse, a Next.js API that turns GitHub data into SVG badges. When a README goes viral, thousands of people view the badge at once. This creates massive traffic.

The cache worked for sequential requests. It failed for concurrent ones.

Here is the problem:

  • Request A checks the cache. It is a miss. Request A starts fetching from GitHub.
  • Request B arrives 5ms later. It checks the cache. It is still a miss because Request A is not finished. Request B starts a second fetch.
  • Request C arrives 10ms later. It also sees a cache miss and starts a third fetch.

This is the thundering herd problem. A cache miss under high load triggers a flood of identical calls to your upstream provider. If you use a rate-limited API like GitHub, this can exhaust your limits instantly.

The solution is request coalescing.

You must track pending requests separately from completed cache entries. I implemented an in-flight Map to manage this:

  • When a request starts, store its Promise in a Map.
  • If a second request arrives for the same key, do not start a new fetch.
  • Instead, return the existing Promise from the Map.
  • Once the request finishes, remove it from the Map and save the result to the cache.

This ensures that no matter how many people request the same data at once, only one call hits the API.

While fixing this, I also resolved three other edge-case bugs in the same file:

  • Missing Token Errors: The system sent "undefined" as a credential. I updated it to validate tokens before making requests.
  • Memory Leaks: Retry logic left stale event listeners on AbortSignals. I added cleanup logic to prevent leaks.
  • URL Injection: Usernames with special characters broke API paths. I added encoding to protect the URL structure.

A cache is not enough. You also need to deduplicate requests that are currently in flight.

Source: https://dev.to/eshaanagrawal/the-cache-was-working-and-still-causing-duplicate-api-calls-3n51