Your database CPU pegs at 100% at three in the morning. Traffic looks normal. The number of requests is not unusually hi

Your database CPU pegs at 100% at three in the morning. Traffic looks normal. The number of requests is not unusually high. Yet your primary store is melting down because every single one of those requests is asking for exactly the same thing at exactly the same time.

This is the thundering herd problem.

Think of a stadium after a concert. Over the course of an hour, the same exit door can comfortably let everyone leave. But if the entire crowd decides to leave through that single door in the same ten-second window, the door does not break. It simply cannot handle the concurrency. The crush is the bug, not the door.

Where the Herd Forms

The pattern shows up whenever many processes synchronize on one event. You do not need malicious traffic. Ordinary systems do this to themselves.

Cache expiration. A hot cache key dies. It might be a product catalog, a feature flag, or a user's permissions list. Thousands of application servers notice the empty slot simultaneously. Each server, acting in good faith, opens its own database connection and runs the same heavy query to rebuild the value. The database was never configured to answer the same expensive question two thousand times in parallel. One expired key takes the whole cluster down.

Connection pool wakeups. In some architectures, many worker processes block on a single condition, waiting for work to arrive. When that condition fires, the operating system wakes every sleeping worker. Only one worker actually acquires the connection or the task. The rest wake up, discover they lost the race, and go back to sleep. That cycle burns CPU in context switches and can starve actual productive work.

Retry storms. A downstream service hiccups. Every client catches the timeout and waits a fixed interval, say exactly one second, before trying again. When the service recovers and accepts traffic again, every client hits it at the same instant. The service, still cold and recovering, goes down again. The cycle repeats.

Cron collisions. If you schedule maintenance, report generation, or cache warming to run at exactly 00:00 UTC across a fleet of instances, you manufacture a traffic spike with clockwork precision. The load is predictable, but the concentration is lethal.

Request Coalescing

The most effective fix for cache stampedes is to stop answering the same question more than once. When a cache miss occurs, you do not want 2,500 threads each running a database query. You want one thread to run the query while the other 2,499 wait for that result.

This pattern is often called request coalescing. In Go, the singleflight package in golang.org/x/sync provides a canonical implementation. The first goroutine to call Do for a given key starts the work. Subsequent callers with the same key block on that same Do call. When the work finishes, all waiters receive the result together. You have executed one query and served 2,500 requests.

You can build similar behavior with an in-memory concurrent map of promises, futures, or channels. The trick is to check and register an in-flight request atomically. If the request fails, everyone gets the error, which is usually the right behavior. If it succeeds, everyone gets the cached value, and subsequent requests hit the cache directly. The database load drops from a vertical spike to a flat line.

Probabilistic Early Expiration

Sometimes you want to avoid the stampede entirely rather than manage it. Enter probabilistic early expiration, implemented through algorithms like XFetch.

Instead of a hard expiration time, each cached value carries a soft expiration window. When a request arrives, the application generates a random number and applies a simple probability check. If the dice roll says yes, that single request refreshes the cache early. If not, the request serves the slightly stale value.

Because each request rolls its own dice, the refreshes spread out across the soft window. One client might refresh at forty seconds before hard expiration, another at twelve seconds, and most others not at all. The work shifts from a single synchronized spike to a gentle