The bot started spitting out the same answer two or three times whenever a user hammered the send button. The duplication only appeared for people who typed fast enough to fire off several messages before the AI began thinking, and it stayed hidden in production for a long time because the pattern was rare. A premature database lock – released a few milliseconds after being taken – left the conversation unprotected, allowing multiple processes to answer the same prompt.

Why the lock failed

The code acquired a lock in a single database call, then immediately handed control back to the request handler. The lock’s lifetime was measured in milliseconds, far shorter than the time the AI model needs to generate a response. By the time the model started work, the lock had already vanished, so nothing prevented a second request from grabbing the same conversation record and issuing another reply.

Two symptoms emerged:

  • Identical replies were sent back-to-back.
  • Slightly re-phrased replies appeared for the same question, as each process built its own prompt from the same user input.

Because most users pause between messages, the bug stayed under the radar. Only rapid typists triggered the race condition, and the cases were rare.

The half-hearted fix that didn’t cut it

The first response was to add a short delay after a message arrived, hoping to “debounce” rapid input. That helped when two messages arrived in quick succession, but it fell apart if a third message showed up while the AI was still generating text.

A second issue surfaced when timers and conversation data lived in the same storage bucket. When the bot finished processing a request, it overwrote the timer record, effectively deleting its own countdown. The system lost track of which messages had already been answered, opening the door to further duplication.

Building a reliable guard: version counters, isolated timers, and a lease

The team redesigned the flow around three pillars:

  • Version counter – each incoming message increments a counter stored with the conversation. The counter tells the system how many messages have arrived since the last reply, making it easy to detect new input while a response is being generated.
  • Dedicated debounce window – timers now live in a separate storage area, insulated from conversation payloads. A hard cap on the debounce duration prevents a user from stalling the bot indefinitely.
  • Session lease – the original lock is replaced with a lease that carries an explicit expiry timestamp. The lease is claimed using a compare-and-swap (CAS) operation: the process reads the current lease value, writes a new one only if the old value matches, and thereby gains exclusive rights to the conversation. If the process crashes, the lease automatically expires, freeing the conversation for the next handler.

How the new pipeline works

  1. Message arrival – the system increments the version counter and (re)sets the debounce timer. It returns to the client immediately, without starting the AI.
  2. Timer expiration – the timer handler attempts to acquire the lease. If the CAS succeeds, the handler proceeds; otherwise it backs off, knowing another process already owns the conversation.
  3. Check for new input – the handler compares the current version counter with the value it recorded when the timer started. If the counter has advanced, it aggregates the pending messages into a single prompt.
  4. Generate a reply – the AI model runs once, producing a single answer that covers all recent user inputs.
  5. Final sanity check – just before the reply is sent, the handler reads the version counter again. If a newer message arrived during generation, the reply is discarded and the process restarts the timer, ensuring no stale answer reaches the user.

This approach eliminates duplicate replies, caps the time a conversation can be stalled, and recovers automatically from process crashes because the lease expires on its own.

Takeaway

A lock that disappears before the critical section begins offers no protection at all. By replacing a fleeting database lock with an explicit, expiring lease and isolating timers from conversation data, the bot now guarantees a single, up-to-date reply even when users type at lightning speed. The episode underscores a timeless lesson: concurrency safeguards must outlast the work they protect, or they become invisible barriers that let bugs slip through.