A developer guide warns that keeping a database transaction open for the duration of an AI-driven chat can corrupt answers and choke the underlying DBMS. The note, aimed at teams building LLM-backed tools, says the practice “should not be done” and offers four short-lived consistency patterns instead.

Why the warning matters

LLM-powered assistants often ask a series of follow-up questions: they read a record, request a detail, then ask for a total. If the underlying data changes between those steps, the assistant can return contradictory figures—one answer will be wrong. The tempting fix is to open a single transaction at the start of the conversation and keep it alive until the chat ends. In practice, that approach ties up row versions, fills tempdb, holds locks, and interferes with connection pooling.

What leads to long-living transactions

  • Multi-turn prompting – LLMs typically generate several prompts before the user sees a response.
  • Tool calls that hit the database – Each turn may invoke a stored procedure, a SELECT, or an UPDATE.
  • Uncontrolled transaction scope – Developers sometimes wrap the whole chat in a BEGIN…COMMIT block, assuming it guarantees consistency.

When the chat stretches, the DB engine must retain the original row versions so that the transaction sees a stable view. Those versions sit in tempdb, consuming space and I/O. Locks held for the same period block concurrent writers, and the idle connection can exhaust the pool, forcing new callers to wait for a free slot.

Four short-lived patterns

The guide recommends treating consistency as a per-tool-call concern rather than a per-conversation one. The four patterns are:

  1. Live statements – Each call runs under the default isolation level, seeing only data that has been committed at the moment of execution. This is the simplest model; the caller accepts that data may have changed since the previous turn.
  2. Bounded transactions – A developer groups a handful of statements inside a single short transaction that finishes before the next LLM turn. It guarantees atomicity for that batch without lingering beyond the tool call.
  3. Snapshot reads – The operation starts with a defined snapshot timestamp, giving a stable view of the database for the duration of the call. All reads within the call see the same data, even if concurrent writes occur.
  4. Materialized reports – The tool reads from a pre-generated, versioned result set that reflects the database at a known cutoff point. Pagination or further calculations then operate on that frozen dataset.

In SQL Server, check if READ_COMMITTED_SNAPSHOT is active. Do not assume the name tells the whole story.

Practical rules for LLM-driven apps

  • Batch what you need – If a question requires many values, compute them in a single tool call rather than issuing separate queries that each start a new transaction.
  • Deterministic pagination – When presenting results across pages, use a stable ordering key, a cursor, or a materialized result set. Never keep a transaction open while the user scrolls.
  • Return evidence – Alongside the data, include metadata that makes the consistency model explicit: the consistency class, snapshot start time, reporting cutoff, data freshness, row count, database identity, and a trace ID.
  • Stress-test with concurrency – Simulate concurrent writes while the LLM is prompting, and verify that the application retries or falls back gracefully.

The bottom line is clear: an AI chat should not dictate the lifespan of a database transaction. By scoping consistency to each tool call, developers keep the database healthy, preserve performance for all users, and still give the LLM enough reliable data to answer accurately.