Your asyncio task can die mid-work

Many Python services use this pattern to run background work:

async def create_order(order): await save(order) asyncio.create_task(send_webhook(order)) return {"status": "created"}

The goal is to send a webhook without making the client wait. This works in development. In production, it fails randomly.

The problem is garbage collection.

When you call asyncio.create_task and do not save the result, the task becomes eligible for garbage collection. The event loop only keeps weak references to tasks. If nothing else points to that task, Python may reclaim it while it is still running.

Your work just stops. There are no errors in your tracker. The code is perfect. The task simply vanishes.

You might see this log in your stderr: Task was destroyed but it is pending!

This error is hard to find because it often appears much later than the actual failure.

How to fix it:

Use a background task set to hold references.

background_tasks = set()

def fire_and_forget(coro): task = asyncio.create_task(coro) background_tasks.add(task) task.add_done_callback(background_tasks.discard) return task

The set keeps the task alive. The callback removes it once the work finishes. This prevents memory leaks and stops the garbage collector from killing your tasks.

For Python 3.11 and newer, you can use TaskGroups. A TaskGroup holds strong references to every task inside it. Note that a TaskGroup will wait for all tasks to finish before exiting the block. Use this if the work must complete before you respond to a user.

For heavy background work, use a supervisor pattern:

  • Create one long-lived task at startup.
  • Use a queue to send work to it.
  • The request handler only adds items to the queue.

This ensures your work survives and allows you to add retries later.

In Python, you must own your tasks. If you do not hold a reference, the runtime will not protect them.

Source: https://dev.to/r9v/your-asyncio-task-can-be-garbage-collected-mid-flight-3kg1