Using Routing Keys and Profile Leases in Playwright Worker Queues

A Playwright error like TimeoutError: page.click failed is often a lie.

The error tells you what the script saw. It does not tell you why the script was in the wrong place to begin with.

I once saw two workers pick up jobs for the same account. Both workers opened the same browser profile at the same time. One worker was waiting on a slow page. The other worker retried, changed the session state, and caused a failure.

The problem was not the code. The problem was the queue.

Most worker queues are designed for stateless jobs.

  • Take a job.
  • Find a free worker.
  • Run the task.
  • Mark it done.

This works for screenshots or API calls. It fails for browser automation.

A browser profile is not stateless. It belongs to one account, one proxy, and one session state. The fastest worker is not always the right worker.

To scale safely, you must shift your logic.

Instead of asking: Which worker is free?

Ask: Which account environment is safe to use now?

You can solve this with three layers:

  1. Routing Keys Do not let any worker grab any job. Use a routing key based on the profile ID or account ID. This ensures only one worker touches a specific profile at a time.

  2. Profile Leases A queue claim means a worker owns a job. A profile lease means a worker is allowed to use a specific browser profile for a set time. Use a lease with a heartbeat. If the task takes longer than expected, the worker must renew the lease.

  3. Fencing Tokens Stale workers are dangerous. If a worker loses its lease due to a network lag but keeps running, it might try to write data. Use a fencing token. The storage layer should reject any writes from an old token.

You must also add a readiness gate. Before you launch Playwright, check these:

  • Is the account paused?
  • Does the profile need human review?
  • Does the proxy region match the job requirements?

A blocked job is not always a failed job. It is often just a job waiting for the right context.

Stop building queues that only prioritize speed. Build queues that prioritize account state.

Source: https://dev.to/web4browser/using-routing-keys-and-profile-leases-in-playwright-worker-queues-a53