𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗔𝗻 𝗘𝘃𝗲𝗻𝘁 𝗕𝘂𝘀 𝗪𝗶𝘁𝗵 𝗘𝘅𝗮𝗰𝘁𝗹𝘆-𝗢𝗻𝗰𝗲 𝗗𝗲𝗹𝗶𝘃𝗲𝗿𝘆
Sending events between microservices is hard. You want events to arrive exactly once. Not zero times. Not twice.
Here is a blueprint for a robust event bus.
The Tech Stack:
- PostgreSQL for durable storage.
- Redis for fast queuing.
- Python for the logic.
The Strategy:
- Use a durable event log.
- Track delivery attempts.
- Use idempotency keys.
How the flow works:
- A publisher sends an event.
- The bus saves it with a unique ID in PostgreSQL.
- A worker puts the event in a Redis queue.
- The consumer receives the event.
- The consumer checks an idempotency table.
- If the ID exists, the consumer skips the work.
- If the ID is new, the consumer processes the event and saves the ID.
Key Tables:
- Events: stores payloads and IDs.
- Delivery Attempts: tracks status and retries.
- Consumer Idempotency: tracks processed events per consumer.
Operational Tips:
- Use a dead-letter queue for permanent failures.
- Apply exponential backoff for retries.
- Monitor success rates with a dashboard.