𝗧𝗵𝗲 𝗣𝗼𝘄𝗲𝗿 𝗼𝗳 𝗣𝗼𝘀𝘁𝗴𝗿𝗲𝗦 𝗟𝗜𝗦𝗧𝗘𝗡/𝗡𝗢𝗧𝗜𝗙𝗬 You need to keep your video metadata up to date. But caching can make this hard.
Here's how we solved this problem:
- We use Postgres to store video metadata.
- When metadata changes, Postgres sends a notification.
- A listener gets this notification and updates the cache.
We use a trigger to send notifications when metadata changes. This trigger runs inside the same transaction as the write. So if the transaction rolls back, the notification is discarded.
The listener is a separate daemon. It holds a Postgres connection open and blocks on incoming notifications. When it gets a notification, it updates the cache.
We also have a reconciliation sweep to catch any missed notifications. This sweep runs periodically and updates the cache for any changed metadata.
Some key lessons we learned:
- Make every consumer action idempotent.
- Keep payloads small - just send identifiers.
- Don't send notifications for view-count changes.
- Use one dedicated connection per listener.