๐ง๐ต๐ฒ ๐ฃ๐ผ๐๐ฒ๐ฟ ๐ผ๐ณ ๐ฃ๐ผ๐๐๐ด๐ฟ๐ฒ๐ฆ ๐๐๐ฆ๐ง๐๐ก/๐ก๐ข๐ง๐๐๐ฌ You need to keep your video metadata up to date. But caching can make this hard.
Here's how we solved this problem:
- We use Postgres to store video metadata.
- When metadata changes, Postgres sends a notification.
- A listener gets this notification and updates the cache.
We use a trigger to send notifications when metadata changes. This trigger runs inside the same transaction as the write. So if the transaction rolls back, the notification is discarded.
The listener is a separate daemon. It holds a Postgres connection open and blocks on incoming notifications. When it gets a notification, it updates the cache.
We also have a reconciliation sweep to catch any missed notifications. This sweep runs periodically and updates the cache for any changed metadata.
Some key lessons we learned:
- Make every consumer action idempotent.
- Keep payloads small - just send identifiers.
- Don't send notifications for view-count changes.
- Use one dedicated connection per listener.