𝗛𝗼𝘄 𝗜 𝗦𝘁𝗼𝗽𝗽𝗲𝗱 𝗟𝗼𝘀𝗶𝗻𝗴 𝗔𝗣𝗜 𝗖𝗮𝗹𝗹𝘀 𝘁𝗼 𝗥𝗮𝘁𝗲 𝗟𝗶𝗺𝗶𝘁𝘀
My app dropped user requests. Logs showed 429 errors. My retry logic made it worse. The system stopped.
I used an AI API. It limited me to 50 requests per minute. I tried a simple sleep timer. It blocked my workers. It created bursts of failures.
I tried these methods first:
- Random waits.
- Local retry libraries.
- Token buckets without locks.
None of these worked.
I fixed it with Redis and asyncio. I separated the rate limiter from the retry logic.
It works like this:
- Redis tracks timestamps in a sliding window.
- The app waits if the limit is hit.
- I added exponential backoff.
- I added jitter to stop burst hits.
Follow these rules for your API calls:
- Only retry 429 and 5xx errors.
- Log every retry attempt.
- Use circuit breakers to stop trying after many failures.
Use libraries like aiolimiter or tenacity for production. They are better.
How do you handle rate limits?