𝗧𝗵𝗲 𝗦𝗲𝗰𝗿𝗲𝘁 𝗧𝗼 𝗠𝗮𝗸𝗶𝗻𝗴 𝗠𝗼𝗿𝗲 𝗥𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗔𝗜 𝗖𝗵𝗮𝘁𝗯𝗼𝘁𝘀

𝗧𝗵𝗲 𝗦𝗲𝗰𝗿𝗲𝘁 𝗧𝗼 𝗠𝗮𝗸𝗶𝗻𝗴 𝗠𝗼𝗿𝗲 𝗥𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗔𝗜 𝗖𝗵𝗮𝘁𝗯𝗼𝘁𝘀 You build a chatbot to answer user questions. But what if the API you use times out or returns errors?

• Your chatbot becomes unresponsive
• Users get frustrated
• You lose trust

I built a chatbot that used an LLM API. But it kept timing out. I tried different API providers, but the issue persisted. Then I realized the problem was not the provider, but how I was calling them.

To fix this, I used a three-layer approach:

• Retry with backoff: respect rate limits and transient errors
• Circuit breaker: stop hammering a failing endpoint
• Fallback to a different model: if the primary API is down, switch to a backup

This approach made my chatbot more reliable. I learned that you should not assume any API is