𝗦𝗺𝗮𝗿𝘁 𝗦𝗲𝗮𝗿𝗰𝗵 𝗪𝗶𝘁𝗵𝗼𝘂𝘁 𝗠𝗼𝗱𝗲𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴
I built a docs site for an internal tool. Users wanted a search with understanding. Keyword search failed. A user asks about login issues. The search finds nothing because the text says authentication failure.
I tried several things.
- Synonyms took too much time.
- NLP libraries made the app too heavy.
- Local embeddings needed a GPU.
I stopped trying to host a model. I used an AI API instead.
Here is the process:
- Split documentation into small paragraphs.
- Create a backend endpoint for queries.
- Send the query and paragraphs to the API.
- Ask the AI to pick the best match.
You do not need to fine-tune the model. A good prompt and a few documents work.
The results:
- Low cost per call.
- Setup takes minutes.
- Support tickets dropped by 30%.
Keep these points in mind:
- Latency stays between 300ms and 1500ms.
- You need a network connection.
- External APIs have privacy risks.
You do not need to be an ML engineer. Use a prompt and an API.
What is your AI setup? Share your experience below.