I Put An AI Agent In Front Of 20 Years Of Support Tickets
We have a helpdesk system with one million tickets. We have 2,400 manuals and two massive codebases.
When a developer asks if we solved a problem before, the answer is always yes. The data is just buried. Senior staff lose hours searching. Juniors lose weeks.
I built an AI agent to sit in front of all this data. It is now in production. Here is what actually happened.
The data was not the hard part. Knowing where to look was. Making the agent stop making things up was even harder.
How it works:
- One agent with natural language input.
- It reads the helpdesk database, project tools, GitLab, and SVN.
- It performs vector searches over manuals and old tickets.
- It can generate files like .xlsx or .docx.
- It picks its own tools and runs them in parallel.
The architecture is simple. It runs in a Node container separate from the main app. I did this for failure isolation. If the AI fails, the helpdesk keeps running.
The biggest lesson was about the prompt. I started with one giant wall of text. It was 40,000 tokens. It was hard to maintain and the agent lost focus.
I changed my approach. I split the prompt into small skill files.
- Ticket search is one file.
- Code history is another.
- Project management is a third.
The main prompt acts as a router. It only loads a skill when it needs it. This dropped our tokens from 40,000 to 8,000 per request. The agent stays on task and the code is easy to update.
I also added user profiles. A developer gets code paths. A project manager gets status summaries. The agent knows who is asking without being told.
It is not perfect. We faced real failures:
- Hallucinations: The agent invented fake URLs. Fix: Pin exact patterns or force a lookup.
- Memory loss: The agent forgot rules in long chats. Fix: Inject a structured to-do list into every turn.
- Infinite loops: The agent made too many tool calls. Fix: Set a strict budget per request.
- Safety: Users tried to trick it. Fix: Do not rely on prompts for safety. Use read-only database credentials in your infrastructure.
The real work is a weekly loop. I read the logs. I score the sessions. I tweak the skills. I update the profiles.
If you skip this loop, you have a demo. If you do it, you have a product.
The goal is not just faster answers. The goal is keeping institutional knowledge in the building. When a person leaves, their knowledge stays in the transcripts.
Summary for your build:
- Start with RAG over your own history.
- Use lazy-loaded skill files instead of one big prompt.
- Use user profiles for personalized context.
- Isolate the AI in its own container.
- Enforce safety through infrastructure, not instructions.
- Read your logs every week.
Optional learning community: https://t.me/GyaanSetuAi