Building FailureDNA: An Agent Memory That Knows When Not To Trust Itself
Most AI agents make a dangerous mistake. They use past incidents to solve new outages. If a new problem looks like an old one, the agent copies the old fix.
This is a problem. Similarity does not mean applicability. A past fix might work for an old database version but fail on your current one. Repeating a failed fix wastes time. Using a stale fix makes the outage worse.
I built FailureDNA to solve this. It is a memory system that tracks real outcomes. It decides if a past experience is useful, needs inspection, or should be avoided.
The system follows one rule: the model selects the action, but deterministic code decides what is valid.
How it works: • The agent embeds incident symptoms. • It searches past incidents using pgvector on Alibaba Cloud RDS. • A deterministic gate checks the results. • The model picks an allowlisted action. • The system executes the action and saves the real outcome.
The gate uses three simple rules:
- If the past outcome was a failure: Avoid it.
- If the past outcome was a success and the environment matches: Use it.
- If the past outcome was a success but the environment changed: Inspect it.
The agent cannot repeat a known failure. The system removes bad actions before the model even sees them. This prevents the model from ignoring warnings. The model provides creativity, but the code provides safety.
Technical details: • I used Qwen Cloud for embeddings and action selection. • text-embedding-v3 creates 1024-d vectors. • Hybrid retrieval combines semantic similarity and keyword overlap. • The backend runs on Alibaba Cloud Function Compute with FastAPI. • Memory stays in ApsaraDB RDS for PostgreSQL.
FailureDNA prevents the agent from repeating historical failures and using stale fixes. It keeps the model creative while ensuring the system stays safe.
Optional learning community: https://t.me/GyaanSetuAi