Building a Local AI Agent for OSM
I spent 21 days building a local AI agent. It turns natural language into OSM filter JSON.
I wanted this to run entirely on a local GPU. No API calls. No privacy risks. No rate limits.
Here is how I built it and what I learned.
The Architecture
I used a RAG pipeline to give the model context.
• Embeddings: I used bge-small-en-v1.5 to turn OSM tag descriptions into vectors. • Database: I stored these embeddings in SQLite. • Retrieval: When you ask a question, the system finds the most relevant OSM tags and filter examples. • Generation: The LLM receives your query plus the retrieved examples to create the JSON.
The Failures
Building this was not smooth. I hit several walls.
• The Stop Token Bug: My first runs returned empty results. The model added a blank line before the JSON. This triggered my stop token and cut the output short. I had to remove the stop token to fix it. • Blind Copying: The LLM often copied examples exactly instead of adapting them. I rewrote the prompts to force the model to synthesize new filters. • Domain Confusion: The model confused maritime tags with land tags. I added a validation step to filter candidate tags by relevance.
The Lessons
• Avoid stop tokens: Unless you are certain, do not use them. They often break your output. • Data quality is king: Embedding natural language sentences works better than embedding raw JSON strings. • The loop is the magic: I added a simple loop. If a filter returns zero results, the agent broadens the request and tries again. This turns a static tool into an adaptive system.
The Result
The system runs on a single GPU with less than 8 GB VRAM. It handles most simple queries and adapts when it fails. It still needs work on complex negation and tag synthesis, but the core works.
How do you handle tag ambiguity in your geospatial workflows?
Source: https://dev.to/gisfromscratch/building-a-local-ai-agent-for-osm-21-days-of-iteration-2fc4
Optional learning community: https://t.me/GyaanSetuAi