𝗙𝗶𝘅𝗶𝗻𝗴 𝗔𝗜 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗶𝗻 𝗠𝗮𝘀𝘁𝗿𝗮
OpenTelemetry is the standard for monitoring modern systems. Traditional traces work for most software. They fail for AI applications.
When you build AI, you need specific answers: • Which model generated the output? • Which provider did you use? • How many tokens did you consume? • Which embedding model processed your documents? • What was the cost of the operation?
These questions matter most in Retrieval-Augmented Generation (RAG) systems.
While contributing to Mastra, I found a gap in RAG embedding observability. Mastra exported metadata for many AI tasks, but RAG embedding spans lacked standard attributes.
Observability tools saw the embedding operation. They did not understand the context. They missed model details, provider info, and token usage.
A RAG pipeline follows these steps: • Documents • Chunking • Embedding Model • Vector Database • Similarity Search • LLM Generation
The embedding stage is vital. If you lack data here, debugging performance becomes hard.
OpenTelemetry uses semantic conventions to create a common language. Instead of every tool using custom names, everyone follows one standard. This allows tools to read attributes like: • gen_ai.system • gen_ai.request.model • gen_ai.usage.input_tokens
I submitted a pull request to map Mastra RAG embedding data to these OpenTelemetry standards.
The work included: • Exporting embedding model metadata • Exporting provider information • Mapping token usage metrics • Aligning attributes with global standards
This lets observability systems understand embeddings without custom code.
Production AI systems need visibility. You need to know which model causes latency or which provider costs the most. Standardized telemetry provides these answers automatically.
Open source teaches a great lesson. Not every good contribution adds a new feature. Sometimes the best work makes existing systems easier to monitor and run.
If you build AI infrastructure, do not ignore observability. The best AI systems are observable.
Optional learning community: https://t.me/GyaanSetuAi