Microsoft’s Foundry team has added OpenTelemetry-based tracing to its agent framework, giving developers a way to see end-to-end execution across heterogeneous LLM-powered agents.

Why multi-agent systems need more than log files

A typical AI-driven incident-response drill uses a commander agent that orchestrates several specialist agents: one parses logs, another detects metric anomalies, a third matches symptoms to runbooks, and a router picks the best language model for each sub-task. Each specialist may call a different model—say, a “gpt-5-mini” variant—and invoke its own tools. When something goes wrong, engineers stare at isolated logs that show what each component did, but no view of how the pieces fit together.

Without a unified trace, the root cause hides in the hand-off between agents. The commander might send a request that the log-reader handles correctly, yet the metric specialist mis-interprets the data and suggests the wrong runbook. Debugging that chain by hand takes time and invites error.

How OpenTelemetry stitches the workflow together

OpenTelemetry defines two core concepts: traces and spans. A trace is a unique identifier that follows a request from entry to final response. A span records a single operation—such as a call to a language model or a tool invocation—within that trace.

When an agent receives a request, it pulls the incoming Trace ID from the request’s metadata and creates a child span that inherits the same ID. The child span logs its start time, duration, attributes (model name, tool used) and any errors. The process repeats for every downstream agent, building a tree that mirrors the logical flow of the overall task.

OpenTelemetry also supports Baggage, a lightweight carrier for custom key-value pairs. By attaching a “drill-id” or other business context to the baggage at the top of the trace, every downstream span automatically inherits that identifier. A span processor then promotes the baggage into regular attributes, making it easy to query all spans belonging to a particular incident drill.

What the new tracing surface looks like

With the instrumentation in place, Azure Monitor (or any OpenTelemetry-compatible backend) renders a visual hierarchy:

  • Agent name / ID – shows which component performed the operation.
  • Tool usage – records which external service or function was called.
  • Model version – logs the exact LLM used, useful for tracking regressions after a model upgrade.
  • Token consumption – captures how many tokens were sent to and received from the model, helping teams manage cost.
  • Latency / duration – highlights where bottlenecks appear, whether in model inference or tool I/O.

In the incident-drill example, the commander’s root span spawns child spans for each specialist, and each specialist spawns further children for its model calls. Clicking any node reveals the full attribute set, so an engineer instantly sees the details of each operation.

The stakes for AI-centric operations

  • Speed of root-cause analysis – Teams trace a failure back to the exact span that threw an error, cutting mean time to resolution.
  • Cost visibility – Token counts sit beside latency, letting finance spot runaway usage before cloud bills balloon.
  • Performance tuning – High-latency spans across agents point to where caching, model selection or tool redesign could boost throughput.

What to watch next

Projects built on LangChain, the OpenAI SDK or other orchestration layers can adopt the same semantic conventions for GenAI, paving the way for traces that flow across cloud providers and on-premise deployments.

Organizations simply enable the OpenTelemetry SDK in their agents and send data to Azure Monitor or an open-source collector.

Takeaway

OpenTelemetry gives multi-agent AI systems the missing glue that turns a scatter of logs into a coherent narrative. By propagating a single Trace ID across heterogeneous LLMs, routers and tool calls, developers locate failures, monitor costs and optimize performance without reinventing tracing infrastructure.