A semi-autonomous, four-agent pipeline can cut hallucinations in exploratory data analysis (EDA) by keeping large language models (LLMs) out of the number-crunching and into the judgment calls only. The design lets the model decide what to examine while plain code, run in a sandbox, does the actual calculations, delivering results that can be verified step by step.
Why a single LLM call is risky
Prompting an LLM to “look at this spreadsheet and tell me what’s interesting” yields a single block of text. The model invents numbers, blends reasoning with output, and offers no trace of how a conclusion was reached. When it hallucinates—a fabricated statistic or a spurious correlation—there is no checkpoint to catch the error before it reaches the user.
The idea behind a semi-autonomous pipeline
The new workflow splits EDA into four fixed stages, each handled by its own agent. The agents follow a “plan-then-act” pattern: they first decide which operation is needed, then hand the operation to sandboxed code that actually runs it. The LLM never performs arithmetic or file manipulation; it only makes the judgment about relevance.
The four agents
- Data-cleaning agent – inspects column types, flags missing values, and decides on imputation or type conversion strategies.
- Column-analysis agent – selects appropriate visualisations (histograms, box plots, etc.) for each variable based on its data type and distribution.
- Relationship-analysis agent – evaluates every pair of columns, chooses which pairs deserve a deeper statistical test, and orders them by potential insight.
- Report-writing agent – translates the generated charts and computed statistics into natural-language explanations, highlighting noteworthy patterns and caveats.
Each stage runs independently, so the column-analysis and relationship-analysis agents can execute in parallel, shaving time off the overall workflow.
How autonomy is dialed back
The pipeline’s safety rule is simple: any code the model writes runs inside a sandbox that isolates it from the host system. If execution fails, the error feeds back to the model, which gets up to two attempts to correct the script before the pipeline aborts. This stops runaway loops and guarantees that the model never directly manipulates files or performs arithmetic.
Because the model’s role is limited to deciding what to compute, the actual numbers always come from deterministic code. If the relationship-analysis agent suspects a link between “Order ID” and “Month,” the sandbox runs a correlation function, returns the precise value, and only then does the model comment on whether the correlation is likely causal or coincidental.
What this solves – and what it costs
Reduced hallucinations. By separating reasoning from calculation, the pipeline eliminates the most common source of fabricated statistics: the model guessing numbers instead of letting code generate them.
Modularity. Adding a new stage—say, a time-series forecasting agent—doesn’t require rewriting the whole prompt. Each agent is a self-contained module that plugs into the fixed sequence.
Speed gains. Parallel execution of independent agents cuts wall-clock time compared with a monolithic LLM call that must serialize every step.
Complexity overhead. The trade-off is a more elaborate architecture. Teams need to maintain the sandbox environment, handle error-return loops, and orchestrate multiple agents.
What to watch next
- Tooling integrations. Open-source frameworks that abstract the sandbox-execution step could lower the engineering burden and make the pattern more accessible.
- Standardised agent contracts. As more teams adopt multi-agent pipelines, common interfaces for “plan-then-act” agents may emerge, easing interoperability.
The core lesson is clear: give the LLM the freedom to think, not the power to compute. By confining its autonomy to judgment and routing every number through isolated code, a semi-autonomous EDA pipeline produces results you can check, trust, and share without fearing phantom figures.
Source: https://dev.to/sraveend/agentic-but-only-semi-autonomous-designing-an-eda-pipeline-you-can-trust-4ha9
Community discussion: https://t.me/GyaanSetuAi