Unleashing the Power of AI: Building On-Demand Tools with an Explainability Agent

The integration of AI into various domains is witnessing a transformative shift, notably in how AI-driven systems approach problem-solving. Traditional agentic architectures primarily rely on predefined tools to execute tasks. However, the emergence of advanced AI models is challenging this norm, advocating for a more dynamic approach where tools are created on demand. This article explores the innovative plan–code–execute architecture, designed to build an end-to-end Explainability Agent, demonstrating the potential of AI to generate tools as required by the task at hand.

The Scenario: Explaining GNN Model Results

The focus of this scenario is on explaining the results of a Graph Neural Network (GNN) model used for demand forecasting. The data signals under consideration include spatial features of nodes (SKUs), temporal features like sales order and production data, and results from GNN models trained for demand forecasting and anomaly detection. The challenge lies in explaining these complex outputs holistically from a domain-aware planner's perspective.

Explainability Agent Architecture

The architecture employs a plan-code-execute workflow consisting of custom-designed agents to ensure interpretability and flexibility in tool generation. Here’s a breakdown of the agents involved:

The Analyst Agent

The Analyst Agent serves as the application's eyes, discovering the environment by examining raw data, model results, and previous outputs. It creates a dynamic schema, forming the context for the planning agent. This prevents assumptions, such as misnaming files, and ensures the planner operates with accurate information.

The Planner

Acting as the brain of the application, the Planner receives high-level goals and schema from the Analyst. It decomposes requests into a dependency graph of steps, generating a JSON plan with flags indicating whether each step requires code (CODE) or text (TEXT). This plan guides the subsequent tool generation and execution.

The Coder

The Coder is the application’s hands, tasked with generating executable Python scripts for each CODE step. It manages library imports, file paths, and error handling, ensuring the creation of self-sufficient software rather than mere code snippets.

The Executor

The Executor runs the generated scripts within a controlled environment, capturing outputs and handling errors. This agent ensures that scripts run smoothly, leveraging the schema's grounding to minimize execution failures.

The Reporter

Finally, the Reporter interprets logs and artifacts produced by the scripts, answering the user’s original queries in the required tone and depth. It acts as the voice of the application, delivering meaningful insights based on the analyses performed.

Experiment Results: A Case Study

The architecture was tested on SKU SOS001L12P, a node with significant sales volume. The goal was to uncover insights about its spatial relationships, sales trends, and production signals, ultimately generating a comprehensive report.

Spatial Analysis

The goal was to analyze the spatial structure around SKU SOS001L12P. The Planner Agent created a sequence of steps to load data, extract relationships, and visualize the node’s connections. The output featured a graph detailing the SKU's relationships with its neighbors, providing insights into shared resources and logistics.

Sales Order Signal Analysis

To assess demand forecasting accuracy, the sales trends of SKU SOS001L12P were analyzed. The Planner generated a plan to calculate rolling averages, identify outliers, and visualize trends. The results showed a persistent decline in sales, prompting further investigation into potential causes.

Production Signal Analysis

The production trends were analyzed to correlate with sales declines. The analysis revealed anomalies in production data, which could potentially explain the sales trend. By identifying these anomalies, the architecture provided actionable insights into supply chain disruptions.

Comprehensive Report

The final step was to correlate the findings from spatial, sales, and production analyses with demand forecasts. The resulting report concluded that while the forecasting model captured sales trends effectively, production struggled to align, suggesting operational disruptions. This comprehensive assessment provided planners with valuable insights for strategic decision-making.

Conclusion

The future of agentic systems moves beyond static tool catalogs towards dynamic tool creation. The plan–code–execute architecture exemplifies how AI can autonomously transform intent into functioning software, underscoring a shift towards software generation as a core capability. As AI models continue to evolve, the value lies in agents that seamlessly convert high-level goals into actionable insights, marking a natural progression in agentic system design.

For further insights into the GNN model for demand forecasting, refer to my previous work. Connect with me on LinkedIn to share your thoughts on this evolving AI frontier.