The increasing prevalence of AI hallucinations, where generative AI models produce inaccurate or fabricated information, poses significant risks to enterprises. These errors can disrupt operations, erode trust, and lead to costly setbacks. A new wave of solutions, centered around "agentic AI," is emerging to combat this challenge, offering a more robust approach to ensuring AI accuracy and reliability.
The Pervasive Problem of AI Hallucinations
AI hallucinations are a common occurrence, with studies indicating they can happen between 0.7% and 29.9% of the time, depending on the specific large language model (LLM) used. The consequences can be severe, ranging from a bank chatbot providing incorrect loan terms to an AI robo-advisor recommending unsuitable investments. In a real-world example, a manufacturer's AI customer support assistant offered troubleshooting advice for products not in its knowledge base, extrapolating solutions or generating false information due to the nuanced nature of the devices.
Agentic AI: A Proactive Defence Against Inaccuracy
Agentic AI refers to systems capable of autonomous action and goal achievement. In the context of preventing hallucinations, agentic AI evaluation tools help developers safeguard their AI applications. These tools enable AI agents to perform self-assessment, evaluating whether a response aligns with known facts or logical reasoning before finalisation. This iterative reasoning process helps identify and correct errors internally.
Key strategies employed by agentic AI include:
Retrieval-Augmented Generation (RAG): Combining RAG with autonomous agentic reasoning allows AI to manage complex tasks, personalise responses, and prioritise relevant information, all while grounding outputs in trusted knowledge sources.
Chain-of-Thought Prompting: Breaking down complex problems into logical steps helps AI agents reason more effectively, particularly in demanding tasks.
Validation and Guardrails: Implementing additional checks and safeguards within AI agents ensures more accurate analysis and matching of information to specific contexts, such as product variants in a troubleshooting scenario.
Cross-referencing Sources: AI agents can verify information by querying multiple sources or datasets to confirm consistency and reliability.
Beyond LLM-as-Judge: The Evolution of Evaluation
Early attempts to combat hallucinations involved using one LLM to evaluate another (LLM-as-judge). However, this approach suffers from limitations like positional bias, verbosity bias, and limited reasoning ability. Furthermore, it often fails to scale for large enterprises due to API rate limits and restrictions. The complexity of AI agents, where a single error can cascade, makes tracing issues back to their source challenging.
The Role of Agentic Evaluation Tools
New tools are being developed to make AI agent systems more predictable and provide necessary guardrails. While open-source libraries like RAGAS and TruLens exist, they often focus on quantitative measurements and may lack the customisation needed for specific use cases. Platforms like Galileo offer a more comprehensive approach, acting as an "AI agent co-pilot" that integrates into developer workflows. These platforms provide default guardrails and allow for custom metric creation, adapting to specific data and use cases through machine learning pipelines and human feedback.
Galileo's approach includes:
ChainPoll: An agentic AI framework that improves upon basic LLM-as-judge techniques for detecting various LLM hallucinations, offering customisation for defining hallucinations.
Luna: A suite of low-latency evaluation models with open weights, designed for high-volume requests and data privacy. These smaller models, fine-tuned for hallucination detection, show promise in outperforming larger, more general evaluation methods.
Data Quality and Future Outlook
While data quantity is important, the quality, verification, and source control of data are paramount in preventing hallucinations. Enterprises must ensure AI models are trained on diverse, accurate, and comprehensive datasets, with regular updates and cleaning. Clear, specific prompts and human oversight in high-stakes scenarios also serve as crucial safeguards. As AI agents become more prolific in 2025, the need for robust testing and evaluation tools like those offered by agentic AI solutions will be more critical than ever to ensure responsible and effective deployment.