During my internship, I worked on building a hallucination detection system for an LLM-based information extraction pipeline. The goal was to determine whether extracted information was present in the original webpage text, even when not written verbatim. By combining fuzzy matching with an LLM-based semantic verification system, we developed a hybrid approach that significantly improved our ability to distinguish true extractions from hallucinations.

The Problem & The Mission

Large language models are incredibly powerful at synthesising structured information from unstructured text. However, they introduce a new problem: hallucinations. When an LLM extracts structured information from a webpage, it may generate information that is plausible and domain-relevant but not actually present in the source text. This is because an LLM does not extract as a programmatic parser would; it estimates what the output should be based on the input text and guiding cues in its prompt, rather than identifying where the relevant text is based on rigorous rules.

In a data extraction pipeline, this creates a difficult verification problem. If we extract fields like a person’s name, role, or date of birth from a webpage, we need to verify that this information actually appears on the page. Traditionally, this is done using fuzzy string matching, which checks whether the extracted text appears in the source text with minor variations, such as spelling differences.

Fuzzy matching works very well when the extraction appears verbatim or nearly verbatim. However, it struggles when the information is semantically present but phrased differently. For example, a page might say:

“John Smith was appointed as Minister of Finance. He was born on Christmas Day, 1981.”

But the extraction might be structured as:

  • Name: John Smith
  • Role: Finance Minister
  • Date of Birth: 25-12-1981

These are semantically correct extractions, but fuzzy matching may fail to find exact string matches. On the other hand, if an LLM produces a completely incorrect extraction, fuzzy matching alone is not always sufficient to confidently classify it as a hallucination due to this same limitation.

The goal of this project was, therefore, to build a hallucination checker that could determine whether extracted information was truly present in the source text, even when not written verbatim.

The Pipeline

The hallucination checker was designed as part of a larger agentic extraction pipeline. At a high level, the pipeline looks like this:

In this setup, fuzzy matching acts as a fast, first-pass filter. Only the cases where fuzzy matching fails are passed to the hallucination checker, which uses an LLM to determine whether the information is semantically present in the source text.

This hybrid approach is both more accurate and more cost-effective than running an LLM on every extraction. Since the majority of extractions do not feature hallucinations, using fuzzy matching to quickly identify verbatim matches significantly reduces computational cost.

Dataset Design: The Hardest Part

The most challenging part of the project was not the model or the prompts, it was designing the dataset to properly evaluate hallucination detection.

To evaluate the system properly, we needed data in four categories:

  1. Exact Match – The extracted information appears verbatim on the page (where fuzzy matching works extremely well).
  2. Semantic Match – The information is present on the page but phrased differently.
  3. Domain-Related but Incorrect – The extraction is related to the page's topic but not actually present on that page (to check if the LLM can distinguish between information presence and general domain content).
  4. Complete Hallucination – The extraction is completely unrelated to the page.

Each category represents a different failure mode or success case for extraction verification systems, and building each required a different approach:

  • Exact Match data came from an existing labeled dataset.
  • Complete Hallucination examples were generated using random or unrelated text.
  • Topically Related errors were created by shuffling extractions between similar websites so that the information domain remained similar, but the data was incorrect for that page.
  • Semantic Match data was the hardest to create. To generate this dataset, we used a state-of-the-art LLM to modify existing correct extractions in realistic ways, such as using: Synonyms, Translations, Abbreviations, Name shortening, Misspellings, and Reworded roles. This allowed us to simulate realistic, non-verbatim extractions that are semantically correct but difficult for fuzzy matching to detect.

This dataset design ended up being one of the most important parts of the project because it allowed us to evaluate different verification approaches across multiple realistic scenarios.

The Hallucination Checker

The hallucination checker itself works by prompting an LLM with:

  • The webpage content
  • The extracted fields
  • Instructions to determine whether each field is semantically present in the text

Instead of asking the model to extract information, we ask it to verify whether information exists, which is a much more constrained and reliable task. This reframes the problem from generation to verification, which significantly reduces the risk of hallucination.

Results & Findings

The below figure shows how various Google Gemini models and fuzzy matching perform in predicting presence of text from the four dataset categories.

One of the more surprising findings was that lightweight language models performed worse than fuzzy matching. A plausible explanation could be that the model's tokenisation process fragmented strings with minor variations in a way that obscured the simple edit-distance patterns the algorithmic method could easily detect.

However, top-tier LLMs significantly outperformed fuzzy matching on semantic matches while performing comparably in all other categories. This means that powerful LLMs are particularly useful for detecting when information is present but phrased differently.

The main conclusion from the experiments was that the best approach is not to replace fuzzy matching but to augment it. The hybrid system works as follows:

  1. Use fuzzy matching for fast exact and near-exact matches.
  2. Use an LLM only for cases where fuzzy matching fails.

This improves semantic detection while keeping costs manageable and leveraging the efficiency of fuzzy matching. This hybrid approach significantly improved the system’s ability to correctly identify semantically true extractions while still filtering out hallucinations. Using the best-performing LLM as a secondary evaluator, the overall accuracy of predicting text presence was 15% higher than using fuzzy matching alone.

My Personal Takeaway

This internship was my first experience working in a real, production-focused environment rather than a purely academic research setting. One of the biggest differences I noticed was in the pace and the decision-making process. In academic research, you often explore every possible direction. In a production environment, if something can be proven to work and delivers value, you ship it.

I also worked with tools and technologies that were new to me, such as HDF5 datasets and asynchronous processing, which are extremely useful when working with large-scale data pipelines and LLM systems. I am very excited to apply what I learned here to my own research workflow.

One of the most valuable technical lessons I learned was the importance of choosing the right tools and architectures for each specific problem rather than trying to use the same approach everywhere. Modern machine learning and data engineering ecosystems provide many specialised tools, and knowing when to use them is a key skill.

On the non-technical side, I significantly improved my ability to communicate technical results to a mixed audience, including both technical and non-technical stakeholders. In a regulated environment, objective reporting and clear communication are extremely important, and learning how to present results clearly and concisely was one of the most valuable parts of the experience.

Overall, this project gave me experience working on real-world data pipelines, designing evaluation frameworks, building datasets from scratch, and contributing to systems that are part of a larger production workflow. It was a very different experience from university projects, and it gave me a much better understanding of how machine learning and data systems are built and deployed in industry.

Conclusion

Hallucination detection is becoming increasingly important as more systems move toward LLM-based information extraction. While traditional methods like fuzzy matching are still very useful, they struggle with semantically correct but non-verbatim information.

This project showed that a hybrid approach combining fuzzy matching and LLM-based semantic verification provides a practical and effective solution. Just as importantly, the project highlighted how crucial dataset design and evaluation methodology are when building and testing LLM systems.

From a personal perspective, this internship was an opportunity to work on real-world machine learning systems, learn new tools and technologies, and understand how research ideas are translated into production environments. It was a challenging but extremely rewarding experience, and it gave me valuable insights into both machine learning engineering and working in a fast-paced, production setting.