Hallucination Leaderboard is a benchmarking tool that ranks large language models by their tendency to introduce hallucinations when summarizing documents.
The tool addresses the problem of evaluating whether LLMs faithfully summarize source material or generate false information. It uses Vectara's Hallucination Evaluation Model, known as HHEM, to measure how often different models produce hallucinations during summarization tasks. The leaderboard computes performance metrics across multiple LLMs and presents results in a comparable format, allowing developers and researchers to see which models perform better at staying faithful to their source documents.
Developers should use this leaderboard when selecting an LLM for applications where hallucination risk matters, particularly in summarization workflows. It suits projects that require high-fidelity document processing, such as legal document analysis, medical record summarization, or research paper condensing. The tool provides an interactive interface hosted on Hugging Face for exploring results, making it accessible for quick model comparisons without requiring local setup.
The project maintains historical versions of the leaderboard as its evaluation model and tested LLMs evolve, allowing users to track how performance has changed across different model iterations. The leaderboard is updated regularly as both the underlying evaluation model and the LLMs being tested improve over time.
