Detect LLM hallucinations using uncertainty quantification techniques with UQLM

UQLM (uncertainty quantification for language models) is an open source Python package for generation time, zero-resource hallucination detection. It leverages state-of-the-art uncertainty quantification (UQ) techniques from the academic literature to compute response-level confidence scores based on response consistency (in multiple responses to the same prompt), token probabilities, LLM-as-a-Judge, or ensembles of these.

UQLM aims to democratize state-of-the-art UQ techniques. By integrating generation and UQ-scoring processes with a user-friendly API, UQLM makes these methods accessible to non-specialized practitioners with minimal engineering effort.

Check it out, share feedback, and contribute if you are interested!

Link: https://github.com/cvs-health/uqlm