garak is a command-line vulnerability scanner for large language models that probes for weaknesses including hallucination, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks. The tool combines static, dynamic, and adaptive probes to explore failure modes in LLMs and dialog systems, operating similarly to network security tools like nmap or Metasploit Framework but targeting generative AI systems instead.
The tool addresses the need to systematically assess LLM safety and robustness before deployment. Rather than relying on manual testing, garak automates the discovery of vulnerabilities by running curated probe sets against a target model. Users can list available probes, select specific ones to run, and configure which vulnerability detectors to apply, giving them control over the scope and depth of assessment.
Developers should adopt garak if they need to evaluate LLM safety as part of their development or deployment pipeline. It suits projects where understanding model failure modes is critical, from research environments to production systems. The tool supports a broad range of LLM backends including Hugging Face Hub models, OpenAI API, AWS Bedrock, Replicate, LiteLLM, and REST-accessible models, as well as local GGUF models, making it flexible across different infrastructure choices. Installation is straightforward via pip or from source with conda, and the project provides documentation and community support through Discord.
Development activity shows consistent engagement with the codebase through regular updates and maintenance. The project maintains active communication channels and documentation resources for users. The tool continues to expand its probe coverage and detector capabilities to address emerging vulnerability classes in LLMs.
