Guidance is a programming language for controlling large language models through structured prompts and constrained generation.
The tool addresses the challenge of obtaining reliable, well-formatted output from language models while minimizing latency and cost. It works by allowing developers to interleave generation with control flow—conditionals, loops, and tool calls—while enforcing output constraints through regular expressions, token selection, and context-free grammars. This means you can guarantee that a model's output conforms to a specific format or vocabulary before it is generated, rather than hoping the model complies after the fact.
Guidance suits projects where output structure matters: parsing structured data, generating code, building multi-turn interactions, or ensuring compliance with a known schema. It works across multiple backends including Transformers, llama.cpp, and OpenAI's API, so you can apply the same Guidance code to different models. The tool provides a Pythonic interface with decorators for composing reusable generation functions, and includes a Mock model for testing constraints offline without making API calls. This is particularly valuable when iterating on grammars or validating logic before running against a live model.
The project maintains a substantial base of real-world adopters, with nearly all open issues coming from outside users rather than the core team. Maintainers typically respond to new issues and pull requests within a few days. Work in the issue tracker centers on bug fixes and enhancements, indicating active refinement of existing functionality.
