Guidance
Control LM outputs. Reduce latency and cost.
Get the LM output you need with a single prompt
Guidance is a proven open-source Python library (opens in new tab) for controlling outputs of any language model (LM). In only one API call, developers express in Python the precise programmatic constraints the model must follow for structured output in JSON, Python, HTML, SQL, whatever the use case requires.
The result: 100% guaranteed output structure—with 30–50% reduction in latency and costs.
Steering the model token by token
Guidance works with most open-source LMs that can be hosted locally as well as with Azure-hosted Phi 3.5 (opens in new tab). Fundamentally different from conventional prompting techniques, Guidance enforces constraints by steering the model token by token in the inference layer to deliver accurate outputs. No need for expensive retries or fine-tuning. The Guidance advantage includes:
Cost savings
Save significantly on runtime, while accelerating inference. In contrast to prompt chaining, Guidance programs are a single API call. A Guidance program batches—instead of generating—any additional text that is added by the user as execution unrolls. See example (opens in new tab).
Flexibility
Get structured LM output in any specified format. Guidance is uniquely flexible compared with alternative technologies, enabling developers to constrain outputs to JSON, Python, HTML, SQL, whatever is required. Enforce other constraints with select (opens in new tab) (i.e., a set of options), regular expressions (opens in new tab), and context-free grammars (opens in new tab).
Elegant workflow
Write constraints in pure Python and Guidance enforces syntax for a smooth developer workflow. The Guidance interface and library functionality are designed to reduce developer pain. Call and deploy tools easily (opens in new tab). Access rich templates with f-strings and prebuilt components (e.g., substrings). More … (opens in new tab)
Building LM-powered apps?
Learn in this video how Guidance works to give you unprecedented control of LM outputs.
Developers love Guidance. With more than 18K GitHub stars, Guidance has a thriving community of developers and researchers.