What is promptfoo?
Latest launch MIT
Summary
promptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps.
- Category
- Agent infra
- Open source
- Yes
- License
- MIT
- Pricing
- Not stated in the sources on this page.
promptfoo describes itself as "an open-source CLI and library for evaluating and red-teaming LLM apps." According to its docs, the goal is "test-driven LLM development, not trial-and-error." The docs list two main areas: red teaming, which scans for security vulnerabilities and compliance risks, and evaluations, which test the quality and accuracy of prompts, models, and applications.
The docs say it can be used as a CLI, library, or in CI/CD, and that evaluations run locally and talk directly with the LLM. They state that it supports OpenAI, Anthropic, Azure, Google, HuggingFace, open-source models like Llama, and custom API providers. The GitHub repository reports an MIT license, and npm lists version 0.124.1 as the latest release, published 2026-10-08, with the executables pf and promptfoo. Pricing is not stated in the cited sources.
For more on this site, see the Agent infra category and the promptfoo 0.124.1 launch. Comparable tools in the category include Langfuse, Phoenix, and Opik. Read the source docs at promptfoo.dev and the GitHub repository.
promptfoo alternatives
| Tool | License | Pricing note |
|---|---|---|
promptfooThis pagepromptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps. | MIT | Not stated |
DeepEvalDeepEval is an Apache-2.0 open-source Python framework for evaluating LLM applications and agents. | Apache-2.0 | Not stated |
LangfuseOpen source platform for tracing, evaluating and improving AI agents and LLM applications. | MIT | Not stated |
PhoenixPhoenix is an Arize-maintained AI observability and evaluation tool with prompt and dataset tooling. | — | Not stated |
OpikOpen-source platform for tracing, evaluating and optimizing LLM applications and agents. | Apache-2.0 | Not stated |
Launches and updates
promptfoo 0.124.1 adds OpenAI Decisions provider support
promptfoopromptfoo 0.124.1, released October 8, 2026, adds OpenAI Decisions support, cost recording without thresholds and catalog token pricing.
Category: Agent infra
Pricing: Open source
Guides covering promptfoo
Questions
What is promptfoo?
Its docs say "promptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps."
Is promptfoo open source?
Yes. Its docs describe it as open-source, and the GitHub API and npm both report the MIT license.
How is promptfoo used?
Its docs say it can be used "as a CLI, library, or in CI/CD". The workflow is to define test cases, configure the evaluation with prompts, test cases and API providers, run it, then analyze results.