promptfoo 0.124.1 adds OpenAI Decisions provider support
Updated promptfoo
Summary
promptfoo 0.124.1, released October 8, 2026, adds OpenAI Decisions support, cost recording without thresholds and catalog token pricing.
- Tool
- promptfoo
- Category
- Agent infra
- Date
- Open source
- Yes
- License
- MIT
- Pricing
- Not stated in the sources we cited.
promptfoo published release 0.124.1 on GitHub on October 8, 2026. According to the release notes, it adds three features around assertions and providers, plus a set of bug fixes in assertions, red teaming and eval. The repository describes the project as a tool to test prompts, agents and RAGs, with red teaming and vulnerability scanning for AI.
Release facts
| Fact | Detail |
|---|---|
| Product | promptfoo |
| Version | 0.124.1 (tag 0.124.1) |
| Release date | October 8, 2026 |
| License | MIT, as reported by the GitHub API |
| Where to get it | GitHub release and repository |
What changed
- OpenAI Decisions support: the release notes list a new provider feature, "add OpenAI Decisions support".
- Cost assertions without a threshold: assertions can now record cost without a threshold.
- Catalog pricing: providers expose catalog text-token pricing.
- Assertion grading fixes: the notes list fixes that preserve async expression grading, keyword-property grading and regex operands after yield.
- Red team fixes: missing target output is now reported as errors, dataset plugin rows are sampled uniformly, and Meta stops on invalid JSON requests.
- Performance: eval indexes extracted tests when filtering results.
Questions
Does this release add a new provider? The release notes list "add OpenAI Decisions support" under providers. They do not give further detail on what it covers.
Is there a security-related fix? Yes. According to the release notes, dependency updates patch Sharp and Codex SDK security vulnerabilities. The notes do not describe the vulnerabilities themselves.
What is the license? The repository license reported by the GitHub API is MIT. The cited sources do not state pricing for any hosted service.
See the promptfoo tool hub, the Agent infra category, and the related Langfuse v4.57.0 launch for more.
Alternatives
| Tool | License | Pricing note |
|---|---|---|
promptfooThis launchpromptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps. | MIT | Not stated |
LangfuseOpen source platform for tracing, evaluating and improving AI agents and LLM applications. | MIT | Not stated |
OpikOpen-source platform for tracing, evaluating and optimizing LLM applications and agents. | Apache-2.0 | Not stated |
PhoenixPhoenix is an Arize-maintained AI observability and evaluation tool with prompt and dataset tooling. | — | Not stated |
Questions
What is promptfoo?
Its docs say "promptfoo is an open-source CLI and library for evaluating and red-teaming LLM apps."
Is promptfoo open source?
Yes. Its docs describe it as open-source, and the GitHub API and npm both report the MIT license.
How is promptfoo used?
Its docs say it can be used "as a CLI, library, or in CI/CD". The workflow is to define test cases, configure the evaluation with prompts, test cases and API providers, run it, then analyze results.
Filed on Agent Radar. Claims in this note are limited to the sources above.