<!--
Sitemap:
- [Installation](/installation): Install the Bento daemon on macOS or Linux, then write your first pipeline to run a coding agent on a GitHub, Linear, or schedule event.
- [Upgrading](/upgrading): Version-specific steps for upgrading an existing Bento install.
- [Concepts](/concepts)
- [Build your first pipeline](/tutorials/pipeline-args)
- [Target a specific issue or PR from a URL](/tutorials/url-targeting)
- [Keep state across runs](/tutorials/pipeline-state)
- [Fire a pipeline on a schedule or on demand](/tutorials/schedule-and-fire)
- [Deploy a box to Railway](/tutorials/deploy-to-railway)
- [Operate a hosted daemon](/tutorials/operate-a-hosted-daemon)
- [Configuration](/configuration): Configure the Bento daemon with daemon.yaml: service name, port, webhooks, sandboxes, knowledge base, and environment variables.
- [Members](/members)
- [Knowledge base](/knowledge-base/)
- [Method and delivery](/knowledge-base/modes): Choose how Bento retrieves knowledge (BM25, vector, or hybrid) and how it delivers that knowledge to a coding agent.
- [Config](/knowledge-base/config)
- [MCP](/knowledge-base/mcp): Give coding agents your team knowledge over MCP. The Bento daemon serves an ask_knowledge search tool and per-file resources.
- [Pipeline configuration reference](/pipelines/config): Reference for Bento pipeline YAML: triggers, agents, skills, instructions, filters, sandboxes, structured results, and outputs.
- [Filters](/pipelines/filters): Filter GitHub, Linear, and webhook events by repository, label, and other fields so only the right Bento pipelines run.
- [Triggers](/triggers/): Start a Bento pipeline from a GitHub, Linear, webhook, schedule, or manual trigger. Compare the trigger types and when each one fires.
- [GitHub trigger](/triggers/github): Run a coding agent on GitHub pull request, issue, and other webhook events. Configure HMAC verification and event patterns.
- [Linear trigger](/triggers/linear): Run a coding agent on Linear issue and comment events. Configure the Linear webhook, signature verification, and Hookdeck relay.
- [Webhook trigger](/triggers/webhook): Fire a Bento pipeline from any HTTP POST to /webhooks/{name}, with bearer-token authentication declared in daemon.yaml.
- [Schedule trigger](/triggers/schedule)
- [Manual trigger](/triggers/manual): Fire any Bento pipeline on demand with bento trigger fire, and pass typed inputs to a pipeline that runs only on request.
- [Traces](/pipelines/traces)
- [Slack](/integrations/slack)
- [Public access](/public-access): Expose the Bento daemon to GitHub webhooks through a Cloudflare Quick Tunnel or Tailscale Funnel, and compare the two options.
- [Context engineering](/context-engineering): Assemble each coding agent prompt from personas, policies, skills, and pipeline instructions. Learn which file each instruction belongs in.
- [Best practices](/best-practices)
- [Troubleshooting](/troubleshooting)
- [Architecture](/architecture/vision)
- [Workspaces](/workspaces)
- [Authentication](/authentication)
- [Identity](/identity)
- [Security](/security)
- [References](/references)
- [Changelog](/changelog): Bento release history.
- [CLI reference](/cli/)
- [Setup](/cli/setup)
- [Secrets](/cli/secrets)
- [Lifecycle](/cli/lifecycle)
- [Sandbox image](/cli/image)
- [Sandboxes](/cli/sandbox)
- [Observability](/cli/observability)
- [Diagnostics](/cli/diagnostics)
- [Triggers](/cli/triggers)
- [Workbench](/cli/workbench)
- [Auth](/cli/auth)
- [Knowledge](/cli/knowledge)
- [Evals](/cli/evals)
- [Bento docs: run coding agents from version-controlled pipelines](/index): Bento runs coding agents from pipelines you keep in your repository. Declare the trigger, sandbox, skill, and guardrails in YAML.
- [Runtime wrapper](/architecture/runtime-wrapper)
- [Skill evolve](/architecture/skill-evolve)
-->

# Evals

List, curate, and compare the answers that pipelines collect through an [`eval:`](/pipelines/config#eval) block.

## `bento evals list`

List collected evals, filterable by status.

```bash
bento evals list --status answered
```

| Flag | Description |
|------|-------------|
| `--status <s>` | One of `requested`, `answered`, `accepted`, `rejected`, `expired`, `skipped`, `superseded` |
| `--limit / --offset` | Page through results (limit ≤ 200, default 50) |
| `--instance <name>` | Target daemon instance (default: cwd config, else the running instance) |

The footer reports `showing N of M`, where `M` is the unpaged number of evals the filters match. Raise `--limit` or step `--offset` when `N` is short of `M`.

## `bento evals accept <id>`

Accept an answered eval — the answer counts as this pipeline's signal.

```bash
bento evals accept eval_abc123
```

| Flag | Description |
|------|-------------|
| `--instance <name>` | Target daemon instance (default: cwd config, else the running instance) |

## `bento evals reject <id>`

Reject an answered eval — the answer does not count as signal.

```bash
bento evals reject eval_abc123 --reason "answered the wrong question"
```

| Flag | Description |
|------|-------------|
| `--reason <text>` | **Required.** Why this answer does not count — recorded as the curator's note |
| `--instance <name>` | Target daemon instance (default: cwd config, else the running instance) |

`accept` and `reject` both apply only to an `answered` eval. The id comes from `bento evals list`.

## `bento evals scorecard`

Compare the routes a pipeline ran on. A cohort is one (pipeline, question, version, runtime, model, effort, credential, access) tuple, and the scorecard prints one row per cohort with its answer distribution, response coverage, cost and latency.

```bash
bento evals scorecard --question "Was the review useful?" --since 7d
```

```
window:     since 2026-08-15T00:00:00.000Z
pipeline:   pr-review
question:   Was the review useful?
version:    v3
runtime:    claude
credential: anthropic
access:     write

MODEL     EFFORT  ACCEPTED  RESPONSE    COST           LATENCY      RETRIED  QUALITY
opus-5    high           5  80% (8/10)  $0.4231 (6/6)  1m12s (6/6)        0  useful 4, meh 1, harmful 0
sonnet-5  medium        2*  67% (2/3)   - (0/3)        45.5s (3/3)        1  useful 2, meh 0, harmful 0
* fewer than 3 accepted answers — not reportable
- cost is unpriced, not zero — codex and orchestrated runs report no cost
RETRIED invocations record only the winning attempt's cost, so their COST is a lower bound
showing 2 of 2 cohorts
```

A cohort key field becomes a column only when the cohorts differ on it. A field they all share is stated once above the table, so two cohorts never render as the same row.

Every figure carries the sample it was computed over. `ACCEPTED` is the number of accepted answers behind `QUALITY`, and a `*` marks a cohort below `--min-sample`. `RESPONSE` is answered over asked. `COST` and `LATENCY` count distinct invocations, not eval rows, and their `(n/d)` is how many of the cohort's invocations carried a figure at all. A `-` cost means no invocation was priced, which is unpriced rather than free.

| Flag | Description |
|------|-------------|
| `--pipeline <name>` | Pipeline name |
| `--version <v>` | Pipeline version |
| `--question <text>` | Exact eval question text, to compare routes under one question |
| `--since <t>` | Only evals requested at or after this — ISO timestamp or shorthand like `7d` |
| `--until <t>` | Only evals requested before this — ISO timestamp or shorthand like `1d` |
| `--min-sample <n>` | Accepted answers a cohort needs to be reportable (default: `3`) |
| `--limit <n>` | Cohorts to show; the footer reports how many there were (default: `50`) |
| `--instance <name>` | Target daemon instance (default: cwd config, else the running instance) |

`bento evals scorecard --json` prints the endpoint payload unrendered, including the `totalUsd` and exemplar handles the table leaves out.
