<!--
Sitemap:
- [Installation](/installation): Install the Bento daemon on macOS or Linux, then write your first pipeline to run a coding agent on a GitHub, Linear, or schedule event.
- [Upgrading](/upgrading): Version-specific steps for upgrading an existing Bento install.
- [Concepts](/concepts)
- [Build your first pipeline](/tutorials/pipeline-args)
- [Target a specific issue or PR from a URL](/tutorials/url-targeting)
- [Keep state across runs](/tutorials/pipeline-state)
- [Fire a pipeline on a schedule or on demand](/tutorials/schedule-and-fire)
- [Deploy a box to Railway](/tutorials/deploy-to-railway)
- [Operate a hosted daemon](/tutorials/operate-a-hosted-daemon)
- [Configuration](/configuration): Configure the Bento daemon with daemon.yaml: service name, port, webhooks, sandboxes, knowledge base, and environment variables.
- [Members](/members)
- [Knowledge base](/knowledge-base/)
- [Method and delivery](/knowledge-base/modes): Choose how Bento retrieves knowledge (BM25, vector, or hybrid) and how it delivers that knowledge to a coding agent.
- [Config](/knowledge-base/config)
- [MCP](/knowledge-base/mcp): Give coding agents your team knowledge over MCP. The Bento daemon serves an ask_knowledge search tool and per-file resources.
- [Pipeline configuration reference](/pipelines/config): Reference for Bento pipeline YAML: triggers, agents, skills, instructions, filters, sandboxes, structured results, and outputs.
- [Filters](/pipelines/filters): Filter GitHub, Linear, and webhook events by repository, label, and other fields so only the right Bento pipelines run.
- [Triggers](/triggers/): Start a Bento pipeline from a GitHub, Linear, webhook, schedule, or manual trigger. Compare the trigger types and when each one fires.
- [GitHub trigger](/triggers/github): Run a coding agent on GitHub pull request, issue, and other webhook events. Configure HMAC verification and event patterns.
- [Linear trigger](/triggers/linear): Run a coding agent on Linear issue and comment events. Configure the Linear webhook, signature verification, and Hookdeck relay.
- [Webhook trigger](/triggers/webhook): Fire a Bento pipeline from any HTTP POST to /webhooks/{name}, with bearer-token authentication declared in daemon.yaml.
- [Schedule trigger](/triggers/schedule)
- [Manual trigger](/triggers/manual): Fire any Bento pipeline on demand with bento trigger fire, and pass typed inputs to a pipeline that runs only on request.
- [Traces](/pipelines/traces)
- [Slack](/integrations/slack)
- [Public access](/public-access): Expose the Bento daemon to GitHub webhooks through a Cloudflare Quick Tunnel or Tailscale Funnel, and compare the two options.
- [Context engineering](/context-engineering): Assemble each coding agent prompt from personas, policies, skills, and pipeline instructions. Learn which file each instruction belongs in.
- [Best practices](/best-practices)
- [Troubleshooting](/troubleshooting)
- [Architecture](/architecture/vision)
- [Workspaces](/workspaces)
- [Authentication](/authentication)
- [Identity](/identity)
- [Security](/security)
- [References](/references)
- [Changelog](/changelog): Bento release history.
- [CLI reference](/cli/)
- [Setup](/cli/setup)
- [Secrets](/cli/secrets)
- [Lifecycle](/cli/lifecycle)
- [Sandbox image](/cli/image)
- [Sandboxes](/cli/sandbox)
- [Observability](/cli/observability)
- [Diagnostics](/cli/diagnostics)
- [Triggers](/cli/triggers)
- [Workbench](/cli/workbench)
- [Auth](/cli/auth)
- [Knowledge](/cli/knowledge)
- [Evals](/cli/evals)
- [Bento docs: run coding agents from version-controlled pipelines](/index): Bento runs coding agents from pipelines you keep in your repository. Declare the trigger, sandbox, skill, and guardrails in YAML.
- [Runtime wrapper](/architecture/runtime-wrapper)
- [Skill evolve](/architecture/skill-evolve)
-->

# Runtime wrapper

The runtime wrapper collects metrics from the pi, claude, and codex runtimes in one format.

Each runtime emits structured JSONL in non-interactive mode, but the event schemas differ. The runtime wrapper converts these events to the common `SessionResult` type in `packages/core/src/runtime.ts`. The daemon, skill-evolve, and other consumers read this common format.

## Interfaces

See `packages/core/src/runtime.ts` for the full TypeScript types.

The wrapper uses these types:

* **`SessionResult`** — the main output: usage, cost, tool calls, file operations, turns, status
* **`RuntimeAdapter`** — interface each runtime implements: `parse(lines, meta) → SessionResult`
* **`TokenUsage`** / **`CostBreakdown`** — normalized token and cost accounting
* **`ToolCall`** — name, args, result, status, duration
* **`Turn`** — one model round-trip with per-turn usage breakdown

## Invocation modes

| Runtime | Command | Output mode flag | Event format |
|---|---|---|---|
| pi | `pi -p` | `--mode json` | Newline-delimited JSON events |
| claude | `claude -p` | `--output-format json` (summary) or `--output-format stream-json` (streaming) | Single JSON object or newline-delimited JSON events |
| codex | `codex exec --experimental-json` | Built-in | Newline-delimited JSON events |

## Event schemas

### Pi (`--mode json`)

Pi emits lifecycle events and usage for each message:

```
{type: "session", version, id, timestamp, cwd}
{type: "agent_start"}
{type: "turn_start"}
{type: "message_start", message: {role: "user", content: [...]}}
{type: "message_end",   message: {role: "user", content: [...]}}
{type: "message_start", message: {role: "assistant", content: [...], model, usage, stopReason}}
  ... message_update events (text deltas) ...
{type: "message_end",   message: {role: "assistant", ..., usage: {input, output, cacheRead, cacheWrite, totalTokens, cost: {input, output, cacheRead, cacheWrite, total}}}}
{type: "tool_execution_start", toolCallId, toolName, args}
{type: "tool_execution_end",   toolCallId, toolName, result, isError}
{type: "message_start", message: {role: "toolResult", content: [...]}}
{type: "message_end",   message: {role: "toolResult", ...}}
{type: "turn_end", message: {role: "assistant", ..., usage}, toolResults: [...]}
{type: "agent_end", messages: [...]}
```

**Key fields for metric extraction:**

| Metric | Location |
|---|---|
| Session ID | `session.id` |
| Model | `message.model` (on assistant messages) |
| Input tokens | `message.usage.input` |
| Output tokens | `message.usage.output` |
| Cache read tokens | `message.usage.cacheRead` |
| Cache write tokens | `message.usage.cacheWrite` |
| Total tokens | `message.usage.totalTokens` |
| Cost (total) | `message.usage.cost.total` |
| Cost (breakdown) | `message.usage.cost.{input,output,cacheRead,cacheWrite}` |
| Stop reason | `message.stopReason` |
| Tool name | `tool_execution_start.toolName` |
| Tool args | `tool_execution_start.args` |
| Tool result | `tool_execution_end.result` |
| Tool error | `tool_execution_end.isError` |
| Turns | Count `turn_start` / `turn_end` pairs |
| Text output | Concatenate `text` content blocks from assistant messages |

### Claude (`--output-format json`)

Claude's JSON mode returns a single summary object after completion:

```json
{
  "type": "result",
  "subtype": "success",
  "is_error": false,
  "duration_ms": 2956,
  "duration_api_ms": 2890,
  "num_turns": 1,
  "result": "Hello!",
  "stop_reason": "end_turn",
  "session_id": "...",
  "total_cost_usd": 0.07,
  "usage": {
    "input_tokens": 5,
    "output_tokens": 8,
    "cache_creation_input_tokens": 9984,
    "cache_read_input_tokens": 14857,
    "server_tool_use": {"web_search_requests": 0}
  },
  "modelUsage": {
    "claude-opus-4-7[1m]": {
      "inputTokens": 5,
      "outputTokens": 8,
      "cacheReadInputTokens": 14857,
      "cacheCreationInputTokens": 9984,
      "costUSD": 0.07
    }
  }
}
```

**Key fields for metric extraction:**

| Metric | Location |
|---|---|
| Session ID | `session_id` |
| Model | First key in `modelUsage` |
| Input tokens | `usage.input_tokens` |
| Output tokens | `usage.output_tokens` |
| Cache read tokens | `usage.cache_read_input_tokens` |
| Cache write tokens | `usage.cache_creation_input_tokens` |
| Cost (total) | `total_cost_usd` |
| Cost (per-model) | `modelUsage[model].costUSD` |
| Duration | `duration_ms` |
| API duration | `duration_api_ms` |
| Turns | `num_turns` |
| Stop reason | `stop_reason` |
| Text output | `result` |
| Tool calls | Not in JSON mode — need `stream-json` for per-tool detail |

**Note:** Claude's `--output-format json` gives aggregated metrics only. For per-tool-call detail (what meta-harness uses), `--output-format stream-json` emits per-event JSONL with `assistant` and `user` (tool\_result) events, similar to the Anthropic Messages API format.

### Claude (`--output-format stream-json`)

Streaming mode emits events matching the Anthropic Messages API shape:

```
{type: "system", ...}
{type: "assistant", message: {content: [{type: "text", text: "..."} | {type: "tool_use", id, name, input}], usage: {input_tokens, output_tokens, ...}}}
{type: "user", message: {content: [{type: "tool_result", tool_use_id, content, is_error}]}}
{type: "result", session_id, total_cost_usd, usage: {...}}
```

Each assistant message contains individual `text` or `tool_use` blocks. The next `user` message carries the `tool_result`. The final `result` event contains the summary.

### Codex (`exec --experimental-json`)

Codex uses a thread/item model via its TypeScript SDK:

```
{type: "thread.started", thread_id: "..."}
{type: "turn.started"}
{type: "item.started", item: {id, type: "agent_message" | "command_execution" | "file_change" | "mcp_tool_call" | ...}}
{type: "item.updated", item: {...}}
{type: "item.completed", item: {...}}
{type: "turn.completed", usage: {input_tokens, cached_input_tokens, output_tokens, reasoning_output_tokens}}
```

**Item types (from codex SDK `items.ts`):**

| Item type | Description | Key fields |
|---|---|---|
| `agent_message` | Text response | `text` |
| `reasoning` | Chain-of-thought | `text` |
| `command_execution` | Shell command | `command`, `aggregated_output`, `exit_code`, `status` |
| `file_change` | File patch | `changes: [{path, kind}]`, `status` |
| `mcp_tool_call` | MCP tool invocation | `server`, `tool`, `arguments`, `result`, `error`, `status` |
| `web_search` | Web search | `query` |
| `todo_list` | Agent's task list | `items: [{text, completed}]` |
| `error` | Non-fatal error | `message` |

**Key fields for metric extraction:**

| Metric | Location |
|---|---|
| Session ID | `thread.started.thread_id` |
| Model | Not in events (passed as config) |
| Input tokens | `turn.completed.usage.input_tokens` |
| Output tokens | `turn.completed.usage.output_tokens` |
| Cache tokens | `turn.completed.usage.cached_input_tokens` |
| Reasoning tokens | `turn.completed.usage.reasoning_output_tokens` |
| Cost | Not in events. Compute it from token counts and pricing |
| Turns | Count `turn.started` / `turn.completed` pairs |
| Stop reason | Infer from last event type |
| Tool calls | `item.completed` where `item.type === "command_execution"` or `"mcp_tool_call"` |
| File changes | `item.completed` where `item.type === "file_change"` |
| Text output | Concatenate `item.completed` where `item.type === "agent_message"` |

**Codex gaps:**

* No cost data in events. Compute it externally from token counts and model pricing.
* No explicit model field in events — passed as config, not echoed back
* `reasoning_output_tokens` is a distinct field (OpenAI counts reasoning separately)

## Field mapping summary

How each runtime field maps to the common `SessionResult`:

| SessionResult field | Pi | Claude (json) | Codex |
|---|---|---|---|
| `sessionId` | `session.id` | `session_id` | `thread.started.thread_id` |
| `model` | `message.model` | `modelUsage` key | config input |
| `status` | infer from `agent_end` | `subtype` | infer from exit |
| `exitCode` | process exit | process exit | process exit |
| `usage.input` | Σ `message.usage.input` | `usage.input_tokens` | Σ `turn.completed.usage.input_tokens` |
| `usage.output` | Σ `message.usage.output` | `usage.output_tokens` | Σ `turn.completed.usage.output_tokens` |
| `usage.cacheRead` | Σ `message.usage.cacheRead` | `usage.cache_read_input_tokens` | Σ `turn.completed.usage.cached_input_tokens` |
| `usage.cacheWrite` | Σ `message.usage.cacheWrite` | `usage.cache_creation_input_tokens` | — |
| `usage.reasoning` | — | — | Σ `turn.completed.usage.reasoning_output_tokens` |
| `cost.total` | last `message.usage.cost.total` | `total_cost_usd` | computed |
| `durationMs` | compute from timestamps | `duration_ms` | compute from timestamps |
| `turns` | count `turn_end` events | `num_turns` | count `turn.completed` events |
| `toolCalls` | `tool_execution_{start,end}` | stream-json `tool_use` blocks | `command_execution` + `mcp_tool_call` items |
| `fileChanges` | infer from tool args | infer from tool args | `file_change` items (native) |
| `text` | assistant text blocks | `result` | `agent_message` items |

## Implementation plan

Each runtime gets an adapter that implements `RuntimeAdapter`:

```
packages/core/src/
  runtime.ts          ← interfaces (done)
  adapters/
    pi.ts             ← parses pi --mode json output
    claude.ts         ← parses claude --output-format json (or stream-json)
    codex.ts          ← parses codex exec --experimental-json output
```

The daemon's `executePipeline` currently does raw `Bun.spawn` → `stdout.log`. It would instead:

1. Spawn the runtime with the appropriate JSON output flag
2. Collect JSONL lines
3. Pass them through the runtime's adapter → `SessionResult`
4. Store the `SessionResult` for querying, logging, and skill-evolve

## References

* [Pi source](https://github.com/nichochar/pi) — `--mode json` event format
* [Claude Code docs](https://docs.anthropic.com/en/docs/claude-code) — `--output-format` options
* [Codex SDK](https://github.com/openai/codex/tree/main/sdk/typescript) — `events.ts` and `items.ts` type definitions
* [Meta-Harness claude\_wrapper.py](https://github.com/stanford-iris-lab/meta-harness) — reference for stream-json parsing
* `packages/core/src/runtime.ts` — the interfaces
* [`docs/skill-evolve.md`](/architecture/skill-evolve) — primary consumer of session metrics
