---
title: "The early start"
description: "Speculative brain dispatch: the gateway starts the brain on the stable partial transcript before the operator has finished talking, holding output and tools; it promotes the answer when the final matches, and forks the pi session and re-dispatches the committed text when it differs."
type: "living"
source: "docs/speculation.md"
verified: "2026-11-14"
group: "The voice path"
---

> **Living doc**, checked against the code 2026-11-14

Speculation overlaps the end-of-utterance wait with the Brain's first token. Once the armed interim transcript is
stable for `speculation.trigger_silence_ms`, the gateway sends it to Pi with output and tools held. When the STT final
commits, a matching final promotes the answer; a different final forks the Pi session at the anchored entry and
re-dispatches the committed text. It is off by default.

## Mechanism

One speculative turn, from the stable interim to the commit's three outcomes:

```mermaid
---
config:
  sequence:
    wrap: true
    width: 110
    actorMargin: 10
    messageFontSize: 13
---
sequenceDiagram
    participant STT as STT (interim, final)
    participant GW as Gateway (ws + SpeculationMachine)
    participant Gate as Gate file (VULPINE_SPECULATION_GATE)
    participant Pi as Pi brain
    participant Ext as fox9-refork extension
    participant TTS as TTS
    Note over GW: Armed capture open, 100 ms tick
    STT->>GW: interim stable for trigger_silence_ms
    GW->>Gate: create (hold output and tools)
    GW->>Pi: speculative prompt (no thinking phase)
    GW->>Pi: get_entries (anchor the user entry)
    Pi-->>GW: deltas held in the session buffer
    Pi->>Ext: tool_call
    Ext->>Gate: poll, park while it exists
    STT->>GW: final commits
    alt final is spoken-equivalent (promote)
        GW->>Gate: remove
        Ext-->>Pi: parked tool_call proceeds
        GW->>GW: phase thinking, replay the buffer
        GW->>TTS: released output
    else final differs (refork)
        GW->>Pi: abort (parked tool_call dies)
        GW->>Pi: fork at the anchored entry
        Pi-->>GW: pi_fork_complete
        GW->>Pi: committed prompt, promoted at once
        GW->>Gate: remove
    else refork bound hit, no anchor, or unsaved session
        GW->>Pi: abort or retire speculation
        GW->>Pi: normal committed dispatch
    end
```

- **Trigger**: while an armed capture is open, a 100 ms session tick watches the interim transcript. Once it has
  been stable for `speculation.trigger_silence_ms` (silence alone decides), the gateway sends it to Pi as a
  speculative prompt. The dispatch is phase-silent: no thinking chip, all brain output held in a session buffer, TTS
  and side effects withheld.
- **Promote** (final matches the speculated text): the speculative generation is the turn. The thinking phase emits
  and the buffer replays through the normal processing path, with no second prompt.
- **Refork** (final differs): the gateway aborts the generation, forks the Pi session with `fork` at the speculative
  user-message entry (anchored from `get_entries` → `pi_session_entries`) and re-dispatches the committed text as the
  real turn. The rewound prefix is byte-identical, so provider prompt caching absorbs most of the cost.
- **Cancel**: a disarm, a brain error or the `max_reforks_per_turn` bound retires speculation for the turn, and the
  committed path runs normally. A final that resolves to a tier-1/2 gateway verb also cancels it
  (`speculation_cancelled {reason: gateway_verb}`); the verb routes with its `verb_tier` row.
- **Hold**: with `[invocation] enrolled_pcm_gate` on, speculation does not dispatch while the request's speaker vote
  is open (`speculation_held {reason: enrolled_gate_pending}`, once per request). An admitted commit dispatches on
  the committed path, so a refused request's text never reaches the brain, and a refused verb writes
  `verb_refused {verb}` instead of its `verb_tier` row. A request the gate cannot judge (no enrollment from its wake)
  is held the same way and refused at its routing frame (`enrolled_gate_request \{decision: refuse, reason: unjudged,
  fresh_ms\}`).
- **Calls**: a call loadout (`dialogue_mode = true`: Vigil) never arms, so its requests get no speculative start by
  default. With the loadout's `[invocation] call_speculation = true` (set by the `vigil` loadout), an open
  call window is a speculation window: the partials gate opens for the call (`partials_gate_opened {reason: call}`)
  and closes at its end, and the trigger runs as above. A final ends the utterance: its interim is dropped, and a
  running speculation gets the disarm grace to meet its commit. Words heard over a reply never speculate. A call's
  request with no head start writes `speculation_skipped {reason, window: call}` before its commit, with reason
  `call_speculation_off`, `speculation_disabled`, `reply_in_flight`, `in_call`, `playback_latch` or `final_first`.
  - While the partials gate is open, the segmenter runs its close tick at speech end, so the ended segment's whole
    words arrive as a `closed` partial and the trigger fires at once, about 250 ms ahead of the final.
  - The call trigger holds while the call take would refuse the line at its final: `in_call` (a phone call is live)
    and `playback_latch` (a reply ended under 3 s ago). A head start never hands the brain words the take then
    refuses.
- **Tool gate**: the `fox9-refork` Pi extension (`crates/voice_gateway/extensions/fox9-refork`) parks every Pi
  `tool_call` while the gateway's gate file exists. The path comes from `VULPINE_SPECULATION_GATE` (the extension
  still reads the old `FOX9_SPECULATION_GATE`), one per gateway process, and the file follows the speculative hold. Promotion releases parked calls, and an uncleared gate refuses
  at 10 s. It fails closed, so a truncated instruction never runs tools.


## Configuration

`[speculation]` is a section of the gateway config (`docs/reference/config.md`). The defaults:

```toml
[speculation]
enabled = false             # true dispatches speculatively
trigger_silence_ms = 500    # longer: fewer reforks, a later head start
max_reforks_per_turn = 3
```

The loadout must carry `builtin:fox9-refork` in `extensions`; the operator loadout does. The Admin's audio section
shows the three `speculation.*` fields.



## Trace vocabulary

`speculation_triggered`, `speculation_brain_dispatch`,
`speculation_entry_observed`, `speculation_promoted`,
`speculation_refork_started`, `speculation_refork_without_entry`,
`speculation_buffer_released`, `speculation_cancelled`,
`speculation_retired_on_brain_error`, `speculation_dispatch_failed`,
`speculation_skipped_brain_backoff`, `speculation_skipped`,
`speculation_fork_timeout`, `speculation_refork_skipped_unsaved_session`,
`speculation_cancel_restarted_brain`, `speculation_held`.

