Skip to content
Vulpine
Esc
↑↓navigate↵open⌘Jpreview
On this page

Hardpoint: text to speech

The tts hardpoint slot 1.0: the built-in qwen3-worker, what a user speech output does (a Wyoming TTS server on loopback: synthesize in, audio-start, audio-chunk, audio-stop out, 16-bit PCM), its probe (one fixed sentence rendered to bytes, never played, timed to first audio against first_text_to_first_audio's p90) and its fallback; catalog/hardpoints/wyoming-kokoro is the first-party entry, bound with [hardpoints] tts = "wyoming-kokoro".

Living doc, checked against the code 2026-11-14

hardpoint.tts.v1 — the tts slot: what a speech-output implementation must do to carry a session’s voice

  • Status: 1.0, built. The slot’s interface contract under hardpoint.v1 and ADR 0059 §2. A minor version only adds.
  • Owner: the gateway, at session bind. Budget leg: first_text_to_first_audio in turn-budget.v1 (p50 400 ms, p90 850 ms).

Built-ins

id what it is
qwen3-worker the default, the TtsAdapter: the gateway’s own worker streaming Qwen3-TTS from the TTS server just models installs (VULPINE_TTS_HTTP_URL, which just bootstrap writes), or without that server the Python worker in runtime/qwen3_tts_worker under an immutable VULPINE_QWEN_UV

A built-in qualifies by its worker’s ready receipt at start, so it costs nothing at bind.

What an implementation does

A tts implementation speaks one reply segment at a time. For each request it answers 16-bit little-endian PCM at the rate it names and then says it is done, or it answers an error. The gateway turns that into the Qwen worker’s frames: audio_start, audio_chunk with its output PCM, audio_done, error. Playback, barge and the turn budget see no difference. Requests render one at a time, in order, because the session has one playback queue. A cancel drops every render in flight.

Transports this build drives

transport wire endpoint
wyoming One TCP connection per request. The gateway sends synthesize {text}; the server answers audio-start {rate, width, channels}, any number of audio-chunk (PCM payload) and audio-stop, or error {text, code} (its words are carried, bounded, into the failure). width must be 2. tcp://<loopback ip>:<port>
wyoming, streaming text When the server’s describe answers info with supports_synthesize_streaming: true on a TTS program, a reply is one connection: synthesize-start, then one synthesize-chunk {text} per sentence as the reply’s text is cut (each ends in a blank line, which Home Assistant’s sentence detector takes as a sentence end at once), then the compatibility synthesize {text} and synthesize-stop when the reply’s text is complete. The server answers each sentence as its own audio-start .. audio-stop, or the whole stream as one span (xn-ptts: its audio then rides the reply’s first request), and ends with synthesize-stopped. A stream with no event and no new text for 1 s is ended the same way, inside a span too (a later sentence opens a new one); once the stop is sent, 10 s of quiet is a hung server. VULPINE_WYOMING_TTS_STREAMING=0 keeps the plain row. the same

Any other transport is refused transport_not_allowed. The endpoint must be a loopback address: a user implementation stays local.

Qualification probe

At every bind that names it, the gateway renders the fixed sentence Hardpoint check. through the implementation and counts the bytes. Nothing is played. The probe passes when audio comes back within 5 s. The measured ms is the time to the first audio chunk, compared with the leg’s p90. Over it, the implementation stays bound and the Admin marks it over budget. A failed probe binds the manifest’s fallback (qwen3-worker), reported degraded {reason: qualification_failed}. With fallback = "none" the bind is refused.

Voice check

A server can answer every request with audio and still not speak its configured voice. macOS, for one, renders an Eloquence voice in its fallback voice, Samantha, and reports no error. An entry may name the voices its server falls back to (optional, at most four):

reference_voices = ["alloy"]

After a passed qualification the gateway checks the voice in the background; the session never waits for it.

  1. It asks the server’s describe for its configured voice. A server that advertises exactly one voice names it; the OpenAI-speech shim advertises its --voice.
  2. It renders Hardpoint check. in the configured voice, the way every reply is rendered.
  3. It renders the same sentence in each reference voice of another name (synthesize {text, voice: {name}}), and stops at the first match.

Two renders match on the same bytes or on the same fingerprint: the same rate, the trimmed length within 1%, and a 32-bin loudness envelope within 2 of 63 per bin. Nothing is played. The Admin’s Hardpoints card shows the outcome under the slot’s state:

verdict meaning
own_voice the configured voice differs from every reference voice compared, or is a reference voice itself
fell_back the server names its configured voice, and it renders the same as a reference voice of another name
same_as_reference it renders the same as a reference voice, and the server does not say which voice it is configured with
unchecked no_reference_voices (the entry names none), or render_failed (a render failed, or 20 s passed)

A voice that is a reference voice under another name reads as fallen back: preowned’s Samantha is alloy. The check costs one render per voice on the server (about 0.3 s each on preowned), after the bind and off the turn.

Naming the voice

A loadout may name the voice its tts implementation speaks in (optional):

[hardpoints]
tts = "xn-ptts"
tts_voice = "operator"
  • On the wire. The gateway sends it as Wyoming’s own voice: {name}: in the qualification probe (synthesize {text: "Hardpoint check.", voice: {name}}), in every plain reply, in a streamed reply’s synthesize-start and its compatibility synthesize, and in the voice check’s configured render. Without tts_voice nothing changes: no voice field, the server’s own voice.
  • Qualification. A server that does not have the voice answers error (xn-ptts: unknown voice '<name>'). The probe fails with those words and the bind falls back to the declared built-in, degraded {reason: qualification_failed}.
  • Bounds. 1 to 64 of [A-Za-z0-9_.-], and only beside a [hardpoints] tts Catalog implementation. A built-in takes its voice from the persona, so the loadout is refused (voice_invalid).
  • What it is not. A name, never a path. The voice’s file (xn-ptts: <name>.safetensors in --voice-dir) and its pace (--voice-condition <name>:duration_delta=<x>) are the server’s launch settings. A voice file made from someone’s recordings never reaches the gateway, the trace or the Admin.
  • Readback. The Admin’s bind row carries voice while the implementation carries the slot: the card reads xn-ptts (yours) · voice operator. The voice check reports it as configured, also when it has no reference voice to compare. hardpoint-check --voice <name> probes an entry in a voice.

The first-party entry

catalog/hardpoints/wyoming-kokoro/hardpoint.toml names runtime/wyoming-kokoro (kokoro-onnx on 127.0.0.1:10210, optional). A loadout binds it with:

[hardpoints]
tts = "wyoming-kokoro"

User entries

catalog/hardpoints/xn-ptts/hardpoint.toml is a user entry: xn-ptts’s own ptts-openai-server with --wyoming-addr 127.0.0.1:10220 --lang en, bound with no shim. The operator builds and runs it; Vulpine does not install it. A trained Phonon voice file binds through it with tts_voice and a per-voice pace patch, as its README describes. The server holds a streamed sentence under 20 characters until more text or the stop, so a short first sentence waits up to the 1 s idle window. VULPINE_WYOMING_TTS_STREAMING=0 speaks it plain, at once.

A server that speaks OpenAI’s POST /v1/audio/speech rather than Wyoming binds through vulpine-openai-speech (crates/openai_speech_shim). It is a loopback Wyoming server. It asks the server for response_format=pcm, passes each piece of body on as one audio-chunk the moment it lands, and hangs up on the server when the gateway does. Two entries use it:

  • catalog/hardpoints/preowned-speech/hardpoint.toml: macOS system voices, one whole buffer per reply, port 10240.
  • catalog/hardpoints/qwentts-cpp-server/hardpoint.toml: qwentts.cpp’s tts-server on CUDA, Vulkan or the CPU, streamed while the model runs, port 10241.

Was this page helpful?