feat: add authenticated voice pipeline API docs
CI / Verify and package (ubuntu-latest) (push) Canceled after 0s
CI / Verify and package (windows-latest) (push) Canceled after 0s

This commit is contained in:
snowspeeder
2026-08-04 19:09:33 -04:00
parent a2e4f1d576
commit 0793fa0ac4
14 changed files with 1874 additions and 9 deletions
+91
View File
@@ -0,0 +1,91 @@
# Voice Conversation API (v1)
`/api/v1/voice` is a server-side device API for native iOS and Echo-style clients. It is separate from the browser voice feature. Azure credentials remain on the PI WEB host.
## Security and provisioning
Create a device token on the PI WEB host:
```sh
pi-web voice-token create \
--project project-id --workspace workspace-id \
--model openai-codex/gpt-5.6-terra --thinking high
```
Each restriction flag may be repeated. Omitting a category grants `*` for that category. A restricted model or thinking scope requires the client to explicitly select an allowed value; it cannot inherit a server default. Project and workspace scopes are both enforced. Token records contain a SHA-256 token hash only and are persisted at mode `0600` in `~/.config/pi-web/voice-api.json`; the plaintext (`pwv1_...`) is printed only once.
```sh
pi-web voice-token list
pi-web voice-token revoke <token-id>
```
Use `Authorization: Bearer pwv1_...` on every HTTP and WebSocket connection. The API requires HTTPS, except loopback for local development/testing. A conversation is owned by the creating token: other valid device tokens receive `404` when they try to inspect, delete, or attach it. A client may select only a registered workspace id, never a filesystem path.
## HTTP
`GET /api/v1/voice/targets` returns only the caller's authorized registered workspaces plus effective scopes.
`POST /api/v1/voice/conversations`
```json
{
"workspaceId": "registered-workspace-id",
"model": "openai-codex/gpt-5.6-terra",
"thinking": "high",
"context": "You are helping with the home-automation project."
}
```
The response is `201` and includes the opaque conversation id, session id, selected workspace, and `input-ready` status. `GET` and `DELETE /api/v1/voice/conversations/:id` inspect and close the API conversation handle.
## WebSocket wire protocol
Connect to `wss://host/api/v1/voice/stream` with the Bearer header. The server first sends:
```json
{
"type":"hello",
"protocol":"pi-web.voice.v1",
"pcm":{"input":"s16le/16000/mono","output":"s16le/24000/mono"},
"binary":{"version":1,"headerBytes":8,"kind":"audio"}
}
```
Attach an owned conversation and wait for `input.ready` before sending a turn:
```json
{"type":"attach","conversationId":"..."}
```
### Text fallback
Send exactly one JSON input while `input-ready`:
```json
{"type":"input.text","text":"Turn on the kitchen lights."}
```
### Audio input
For speech input, send binary WebSocket frames containing raw signed little-endian 16-bit, 16 kHz, mono PCM (no WAV header). Frames must be nonempty, <=64 KiB, aligned to two bytes, and an utterance is limited to two minutes. Finish with:
```json
{"type":"input.end"}
```
Azure push-stream recognition emits `transcript.partial` and `transcript.final` JSON events. The final transcript is submitted to Pi. This Voice API never returns an Azure token or key to a device.
### Turn output and lifecycle
For either input form, the server emits `agent.working`, then `agent.accepted` after Pi accepted the prompt. It subscribes to the session event stream before submission and waits for Pi's native `agent.settled` event (not merely `agent.end`), with a two-minute turn timeout. `assistant.delta` is streamed while Pi answers; `assistant.final` contains only assistant text, never STT transcript.
Azure synthesis output is signed little-endian 16-bit, 24 kHz, mono PCM. Every binary server frame has this eight-byte header followed by PCM:
| Bytes | Meaning |
|---|---|
| 0 | protocol version: `1` |
| 1 | kind: `1` (audio) |
| 23 | reserved: `0` (big endian) |
| 47 | monotonically increasing frame sequence (uint32 big endian) |
The server then emits `{"type":"audio.end"}` and a fresh `input.ready`. Send `{"type":"close"}` or close the WebSocket to end; disconnecting closes STT/event resources and clears the handle's working state.