feat: speak answers sentence by sentence and never speak reasoning

Segment the final answer as it streams and synthesize each completed sentence immediately, so a listener hears the reply begin while the model is still writing it. Sentences share the progress clips' FIFO chain and output-sequence counter; only text left unspoken (a hit budget) is synthesized after settle, so audio is never sent twice. A message that grows a tool call stops answer-streaming without re-speaking what already played. Reasoning models inline their chain of thought as <think> blocks in the same delta stream. Strip it while streaming -- carrying a partial tag across delta boundaries -- so it is neither spoken nor shown as answer text. The fixed "Hang on while I work on that." is now a rotation of short natural phrases that never repeats twice running, and a long silent think gets a spoken "still on it" roughly every 45 seconds until real speech begins. Turn timeout rises to five minutes to match the client. Co-Authored-By: Claude Fable 5 <[email protected]>
feat: speak a progress update during long voice turns
2026-08-07 14:20:21 -04:00 · 2026-08-06 23:10:07 -04:00 · 2026-08-06 12:09:34 -04:00 · 2026-08-06 00:42:55 -04:00 · 2026-08-04 19:09:33 -04:00 · 2026-08-04 16:25:32 -04:00
31 changed files with 4413 additions and 71 deletions
@@ -14,3 +14,22 @@ dist/

 # Local runtime attachment uploads (created by the chat composer "save to folder" mode).
 .pi-web/
+
+# Local PI subagent run transcripts and artifacts.
+.pi-subagents/
+
+# Test, build, and browser-automation output.
+coverage/
+test-results/
+playwright-report/
+.vite/
+*.tsbuildinfo
+
+# Local credentials and TLS material. Keep examples explicitly versionable.
+.env.*
+!.env.example
+*.pem
+*.key
+*.crt
+*.p12
+*.pfx
@@ -1,5 +1,16 @@
 # Agent Notes

+## Fork initialization and upstream sync
+
+This repository is a personal fork maintained under the `snowspeeder` Gitea account.
+
+- `origin` is the writable Gitea repository: `snowspeeder/pi-web`.
+- `upstream` is the canonical project: `https://github.com/jmfederico/pi-web.git`.
+- Keep fork-specific work in focused commits and branches. Do not force-push shared branches.
+- To bring in project updates, run `git fetch upstream`, merge or rebase `upstream/main` into the relevant local branch, resolve conflicts, test, then push to `origin`.
+- Do not push changes to `upstream`.
+
+
 This project is expected to run locally using split systemd user services:

 - `pi-web-sessiond.service` runs `npm run start:sessiond` in non-autoreload, non-auto-restart mode.
@@ -0,0 +1,113 @@
+# Voice Conversation API (v1)
+
+`/api/v1/voice` is a server-side device API for native iOS and Echo-style clients. It is separate from the browser voice feature. Azure credentials remain on the PI WEB host. Every voice turn automatically includes server-side instructions to produce concise, natural DragonHD-friendly speech without Markdown or visual-only formatting.
+
+## Security and provisioning
+
+Create a device token on the PI WEB host:
+
+```sh
+pi-web voice-token create \
+  --project project-id --workspace workspace-id \
+  --model openai-codex/gpt-5.6-terra --thinking high
+```
+
+Each restriction flag may be repeated. Omitting a category grants `*` for that category. A restricted model or thinking scope requires the client to explicitly select an allowed value; it cannot inherit a server default. Project and workspace scopes are both enforced. Token records contain a SHA-256 token hash only and are persisted at mode `0600` in `~/.config/pi-web/voice-api.json`; the plaintext (`pwv1_...`) is printed only once.
+
+```sh
+pi-web voice-token list
+pi-web voice-token revoke <token-id>
+```
+
+Use `Authorization: Bearer pwv1_...` on every HTTP and WebSocket connection. The API requires HTTPS, except loopback for local development/testing. A conversation is owned by the creating token: other valid device tokens receive `404` when they try to inspect, delete, or attach it. A client may select only a registered workspace id, never a filesystem path.
+
+## HTTP
+
+`GET /api/v1/voice/targets` returns only the caller's authorized registered workspaces plus effective scopes. Use its `workspaces[].id` to choose a workspace; paths are returned for display but are never accepted as an API input.
+
+`GET /api/v1/voice/conversations?workspaceId=<workspace-id>` lists prior Pi sessions rooted at that authorized workspace. This lets a device show a conversation picker without learning about sessions outside its workspace scope.
+
+`POST /api/v1/voice/conversations`
+
+```json
+{
+  "workspaceId": "registered-workspace-id",
+  "model": "openai-codex/gpt-5.6-terra",
+  "thinking": "high",
+  "context": "You are helping with the home-automation project."
+}
+```
+
+For an existing directory beneath the device-safe root `/home/hope/workspaces` that is not yet registered as a PI WEB workspace, use `POST /api/v1/voice/conversations/path` with `path` instead of `workspaceId`. The path must already exist, resolve beneath that root, and is validated before a session starts.
+
+The response is `201` and includes the opaque conversation id, session id, selected workspace, and `input-ready` status. `GET /api/v1/voice/conversations/:id/models` lists the models currently available to that conversation's Pi session, filtered by the device token's model scope. `GET` and `DELETE /api/v1/voice/conversations/:id` inspect and close the API conversation handle.
+
+To resume a listed session, create a new token-owned voice handle:
+
+```json
+POST /api/v1/voice/conversations/resume
+{
+  "workspaceId": "registered-workspace-id",
+  "sessionId": "previous-pi-session-id"
+}
+```
+
+The session must be in the selected workspace and within the caller's token scope.
+
+## WebSocket wire protocol
+
+Connect to `wss://host/api/v1/voice/stream` with the Bearer header. The server first sends:
+
+```json
+{
+  "type":"hello",
+  "protocol":"pi-web.voice.v1",
+  "pcm":{"input":"s16le/16000/mono","output":"s16le/24000/mono"},
+  "binary":{"version":1,"headerBytes":8,"kind":"audio"}
+}
+```
+
+Attach an owned conversation and wait for `input.ready` before sending a turn:
+
+```json
+{"type":"attach","conversationId":"..."}
+```
+
+### Text fallback
+
+Send exactly one JSON input while `input-ready`:
+
+```json
+{"type":"input.text","text":"Turn on the kitchen lights."}
+```
+
+### Audio input
+
+For speech input, send binary WebSocket frames containing raw signed little-endian 16-bit, 16 kHz, mono PCM (no WAV header). Frames must be nonempty, <=64 KiB, aligned to two bytes, and an utterance is limited to two minutes. Finish with:
+
+```json
+{"type":"input.end"}
+```
+
+Azure push-stream recognition emits `transcript.partial` and `transcript.final` JSON events. The final transcript is submitted to Pi. This Voice API never returns an Azure token or key to a device.
+
+### Turn output and lifecycle
+
+For either input form, the server emits `agent.working`, then `agent.accepted` after Pi accepted the prompt. If work is still in progress after two seconds and nothing has been spoken yet (see below), it sends one `agent.progress` event with a short spoken status clip, picked pseudo-randomly from a small set of natural filler phrases (“One sec.”, “Let me take a look.”, “Hmm, checking now.”, “Working on it.”, “Give me a moment.”, “Let me look into that.”). The server never repeats the same phrase on two consecutive turns. Ordinary screen-only `agent.working` heartbeats continue every 15 seconds regardless.
+
+During a long turn, Pi may emit several assistant messages between tool calls before its final answer — each one ends with Pi's `message.end` event. Whenever a completed message includes a tool call (i.e., it is not the turn's final answer), the server also speaks it: it sends `{"type":"agent.progress","text":"<message text>","audio":true}` — `text` is always the message's full, untruncated text — immediately followed by that clip's binary audio frames, unless every word of that text was already spoken live as answer sentences (see below), in which case the clip is skipped entirely, or only its unspoken remainder is synthesized if part of it was spoken live and part was not. `agent.progress` may therefore occur multiple times per turn, always in the order the messages completed, and each is optionally followed by its own clip's binary frames. For speech only, a message longer than about 300 characters is truncated at a sentence boundary where possible (the JSON `text` field is never truncated). Progress clips are sent one at a time — a clip's audio always finishes sending before the next `agent.progress` frame is sent — and a turn's total spoken progress narration is capped at roughly 60 seconds of audio or 8 clips, whichever comes first; the fallback phrase counts toward this same cap. Once the turn's answer has audibly begun (the first `answer.sentence` has been sent — see below), a brand-new message that had no live narration of its own is left quiet rather than starting fresh progress narration that would talk over the answer; a message that was already mid-narration when a tool call appeared in it still gets its own unspoken leftover finished so it isn't cut off abruptly. `agent.progress` frames, spoken or not, always stop before `assistant.final`.
+
+While the final answer streams in as Pi generates it, the server does not wait for it to fully finish before speaking it: as each complete sentence of the in-progress message accumulates (split on `.`/`!`/`?` — plus a closing quote or bracket — followed by whitespace; a decimal like “3.5” never triggers a false split, since the period is not followed by whitespace), the server sends `{"type":"answer.sentence","text":"<sentence>"}` immediately followed by that sentence's own binary audio frames, serialized through the same ordered chain as `agent.progress` clips so nothing interleaves or reorders on the single audio output. Markdown-decoration-only fragments (e.g. `---`, `` ``` ``, `**`) are skipped silently. Because a message might still turn out to include a tool call (making it narration, not the final answer), every message is optimistically streamed this way until either a tool call is detected in it or it ends; once a tool call appears, sentence-streaming for that message stops (already-spoken sentences remain spoken — that's fine acoustically) and any unspoken remainder is handled by the `agent.progress` mechanism above instead. A message's trailing fragment with no terminal punctuation yet is held back until the message ends, then flushed as one final `answer.sentence`. Spoken answer audio is capped at roughly 120 seconds or 40 sentences per turn, whichever comes first; once agent.settled fires, `assistant.final` (the full, untruncated answer text) is always sent next, and any part of the answer that was never spoken live because a cap was hit is synthesized once, as ordinary audio frames, between `assistant.final` and `audio.end` — a turn where every sentence streamed live sends zero audio frames in that window. `audio.end` always ends the turn's audio either way.
+
+While it waits for Pi's native `agent.settled` event (not merely `agent.end`), it repeats the existing `agent.working` frame every 15 seconds; those progress frames stop before `assistant.final` and never overlap synthesized audio. The wait has a two-minute turn timeout. `assistant.delta` is streamed while Pi answers; `assistant.final` contains only assistant text, never STT transcript.
+
+Azure synthesis output is signed little-endian 16-bit, 24 kHz, mono PCM. Every binary server frame has this eight-byte header followed by PCM:
+
+| Bytes | Meaning |
+|---|---|
+| 0 | protocol version: `1` |
+| 1 | kind: `1` (audio) |
+| 2–3 | reserved: `0` (big endian) |
+| 4–7 | monotonically increasing frame sequence (uint32 big endian) |
+
+The server then emits `{"type":"audio.end"}` and a fresh `input.ready`. Send `{"type":"close"}` or close the WebSocket to end; disconnecting closes STT/event resources and clears the handle's working state.
@@ -24,6 +24,8 @@
        "@codemirror/view": "^6.43.6",
        "@fastify/compress": "^9.0.0",
        "@fastify/static": "^9.3.0",
+        "@fastify/swagger": "^9.8.1",
+        "@fastify/swagger-ui": "^5.2.6",
        "@fastify/websocket": "^11.3.0",
        "@xterm/addon-fit": "^0.11.0",
        "@xterm/xterm": "^6.0.0",
@@ -31,6 +33,7 @@
        "fastify": "^5.10.0",
        "lit": "^3.3.3",
        "marked": "^18.0.6",
+        "microsoft-cognitiveservices-speech-sdk": "^1.51.0",
        "node-pty": "^1.1.0",
        "typebox": "1.3.6",
        "ws": "^8.21.0"
@@ -52,6 +55,7 @@
        "globals": "^17.7.0",
        "happy-dom": "^20.11.1",
        "knip": "^6.25.0",
+        "openapi-types": "^12.1.3",
        "tsx": "^4.23.0",
        "typescript": "^6.0.3",
        "typescript-eslint": "^8.63.0",
@@ -547,6 +551,46 @@
        "node": ">=18.0.0"
      }
    },
+    "node_modules/@azure/abort-controller": {
+      "version": "2.2.0",
+      "resolved": "https://registry.npmjs.org/@azure/abort-controller/-/abort-controller-2.2.0.tgz",
+      "integrity": "sha512-fNAjWnA/nZ2jz31kxR/AqRaUT8ewHBw/WuBIosK0moMy1C9e5ValbDfFdIxJzVOOYaYkV/b2F1S4H/aHiqfVQg==",
+      "license": "MIT",
+      "dependencies": {
+        "tslib": "^2.6.2"
+      },
+      "engines": {
+        "node": ">=22.0.0"
+      }
+    },
+    "node_modules/@azure/core-auth": {
+      "version": "1.11.0",
+      "resolved": "https://registry.npmjs.org/@azure/core-auth/-/core-auth-1.11.0.tgz",
+      "integrity": "sha512-IUZydyTUkDnYdstOW9pFOOUQlBjAepK5teihDE3x6yxsPJs/hsAaaYpeGxdxrgtOiJbBKSjKW7MDk7AEhb4LRg==",
+      "license": "MIT",
+      "dependencies": {
+        "@azure/abort-controller": "^2.1.2",
+        "@azure/core-util": "^1.13.0",
+        "tslib": "^2.6.2"
+      },
+      "engines": {
+        "node": ">=22.0.0"
+      }
+    },
+    "node_modules/@azure/core-util": {
+      "version": "1.14.0",
+      "resolved": "https://registry.npmjs.org/@azure/core-util/-/core-util-1.14.0.tgz",
+      "integrity": "sha512-9n2pWK61veAuN0V20t9lOuoV4CFMdyAZ1ygZzvBGk/pBBJRib/PjL9PLXa/aI2CcPpyHfqVsxxqLCYl6uZlfDw==",
+      "license": "MIT",
+      "dependencies": {
+        "@azure/abort-controller": "^2.1.2",
+        "@typespec/ts-http-runtime": "^0.3.0",
+        "tslib": "^2.6.2"
+      },
+      "engines": {
+        "node": ">=22.0.0"
+      }
+    },
    "node_modules/@babel/runtime": {
      "version": "7.29.7",
      "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.7.tgz",
@@ -3866,6 +3910,68 @@
        "glob": "^13.0.0"
      }
    },
+    "node_modules/@fastify/swagger": {
+      "version": "9.8.1",
+      "resolved": "https://registry.npmjs.org/@fastify/swagger/-/swagger-9.8.1.tgz",
+      "integrity": "sha512-VpHMnqZTY8iBZYJE8WWkbKPrXIYWy2rDfIf5qLr6DzZSpQYZ+KxQVcJFiq/AMlvNwI4gCBd66++iUlxXXGT0IQ==",
+      "funding": [
+        {
+          "type": "github",
+          "url": "https://github.com/sponsors/fastify"
+        },
+        {
+          "type": "opencollective",
+          "url": "https://opencollective.com/fastify"
+        }
+      ],
+      "license": "MIT",
+      "dependencies": {
+        "fastify-plugin": "^6.0.0",
+        "json-schema-resolver": "^3.0.0",
+        "openapi-types": "^12.1.3",
+        "rfdc": "^1.3.1",
+        "yaml": "^2.4.2"
+      }
+    },
+    "node_modules/@fastify/swagger-ui": {
+      "version": "5.2.6",
+      "resolved": "https://registry.npmjs.org/@fastify/swagger-ui/-/swagger-ui-5.2.6.tgz",
+      "integrity": "sha512-OMnms0O5s9wb6wis/K5nlrAMLsgUbr1GA8uphM41IasWe3AFdgxz6r/3bA9HTxlDNUYc2FGGKeqMp3ntxmSiNA==",
+      "funding": [
+        {
+          "type": "github",
+          "url": "https://github.com/sponsors/fastify"
+        },
+        {
+          "type": "opencollective",
+          "url": "https://opencollective.com/fastify"
+        }
+      ],
+      "license": "MIT",
+      "dependencies": {
+        "@fastify/static": "^9.1.2",
+        "fastify-plugin": "^5.0.0",
+        "openapi-types": "^12.1.3",
+        "rfdc": "^1.3.1",
+        "yaml": "^2.4.1"
+      }
+    },
+    "node_modules/@fastify/swagger-ui/node_modules/fastify-plugin": {
+      "version": "5.1.0",
+      "resolved": "https://registry.npmjs.org/fastify-plugin/-/fastify-plugin-5.1.0.tgz",
+      "integrity": "sha512-FAIDA8eovSt5qcDgcBvDuX/v0Cjz0ohGhENZ/wpc3y+oZCY2afZ9Baqql3g/lC+OHRnciQol4ww7tuthOb9idw==",
+      "funding": [
+        {
+          "type": "github",
+          "url": "https://github.com/sponsors/fastify"
+        },
+        {
+          "type": "opencollective",
+          "url": "https://opencollective.com/fastify"
+        }
+      ],
+      "license": "MIT"
+    },
    "node_modules/@fastify/websocket": {
      "version": "11.3.0",
      "resolved": "https://registry.npmjs.org/@fastify/websocket/-/websocket-11.3.0.tgz",
@@ -4445,9 +4551,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4465,9 +4568,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4485,9 +4585,6 @@
        "ppc64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4505,9 +4602,6 @@
        "riscv64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4525,9 +4619,6 @@
        "riscv64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4545,9 +4636,6 @@
        "s390x"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4565,9 +4653,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -4585,9 +4670,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5153,9 +5235,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5173,9 +5252,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5193,9 +5269,6 @@
        "ppc64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5213,9 +5286,6 @@
        "s390x"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5233,9 +5303,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5253,9 +5320,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MIT",
      "optional": true,
      "os": [
@@ -5550,6 +5614,12 @@
      "integrity": "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==",
      "license": "MIT"
    },
+    "node_modules/@types/webrtc": {
+      "version": "0.0.37",
+      "resolved": "https://registry.npmjs.org/@types/webrtc/-/webrtc-0.0.37.tgz",
+      "integrity": "sha512-JGAJC/ZZDhcrrmepU4sPLQLIOIAgs5oIK+Ieq90K8fdaNMhfdfqmYatJdgif1NDQtvrSlTOGJDUYHIDunuufOg==",
+      "license": "MIT"
+    },
    "node_modules/@types/whatwg-mimetype": {
      "version": "3.0.2",
      "resolved": "https://registry.npmjs.org/@types/whatwg-mimetype/-/whatwg-mimetype-3.0.2.tgz",
@@ -5787,6 +5857,20 @@
        "url": "https://opencollective.com/typescript-eslint"
      }
    },
+    "node_modules/@typespec/ts-http-runtime": {
+      "version": "0.3.8",
+      "resolved": "https://registry.npmjs.org/@typespec/ts-http-runtime/-/ts-http-runtime-0.3.8.tgz",
+      "integrity": "sha512-bLMpVcWZNzq6lYOybwFwOAR1IXKcHnhUNqYeHjl1bET/qE3jFPFH+p8Wrh3rU4xwdnifPxmKNESBYnvnmc75aA==",
+      "license": "MIT",
+      "dependencies": {
+        "http-proxy-agent": "^7.0.0",
+        "https-proxy-agent": "^7.0.0",
+        "tslib": "^2.6.2"
+      },
+      "engines": {
+        "node": ">=22.0.0"
+      }
+    },
    "node_modules/@vitest/expect": {
      "version": "4.1.10",
      "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.10.tgz",
@@ -5960,7 +6044,6 @@
      "version": "7.1.4",
      "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz",
      "integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==",
-      "dev": true,
      "license": "MIT",
      "engines": {
        "node": ">= 14"
@@ -6127,6 +6210,17 @@
      ],
      "license": "MIT"
    },
+    "node_modules/bent": {
+      "version": "7.3.12",
+      "resolved": "https://registry.npmjs.org/bent/-/bent-7.3.12.tgz",
+      "integrity": "sha512-T3yrKnVGB63zRuoco/7Ybl7BwwGZR0lceoVG5XmQyMIH9s19SV5m+a8qam4if0zQuAmOQTyPTPmsQBdAorGK3w==",
+      "license": "Apache-2.0",
+      "dependencies": {
+        "bytesish": "^0.4.1",
+        "caseless": "~0.12.0",
+        "is-stream": "^2.0.0"
+      }
+    },
    "node_modules/better-path-resolve": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/better-path-resolve/-/better-path-resolve-1.0.0.tgz",
@@ -6232,6 +6326,18 @@
        "node": ">=4.0"
      }
    },
+    "node_modules/bytesish": {
+      "version": "0.4.4",
+      "resolved": "https://registry.npmjs.org/bytesish/-/bytesish-0.4.4.tgz",
+      "integrity": "sha512-i4uu6M4zuMUiyfZN4RU2+i9+peJh//pXhd9x1oSe1LBkZ3LEbCoygu8W0bXTukU1Jme2txKuotpCZRaC3FLxcQ==",
+      "license": "(Apache-2.0 AND MIT)"
+    },
+    "node_modules/caseless": {
+      "version": "0.12.0",
+      "resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz",
+      "integrity": "sha512-4tYFyifaFfGacoiObjJegolkwSU4xQNGbVgUiNYVUxbQ2x2lUsFvY4hVgVzGiIe6WLOPqycWXA40l+PWsxthUw==",
+      "license": "Apache-2.0"
+    },
    "node_modules/chai": {
      "version": "6.2.2",
      "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz",
@@ -6323,7 +6429,6 @@
      "version": "4.4.3",
      "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
      "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
-      "dev": true,
      "license": "MIT",
      "dependencies": {
        "ms": "^2.1.3"
@@ -7451,7 +7556,6 @@
      "version": "7.0.2",
      "resolved": "https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-7.0.2.tgz",
      "integrity": "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==",
-      "dev": true,
      "license": "MIT",
      "dependencies": {
        "agent-base": "^7.1.0",
@@ -7465,7 +7569,6 @@
      "version": "7.0.6",
      "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
      "integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==",
-      "dev": true,
      "license": "MIT",
      "dependencies": {
        "agent-base": "^7.1.2",
@@ -7590,6 +7693,18 @@
        "node": ">=0.12.0"
      }
    },
+    "node_modules/is-stream": {
+      "version": "2.0.1",
+      "resolved": "https://registry.npmjs.org/is-stream/-/is-stream-2.0.1.tgz",
+      "integrity": "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg==",
+      "license": "MIT",
+      "engines": {
+        "node": ">=8"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/sindresorhus"
+      }
+    },
    "node_modules/is-subdir": {
      "version": "1.2.0",
      "resolved": "https://registry.npmjs.org/is-subdir/-/is-subdir-1.2.0.tgz",
@@ -7695,6 +7810,23 @@
        "dequal": "^2.0.3"
      }
    },
+    "node_modules/json-schema-resolver": {
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/json-schema-resolver/-/json-schema-resolver-3.0.0.tgz",
+      "integrity": "sha512-HqMnbz0tz2DaEJ3ntsqtx3ezzZyDE7G56A/pPY/NGmrPu76UzsWquOpHFRAf5beTNXoH2LU5cQePVvRli1nchA==",
+      "license": "MIT",
+      "dependencies": {
+        "debug": "^4.1.1",
+        "fast-uri": "^3.0.5",
+        "rfdc": "^1.1.4"
+      },
+      "engines": {
+        "node": ">=20"
+      },
+      "funding": {
+        "url": "https://github.com/Eomm/json-schema-resolver?sponsor=1"
+      }
+    },
    "node_modules/json-schema-to-ts": {
      "version": "3.1.1",
      "resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
@@ -7999,9 +8131,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MPL-2.0",
      "optional": true,
      "os": [
@@ -8023,9 +8152,6 @@
        "arm64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MPL-2.0",
      "optional": true,
      "os": [
@@ -8047,9 +8173,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "glibc"
-      ],
      "license": "MPL-2.0",
      "optional": true,
      "os": [
@@ -8071,9 +8194,6 @@
        "x64"
      ],
      "dev": true,
-      "libc": [
-        "musl"
-      ],
      "license": "MPL-2.0",
      "optional": true,
      "os": [
@@ -8255,6 +8375,55 @@
        "url": "https://github.com/sponsors/jonschlinkert"
      }
    },
+    "node_modules/microsoft-cognitiveservices-speech-sdk": {
+      "version": "1.51.0",
+      "resolved": "https://registry.npmjs.org/microsoft-cognitiveservices-speech-sdk/-/microsoft-cognitiveservices-speech-sdk-1.51.0.tgz",
+      "integrity": "sha512-BLLovv5PegOr5Lp52h4CSgL2c/ViFAh9LoU49ILNPyb/Z0giGI7nPV3xNLcmTtHLT+kfYNRGUIA62vORLzP8TA==",
+      "license": "MIT",
+      "dependencies": {
+        "@azure/core-auth": "^1.9.0",
+        "@types/webrtc": "^0.0.37",
+        "agent-base": "^6.0.1",
+        "bent": "^7.3.12",
+        "https-proxy-agent": "^4.0.0",
+        "uuid": "^11.1.1",
+        "ws": "^8.21.0"
+      }
+    },
+    "node_modules/microsoft-cognitiveservices-speech-sdk/node_modules/agent-base": {
+      "version": "6.0.2",
+      "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
+      "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
+      "license": "MIT",
+      "dependencies": {
+        "debug": "4"
+      },
+      "engines": {
+        "node": ">= 6.0.0"
+      }
+    },
+    "node_modules/microsoft-cognitiveservices-speech-sdk/node_modules/https-proxy-agent": {
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-4.0.0.tgz",
+      "integrity": "sha512-zoDhWrkR3of1l9QAL8/scJZyLu8j/gBkcwcaQOZh7Gyh/+uJQzGVETdgT30akuwkpL8HTRfssqI3BZuV18teDg==",
+      "license": "MIT",
+      "dependencies": {
+        "agent-base": "5",
+        "debug": "4"
+      },
+      "engines": {
+        "node": ">= 6.0.0"
+      }
+    },
+    "node_modules/microsoft-cognitiveservices-speech-sdk/node_modules/https-proxy-agent/node_modules/agent-base": {
+      "version": "5.1.1",
+      "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-5.1.1.tgz",
+      "integrity": "sha512-TMeqbNl2fMW0nMjTEPOwe3J/PRFP4vqeoNuQMG0HlMrtm5QxKqdvAkZ1pRBQ/ulIyDD5Yq0nJ7YbdD8ey0TO3g==",
+      "license": "MIT",
+      "engines": {
+        "node": ">= 6.0.0"
+      }
+    },
    "node_modules/mime": {
      "version": "3.0.0",
      "resolved": "https://registry.npmjs.org/mime/-/mime-3.0.0.tgz",
@@ -8314,7 +8483,6 @@
      "version": "2.1.3",
      "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
      "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
-      "dev": true,
      "license": "MIT"
    },
    "node_modules/nanoid": {
@@ -8453,6 +8621,12 @@
        }
      }
    },
+    "node_modules/openapi-types": {
+      "version": "12.1.3",
+      "resolved": "https://registry.npmjs.org/openapi-types/-/openapi-types-12.1.3.tgz",
+      "integrity": "sha512-N4YtSYJqghVu4iek2ZUvcN/0aqH1kRDuNqzcycDxhOUpg7GdvLa2F3DgS6yBNhInhv2r/6I0Flkn7CqL8+nIcw==",
+      "license": "MIT"
+    },
    "node_modules/optionator": {
      "version": "0.9.4",
      "resolved": "https://registry.npmjs.org/optionator/-/optionator-0.9.4.tgz",
@@ -9632,7 +9806,6 @@
      "version": "2.8.1",
      "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
      "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
-      "dev": true,
      "license": "0BSD"
    },
    "node_modules/tsx": {
@@ -9754,6 +9927,19 @@
      "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
      "license": "MIT"
    },
+    "node_modules/uuid": {
+      "version": "11.1.1",
+      "resolved": "https://registry.npmjs.org/uuid/-/uuid-11.1.1.tgz",
+      "integrity": "sha512-vIYxrBCC/N/K+Js3qSN88go7kIfNPssr/hHCesKCQNAjmgvYS2oqr69kIufEG+O4+PfezOH4EbIeHCfFov8ZgQ==",
+      "funding": [
+        "https://github.com/sponsors/broofa",
+        "https://github.com/sponsors/ctavan"
+      ],
+      "license": "MIT",
+      "bin": {
+        "uuid": "dist/esm/bin/uuid"
+      }
+    },
    "node_modules/vite": {
      "version": "8.1.4",
      "resolved": "https://registry.npmjs.org/vite/-/vite-8.1.4.tgz",
@@ -10041,7 +10227,6 @@
      "version": "2.9.0",
      "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
      "integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
-      "dev": true,
      "license": "ISC",
      "bin": {
        "yaml": "bin.mjs"
@@ -70,6 +70,8 @@
    "@codemirror/view": "^6.43.6",
    "@fastify/compress": "^9.0.0",
    "@fastify/static": "^9.3.0",
+    "@fastify/swagger": "^9.8.1",
+    "@fastify/swagger-ui": "^5.2.6",
    "@fastify/websocket": "^11.3.0",
    "@xterm/addon-fit": "^0.11.0",
    "@xterm/xterm": "^6.0.0",
@@ -77,6 +79,7 @@
    "fastify": "^5.10.0",
    "lit": "^3.3.3",
    "marked": "^18.0.6",
+    "microsoft-cognitiveservices-speech-sdk": "^1.51.0",
    "node-pty": "^1.1.0",
    "typebox": "1.3.6",
    "ws": "^8.21.0"
@@ -93,6 +96,7 @@
    "globals": "^17.7.0",
    "happy-dom": "^20.11.1",
    "knip": "^6.25.0",
+    "openapi-types": "^12.1.3",
    "tsx": "^4.23.0",
    "typescript": "^6.0.3",
    "typescript-eslint": "^8.63.0",
@@ -26,8 +26,12 @@ describe("production build contents", () => {
    try {
      const fixtureDist = join(fixtureRoot, "dist", "server");
      await mkdir(fixtureDist, { recursive: true });
+      // npm pack runs the package's prepare lifecycle even with --ignore-scripts
+      // on the npm version used in CI, so include its harmless fixture script.
+      await mkdir(join(fixtureRoot, "scripts"), { recursive: true });
      await Promise.all([
        copyFile(join(repoRoot, "package.json"), join(fixtureRoot, "package.json")),
+        copyFile(join(repoRoot, "scripts", "install-git-hooks.mjs"), join(fixtureRoot, "scripts", "install-git-hooks.mjs")),
        writeFile(join(fixtureDist, "app.js"), "export {};\n", "utf8"),
        writeFile(join(fixtureDist, "app.testSupport.js"), "export {};\n", "utf8"),
        writeFile(join(fixtureDist, "app.testSupport.js.map"), "{}\n", "utf8"),
@@ -41,6 +41,7 @@ import {
  nativeServicePrerequisiteShellCheck,
 } from "./nativeServices/serviceProbe.js";
 import { renderLaunchdPlist, renderSystemdUnit } from "./nativeServices/serviceRendering.js";
+import { createVoiceToken, listVoiceTokens, revokeVoiceToken } from "./server/voiceApi.js";

 const PI_WEB_PACKAGE_NAME = "@jmfederico/pi-web";

@@ -1060,6 +1061,9 @@ Usage:
  pi-web start|stop|restart|status|logs
  pi-web doctor
  pi-web version
+  pi-web voice-token create [--model provider/model] [--thinking level] [--project project-id] [--workspace workspace-id]
+  pi-web voice-token list
+  pi-web voice-token revoke <token-id>

 Recommended install:
  npm install -g @jmfederico/pi-web --allow-scripts=node-pty
@@ -1078,11 +1082,55 @@ async function main(): Promise<void> {
  else if (command === "logs") logs();
  else if (command === "doctor") await doctor();
  else if (command === "version") await printPiWebVersionReport();
+  else if (command === "voice-token") voiceToken(args);
  else if (command === "--version" || command === "-v") console.log(packageVersion());
  else if (command === "help" || command === "--help" || command === "-h") help();
  else throw new Error(`Unknown command: ${command}`);
 }

+function voiceToken(args: string[]): void {
+  const [action, ...rest] = args;
+  if (action === "list") { console.log(JSON.stringify(listVoiceTokens(), null, 2)); return; }
+  if (action === "revoke") {
+    const id = rest[0];
+    if (id === undefined || !revokeVoiceToken(id)) throw new Error("Voice token was not found or is already revoked");
+    console.log(`Revoked voice token ${id}`);
+    return;
+  }
+  if (action === "create") {
+    const models: string[] = [];
+    const thinking: string[] = [];
+    const projects: string[] = [];
+    const workspaces: string[] = [];
+    for (let index = 0; index < rest.length; index += 1) {
+      const flag = rest[index];
+      const value = rest[index + 1];
+      if (
+        (flag !== "--model" &&
+          flag !== "--thinking" &&
+          flag !== "--project" &&
+          flag !== "--workspace") ||
+        value === undefined
+      )
+        throw new Error("Usage: pi-web voice-token create [--model provider/model] [--thinking level] [--project project-id] [--workspace workspace-id]");
+      if (flag === "--model") models.push(value);
+      else if (flag === "--thinking") thinking.push(value);
+      else if (flag === "--project") projects.push(value);
+      else workspaces.push(value);
+      index += 1;
+    }
+    const created = createVoiceToken({
+      ...(models.length === 0 ? {} : { models }),
+      ...(thinking.length === 0 ? {} : { thinking }),
+      ...(projects.length === 0 ? {} : { projects }),
+      ...(workspaces.length === 0 ? {} : { workspaces }),
+    });
+    console.log(`Voice token ${created.id} (shown once): ${created.token}`);
+    return;
+  }
+  throw new Error("Usage: pi-web voice-token create|list|revoke");
+}
+
 export function isCliEntrypoint(entrypoint: string | undefined = process.argv[1], modulePath: string = fileURLToPath(import.meta.url)): boolean {
  if (entrypoint === undefined) return false;
  if (entrypoint === modulePath) return true;
@@ -1,5 +1,5 @@
-export { activityApi, api, configApi, filesApi, gitApi, machinesApi, piPackagesApi, piWebApi, pluginsApi, projectsApi, sessionsApi, terminalsApi, workspacesApi } from "./api/clients";
+export { activityApi, api, azureSpeechApi, configApi, filesApi, gitApi, machinesApi, piPackagesApi, piWebApi, pluginsApi, projectsApi, sessionsApi, terminalsApi, workspacesApi } from "./api/clients";
 export { globalSessionEvents, realtimeEvents, sessionEvents, terminalSocket } from "./api/sockets";
 export { DEFAULT_WORKSPACE_UPLOADS_FOLDER, effectiveWorkspaceUploadFolder, uploadWorkspaceFile, uploadWorkspaceFiles, workspaceEffectiveUploadFolder, workspaceUploadPath, WorkspaceUploadBatchError, WorkspaceUploadCancelledError } from "./api/workspaceUploads";
 export type { UploadWorkspaceFileOptions, UploadWorkspaceFilesOptions, WorkspaceFileUploadProgress, WorkspaceUploadBatchFileProgress, WorkspaceUploadBatchProgress, WorkspaceUploadFileFailure, WorkspaceUploadFileInput, WorkspaceUploadFolderConfig, WorkspaceUploadTask, WorkspaceUploadXhr, WorkspaceUploadXhrFactory } from "./api/workspaceUploads";
-export type { ActiveAgentProfileDescriptor, ArchiveSessionsResponse, AskUserCloseResponse, AskUserQuestion, AskUserSubmission, PendingAskUser, PendingExtensionDialog, AuthProviderOption, AuthProviderStatus, AuthProvidersResponse, AuthStatusSource, AuthType, CommandOption, CommandResult, DeleteWorkspaceFileResponse, ExtensionDialogAnswer, ExtensionDialogCloseReason, ExtensionDialogCloseResponse, ExtensionDialogKind, ExtensionDialogOutcome, FileContentMediaType, FileContentResponse, FileSuggestion, FileTreeEntry, FileTreeResponse, GitDiffResponse, GitFileState, GitStatusFile, GitStatusResponse, Machine, MachineHealth, MachineKind, MachineRuntime, MachineStatus, MessagePage, ModelSelectionResponse, MoveWorkspaceFileOptions, MoveWorkspaceFileResponse, OAuthFlowState, PiPackageInfo, PiPackageInstallRequest, PiPackageMutationAction, PiPackageMutationResponse, PiPackageRemoveRequest, PiPackageScope, PiPackageUpdateRequest, PiPackagesResponse, PiWebAgentDirEnvSource, PiWebCapability, PiWebComponentStatus, PiWebConfigEnvOverrides, PiWebConfigResponse, PiWebConfigValues, PiWebDockerMode, PiWebInstallationInfo, PiWebInstallationKind, PiWebPluginConfig, PiWebPluginConfigMap, PiWebPluginInfo, PiWebPluginsResponse, PiWebPluginScope, PiWebPluginSettings, PiWebReleaseStatus, PiWebRuntimeComponent, PiWebRuntimeResponse, PiWebShortcutConfig, PiWebStatusMessage, PiWebStatusResponse, PiWebUploadsConfig, Project, PromptAttachment, QueuedSessionMessage, RealtimeEvent, RunTerminalCommandInput, SavedPromptAttachment, SessionActivity, SessionBulkArchiveResponse, SessionBulkDeleteArchivedResponse, SessionBulkFailure, SessionBulkMutationRef, SessionBulkMutationRequest, SessionCleanupExecuteResponse, SessionCleanupPreviewResponse, SessionCleanupProjectSummary, SessionCleanupRequest, SessionCleanupThresholds, SessionCleanupTotals, SessionInfo, SessionModel, SessionRef, SessionStatus, SessionStreamSnapshot, SessionUnreadAcknowledgeRequest, SessionUnreadCatalogSnapshot, SessionUnreadEvent, SessionUnreadSummary, SessionTreeNavigateRequest, SessionTreeNavigateResult, SessionTreeNode, SessionTreeNodeKind, SessionTreeSnapshot, SessionTreeSummaryChoice, SessionWarning, SessionWarningSeverity, SlashCommand, SessionUiEvent, TerminalCommandRun, TerminalCommandRunFilter, TerminalCommandRunHandle, TerminalCommandRunStatus, TerminalInfo, TerminalUiEvent, ThinkingLevel, ThinkingLevelsResponse, WriteWorkspaceFileOptions, WriteWorkspaceFileResponse, Workspace, WorkspaceActivity, WorkspaceActivityResponse, WorkspaceActivityUiEvent } from "../../shared/apiTypes";
+export type { ActiveAgentProfileDescriptor, ArchiveSessionsResponse, AzureSpeechSettings, AzureSpeechSettingsUpdate, AzureSpeechToken, AskUserCloseResponse, AskUserQuestion, AskUserSubmission, PendingAskUser, PendingExtensionDialog, AuthProviderOption, AuthProviderStatus, AuthProvidersResponse, AuthStatusSource, AuthType, CommandOption, CommandResult, DeleteWorkspaceFileResponse, ExtensionDialogAnswer, ExtensionDialogCloseReason, ExtensionDialogCloseResponse, ExtensionDialogKind, ExtensionDialogOutcome, FileContentMediaType, FileContentResponse, FileSuggestion, FileTreeEntry, FileTreeResponse, GitDiffResponse, GitFileState, GitStatusFile, GitStatusResponse, Machine, MachineHealth, MachineKind, MachineRuntime, MachineStatus, MessagePage, ModelSelectionResponse, MoveWorkspaceFileOptions, MoveWorkspaceFileResponse, OAuthFlowState, PiPackageInfo, PiPackageInstallRequest, PiPackageMutationAction, PiPackageMutationResponse, PiPackageRemoveRequest, PiPackageScope, PiPackageUpdateRequest, PiPackagesResponse, PiWebAgentDirEnvSource, PiWebCapability, PiWebComponentStatus, PiWebConfigEnvOverrides, PiWebConfigResponse, PiWebConfigValues, PiWebDockerMode, PiWebInstallationInfo, PiWebInstallationKind, PiWebPluginConfig, PiWebPluginConfigMap, PiWebPluginInfo, PiWebPluginsResponse, PiWebPluginScope, PiWebPluginSettings, PiWebReleaseStatus, PiWebRuntimeComponent, PiWebRuntimeResponse, PiWebShortcutConfig, PiWebStatusMessage, PiWebStatusResponse, PiWebUploadsConfig, Project, PromptAttachment, QueuedSessionMessage, RealtimeEvent, RunTerminalCommandInput, SavedPromptAttachment, SessionActivity, SessionBulkArchiveResponse, SessionBulkDeleteArchivedResponse, SessionBulkFailure, SessionBulkMutationRef, SessionBulkMutationRequest, SessionCleanupExecuteResponse, SessionCleanupPreviewResponse, SessionCleanupProjectSummary, SessionCleanupRequest, SessionCleanupThresholds, SessionCleanupTotals, SessionInfo, SessionModel, SessionRef, SessionStatus, SessionStreamSnapshot, SessionUnreadAcknowledgeRequest, SessionUnreadCatalogSnapshot, SessionUnreadEvent, SessionUnreadSummary, SessionTreeNavigateRequest, SessionTreeNavigateResult, SessionTreeNode, SessionTreeNodeKind, SessionTreeSnapshot, SessionTreeSummaryChoice, SessionWarning, SessionWarningSeverity, SlashCommand, SessionUiEvent, TerminalCommandRun, TerminalCommandRunFilter, TerminalCommandRunHandle, TerminalCommandRunStatus, TerminalInfo, TerminalUiEvent, ThinkingLevel, ThinkingLevelsResponse, WriteWorkspaceFileOptions, WriteWorkspaceFileResponse, Workspace, WorkspaceActivity, WorkspaceActivityResponse, WorkspaceActivityUiEvent } from "../../shared/apiTypes";
@@ -1,9 +1,11 @@
-import type { AskUserSubmission, DeleteWorkspaceFileResponse, ExtensionDialogAnswer, FileSuggestion, MoveWorkspaceFileOptions, PiPackageInstallRequest, PiPackageRemoveRequest, PiPackageScope, PiPackageUpdateRequest, PiWebConfigValues, PromptAttachment, RunTerminalCommandInput, SessionBulkMutationRef, SessionCleanupRequest, SessionNotificationDismissThrough, SessionRef, SessionTreeNavigateRequest, SessionUnreadAcknowledgeRequest, TerminalCommandRun, TerminalCommandRunFilter, WriteWorkspaceFileOptions } from "../../../shared/apiTypes";
+import type { AskUserSubmission, AzureSpeechSettings, AzureSpeechSettingsUpdate, AzureSpeechToken, DeleteWorkspaceFileResponse, ExtensionDialogAnswer, FileSuggestion, MoveWorkspaceFileOptions, PiPackageInstallRequest, PiPackageRemoveRequest, PiPackageScope, PiPackageUpdateRequest, PiWebConfigValues, PromptAttachment, RunTerminalCommandInput, SessionBulkMutationRef, SessionCleanupRequest, SessionNotificationDismissThrough, SessionRef, SessionTreeNavigateRequest, SessionUnreadAcknowledgeRequest, TerminalCommandRun, TerminalCommandRunFilter, WriteWorkspaceFileOptions } from "../../../shared/apiTypes";
 import { resolveAppUrl } from "../appUrl";
 import { request } from "./http";
 import {
  arrayOf,
  parseAborted,
+  parseAzureSpeechSettings,
+  parseAzureSpeechToken,
  parseAskUserCloseResponse,
  parseAccepted,
  parseArchived,
@@ -137,6 +139,12 @@ export const configApi = {
  saveConfig: (config: PiWebConfigValues, machineId?: string) => request(configPath(machineId), parsePiWebConfigResponse, { method: "PUT", body: JSON.stringify({ config }) }),
 };

+export const azureSpeechApi = {
+  settings: (): Promise<AzureSpeechSettings> => request("api/azure-speech", parseAzureSpeechSettings),
+  saveSettings: (settings: AzureSpeechSettingsUpdate): Promise<AzureSpeechSettings> => request("api/azure-speech", parseAzureSpeechSettings, { method: "PUT", body: JSON.stringify(settings) }),
+  token: (): Promise<AzureSpeechToken> => request("api/azure-speech/token", parseAzureSpeechToken, { method: "POST" }),
+};
+
 export const pluginsApi = {
  plugins: (machineId?: string) => request(pluginsPath(machineId), parsePiWebPluginsResponse),
 };
@@ -1,4 +1,4 @@
-import { ASK_USER_ID_MAX_LENGTH, ASK_USER_OPTION_LIMIT, ASK_USER_OTHER_TEXT_MAX_LENGTH, ASK_USER_QUESTION_LIMIT, ASK_USER_TEXT_MAX_LENGTH, EXTENSION_DIALOG_ID_MAX_LENGTH, EXTENSION_DIALOG_INPUT_MAX_LENGTH, EXTENSION_DIALOG_OPTION_LIMIT, EXTENSION_DIALOG_TEXT_MAX_LENGTH, SESSION_NOTIFICATION_LIMIT, SESSION_NOTIFICATION_MESSAGE_BYTES, SESSION_UNREAD_CATALOG_ID_MAX_LENGTH, SESSION_UNREAD_COMPLETED_AT_MAX_LENGTH, SESSION_UNREAD_CWD_MAX_LENGTH, SESSION_UNREAD_LIMIT, SESSION_UNREAD_SESSION_ID_MAX_LENGTH, type ArchiveSessionsResponse, type AskUserCloseReason, type AskUserCloseResponse, type AskUserOutcome, type AskUserQuestion, type AskUserQuestionOption, type AskUserQuestionRecord, type PendingAskUser, type PendingExtensionDialog, type AuthProviderOption, type AuthProviderStatus, type AuthProvidersResponse, type AuthStatusSource, type AuthType, type CommandOption, type CommandResult, type DeleteWorkspaceFileResponse, type ExtensionDialogAnswer, type ExtensionDialogCloseReason, type ExtensionDialogCloseResponse, type ExtensionDialogKind, type ExtensionDialogOutcome, type FileContentResponse, type FileSuggestion, type FileTreeEntry, type FileTreeResponse, type GitDiffResponse, type GitFileState, type GitStatusFile, type GitStatusResponse, type Machine, type MachineHealth, type MachineKind, type MachineRuntime, type MachineStatus, type MessagePage, type ModelSelectionResponse, type MoveWorkspaceFileResponse, type OAuthFlowState, type PiWebAgentDirEnvSource, type PiWebCapability, type PiWebComponentStatus, type PiWebConfigEnvOverrides, type PiWebConfigResponse, type PiWebConfigValues, type PiWebInstallationInfo, type PiWebPluginConfigMap, type PiWebPluginInfo, type PiWebPluginsResponse, type PiWebPluginScope, type PiWebReleaseStatus, type PiWebRuntimeComponent, type PiWebRuntimeResponse, type PiWebServiceComponent, type PiWebShortcutConfig, type PiWebStatusMessage, type PiWebStatusResponse, type PiWebStatusSeverity, type Project, type QueuedSessionMessage, type SavedPromptAttachment, type SessionBulkArchiveResponse, type SessionBulkDeleteArchivedResponse, type SessionBulkFailure, type SessionCleanupExecuteResponse, type SessionCleanupPreviewResponse, type SessionCleanupProjectSummary, type SessionCleanupThresholds, type SessionCleanupTotals, type SessionInfo, type SessionModel, type SessionNotification, type SessionNotificationClearReason, type SessionNotificationDismissThrough, type SessionNotificationInboxDelta, type SessionNotificationInboxEvent, type SessionNotificationInboxSnapshot, type SessionNotificationSeverity, type SessionNotificationSummary, type SessionStatus, type SessionStreamSnapshot, type SessionUnreadCatalogSnapshot, type SessionUnreadEvent, type SessionUnreadSummary, type SessionWarning, type SessionWarningSeverity, type SlashCommand, type TerminalCommandRun, type TerminalCommandRunStatus, type TerminalInfo, type ThinkingLevelsResponse, type WriteWorkspaceFileResponse, type Workspace, type WorkspaceActivity, type WorkspaceActivityResponse } from "../../../shared/apiTypes";
+import { ASK_USER_ID_MAX_LENGTH, ASK_USER_OPTION_LIMIT, ASK_USER_OTHER_TEXT_MAX_LENGTH, ASK_USER_QUESTION_LIMIT, ASK_USER_TEXT_MAX_LENGTH, EXTENSION_DIALOG_ID_MAX_LENGTH, EXTENSION_DIALOG_INPUT_MAX_LENGTH, EXTENSION_DIALOG_OPTION_LIMIT, EXTENSION_DIALOG_TEXT_MAX_LENGTH, SESSION_NOTIFICATION_LIMIT, SESSION_NOTIFICATION_MESSAGE_BYTES, SESSION_UNREAD_CATALOG_ID_MAX_LENGTH, SESSION_UNREAD_COMPLETED_AT_MAX_LENGTH, SESSION_UNREAD_CWD_MAX_LENGTH, SESSION_UNREAD_LIMIT, SESSION_UNREAD_SESSION_ID_MAX_LENGTH, type ArchiveSessionsResponse, type AzureSpeechSettings, type AzureSpeechToken, type AskUserCloseReason, type AskUserCloseResponse, type AskUserOutcome, type AskUserQuestion, type AskUserQuestionOption, type AskUserQuestionRecord, type PendingAskUser, type PendingExtensionDialog, type AuthProviderOption, type AuthProviderStatus, type AuthProvidersResponse, type AuthStatusSource, type AuthType, type CommandOption, type CommandResult, type DeleteWorkspaceFileResponse, type ExtensionDialogAnswer, type ExtensionDialogCloseReason, type ExtensionDialogCloseResponse, type ExtensionDialogKind, type ExtensionDialogOutcome, type FileContentResponse, type FileSuggestion, type FileTreeEntry, type FileTreeResponse, type GitDiffResponse, type GitFileState, type GitStatusFile, type GitStatusResponse, type Machine, type MachineHealth, type MachineKind, type MachineRuntime, type MachineStatus, type MessagePage, type ModelSelectionResponse, type MoveWorkspaceFileResponse, type OAuthFlowState, type PiWebAgentDirEnvSource, type PiWebCapability, type PiWebComponentStatus, type PiWebConfigEnvOverrides, type PiWebConfigResponse, type PiWebConfigValues, type PiWebInstallationInfo, type PiWebPluginConfigMap, type PiWebPluginInfo, type PiWebPluginsResponse, type PiWebPluginScope, type PiWebReleaseStatus, type PiWebRuntimeComponent, type PiWebRuntimeResponse, type PiWebServiceComponent, type PiWebShortcutConfig, type PiWebStatusMessage, type PiWebStatusResponse, type PiWebStatusSeverity, type Project, type QueuedSessionMessage, type SavedPromptAttachment, type SessionBulkArchiveResponse, type SessionBulkDeleteArchivedResponse, type SessionBulkFailure, type SessionCleanupExecuteResponse, type SessionCleanupPreviewResponse, type SessionCleanupProjectSummary, type SessionCleanupThresholds, type SessionCleanupTotals, type SessionInfo, type SessionModel, type SessionNotification, type SessionNotificationClearReason, type SessionNotificationDismissThrough, type SessionNotificationInboxDelta, type SessionNotificationInboxEvent, type SessionNotificationInboxSnapshot, type SessionNotificationSeverity, type SessionNotificationSummary, type SessionStatus, type SessionStreamSnapshot, type SessionUnreadCatalogSnapshot, type SessionUnreadEvent, type SessionUnreadSummary, type SessionWarning, type SessionWarningSeverity, type SlashCommand, type TerminalCommandRun, type TerminalCommandRunStatus, type TerminalInfo, type ThinkingLevelsResponse, type WriteWorkspaceFileResponse, type Workspace, type WorkspaceActivity, type WorkspaceActivityResponse } from "../../../shared/apiTypes";
 import type { PiPackageInfo, PiPackageMutationAction, PiPackageMutationResponse, PiPackageScope, PiPackagesResponse, SessionActivity, SessionStartupProgressEvent, SessionTreeNavigateResult, SessionTreeNode, SessionTreeNodeKind, SessionTreeSnapshot } from "../../../shared/apiTypes";
 import { parseActiveAgentProfileDescriptor } from "../../../shared/activeAgentProfile";
 import { parseKnownPiWebCapabilities } from "../../../shared/capabilities";
@@ -1207,6 +1207,16 @@ export function parseWorkspaceActivityResponse(value: unknown): WorkspaceActivit
  return { workspaces: arrayOf(parseWorkspaceActivity)(record["workspaces"]), generatedAt: requireString(record, "generatedAt") };
 }

+export function parseAzureSpeechSettings(value: unknown): AzureSpeechSettings {
+  const record = requireRecord(value);
+  return { region: requireString(record, "region"), voice: requireString(record, "voice"), hasApiKey: requireBoolean(record, "hasApiKey") };
+}
+
+export function parseAzureSpeechToken(value: unknown): AzureSpeechToken {
+  const record = requireRecord(value);
+  return { token: requireString(record, "token"), region: requireString(record, "region"), voice: requireString(record, "voice") };
+}
+
 export function parsePiWebConfigResponse(value: unknown): PiWebConfigResponse {
  const record = requireRecord(value);
  return {
@@ -0,0 +1,204 @@
+import * as SpeechSDK from "microsoft-cognitiveservices-speech-sdk";
+import { azureSpeechApi } from "./api";
+import { transitionVoiceRecognition, type VoiceRecognitionPhase } from "./voiceRecognitionLifecycle";
+
+const VOICE_MODE_STORAGE_KEY = "pi-web.azure-speech.voice-mode";
+
+export function voiceModeEnabled(): boolean {
+  return window.localStorage.getItem(VOICE_MODE_STORAGE_KEY) === "true";
+}
+
+export function setVoiceModeEnabled(enabled: boolean): void {
+  window.localStorage.setItem(VOICE_MODE_STORAGE_KEY, String(enabled));
+}
+
+/** Call from the microphone tap handler so iOS permits later response playback. */
+export function primeVoiceAudio(): void {
+  AzureSpeechClient.primeSpeaker();
+}
+
+export class AzureSpeechClient {
+  static speaker: HTMLAudioElement | undefined;
+  private recognizer: SpeechSDK.SpeechRecognizer | undefined;
+  private stream: MediaStream | undefined;
+  private phase: VoiceRecognitionPhase = "idle";
+  private generation = 0;
+  private synthesizer: SpeechSDK.SpeechSynthesizer | undefined;
+
+  static primeSpeaker(): void {
+    const speaker = this.speaker ?? new Audio();
+    this.speaker = speaker;
+    // A muted, zero-length WAV played directly from the mic button's user
+    // gesture unlocks this element for later Azure response playback on iOS.
+    speaker.muted = true;
+    speaker.src = "data:audio/wav;base64,UklGRiQAAABXQVZFZm10IBAAAAABAAEAQB8AAEAfAAABAAgAZGF0YQAAAAA=";
+    void speaker.play().then(() => { speaker.pause(); }).catch(() => undefined);
+  }
+
+  get recording(): boolean {
+    return this.phase !== "idle";
+  }
+
+  /**
+   * Begins one persistent microphone session. Its MediaStream is acquired in
+   * the button's user gesture and is retained between turns, which is required
+   * for iOS to accept follow-up speech without another tap.
+   */
+  async startRecognition(options: { onReady: () => void; onInterim: (text: string) => void; onFinal: (text: string) => void; onStopped: () => void; }): Promise<void> {
+    if (this.phase !== "idle") return;
+    if (!window.isSecureContext || !hasMicrophoneAccess(navigator)) {
+      throw new Error("Voice input requires PI WEB to be opened over HTTPS. iPhone browsers do not expose the microphone to an http:// LAN address.");
+    }
+
+    const generation = ++this.generation;
+    // This must be the first asynchronous operation so getUserMedia starts in
+    // the microphone button's gesture, not after a token/network round trip.
+    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+    this.stream = stream;
+    try {
+      const settings = await azureSpeechApi.token();
+      if (generation !== this.generation) { stopStream(stream); return; }
+
+      const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(settings.token, settings.region);
+      speechConfig.speechRecognitionLanguage = navigator.language || "en-US";
+      speechConfig.setProperty(SpeechSDK.PropertyId.Speech_SegmentationSilenceTimeoutMs, "1200");
+      const recognizer = new SpeechSDK.SpeechRecognizer(speechConfig, SpeechSDK.AudioConfig.fromStreamInput(stream));
+      this.recognizer = recognizer;
+      this.phase = transitionVoiceRecognition(this.phase, "start");
+      recognizer.recognizing = (_sender, event) => {
+        if (this.generation === generation && this.phase === "listening") options.onInterim(event.result.text);
+      };
+      recognizer.recognized = (_sender, event) => {
+        if (this.generation !== generation || this.phase !== "listening") return;
+        if (event.result.reason !== SpeechSDK.ResultReason.RecognizedSpeech || event.result.text.trim() === "") return;
+        // Gate the already-authorized stream before notifying the UI. This
+        // prevents another utterance or TTS from being captured while PI works.
+        this.setPhase(transitionVoiceRecognition(this.phase, "final"));
+        options.onFinal(event.result.text);
+      };
+      recognizer.canceled = () => {
+        if (this.generation !== generation || this.phase === "idle") return;
+        this.releaseRecognition();
+        options.onStopped();
+      };
+      await callbackPromise((resolve, reject) => { recognizer.startContinuousRecognitionAsync(resolve, reject); });
+      if (this.generation !== generation) return;
+      options.onReady();
+    } catch (error) {
+      if (generation === this.generation) {
+        if (this.stream !== stream) stopStream(stream);
+        this.releaseRecognition();
+      }
+      throw error;
+    }
+  }
+
+  /** Re-enable the existing, user-authorized microphone after response audio. */
+  resumeRecognition(): void {
+    if (this.phase !== "paused") return;
+    this.setPhase(transitionVoiceRecognition(this.phase, "resume"));
+  }
+
+  async stopRecognition(): Promise<void> {
+    if (this.phase === "idle" && this.stream === undefined) return;
+    ++this.generation;
+    const recognizer = this.recognizer;
+    this.releaseRecognition();
+    if (recognizer !== undefined) {
+      await callbackPromise((resolve, reject) => { recognizer.stopContinuousRecognitionAsync(resolve, reject); }).catch(() => undefined);
+      recognizer.close();
+    }
+  }
+
+  async speak(text: string): Promise<void> {
+    const spoken = text.trim();
+    if (spoken === "") return;
+    this.stopSpeaking();
+    const settings = await azureSpeechApi.token();
+    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(settings.token, settings.region);
+    if (settings.voice !== "") speechConfig.speechSynthesisVoiceName = settings.voice;
+    speechConfig.speechSynthesisOutputFormat = SpeechSDK.SpeechSynthesisOutputFormat.Audio24Khz48KBitRateMonoMp3;
+    const synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig, null);
+    this.synthesizer = synthesizer;
+    try {
+      const audioData = await synthesisAudio(synthesizer, spoken);
+      await playAudioData(audioData);
+    } finally {
+      if (this.synthesizer === synthesizer) this.synthesizer = undefined;
+      synthesizer.close();
+    }
+  }
+
+  stopSpeaking(): void {
+    const synthesizer = this.synthesizer;
+    this.synthesizer = undefined;
+    synthesizer?.close();
+  }
+
+  dispose(): void {
+    void this.stopRecognition();
+    this.stopSpeaking();
+  }
+
+  private setPhase(phase: VoiceRecognitionPhase): void {
+    this.phase = phase;
+    for (const track of this.stream?.getAudioTracks() ?? []) track.enabled = phase === "listening";
+  }
+
+  private releaseRecognition(): void {
+    this.phase = transitionVoiceRecognition(this.phase, "stop");
+    this.recognizer = undefined;
+    stopStream(this.stream);
+    this.stream = undefined;
+  }
+}
+
+function hasMicrophoneAccess(value: Navigator): boolean {
+  const mediaDevices: unknown = Reflect.get(value, "mediaDevices");
+  return typeof mediaDevices === "object" && mediaDevices !== null && typeof Reflect.get(mediaDevices, "getUserMedia") === "function";
+}
+
+function stopStream(stream: MediaStream | undefined): void {
+  for (const track of stream?.getTracks() ?? []) track.stop();
+}
+
+function synthesisAudio(synthesizer: SpeechSDK.SpeechSynthesizer, text: string): Promise<ArrayBuffer> {
+  return new Promise((resolve, reject) => {
+    synthesizer.speakTextAsync(
+      text,
+      (result) => {
+        if (result.reason !== SpeechSDK.ResultReason.SynthesizingAudioCompleted) {
+          reject(new Error(`Azure Speech synthesis failed: ${result.errorDetails || String(result.reason)}`));
+          return;
+        }
+        resolve(result.audioData);
+      },
+      (error) => { reject(new Error(error)); },
+    );
+  });
+}
+
+function playAudioData(audioData: ArrayBuffer): Promise<void> {
+  const speaker = AzureSpeechClient.speaker ?? new Audio();
+  AzureSpeechClient.speaker = speaker;
+  speaker.muted = false;
+  const url = URL.createObjectURL(new Blob([audioData], { type: "audio/mpeg" }));
+  speaker.src = url;
+  return new Promise((resolve, reject) => {
+    const releaseAudioSession = () => {
+      speaker.onended = null;
+      speaker.onerror = null;
+      speaker.pause();
+      speaker.removeAttribute("src");
+      speaker.load();
+      URL.revokeObjectURL(url);
+    };
+    speaker.onended = () => { releaseAudioSession(); resolve(); };
+    speaker.onerror = () => { releaseAudioSession(); reject(new Error("Unable to play Azure Speech audio.")); };
+    void speaker.play().catch((error: unknown) => { releaseAudioSession(); reject(error instanceof Error ? error : new Error(String(error))); });
+  });
+}
+
+function callbackPromise(start: (resolve: () => void, reject: (error: string) => void) => void): Promise<void> {
+  return new Promise((resolve, reject) => { start(resolve, (error) => { reject(new Error(error)); }); });
+}
@@ -76,6 +76,7 @@ import { shouldShowMachinesSection, type AppNavigationPanel, type NavigationFocu
 import "./appShell/AppPanelEdgeControl";
 import "./appShell/AppRefreshControl";
 import { appStyles } from "./shared";
+import { AzureSpeechClient, voiceModeEnabled } from "../azureSpeechClient";


 const PI_WEB_STATUS_REFRESH_MS = 15 * 60 * 1000;
@@ -110,6 +111,7 @@ export class PiWebApp extends LitElement {
  @query("#navigation-panel") private navigationPanelFrame?: HTMLElement;
  @query("#workspace-panel") private workspacePanelFrame?: HTMLElement;

+  private readonly azureSpeech = new AzureSpeechClient();
  private readonly sessionUnread = new SessionUnreadController({
    onChange: (machineId) => {
      this.syncUnreadPresence();
@@ -142,6 +144,12 @@ export class PiWebApp extends LitElement {
      onSelectedSessionReady: ({ machineId, session }) => {
        void this.commitReadyChatAfterRender(machineId, session);
      },
+      onFinalAssistantResponse: ({ text }) => {
+        if (!voiceModeEnabled()) return;
+        void this.azureSpeech.speak(text)
+          .catch((error: unknown) => { console.warn("Azure Speech synthesis failed", error); })
+          .finally(() => { this.promptEditor?.resumeVoiceInput(); });
+      },
      replacePromptEditorText: async ({ machineId, sessionId, text }) => {
        await this.updateComplete;
        if (selectedMachineId(this.state) !== machineId || this.state.selectedSession?.id !== sessionId) return;
@@ -397,6 +405,7 @@ export class PiWebApp extends LitElement {
    this.systemLightThemeMedia?.removeEventListener("change", this.onSystemLightThemeChange);
    this.keyboard.reset();
    this.auth.dispose();
+    this.azureSpeech.dispose();
    this.sessions.dispose();
    this.notifications.dispose();
    this.realtime.close();
@@ -6,6 +6,7 @@ import { defaultHighlightStyle, indentOnInput, indentUnit, syntaxHighlighting }
 import { LitElement, html, type PropertyValues } from "lit";
 import { customElement, property, query, state } from "lit/decorators.js";
 import { api, type FileSuggestion, type PromptAttachment, type SessionModel, type SessionStatus, type SlashCommand } from "../api";
+import { AzureSpeechClient, primeVoiceAudio, setVoiceModeEnabled } from "../azureSpeechClient";
 import type { PromptAttachmentDelivery } from "../../../shared/apiTypes";
 import { capturePromptAttachments, effectivePromptAttachmentDelivery, isInlinePromptAttachment, promptAttachmentsCanUseInlineDelivery, type CapturedAttachment } from "../promptAttachmentCapture";
 import { inputModeForDraft, inputModesEqual, type InputMode } from "../inputModes";
@@ -15,7 +16,7 @@ import { clearDraft, loadDraft, saveDraft } from "../promptDraftStorage";
 import { loadAttachmentDelivery, saveAttachmentDelivery } from "../attachmentPreferences";
 import { createMobilePromptEnterMedia, readPromptEnterPreference, shouldSendPromptOnEnterShortcut, shouldUsePromptEnterShiftShortcut } from "../promptEnterBehavior";
 import { promptEditorStyles, type CompletionItem } from "./shared";
-import { renderAttachIcon, renderSendIcon, renderQueueIcon, renderSteerIcon, renderStopIcon, renderThinkingGauge } from "./promptEditorIcons";
+import { renderAttachIcon, renderSendIcon, renderQueueIcon, renderSteerIcon, renderStopIcon, renderThinkingGauge, renderVoiceIcon } from "./promptEditorIcons";
 import { thinkingGauge, thinkingLevelLabel } from "../../../shared/thinkingLevels";
 import "./AutocompleteMenu";

@@ -55,6 +56,11 @@ export class PromptEditor extends LitElement {
  @state() private attachments: PendingAttachment[] = [];
  @state() private attachmentDelivery: PromptAttachmentDelivery = loadAttachmentDelivery();
  @state() private attachmentError: string | undefined = undefined;
+  @state() private voiceRecording = false;
+  @state() private voiceError: string | undefined = undefined;
+  private readonly azureSpeech = new AzureSpeechClient();
+  private voiceTranscript = "";
+  private voicePrefix = "";
  private attachmentSeq = 0;
  private requestVersion = 0;
  private editor: EditorView | undefined;
@@ -97,6 +103,7 @@ export class PromptEditor extends LitElement {
  }

  override disconnectedCallback(): void {
+    this.azureSpeech.dispose();
    this.editor?.destroy();
    this.editor = undefined;
    super.disconnectedCallback();
@@ -116,10 +123,12 @@ export class PromptEditor extends LitElement {
          ${shellMode ? html`<div class="mode-hint">Shell command${shellInputMode.excludeFromContext ? " · excluded from context" : ""}</div>` : null}
          ${this.isCompacting && !shellMode ? html`<div class="mode-hint">Compacting history · message will be queued</div>` : null}
          ${this.renderAttachments()}
+          ${this.voiceError === undefined ? null : html`<div class="mode-hint voice-error" role="alert">${this.voiceError}</div>`}
          <autocomplete-menu .items=${this.completions} .selectedIndex=${this.selectedIndex} .onPick=${(item: CompletionItem) => { this.pick(item); }}></autocomplete-menu>
        </div>
        <div class="actions">
          ${this.renderCompactStatus()}
+          <button class=${`icon-button voice-button${this.voiceRecording ? " voice-recording" : ""}`} ?disabled=${busy} title=${this.voiceRecording ? "Cancel voice input" : "Start voice message"} aria-label=${this.voiceRecording ? "Cancel voice input" : "Start voice message"} @click=${() => { void this.toggleVoice(); }}>${renderVoiceIcon(this.voiceRecording)}</button>
          <button class="icon-button send-button" ?disabled=${busy} title=${queuesInput ? "Queue until the current activity finishes" : "Send message"} aria-label=${queuesInput ? "Queue message" : "Send message"} @click=${() => { this.send("followUp"); }}>${queuesInput ? renderQueueIcon() : renderSendIcon()}</button>
          ${this.canSteer && !this.isCompacting ? html`<button class="icon-button steer-button" ?disabled=${busy} title="Steer the current response before the next model call" aria-label="Steer current response" @click=${() => { this.send("steer"); }}>${renderSteerIcon()}</button>` : null}
          <button class="icon-button stop-button" ?disabled=${this.disabled || !this.canStop} title=${this.canStop ? "Stop current work and clear queued messages" : "Nothing running"} aria-label="Stop current work" @click=${() => this.onStop?.()}>${renderStopIcon()}</button>
@@ -467,6 +476,55 @@ export class PromptEditor extends LitElement {
    this.completions = [];
  }

+  private async toggleVoice(): Promise<void> {
+    this.voiceError = undefined;
+    if (this.voiceRecording) {
+      this.voiceRecording = false;
+      setVoiceModeEnabled(false);
+      await this.azureSpeech.stopRecognition();
+      return;
+    }
+    // Must run directly in the button gesture so iOS authorizes this stream
+    // for every follow-up turn, not just the first one.
+    primeVoiceAudio();
+    this.voiceRecording = true;
+    setVoiceModeEnabled(true);
+    this.voicePrefix = this.draft.trim();
+    this.voiceTranscript = "";
+    try {
+      await this.azureSpeech.startRecognition({
+        onReady: prepareVoiceReadyChime(),
+        onInterim: (text) => { this.replaceText(this.voiceText(text)); },
+        onFinal: (text) => {
+          this.voiceTranscript = text;
+          this.replaceText(this.voiceText());
+          this.send("followUp");
+        },
+        onStopped: () => {
+          if (!this.voiceRecording) return;
+          this.voiceRecording = false;
+          setVoiceModeEnabled(false);
+        },
+      });
+    } catch (error) {
+      this.voiceError = errorMessage(error);
+      this.voiceRecording = false;
+      setVoiceModeEnabled(false);
+    }
+  }
+
+  /** Called by PiWebApp after final TTS playback (or a synthesis failure). */
+  resumeVoiceInput(): void {
+    if (!this.voiceRecording) return;
+    this.voicePrefix = this.draft.trim();
+    this.voiceTranscript = "";
+    this.azureSpeech.resumeRecognition();
+  }
+
+  private voiceText(interim = ""): string {
+    return [this.voicePrefix, this.voiceTranscript, interim].filter((part) => part.trim() !== "").join(" ");
+  }
+
  private send(streamingBehavior?: "steer" | "followUp") {
    if (this.disabled || this.sending) return;
    const text = this.draft.trim();
@@ -584,6 +642,36 @@ const codeLikeInputAssistanceAttributes: Record<string, string> = {
  dir: "auto",
 };

+function prepareVoiceReadyChime(): () => void {
+  // Create and resume the context in the tap handler. That keeps iOS' user-
+  // gesture audio permission alive until Azure confirms the microphone session.
+  const context = new AudioContext();
+  void context.resume();
+  let played = false;
+  return () => {
+    if (played) return;
+    played = true;
+    const start = context.currentTime;
+    for (const [offset, frequency] of [[0, 523.25], [0.09, 783.99]] as const) {
+      const oscillator = context.createOscillator();
+      const gain = context.createGain();
+      oscillator.type = "sine";
+      oscillator.frequency.setValueAtTime(frequency, start + offset);
+      gain.gain.setValueAtTime(0.0001, start + offset);
+      gain.gain.exponentialRampToValueAtTime(0.055, start + offset + 0.018);
+      gain.gain.exponentialRampToValueAtTime(0.0001, start + offset + 0.24);
+      oscillator.connect(gain).connect(context.destination);
+      oscillator.start(start + offset);
+      oscillator.stop(start + offset + 0.25);
+    }
+    window.setTimeout(() => { void context.close(); }, 500);
+  };
+}
+
+function errorMessage(error: unknown): string {
+  return error instanceof Error ? error.message : String(error);
+}
+
 function inputAssistanceContentAttributes(draftBeforeCursor: string): Record<string, string> {
  // CodeMirror is optimized for code and disables these by default, but the chat prompt is usually prose.
  return inputModeForDraft(draftBeforeCursor).kind === "normal" ? proseInputAssistanceAttributes : codeLikeInputAssistanceAttributes;
@@ -4,6 +4,7 @@ import type { AppAction } from "../actions";
 import { configApi, piPackagesApi, pluginsApi, type Machine, type MachineRuntime, type PiPackageMutationResponse, type PiPackageScope, type PiPackagesResponse, type PiWebConfigResponse, type PiWebConfigValues, type PiWebPluginsResponse } from "../api";
 import type { SettingsSection } from "../settingsRoute";
 import "./settings/SettingsGeneralPanel";
+import "./settings/SettingsAzureSpeechPanel";
 import "./settings/SettingsSessiondPanel";
 import "./settings/SettingsPackagesPanel";
 import "./settings/SettingsPluginsPanel";
@@ -118,6 +119,7 @@ export class SettingsDialog extends LitElement {
              ${this.renderNavButton("packages", "Pi packages", "Selected machine")}
              ${this.renderNavButton("plugins", "PI WEB plugins", "Selected machine")}
              ${this.renderNavButton("shortcuts", "Keyboard", "Gateway shortcuts")}
+              ${this.renderNavButton("azure-speech", "Azure Speech", "Voice input + replies")}
            </nav>
            <main class="settings-content">
              ${this.renderActiveSection()}
@@ -148,6 +150,9 @@ export class SettingsDialog extends LitElement {
        ></settings-sessiond-panel>
      `;
    }
+    if (this.section === "azure-speech") {
+      return html`<settings-azure-speech-panel></settings-azure-speech-panel>`;
+    }
    if (this.section === "shortcuts") {
      return html`
        <settings-shortcuts-panel
@@ -686,7 +691,8 @@ export type SettingsPanelTag =
  | "settings-sessiond-panel"
  | "settings-packages-panel"
  | "settings-plugins-panel"
-  | "settings-shortcuts-panel";
+  | "settings-shortcuts-panel"
+  | "settings-azure-speech-panel";

 /**
 * The single custom-element panel the settings dialog renders for a section.
@@ -706,6 +712,8 @@ export function activeSettingsPanelTag(section: SettingsSection): SettingsPanelT
      return "settings-plugins-panel";
    case "shortcuts":
      return "settings-shortcuts-panel";
+    case "azure-speech":
+      return "settings-azure-speech-panel";
    case "general":
      return "settings-general-panel";
  }
@@ -13,6 +13,17 @@ export function renderAttachIcon(): TemplateResult {
  `;
 }

+export function renderVoiceIcon(recording = false): TemplateResult {
+  return svg`
+    <svg class=${`prompt-action-icon${recording ? " prompt-action-icon-filled" : ""}`} viewBox="0 0 24 24" aria-hidden="true" focusable="false">
+      <rect x="9" y="3" width="6" height="11" rx="3"></rect>
+      <path d="M6 11a6 6 0 0 0 12 0"></path>
+      <path d="M12 17v4"></path>
+      <path d="M8 21h8"></path>
+    </svg>
+  `;
+}
+
 export function renderSendIcon(): TemplateResult {
  return svg`
    <svg class="prompt-action-icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false">
@@ -0,0 +1,123 @@
+import { css, html, LitElement, type TemplateResult } from "lit";
+import { customElement, state } from "lit/decorators.js";
+import { azureSpeechApi, type AzureSpeechSettings } from "../../api";
+import "./SettingsPanelFrame";
+import type { SettingsNotice } from "./SettingsPanelFrame";
+
+@customElement("settings-azure-speech-panel")
+export class SettingsAzureSpeechPanel extends LitElement {
+  @state() private settings: AzureSpeechSettings | undefined;
+  @state() private region = "eastus";
+  @state() private voice = "";
+  @state() private apiKey = "";
+  @state() private loading = true;
+  @state() private saving = false;
+  @state() private error = "";
+  @state() private saved = "";
+
+  override connectedCallback(): void {
+    super.connectedCallback();
+    void this.load();
+  }
+
+  override render(): TemplateResult {
+    return html`
+      <settings-panel-frame
+        heading="Azure Speech"
+        description="Real-time speech recognition and spoken final replies. Your key is stored only on this PI WEB host and is never returned to the browser."
+        actionLabel="Reload"
+        .actionDisabled=${this.loading || this.saving}
+        .notices=${this.notices()}
+        .onAction=${() => { void this.load(); }}
+      >
+        <section class="settings-card">
+          <div class="card-heading">
+            <h3>Azure AI Speech</h3>
+            <p>Use the region for your Speech resource and the exact Azure voice name you want for text-to-speech. Leave the voice empty to use Azure's default voice.</p>
+          </div>
+          ${this.loading ? html`<div class="loading-card">Loading Azure Speech settings…</div>` : html`
+            <form class="config-form" @submit=${(event: Event) => { void this.save(event); }}>
+              <label class="field">
+                <span>Speech region</span>
+                <input .value=${this.region} required autocomplete="off" spellcheck="false" placeholder="eastus" @input=${(event: Event) => { this.region = inputValue(event); this.error = ""; }}>
+                <small>For example, <code>eastus</code>. This must match the Azure Speech resource.</small>
+              </label>
+              <label class="field">
+                <span>DragonHD voice name</span>
+                <input .value=${this.voice} autocomplete="off" spellcheck="false" placeholder="en-US-Andrew:DragonHDLatestNeural" @input=${(event: Event) => { this.voice = inputValue(event); this.error = ""; }}>
+                <small>Paste the exact voice identifier available to your resource.</small>
+              </label>
+              <label class="field">
+                <span>Speech API key</span>
+                <input type="password" .value=${this.apiKey} autocomplete="new-password" spellcheck="false" placeholder=${this.settings?.hasApiKey === true ? "Saved — enter a replacement or leave blank" : "Azure AI Speech key"} @input=${(event: Event) => { this.apiKey = inputValue(event); this.error = ""; }}>
+                <small>${this.settings?.hasApiKey === true ? "A key is already saved. Leaving this blank keeps it unchanged; entering a new value replaces it." : "The key is sent once over your PI WEB connection and stored in a separate owner-only file."}</small>
+              </label>
+              <footer class="form-actions"><button class="primary" ?disabled=${this.saving}>${this.saving ? "Saving…" : "Save Azure Speech settings"}</button></footer>
+            </form>
+          `}
+        </section>
+      </settings-panel-frame>
+    `;
+  }
+
+  private notices(): readonly SettingsNotice[] {
+    return [
+      ...(this.error === "" ? [] : [{ type: "error" as const, content: this.error }]),
+      ...(this.saved === "" ? [] : [{ type: "success" as const, content: this.saved }]),
+    ];
+  }
+
+  private async load(): Promise<void> {
+    this.loading = true;
+    this.error = "";
+    try {
+      const settings = await azureSpeechApi.settings();
+      this.settings = settings;
+      this.region = settings.region;
+      this.voice = settings.voice;
+      this.apiKey = "";
+    } catch (error) {
+      this.error = errorMessage(error);
+    } finally {
+      this.loading = false;
+    }
+  }
+
+  private async save(event: Event): Promise<void> {
+    event.preventDefault();
+    this.saving = true;
+    this.error = "";
+    this.saved = "";
+    try {
+      this.settings = await azureSpeechApi.saveSettings({ region: this.region, voice: this.voice, ...(this.apiKey === "" ? {} : { apiKey: this.apiKey }) });
+      this.apiKey = "";
+      this.saved = "Azure Speech settings saved.";
+    } catch (error) {
+      this.error = errorMessage(error);
+    } finally {
+      this.saving = false;
+    }
+  }
+
+  static override styles = css`
+    :host { display: block; }
+    .settings-card, .loading-card { display: grid; gap: 14px; border: 1px solid var(--pi-border); border-radius: 10px; background: var(--pi-surface); padding: 12px; }
+    .card-heading, .config-form, .field { display: grid; gap: 7px; }
+    h3, p { margin: 0; } p, small { color: var(--pi-muted); line-height: 1.45; }
+    .field > span { color: var(--pi-muted); font-size: 12px; font-weight: 700; text-transform: uppercase; }
+    input, button { font: inherit; } input { box-sizing: border-box; width: 100%; border: 1px solid var(--pi-border); border-radius: 8px; background: var(--pi-bg); color: var(--pi-text); padding: 9px 10px; }
+    input:focus { outline: none; border-color: var(--pi-accent); box-shadow: 0 0 0 1px var(--pi-accent-border); }
+    code { font: 12px ui-monospace, monospace; }
+    .form-actions { display: flex; justify-content: flex-end; padding-top: 2px; }
+    button { border: 1px solid var(--pi-border); border-radius: 8px; background: var(--pi-surface); color: var(--pi-text); padding: 7px 9px; cursor: pointer; }
+    button:disabled { opacity: .55; cursor: not-allowed; } .primary { border-color: var(--pi-accent); background: var(--pi-selection-bg); color: var(--pi-text-bright); }
+  `;
+}
+
+function inputValue(event: Event): string {
+  return event.target instanceof HTMLInputElement ? event.target.value : "";
+}
+
+function errorMessage(error: unknown): string {
+  return error instanceof Error ? error.message : String(error);
+}
@@ -4,6 +4,37 @@ import { SessionController } from "./sessionController";
 import { defaultApi, EmitSocket, emptyPage, FakeSocket, oldSession, runPendingAnimationFrames, status, workspace, type AppState, type SessionActivity, type SessionInfo } from "./sessionController.testSupport";

 describe("SessionController live events", () => {
+  it("emits only final assistant text after agent.end", async () => {
+    const socket = new EmitSocket();
+    const replies: string[] = [];
+    let state: AppState = {
+      ...initialAppState(),
+      selectedWorkspace: workspace,
+      selectedSession: oldSession,
+      sessions: [oldSession],
+      messages: [{ role: "assistant", parts: [{ type: "thinking", text: "private" }, { type: "text", text: "Final answer" }] }],
+    };
+    const api: typeof defaultApi = {
+      ...defaultApi,
+      messages: () => Promise.resolve(emptyPage),
+      status: () => Promise.resolve(status(oldSession.id)),
+      streamSnapshot: () => Promise.resolve({ seq: 0, partial: null }),
+    };
+    const controller = new SessionController(
+      () => state,
+      (patch) => { state = { ...state, ...patch }; },
+      () => undefined,
+      undefined,
+      { api, socket, onFinalAssistantResponse: ({ text }) => { replies.push(text); } },
+    );
+    await controller.selectSession(oldSession, { updateUrl: false });
+    state = { ...state, messages: [{ role: "assistant", parts: [{ type: "thinking", text: "private" }, { type: "text", text: "Final answer" }] }] };
+
+    socket.emit({ type: "agent.end" });
+
+    expect(replies).toEqual(["Final answer"]);
+  });
+
  it("coalesces rapid status updates into a single state write per frame", () => {
    const setStateCalls: Partial<AppState>[] = [];
    let state: AppState = { ...initialAppState(), selectedSession: oldSession, sessions: [oldSession] };
@@ -52,6 +52,12 @@ export interface SelectedSessionReady {
  session: SessionInfo;
 }

+export interface FinalAssistantResponse {
+  machineId: string;
+  sessionId: string;
+  text: string;
+}
+
 export interface SessionControllerDependencies {
  api?: typeof defaultApi;
  socket?: SessionEventSocket;
@@ -59,6 +65,7 @@ export interface SessionControllerDependencies {
  notifications?: SessionNotificationSessionBridge;
  replacePromptEditorText?: (replacement: PromptEditorTextReplacement) => void | Promise<void>;
  onSelectedSessionReady?: (selection: SelectedSessionReady) => void;
+  onFinalAssistantResponse?: (response: FinalAssistantResponse) => void;
 }

 interface BulkSessionMutationResult {
@@ -111,6 +118,7 @@ export class SessionController {
  private readonly notifications: SessionNotificationSessionBridge | undefined;
  private readonly replacePromptEditorText: SessionControllerDependencies["replacePromptEditorText"];
  private readonly onSelectedSessionReady: SessionControllerDependencies["onSelectedSessionReady"];
+  private readonly onFinalAssistantResponse: SessionControllerDependencies["onFinalAssistantResponse"];
  private selectionSeq = 0;
  private disposed = false;
  // Join-time stream watermark for the selected session. `seq` is the
@@ -142,6 +150,7 @@ export class SessionController {
    this.notifications = deps.notifications;
    this.replacePromptEditorText = deps.replacePromptEditorText;
    this.onSelectedSessionReady = deps.onSelectedSessionReady;
+    this.onFinalAssistantResponse = deps.onFinalAssistantResponse;
  }

  applyGlobalEvent(event: GlobalSessionEvent): void {
@@ -1489,6 +1498,7 @@ export class SessionController {
    }

    this.flushPendingUpdates();
+    if (event.type === "agent.end") this.emitFinalAssistantResponse();
    // Ask frames are applied after the buffered status they were published with,
    // so the card follows the daemon's own open/close order.
    if (event.type === "ask.opened") {
@@ -1680,6 +1690,14 @@ export class SessionController {
    }
  }

+  private emitFinalAssistantResponse(): void {
+    const session = this.getState().selectedSession;
+    if (session === undefined) return;
+    const text = finalAssistantReply(this.getState().messages);
+    if (text === "") return;
+    this.onFinalAssistantResponse?.({ machineId: selectedMachineId(this.getState()), sessionId: session.id, text });
+  }
+
  private clearPendingUpdates(): void {
    this.pendingTranscriptEvents = [];
    this.pendingStatusBySession.clear();
@@ -1701,6 +1719,11 @@ export class SessionController {
  }
 }

+function finalAssistantReply(messages: AppState["messages"]): string {
+  const message = [...messages].reverse().find((line) => line.role === "assistant");
+  return message?.parts.filter((part) => part.type === "text").map((part) => part.text).join("\n").trim() ?? "";
+}
+
 function omitSessionActivity(activities: Record<string, SessionActivity>, sessionId: string): Record<string, SessionActivity> {
  return omitKey(activities, sessionId);
 }
@@ -1,4 +1,4 @@
-export type SettingsSection = "general" | "sessiond" | "packages" | "plugins" | "shortcuts";
+export type SettingsSection = "general" | "sessiond" | "packages" | "plugins" | "shortcuts" | "azure-speech";

 export function readSettingsSection(): SettingsSection | undefined {
  return parseSettingsSection(new URLSearchParams(window.location.search).get("settings"));
@@ -21,5 +21,6 @@ export function parseSettingsSection(value: string | null): SettingsSection | un
  if (value === "packages" || value === "pi-packages") return "packages";
  if (value === "plugins") return "plugins";
  if (value === "shortcuts" || value === "keyboard" || value === "keyboard-shortcuts") return "shortcuts";
+  if (value === "azure-speech" || value === "voice") return "azure-speech";
  return undefined;
 }
@@ -0,0 +1,19 @@
+import { describe, expect, it } from "vitest";
+import { transitionVoiceRecognition } from "./voiceRecognitionLifecycle";
+
+describe("voice recognition lifecycle", () => {
+  it("keeps the microphone allocated while a final turn waits for TTS", () => {
+    expect(transitionVoiceRecognition("idle", "start")).toBe("listening");
+    expect(transitionVoiceRecognition("listening", "final")).toBe("paused");
+    expect(transitionVoiceRecognition("paused", "resume")).toBe("listening");
+  });
+
+  it("ignores duplicate finals until the response resumes listening", () => {
+    expect(transitionVoiceRecognition("paused", "final")).toBe("paused");
+  });
+
+  it("stops from every active phase", () => {
+    expect(transitionVoiceRecognition("listening", "stop")).toBe("idle");
+    expect(transitionVoiceRecognition("paused", "stop")).toBe("idle");
+  });
+});
@@ -0,0 +1,14 @@
+export type VoiceRecognitionPhase = "idle" | "listening" | "paused";
+export type VoiceRecognitionEvent = "start" | "final" | "resume" | "stop";
+
+/**
+ * The microphone remains allocated across paused turns. A final result is
+ * accepted once only; subsequent events are ignored until TTS resumes it.
+ */
+export function transitionVoiceRecognition(phase: VoiceRecognitionPhase, event: VoiceRecognitionEvent): VoiceRecognitionPhase {
+  if (event === "stop") return "idle";
+  if (event === "start" && phase === "idle") return "listening";
+  if (event === "final" && phase === "listening") return "paused";
+  if (event === "resume" && phase === "paused") return "listening";
+  return phase;
+}
@@ -2,9 +2,12 @@ import { existsSync } from "node:fs";
 import { dirname, join } from "node:path";
 import { fileURLToPath } from "node:url";
 import Fastify, { type FastifyInstance, type FastifyReply, type FastifyServerOptions } from "fastify";
+import type { ServerOptions as HttpsServerOptions } from "node:https";
 import fastifyCompress from "@fastify/compress";
 import fastifyStatic from "@fastify/static";
 import fastifyWebsocket from "@fastify/websocket";
+import fastifySwagger from "@fastify/swagger";
+import fastifySwaggerUi from "@fastify/swagger-ui";
 import { ProjectStore } from "./storage/projectStore.js";
 import { ProjectService } from "./projects/projectService.js";
 import { WorkspaceService } from "./workspaces/workspaceService.js";
@@ -20,6 +23,9 @@ import { registerGitRoutes } from "./gitRoutes.js";
 import { registerTerminalProxyRoutes } from "./terminalProxyRoutes.js";
 import { registerWorkspaceDeletionRoutes } from "./workspaces/workspaceDeletionRoutes.js";
 import { createFilePiWebConfigService, registerConfigRoutes, registerLocalMachineConfigRoutes, type PiWebConfigService } from "./configRoutes.js";
+import { registerAzureSpeechRoutes } from "./azureSpeechRoutes.js";
+import { voiceApiOpenApi } from "./voiceApiOpenApi.js";
+import { registerVoiceApiRoutes } from "./voiceApi.js";
 import { PiWebPluginService } from "./piWebPluginService.js";
 import { createActiveProfilePiPackageService, type PiPackageService } from "./piPackageService.js";
 import { registerPiPackageRoutes } from "./piPackageRoutes.js";
@@ -51,6 +57,7 @@ export interface AppDependencies {
  logger?: FastifyServerOptions["logger"];
  /** Maximum accepted HTTP request body size in bytes. */
  bodyLimit?: number;
+  https?: HttpsServerOptions;
 }

 interface LocalProjectRouteOptions {
@@ -151,8 +158,12 @@ async function withProfileDependency<T>(reply: FastifyReply, operation: () => Pr
  }
 }

-export async function buildApp(deps: AppDependencies = {}): Promise<FastifyInstance> {
-  const app = Fastify({ logger: deps.logger ?? true, ...(deps.bodyLimit === undefined ? {} : { bodyLimit: deps.bodyLimit }) });
+export async function buildApp(deps: AppDependencies = {}) {
+  const options = {
+    logger: deps.logger ?? true,
+    ...(deps.bodyLimit === undefined ? {} : { bodyLimit: deps.bodyLimit }),
+  };
+  const app: FastifyInstance = deps.https === undefined ? Fastify(options) : Fastify({ ...options, https: deps.https });
  // Vite proxies development API requests here, while production and machine-scoped
  // API requests already terminate here, so this is the shared browser HTTP edge.
  await app.register(fastifyCompress, {
@@ -161,6 +172,11 @@ export async function buildApp(deps: AppDependencies = {}): Promise<FastifyInsta
    threshold: 1024,
  });
  await app.register(fastifyWebsocket);
+  await app.register(fastifySwagger, { mode: "static", specification: { document: voiceApiOpenApi } });
+  await app.register(fastifySwaggerUi, {
+    routePrefix: "/api/v1/voice/docs",
+    uiConfig: { docExpansion: "list", persistAuthorization: true },
+  });

  const projects = deps.projects ?? new ProjectService(new ProjectStore());
  const workspaces = deps.workspaces ?? new WorkspaceService();
@@ -214,6 +230,9 @@ export async function buildApp(deps: AppDependencies = {}): Promise<FastifyInsta
  const invalidatingConfigService = invalidatePiWebStatusOnWrite(configService, piWebStatusCache);
  registerConfigRoutes(app, invalidatingConfigService);
  registerLocalMachineConfigRoutes(app, invalidatingConfigService);
+  registerAzureSpeechRoutes(app);
+  registerAzureSpeechRoutes(app, undefined, "/api/machines/local");
+  registerVoiceApiRoutes(app, { projects, workspaces, daemon: sessionDaemon });

  registerMachineRoutes(app, machines);
  registerMachinePluginProxyRoutes(app, machines);
@@ -0,0 +1,79 @@
+import { describe, expect, it, vi } from "vitest";
+import Fastify from "fastify";
+import {
+  registerAzureSpeechRoutes,
+  type AzureSpeechService,
+} from "./azureSpeechRoutes.js";
+
+function service(): AzureSpeechService {
+  return {
+    settings: vi.fn(() => ({
+      region: "eastus",
+      voice: "en-US-Andrew:DragonHDLatestNeural",
+      hasApiKey: true,
+    })),
+    update: vi.fn(() => ({
+      region: "eastus",
+      voice: "en-US-Andrew:DragonHDLatestNeural",
+      hasApiKey: true,
+    })),
+    token: vi.fn(() =>
+      Promise.resolve({
+        token: "short-lived-token",
+        region: "eastus",
+        voice: "en-US-Andrew:DragonHDLatestNeural",
+      })
+    ),
+  };
+}
+
+describe("Azure Speech routes", () => {
+  it("returns redacted settings and accepts a write-only API key", async () => {
+    const app = Fastify();
+    const azure = service();
+    registerAzureSpeechRoutes(app, azure);
+
+    const get = await app.inject({ method: "GET", url: "/api/azure-speech" });
+    expect(get.statusCode).toBe(200);
+    expect(get.json()).toEqual({
+      region: "eastus",
+      voice: "en-US-Andrew:DragonHDLatestNeural",
+      hasApiKey: true,
+    });
+    expect(get.body).not.toContain("apiKey");
+
+    const put = await app.inject({
+      method: "PUT",
+      url: "/api/azure-speech",
+      payload: {
+        region: "eastus",
+        voice: "en-US-Andrew:DragonHDLatestNeural",
+        apiKey: "a".repeat(32),
+      },
+    });
+    expect(put.statusCode).toBe(200);
+    // eslint-disable-next-line @typescript-eslint/unbound-method
+    expect(azure.update).toHaveBeenCalledWith({
+      region: "eastus",
+      voice: "en-US-Andrew:DragonHDLatestNeural",
+      apiKey: "a".repeat(32),
+    });
+    expect(put.body).not.toContain("apiKey");
+    await app.close();
+  });
+
+  it("rejects malformed settings before they reach the service", async () => {
+    const app = Fastify();
+    const azure = service();
+    registerAzureSpeechRoutes(app, azure);
+    const response = await app.inject({
+      method: "PUT",
+      url: "/api/azure-speech",
+      payload: { region: "eastus!", voice: "voice" },
+    });
+    expect(response.statusCode).toBe(400);
+    // eslint-disable-next-line @typescript-eslint/unbound-method
+    expect(azure.update).not.toHaveBeenCalled();
+    await app.close();
+  });
+});
@@ -0,0 +1,144 @@
+import { chmodSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import type { FastifyInstance } from "fastify";
+import { piWebConfigPath } from "../config.js";
+import type { AzureSpeechSettings, AzureSpeechSettingsUpdate, AzureSpeechToken } from "../shared/apiTypes.js";
+
+interface AzureSpeechSecret {
+  region: string;
+  voice: string;
+  apiKey?: string;
+}
+
+/** Server-only Azure credentials. Never expose this shape through an HTTP route. */
+export interface AzureSpeechServerSettings { region: string; voice: string; apiKey: string }
+
+export function readAzureSpeechServerSettings(path = join(dirname(piWebConfigPath()), "azure-speech.json")): AzureSpeechServerSettings {
+  const secret = readSecret(path);
+  if (secret.apiKey === undefined || secret.region === "") throw new AzureSpeechConfigurationError("Azure Speech is not configured. Add a Speech key and region in Settings → Azure Speech.");
+  return { region: secret.region, voice: secret.voice, apiKey: secret.apiKey };
+}
+
+export interface AzureSpeechService {
+  settings(): AzureSpeechSettings;
+  update(input: AzureSpeechSettingsUpdate): AzureSpeechSettings;
+  token(): Promise<AzureSpeechToken>;
+}
+
+export function createFileAzureSpeechService(): AzureSpeechService {
+  const path = join(dirname(piWebConfigPath()), "azure-speech.json");
+  return {
+    settings: () => publicSettings(readSecret(path)),
+    update: (input) => {
+      const current = readSecret(path);
+      const next: AzureSpeechSecret = {
+        region: requireRegion(input.region),
+        voice: requireVoice(input.voice),
+        ...(input.apiKey === undefined ? (current.apiKey === undefined ? {} : { apiKey: current.apiKey }) : input.apiKey === "" ? {} : { apiKey: requireApiKey(input.apiKey) }),
+      };
+      writeSecret(path, next);
+      return publicSettings(next);
+    },
+    token: async () => {
+      const secret = readSecret(path);
+      if (secret.apiKey === undefined || secret.region === "") throw new AzureSpeechConfigurationError("Azure Speech is not configured. Add a Speech key and region in Settings → Azure Speech.");
+      const response = await fetch(`https://${encodeURIComponent(secret.region)}.api.cognitive.microsoft.com/sts/v1.0/issueToken`, {
+        method: "POST",
+        headers: { "Ocp-Apim-Subscription-Key": secret.apiKey },
+        // Let the runtime calculate Content-Length. PI's installed Undici dispatcher
+        // rejects a manually supplied header, while Azure requires a zero-length POST body.
+        body: "",
+      });
+      if (!response.ok) throw new AzureSpeechTokenError(`Azure Speech token request failed (${String(response.status)}). Check the key and region.`);
+      const token = (await response.text()).trim();
+      if (token === "") throw new AzureSpeechTokenError("Azure Speech returned an empty authorization token.");
+      return { token, region: secret.region, voice: secret.voice };
+    },
+  };
+}
+
+export function registerAzureSpeechRoutes(app: FastifyInstance, service: AzureSpeechService = createFileAzureSpeechService(), prefix = "/api"): void {
+  app.get(`${prefix}/azure-speech`, async (_request, reply) => {
+    try {
+      return service.settings();
+    } catch (error) {
+      return reply.code(500).send({ error: errorMessage(error) });
+    }
+  });
+  app.put<{ Body: unknown }>(`${prefix}/azure-speech`, async (request, reply) => {
+    try {
+      return service.update(parseSettingsUpdate(request.body));
+    } catch (error) {
+      return reply.code(400).send({ error: errorMessage(error) });
+    }
+  });
+  app.post(`${prefix}/azure-speech/token`, async (_request, reply) => {
+    try {
+      return await service.token();
+    } catch (error) {
+      const status = error instanceof AzureSpeechConfigurationError ? 400 : error instanceof AzureSpeechTokenError ? 502 : 500;
+      app.log.error({ err: error }, "Azure Speech token request failed");
+      return reply.code(status).send({ error: errorMessage(error) });
+    }
+  });
+}
+
+function readSecret(path: string): AzureSpeechSecret {
+  if (!existsSync(path)) return { region: "eastus", voice: "" };
+  const value: unknown = JSON.parse(readFileSync(path, "utf8"));
+  if (!isRecord(value)) throw new Error(`Azure Speech settings must be a JSON object: ${path}`);
+  return {
+    region: value["region"] === undefined ? "eastus" : requireRegion(value["region"]),
+    voice: value["voice"] === undefined ? "" : requireVoice(value["voice"]),
+    ...(value["apiKey"] === undefined ? {} : { apiKey: requireApiKey(value["apiKey"]) }),
+  };
+}
+
+function writeSecret(path: string, secret: AzureSpeechSecret): void {
+  mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
+  writeFileSync(path, `${JSON.stringify(secret, null, 2)}\n`, { encoding: "utf8", mode: 0o600 });
+  chmodSync(path, 0o600);
+}
+
+function publicSettings(secret: AzureSpeechSecret): AzureSpeechSettings {
+  return { region: secret.region, voice: secret.voice, hasApiKey: secret.apiKey !== undefined };
+}
+
+function parseSettingsUpdate(value: unknown): AzureSpeechSettingsUpdate {
+  if (!isRecord(value)) throw new Error("Azure Speech settings update must be an object");
+  return {
+    region: requireRegion(value["region"]),
+    voice: requireVoice(value["voice"]),
+    ...(value["apiKey"] === undefined ? {} : { apiKey: typeof value["apiKey"] === "string" ? value["apiKey"] : fail("Azure Speech API key must be a string") }),
+  };
+}
+
+function requireRegion(value: unknown): string {
+  if (typeof value !== "string" || !/^[a-z0-9-]{2,64}$/i.test(value.trim())) throw new Error("Azure Speech region must contain only letters, digits, and hyphens");
+  return value.trim().toLowerCase();
+}
+
+function requireVoice(value: unknown): string {
+  if (typeof value !== "string" || value.trim().length > 256) throw new Error("Azure Speech voice must be a string up to 256 characters");
+  return value.trim();
+}
+
+function requireApiKey(value: unknown): string {
+  if (typeof value !== "string" || value.trim().length < 16 || value.trim().length > 512) throw new Error("Azure Speech API key must be between 16 and 512 characters");
+  return value.trim();
+}
+
+function fail(message: string): never {
+  throw new Error(message);
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return typeof value === "object" && value !== null;
+}
+
+function errorMessage(error: unknown): string {
+  return error instanceof Error ? error.message : String(error);
+}
+
+class AzureSpeechConfigurationError extends Error {}
+class AzureSpeechTokenError extends Error {}
@@ -1,7 +1,21 @@
 #!/usr/bin/env node
+import { readFileSync } from "node:fs";
 import { effectivePiWebConfig, maxUploadBytes } from "../config.js";
 import { buildApp } from "./app.js";

 const { config } = effectivePiWebConfig();
-const app = await buildApp({ bodyLimit: maxUploadBytes(process.env, config) });
+const app = await buildApp({
+  bodyLimit: maxUploadBytes(process.env, config),
+  ...tlsOptions(process.env),
+});
 await app.listen({ port: config.port ?? 8504, host: config.host ?? "127.0.0.1" });
+
+function tlsOptions(env: NodeJS.ProcessEnv): { https?: { key: Buffer; cert: Buffer } } {
+  const keyPath = env["PI_WEB_TLS_KEY"];
+  const certPath = env["PI_WEB_TLS_CERT"];
+  if ((keyPath === undefined || keyPath === "") && (certPath === undefined || certPath === "")) return {};
+  if (keyPath === undefined || keyPath === "" || certPath === undefined || certPath === "") {
+    throw new Error("Set both PI_WEB_TLS_KEY and PI_WEB_TLS_CERT to enable HTTPS.");
+  }
+  return { https: { key: readFileSync(keyPath), cert: readFileSync(certPath) } };
+}
@@ -65,6 +65,21 @@ describe("PiSessionService", () => {
      await service.dispose();
    });

+    it("forwards Pi's native agent_settled event without inferring it from agent_end", async () => {
+      const { fake, service, events } = messagesService([]);
+      await service.status(sessionRef("session-1"));
+
+      fake.emit({ type: "agent_end" });
+      await new Promise<void>((resolve) => setImmediate(resolve));
+      expect(events.sessionEvents.map(({ event }) => event).filter((event) => event.type === "agent.settled")).toEqual([]);
+
+      fake.emit({ type: "agent_settled" });
+      expect(events.sessionEvents.map(({ event }) => event).filter((event) => event.type === "agent.settled")).toEqual([
+        { type: "agent.settled" },
+      ]);
+      await service.dispose();
+    });
+
    it("annotates the join-time stream snapshot partial with the current thinking level", async () => {
      const streamingMessage = {
        role: "assistant",
@@ -3224,6 +3224,9 @@ export class PiSessionService implements SessionRouteService {
      this.publishActivityForEvent(session, event);
      const eventType = getString(event, "type");
      if (eventType === "agent_end") this.abortRunScopedExtensionDialogs(session.sessionId);
+      // Pi itself emits agent_settled only after automatic retries, compaction,
+      // and queued continuations are exhausted. Preserve that native lifecycle
+      // event; never infer it from agent_end or mutable runtime flags.
      if (eventType === "compaction_end") this.scheduleCompactionQueueDrain(session.sessionId);
      if (eventType === "agent_start" || eventType === "agent_end") this.scheduleCompactionQueueDrain(session.sessionId);
      this.publishStatus(session);
@@ -4209,6 +4212,7 @@ function toClientEvent(event: unknown, thinkingLevel?: string): SessionUiEvent {
  }
  if (eventType === "agent_start") return { type: "agent.start" };
  if (eventType === "agent_end") return { type: "agent.end" };
+  if (eventType === "agent_settled") return { type: "agent.settled" };
  if (eventType === "message_end") {
    const message = getProperty(event, "message");
    if (message === undefined) return { type: "message.end" };
@@ -0,0 +1,49 @@
+import type { OpenAPIV3 } from "openapi-types";
+
+export const voiceApiOpenApi: OpenAPIV3.Document = {
+  openapi: "3.0.3",
+  info: {
+    title: "PI WEB Voice Pipeline API",
+    version: "1.0.0",
+    description: "Authenticated, server-side Azure Speech voice conversations for native and embedded clients. Audio is sent over the documented WebSocket protocol; Azure credentials never leave PI WEB.",
+  },
+  servers: [{ url: "/", description: "Current PI WEB HTTPS origin" }],
+  components: {
+    securitySchemes: {
+      bearerAuth: { type: "http", scheme: "bearer", bearerFormat: "pwv1 device token" },
+    },
+    schemas: {
+      Error: { type: "object", required: ["error"], properties: { error: { type: "string" } } },
+      VoiceTarget: { type: "object", required: ["id", "projectId", "label", "path"], properties: { id: { type: "string" }, projectId: { type: "string" }, label: { type: "string" }, path: { type: "string", description: "Registered workspace directory" } } },
+      CreateConversation: { type: "object", required: ["workspaceId"], properties: { workspaceId: { type: "string" }, model: { type: "string", example: "openai-codex/gpt-5.6-terra" }, thinking: { type: "string", example: "high" }, context: { type: "string", maxLength: 12000, description: "Additional instructions appended to the turn" } } },
+      Conversation: { type: "object", required: ["id", "sessionId", "workspaceId", "cwd", "status"], properties: { id: { type: "string" }, sessionId: { type: "string" }, workspaceId: { type: "string" }, cwd: { type: "string" }, status: { type: "string", enum: ["input-ready", "working", "closed"] }, model: { type: "string" }, thinking: { type: "string" } } },
+    },
+  },
+  security: [{ bearerAuth: [] }],
+  paths: {
+    "/api/v1/voice/targets": {
+      get: { summary: "List authorized voice targets", responses: { "200": { description: "Registered workspaces permitted by this device token", content: { "application/json": { schema: { type: "object", required: ["workspaces", "scopes"], properties: { workspaces: { type: "array", items: { $ref: "#/components/schemas/VoiceTarget" } }, scopes: { type: "object" } } } } } }, "401": { description: "Missing or invalid token", content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } } },
+    },
+    "/api/v1/voice/conversations": {
+      get: { summary: "List prior sessions in an authorized workspace", parameters: [{ name: "workspaceId", in: "query", required: true, schema: { type: "string" } }], responses: { "200": { description: "Previous PI sessions rooted in the workspace" }, "400": { description: "Invalid or unauthorized workspace" } } },
+      post: { summary: "Create a voice conversation", requestBody: { required: true, content: { "application/json": { schema: { $ref: "#/components/schemas/CreateConversation" } } } }, responses: { "201": { description: "Conversation created", content: { "application/json": { schema: { $ref: "#/components/schemas/Conversation" } } } }, "400": { description: "Invalid or unauthorized requested scope", content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } } },
+    },
+    "/api/v1/voice/conversations/path": {
+      post: { summary: "Create a conversation in an existing directory beneath the device-safe root", requestBody: { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", example: "/home/hope/workspaces/new-project" }, model: { type: "string" }, thinking: { type: "string" }, context: { type: "string" } } } } } }, responses: { "201": { description: "Conversation created" }, "400": { description: "Path is missing, outside the safe root, or not a directory" } } },
+    },
+    "/api/v1/voice/conversations/resume": {
+      post: { summary: "Resume a prior workspace session", requestBody: { required: true, content: { "application/json": { schema: { type: "object", required: ["workspaceId", "sessionId"], properties: { workspaceId: { type: "string" }, sessionId: { type: "string" } } } } } }, responses: { "201": { description: "Conversation handle attached to prior session", content: { "application/json": { schema: { $ref: "#/components/schemas/Conversation" } } } }, "400": { description: "Session is not in the authorized workspace" } } },
+    },
+    "/api/v1/voice/conversations/{id}/models": {
+      get: { summary: "List available models for a conversation", parameters: [{ name: "id", in: "path", required: true, schema: { type: "string" } }], responses: { "200": { description: "Models available to the session and permitted by this device token" }, "404": { description: "Conversation not found or not owned by this token" } } },
+    },
+    "/api/v1/voice/conversations/{id}": {
+      parameters: [{ name: "id", in: "path", required: true, schema: { type: "string" } }],
+      get: { summary: "Read a conversation", responses: { "200": { description: "Conversation", content: { "application/json": { schema: { $ref: "#/components/schemas/Conversation" } } } }, "404": { description: "Not found or not owned by this token" } } },
+      delete: { summary: "Close a conversation", responses: { "204": { description: "Closed" }, "404": { description: "Not found or not owned by this token" } } },
+    },
+    "/api/v1/voice/stream": {
+      get: { summary: "Voice audio WebSocket", description: "Upgrade to WebSocket using the bearer token. Send `attach`, then binary signed 16-bit little-endian PCM at 16 kHz mono, followed by `input.end`. Server emits transcript JSON, `assistant.final`, and raw 24 kHz mono PCM binary frames. See docs/voice-api.md for the binary frame layout.", responses: { "101": { description: "WebSocket upgraded" }, "401": { description: "Invalid token" } } },
+    },
+  },
+};
@@ -77,6 +77,26 @@ export interface PiWebAgentConfig {
  dir?: string;
 }

+export interface AzureSpeechSettings {
+  region: string;
+  voice: string;
+  /** Deliberately the only key-related field exposed to the browser. */
+  hasApiKey: boolean;
+}
+
+export interface AzureSpeechSettingsUpdate {
+  region: string;
+  voice: string;
+  /** Omit to retain the stored key; an empty string clears it. */
+  apiKey?: string;
+}
+
+export interface AzureSpeechToken {
+  token: string;
+  region: string;
+  voice: string;
+}
+
 export interface PiWebConfigValues {
  host?: string;
  port?: number;
@@ -1263,6 +1283,8 @@ type SessionUiEventBody =
  | { type: "shell.end"; output?: string; exitCode?: number | null; cancelled?: boolean; truncated?: boolean; fullOutputPath?: string; isError?: boolean }
  | { type: "agent.start" }
  | { type: "agent.end" }
+  /** Pi-native completion: no retry, compaction retry, or queued continuation remains. */
+  | { type: "agent.settled" }
  | { type: "message.end"; message?: unknown }
  | { type: "status.update"; status: SessionStatus }
  | { type: "activity.update"; activity: SessionActivity }
Author	SHA1	Message	Date
snowspeederandClaude Fable 5	28a2629c43	feat: speak answers sentence by sentence and never speak reasoning CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details Segment the final answer as it streams and synthesize each completed sentence immediately, so a listener hears the reply begin while the model is still writing it. Sentences share the progress clips' FIFO chain and output-sequence counter; only text left unspoken (a hit budget) is synthesized after settle, so audio is never sent twice. A message that grows a tool call stops answer-streaming without re-speaking what already played. Reasoning models inline their chain of thought as <think> blocks in the same delta stream. Strip it while streaming -- carrying a partial tag across delta boundaries -- so it is neither spoken nor shown as answer text. The fixed "Hang on while I work on that." is now a rotation of short natural phrases that never repeats twice running, and a long silent think gets a spoken "still on it" roughly every 45 seconds until real speech begins. Turn timeout rises to five minutes to match the client. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-08-07 14:20:21 -04:00
snowspeederandClaude Fable 5	a7033f9076	feat: speak a progress update during long voice turns CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details After two seconds of agent work, send one agent.progress frame with a synthesized "Hang on while I work on that." clip over the same binary audio channel (shared output-sequence counter), at most once per turn; screen-only agent.working heartbeats continue every 15 seconds. This is the change the 15:11 rebuild already shipped to the running server — committing it pins the protocol addition to the client that now tolerates and plays it. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-08-06 23:10:07 -04:00
snowspeeder	a71cb8334a	feat: extend voice API workspace and speech controls CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details	2026-08-06 12:09:34 -04:00
snowspeeder	1adb459e04	feat: harden voice API conversations and progress	2026-08-06 00:42:55 -04:00
snowspeeder	0793fa0ac4	feat: add authenticated voice pipeline API docs CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details	2026-08-04 19:09:33 -04:00
snowspeeder	a2e4f1d576	chore: document fork workflow and ignore local artifacts CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details	2026-08-04 16:25:32 -04:00
snowspeeder	9627453e23	fix: keep Azure voice mode active between turns CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details	2026-08-04 16:22:55 -04:00
snowspeeder	c3e0011378	feat: add Azure Speech voice mode CI / Verify and package (ubuntu-latest) (push) Canceled after 0s Details CI / Verify and package (windows-latest) (push) Canceled after 0s Details	2026-08-04 07:28:48 -04:00