Redesign Hope iOS UI, harden playback, and document project handoff

This commit is contained in:
snowspeeder committed 2026-09-09 20:23:44 -04:00
1 parent 40aaf4fea7
commit 24f95639dc
24 files changed
+1470 -594

No files matched your search

+13 -12
View File
@@ -2,11 +2,11 @@
Native SwiftUI iOS app (iOS 17+) that talks to the [Hope Voice](https://github.com/) agent:
voice/text conversation, transcript history, voice & model pickers, media player, and a
background-tasks panel. Mirrors the core web UI of `hope-voice-api`; no server-side changes.
background-tasks panel. Uses `hope-voice-api`, including cached AAC conversion for iOS music playback.
> **Status:** Core functionality is implemented and verified in simulator to the extent the
> environment allows. Full device qualification (mic capture, audible media playback,
> interruption behavior) and the Hope visual redesign (`ui-ux-spec.md`) remain pending —
> interruption behavior) remains pending. The native Talk/Music/Activity redesign is implemented —
> see [PLAN.md](PLAN.md) and [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md).
## Prerequisites
@@ -37,9 +37,9 @@ session when idle). The app derives LiveKit signaling and media endpoints from t
### TLS cert pinning
The deployment uses a self-signed cert that is **regenerated at container start**, so the
pinned SHA-256 in `Sources/Networking/TLSPinning.swift` breaks on every cert rotation.
Re-export after a server restart:
The deployment uses a persistent self-signed cert. The entrypoint regenerates it if
missing or issued for a different IP. The pinned SHA-256 in
`Sources/Networking/TLSPinning.swift` must match after any actual cert rotation:
```sh
openssl s_client -connect 192.168.86.2:9178 </dev/null 2>/dev/null \
@@ -53,7 +53,7 @@ fail with a pin mismatch (not a network error).
- **Conversation** — LiveKit room with the Hope agent; voice mode (mic + TTS) and text mode
(composer), toggleable and persisted. State machine (idle/connecting/listening/thinking/
speaking) driven by audio levels, orb animation, mic meter, barge-in hint.
speaking) driven by audio levels, with a compact status indicator and mic meter.
- **Transcripts** — partials updated in place, finals with timestamps, delivery-marker
stripping, last-50 history persisted to UserDefaults, clear button.
- **Voice & model pickers** — 13 Azure TTS voices; LLM models from the agent's `models`
@@ -62,23 +62,24 @@ fail with a pin mismatch (not a network error).
±10 s, stop), seek + volume sliders, search with "in library" tags, agent-offered choices,
library list (play/remove), add-stream form. AVPlayer-backed with a pinned-TLS Range proxy;
1.5 s `/state` polling and full web-parity reconciliation/reporting.
- **Background tasks panel** — live `task_event` cards (steps, result, error) with an active
count badge in the toolbar.
- **Activity tab** — live `task_event` updates (steps, result, error) with an active
count badge on the tab.
## Known limitations
- **Simulator mic**: CoreAudio returns `-4010` (no usable input source) in this environment;
voice capture must be verified on a real device. Text mode works in the simulator.
- **webm media**: iOS AVPlayer cannot decode webm/opus (`-11828`). Library tracks from yt-dlp
are webm, so audible playback requires the server to serve m4a/aac/mp3. Everything up to
decode (pinning, Range proxy, reconciliation, UI) is verified; no client change needed once
a supported format exists.
- **Music compatibility**: library WebM files are requested as cached AAC on iOS.
This requires the deployed backend compatibility patch; see [deployment](docs/DEPLOYMENT.md).
- **Cert pin rotation**: see above — re-export the pin after each server cert regeneration.
- **LiveKit signaling** uses plain `ws://192.168.86.2:7880` (not wss) because the Swift SDK
cannot trust the self-signed cert on its WebSocket; token + media still go over pinned TLS.
## Docs
- [docs/PROJECT_STATUS.md](docs/PROJECT_STATUS.md) — current handoff, decisions, and remaining work; read first.
- [docs/VERIFICATION.md](docs/VERIFICATION.md) — test evidence and remaining qualification.
- [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) — live NAS configuration and rollback.
- [PLAN.md](PLAN.md) — phase-by-phase progress, verified facts, open items.
- [docs/DEVICE_TESTING.md](docs/DEVICE_TESTING.md) — real-device qualification checklist.
- [UPDATE_PATH.md](UPDATE_PATH.md) — original plan & protocol reference (read-only).