Voice — iOS client for Hope Voice

Native SwiftUI iOS app (iOS 17+) that talks to the Hope Voice agent: voice/text conversation, transcript history, voice & model pickers, media player, and a background-tasks panel. Mirrors the core web UI of hope-voice-api; no server-side changes.

Status: Core functionality is implemented and verified in simulator to the extent the environment allows. Full device qualification (mic capture, audible media playback, interruption behavior) and the Hope visual redesign (ui-ux-spec.md) remain pending — see PLAN.md and docs/DEVICE_TESTING.md.

Prerequisites

Xcode 16+ (Swift 6), iOS 17 simulator or a real iPhone on the LAN.
xcodegen (brew install xcodegen).
The voice container running on Unraid (LiveKit + token server + media API).
Mic: the simulator mic is unreliable in this environment (CoreAudio -4010, 0 input channels) — voice mode needs a real device. If the simulator was wiped, grant permission first: xcrun simctl privacy booted grant microphone dev.shane.voice.

Build & run

xcodegen generate   # regenerate Voice.xcodeproj after file changes
xcodebuild -project Voice.xcodeproj -scheme Voice_iOS \
  -destination 'generic/platform=iOS Simulator' build

The scheme is Voice_iOS (not Voice). Bundle id: dev.shane.voice. To run in the simulator, use Xcode or xcrun simctl with the built app.

Server configuration

Default base URL is https://192.168.86.2:9178 (see Sources/App/SettingsStore.swift). Change it at runtime in Settings → Server URL (the app disconnects and recreates the session when idle). The app derives LiveKit signaling and media endpoints from this base URL.

TLS cert pinning

The deployment uses a self-signed cert that is regenerated at container start, so the pinned SHA-256 in Sources/Networking/TLSPinning.swift breaks on every cert rotation. Re-export after a server restart:

openssl s_client -connect 192.168.86.2:9178 </dev/null 2>/dev/null \
  | openssl x509 -outform der | openssl dgst -sha256

Paste the new hash into TLSPinning.pinnedCertSHA256Hex. Until you do, token/media calls fail with a pin mismatch (not a network error).

Features

Conversation — LiveKit room with the Hope agent; voice mode (mic + TTS) and text mode (composer), toggleable and persisted. State machine (idle/connecting/listening/thinking/ speaking) driven by audio levels, orb animation, mic meter, barge-in hint.
Transcripts — partials updated in place, finals with timestamps, delivery-marker stripping, last-50 history persisted to UserDefaults, clear button.
Voice & model pickers — 13 Azure TTS voices; LLM models from the agent's models message. Both persist and sync on join (set_voice/set_model).
Media player — now-playing card (art/title/channel/status), transport (play/pause, ±10 s, stop), seek + volume sliders, search with "in library" tags, agent-offered choices, library list (play/remove), add-stream form. AVPlayer-backed with a pinned-TLS Range proxy; 1.5 s /state polling and full web-parity reconciliation/reporting.
Background tasks panel — live task_event cards (steps, result, error) with an active count badge in the toolbar.

Known limitations

Simulator mic: CoreAudio returns -4010 (no usable input source) in this environment; voice capture must be verified on a real device. Text mode works in the simulator.
webm media: iOS AVPlayer cannot decode webm/opus (-11828). Library tracks from yt-dlp are webm, so audible playback requires the server to serve m4a/aac/mp3. Everything up to decode (pinning, Range proxy, reconciliation, UI) is verified; no client change needed once a supported format exists.
Cert pin rotation: see above — re-export the pin after each server cert regeneration.
LiveKit signaling uses plain ws://192.168.86.2:7880 (not wss) because the Swift SDK cannot trust the self-signed cert on its WebSocket; token + media still go over pinned TLS.

Docs

PLAN.md — phase-by-phase progress, verified facts, open items.
docs/DEVICE_TESTING.md — real-device qualification checklist.
UPDATE_PATH.md — original plan & protocol reference (read-only).

4.3 KiB Raw Blame History