Archived
feat: voice and model pickers with sync-on-join (Phase 4)
- Voices.swift: 13 Azure voices, default en-US-Ava - SettingsStore: persist voiceID/modelID, fallback to default voice - VoiceSession: sync set_voice/set_model on agent_ready, selectVoice/selectModel, track models + model_changed - SettingsView: voice and model picker sheets - ControlChannel: publish via publish(data:) reliable data channel instead of sendText DataStream (agent listens on legacy data_received; DataStream text was dropped)
This commit is contained in:
1 parent
5483e6ed48
commit
ca2d26f171
8 files changed
+256
-14
No files matched your search
@@ -22,10 +22,10 @@ Goal: full-parity native iOS app per UPDATE_PATH.md, verified in simulator via X
|
||||
- [x] Phase 0: scaffold (commit 297fbaa)
|
||||
- [x] Phase 1: ServerConfig/TokenClient/TLSPinning/ConnectionTest (commit 378b8cb)
|
||||
- [x] Phase 2: LiveKit session spike — commit 4986b6d (mic -4010 is sim env limitation; verify on device)
|
||||
- [x] Phase 3: state machine, audio levels, wake lock, history (commit 70aef02)
|
||||
- [ ] Phase 4: media playback reconciliation
|
||||
- [ ] Phase 5: AVPlayer + local reverse proxy for self-signed https media
|
||||
- [ ] Phase 6: settings parity (voice/model pickers, text mode)
|
||||
- [x] Phase 3: state machine, audio levels, wake lock, history (commit 5483e6e)
|
||||
- [x] Phase 4: voice/model pickers with sync-on-join (commit pending)
|
||||
- [ ] Phase 5: media playback reconciliation + AVPlayer + local reverse proxy for self-signed https media
|
||||
- [ ] Phase 6: settings parity (text mode done in Phase 3; remaining polish)
|
||||
- [ ] Phase 7: polish/edge cases
|
||||
- [ ] Phase 8: real-device qualification checklist
|
||||
- [ ] Phase 9: docs/final
|
||||
@@ -75,6 +75,23 @@ Needs device verification:
|
||||
- Live voice-driven states (listening while user speaks, thinking gap >2s chip, barge-in) — sim mic is silent (-4010), so local audio level never rises in sim; only remote (agent) audio drove the machine here.
|
||||
- Mic meter movement with real speech; wake lock behavior on physical screen-off.
|
||||
|
||||
## Phase 4 status (2026-08-30 ~11:50)
|
||||
Done:
|
||||
- `Voices.swift`: 13 Azure voices from web app.js; default en-US-Ava:DragonHDOmniLatestNeural.
|
||||
- `SettingsStore`: voiceID/modelID persisted to UserDefaults; invalid stored voice falls back to default.
|
||||
- `VoiceSession`: sync-on-join on agent_ready sends set_voice + set_model (if known); selectVoice/selectModel send immediately and persist; models list + model_changed acks tracked (`@Published models`, `selectedModel`).
|
||||
- `SettingsView`: VoicePickerView sheet (13 voices), ModelPickerView sheet (models from session, "Start a conversation..." when not connected).
|
||||
- `ConversationView`: shows current voice/model in status header when ready.
|
||||
|
||||
### SENDTEXT BUG — FIXED (2026-08-30 ~11:47)
|
||||
Root cause: ControlChannel used `localParticipant.sendText(for:)` (LiveKit DataStream protocol, v2.16). Agent listens on legacy data channel (`ctx.room.on("data_received")`), which never receives DataStream text → "ignoring text stream with topic 'voice-control', no callback attached" in agent logs; set_voice/set_model silently dropped, no model_changed ack.
|
||||
Fix: `ControlChannel.publish` now uses `localParticipant.publish(data:options:)` with `DataPublishOptions(topic:, reliable: true)` — same path as web UI's `publishData`.
|
||||
Verified (sim + agent docker logs, room voice-shane-7bf126cf):
|
||||
- iOS: sync-on-join set_voice sent: en-US-JennyNeural; set_model sent: gemma-4-e4b → model_changed ack received and applied; models list count=30.
|
||||
- Agent: "Switching TTS voice to en-US-JennyNeural" / "Voice updated" / "LLM model change: qwen2.5-vl-3b -> gemma-4-e4b".
|
||||
- Persistence: relaunch → sync-on-join sent previously selected Jenny + gemma-4-e4b (survived UserDefaults).
|
||||
Note: `sendText lk.chat` in VoiceSession is also DataStream-based and the agent has no lk.chat handler — text chat path is unverified server-side, out of scope here.
|
||||
|
||||
## Subagent notes
|
||||
- Small context model: keep prompts self-contained, point at this file + UPDATE_PATH.md
|
||||
- Watch for loops (repeated identical commands) — kill and redirect if seen
|
||||
|
||||
Reference in new issue
Block a user