feat: conversation state machine, mic meter, history, wake lock (Phase 3)
This commit is contained in:
1 parent
4986b6d7dc
commit
5483e6ed48
8 files changed
+348
-29
No files matched your search
@@ -21,8 +21,8 @@ Goal: full-parity native iOS app per UPDATE_PATH.md, verified in simulator via X
|
||||
## Phases
|
||||
- [x] Phase 0: scaffold (commit 297fbaa)
|
||||
- [x] Phase 1: ServerConfig/TokenClient/TLSPinning/ConnectionTest (commit 378b8cb)
|
||||
- [ ] Phase 2: LiveKit session spike — **ACTIVE**
|
||||
- [ ] Phase 3: state machine, audio levels, wake lock, history
|
||||
- [x] Phase 2: LiveKit session spike — commit 4986b6d (mic -4010 is sim env limitation; verify on device)
|
||||
- [x] Phase 3: state machine, audio levels, wake lock, history (commit 70aef02)
|
||||
- [ ] Phase 4: media playback reconciliation
|
||||
- [ ] Phase 5: AVPlayer + local reverse proxy for self-signed https media
|
||||
- [ ] Phase 6: settings parity (voice/model pickers, text mode)
|
||||
@@ -51,10 +51,30 @@ Conclusion: simulator microphone hardware/source unavailable here. App code path
|
||||
|
||||
### After connect bug fixed
|
||||
- [x] Verify ready state reached (state=ready logged; UI transitions to conversation view)
|
||||
- [ ] Verify transcript arrival + disconnect cleanup (blocked on mic/audio in sim; do on device)
|
||||
- [x] Verify transcript arrival + disconnect cleanup (verified in sim via text mode: user+agent finals arrived, hang-up cleaned up state)
|
||||
- [x] Fix Swift 6 warnings: ControlChannel @MainActor, VoiceSession closure, duplicate TranscriptHandler removed from ConversationView (build now warning-free)
|
||||
- [ ] Commit Phase 2, mark done
|
||||
|
||||
## Phase 3 status
|
||||
Done:
|
||||
- `ConversationStateModel` (Sources/LiveKit/ConversationState.swift): ports app.js inferState — EMA smoothing `prev*0.6 + raw*0.4`, user threshold 0.035, agent threshold 0.02, speaking→thinking when both silent, thinking chip after 2s in thinking phase. Driven by `RoomDelegate.didUpdateSpeakingParticipants` (participant.audioLevel), ticked only while session ready.
|
||||
- Mic failure is now NON-FATAL: after 3 retries the error is surfaced via `VoiceSession.micError` (orange banner) and state stays `.ready` — text mode still works (also matches Phase 8 mic-denied fallback).
|
||||
- Wake lock: `isIdleTimerDisabled=true` on room connect, false on disconnect.
|
||||
- Text mode toggle: enables → setMuted(true) + `set_mode:text`; disables → `set_mode:voice` + re-enable mic. Persisted via SettingsStore.textMode.
|
||||
- Transcript list: per-row timestamps, delivery-marker strip (web DELIVERY_MARKERS regex), Clear button, history of last 50 finals persisted to UserDefaults JSON (`TranscriptHistory`, key "voice_history").
|
||||
- Barge-in hint line shown while in text mode; mic meter bar under status line in voice mode.
|
||||
|
||||
Verified in simulator (log evidence):
|
||||
- Start → token→connect(retry)→room connected→agent_ready→state=ready.
|
||||
- Mic -4010 x3 retries → error logged, session stayed ready, banner shown (non-fatal).
|
||||
- Text mode toggle → log `set_mode sent: text`; mic muted.
|
||||
- Composer send → log `sendText lk.chat chars=24` → user transcript final arrived → agent TTS audio drove state machine `connecting -> speaking` (log `[dev.shane.voice:state]`) → agent transcript final "Hello! How can I assist you today?" arrived in list.
|
||||
- Hang up → `speaking -> idle`, UI back to Start button.
|
||||
|
||||
Needs device verification:
|
||||
- Live voice-driven states (listening while user speaks, thinking gap >2s chip, barge-in) — sim mic is silent (-4010), so local audio level never rises in sim; only remote (agent) audio drove the machine here.
|
||||
- Mic meter movement with real speech; wake lock behavior on physical screen-off.
|
||||
|
||||
## Subagent notes
|
||||
- Small context model: keep prompts self-contained, point at this file + UPDATE_PATH.md
|
||||
- Watch for loops (repeated identical commands) — kill and redirect if seen
|
||||
|
||||
Reference in new issue
Block a user