Remove embedded music player

The music feature is retired: delete the Media tab, MediaAPI,
MediaPlayerController, PinnedMediaResourceLoader, MediaPanelView, the
media_state control publish, media scope handling in VoiceSession, the
volume setting, and all media tests. App is now Talk + Activity tabs.

Backend media API in hope-voice-api is untouched (still serves the web UI).
This commit is contained in:
snowspeeder committed 2026-09-12 06:26:58 -04:00
1 parent 3620a2387b
commit ed329213de
18 files changed
+55 -2393

No files matched your search

+6 -17
View File
@@ -1,7 +1,7 @@
# Device testing checklist
Simulator verification covered connect/disconnect, transcripts (text mode), media
reconciliation/UI, tasks panel, and the hardening pass. The items below need a **real
Simulator verification covered connect/disconnect, transcripts (text mode), tasks
panel, and the hardening pass. The items below need a **real
iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capture,
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
0 input channels).
@@ -31,29 +31,18 @@ ensure the server cert pin is current (see README — re-export after any server
side button to lock the device. It only disables the automatic idle timeout.
- [ ] After hang-up, auto-lock returns (idle timer re-enabled).
## 4. Audible media playback
- [ ] The deployed server now serves cached AAC for library tracks requested with
`?format=m4a`; verify actual speaker/headphone output on hardware.
- [ ] Play a library track: audio is audible, now-playing card shows art/title/status.
- [ ] Pause / ±10 s / stop / seek all work; agent mics off while playing
(`media_state` published), mics back on when stopped.
- [ ] End conversation while music was playing before → playback resumes.
## 5. Audio interruption (phone call / Siri)
## 4. Audio interruption (phone call / Siri)
- [ ] Mid-conversation, trigger an interruption (incoming call or "Hey Siri").
- [ ] Mic is muted on interruption begin; conversation/media pause cleanly.
- [ ] On end (with `shouldResume`), mic and media resume automatically.
- [ ] Media panel: AVPlayer pauses on interruption and resumes after.
- [ ] On end (with `shouldResume`), mic and conversation resume automatically.
## 6. Mic permission denied fallback
## 5. Mic permission denied fallback
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
banner shows the mic error; text mode remains fully usable.
## 7. General device sanity
## 6. General device sanity
- [ ] Connect / hang-up / reconnect matrix (3×) — no crashes, state resets cleanly.
- [ ] Change server URL in Settings while idle → session recreates without crash.
+32 -25
View File
@@ -1,13 +1,20 @@
# Project status and handoff
Last updated: 2026-09-09.
Last updated: 2026-09-12.
## Current direction
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
a ChatGPT subscription/device-token login. Do not revive that migration because
an old saved goal mentions it. Current work is the native iOS UI/UX redesign,
restoring the music/AI server, and end-to-end qualification.
an old saved goal mentions it. Current work is the native iOS UI/UX redesign
and end-to-end qualification.
**2026-09-12: the embedded music player was removed** by user decision — the
feature is gone from the app (Media tab, `Sources/Media/`, `MediaPanelView`,
`MediaAPI`, `MediaPlayerController`, `PinnedMediaResourceLoader`, media tests,
and the `media_state` control publish). The backend media API still exists and
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
the iOS app but remains deployed.
The UI and playback changes are implemented, and the NAS deployment was verified
on September 8. Full app qualification is **not complete**: physical-device tests
@@ -19,6 +26,8 @@ server health and device availability must be rechecked when resuming.
### Native interface and usability
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
(The Music tab was removed again on 2026-09-12 by user decision; the app now has
**Talk** and **Activity** tabs.)
- Replaced the large animated orb with a compact waveform/status presentation;
added a clearer welcome screen, Voice/Type selection before connecting, explicit
start/end controls, microphone meter, and recoverable connection errors.
@@ -27,7 +36,7 @@ server health and device availability must be rechecked when resuming.
- Reworked Settings around voice, input mode, model, and connection; server changes
require an explicit validated save. Voice selection persists.
- Improved Music library/search navigation, now-playing controls, seek/volume,
empty/loading/error states, and validated station-entry sheet.
empty/loading/error states, and validated station-entry sheet. (Removed 2026-09-12.)
- Improved Activity hierarchy and task detail presentation with an active-task badge.
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
and larger interaction targets, informed by the Impeccable native-iOS guidance.
@@ -35,14 +44,11 @@ server health and device availability must be rechecked when resuming.
### Functional fixes
- Observed nested transcript/state/task objects so asynchronous updates render.
- Preserved latest-wins server replacement; clear old media when switching servers.
- Sent standalone media controls and reports to the default server room.
- Refreshed track metadata as downloads become ready; exposed request/decoder errors.
- Loaded artwork through pinned TLS for the NAS self-signed certificate.
- Requested cached AAC (`?format=m4a`) for YouTube library tracks on iOS.
- Corrected AVFoundation's content type to a UTI rather than an HTTP MIME string.
- Changed the pinned media loader to deliver 256 KB byte ranges instead of buffering
an entire song before delivering usable audio, addressing a reproduced timeout.
- Preserved latest-wins server replacement.
(All media-related behaviors — media clearing on server switch, standalone media
controls/reports, track metadata refresh, pinned-TLS artwork, cached-AAC requests,
UTI content-type fix, bounded-range media loader — were removed with the music
player on 2026-09-12.)
### Tests and tooling
@@ -50,9 +56,10 @@ server health and device availability must be rechecked when resuming.
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
preferences are not reset by those UI-test launch arguments.
- Added media-control/error/metadata tests and opt-in real-NAS UI tests.
- Added media-control/error/metadata tests and opt-in real-NAS UI tests. (Media tests
were removed with the music player on 2026-09-12.)
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
they initiated even when an assertion fails.
they initiated even when an assertion fails. (Music playback test removed 2026-09-12.)
## Backend deployment and cross-repository dependency
@@ -87,15 +94,14 @@ Do not copy credentials or `.env` contents into documentation or commits.
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
- 68 unit tests passed; five offline UI tests passed (navigation/error recovery,
settings persistence/validation, music validation, transcript/task UI, large text).
- 49 unit tests passed (after the 2026-09-12 music-player removal; previously 68 with
media tests); four offline UI tests passed (navigation/error recovery,
settings persistence/validation, transcript/task UI, large text).
- Real NAS tests verified Qwen replies, background task completion, reconnect,
library loading, advancing playback time, pause, forward-seek control and stop.
(Playback evidence predates the music-player removal; the feature no longer exists
in the app.)
- Qwen/task/reconnect passed three consecutive iterations.
- Music initially failed intermittently. Logs captured an AVPlayer timeout while
buffering a whole 6 MB response. After bounded-range delivery, three consecutive
live playback iterations passed. The earlier isolated agent-join failure remains
unexplained; do not claim all intermittent connection issues are resolved.
- Backend integration: 16 passed, 7 skipped, 2 deselected.
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
and byte ranges); deployed range request returned HTTP 206.
@@ -115,19 +121,20 @@ Artifacts live on this Mac under the XcodeBuildMCP workspace, not in Git.
2. **Build, sign, install and run on that iPhone.** No successful device install is
recorded. Check Developer Mode, trust, signing team/profile, and LAN access.
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
barge-in, media/voice handoff, Siri/phone interruptions, lock/background behavior,
barge-in, Siri/phone interruptions, lock/background behavior,
microphone-permission-denied fallback, and network-loss recovery. Some checks
require the user's speech/listening or a real interruption; simulator state tests
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
4. **Broader regression:** rerun the full suite after the range-loader change; test
longer playback, seek near EOF, slow network/cancellation and server replacement.
Exercise new downloads, test-owned station add/play/remove, and voice/model
selection against the live agent. Do not delete the user's existing library song.
4. **Broader regression:** rerun the full suite after the music-player removal;
test slow network/cancellation and server replacement. Exercise voice/model
selection against the live agent.
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
and dark mode, large text and small-device layouts; test VoiceOver and focus.
The existing large-text navigation test is not a full accessibility audit.
6. **Publish the backend compatibility source safely** in the sibling repository
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
(Lower priority now: the iOS app no longer consumes the media API, but the deployed
container still serves it for the web UI.)
7. Update this report with results and remaining limitations. Do not mark the whole
app qualified until the outstanding end-to-end gates have evidence.