Archived
Remove embedded music player
The music feature is retired: delete the Media tab, MediaAPI, MediaPlayerController, PinnedMediaResourceLoader, MediaPanelView, the media_state control publish, media scope handling in VoiceSession, the volume setting, and all media tests. App is now Talk + Activity tabs. Backend media API in hope-voice-api is untouched (still serves the web UI).
This commit is contained in:
1 parent
3620a2387b
commit
ed329213de
18 files changed
+55
-2393
No files matched your search
+6
-17
@@ -1,7 +1,7 @@
|
||||
# Device testing checklist
|
||||
|
||||
Simulator verification covered connect/disconnect, transcripts (text mode), media
|
||||
reconciliation/UI, tasks panel, and the hardening pass. The items below need a **real
|
||||
Simulator verification covered connect/disconnect, transcripts (text mode), tasks
|
||||
panel, and the hardening pass. The items below need a **real
|
||||
iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capture,
|
||||
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
|
||||
0 input channels).
|
||||
@@ -31,29 +31,18 @@ ensure the server cert pin is current (see README — re-export after any server
|
||||
side button to lock the device. It only disables the automatic idle timeout.
|
||||
- [ ] After hang-up, auto-lock returns (idle timer re-enabled).
|
||||
|
||||
## 4. Audible media playback
|
||||
|
||||
- [ ] The deployed server now serves cached AAC for library tracks requested with
|
||||
`?format=m4a`; verify actual speaker/headphone output on hardware.
|
||||
- [ ] Play a library track: audio is audible, now-playing card shows art/title/status.
|
||||
- [ ] Pause / ±10 s / stop / seek all work; agent mics off while playing
|
||||
(`media_state` published), mics back on when stopped.
|
||||
- [ ] End conversation while music was playing before → playback resumes.
|
||||
|
||||
## 5. Audio interruption (phone call / Siri)
|
||||
## 4. Audio interruption (phone call / Siri)
|
||||
|
||||
- [ ] Mid-conversation, trigger an interruption (incoming call or "Hey Siri").
|
||||
- [ ] Mic is muted on interruption begin; conversation/media pause cleanly.
|
||||
- [ ] On end (with `shouldResume`), mic and media resume automatically.
|
||||
- [ ] Media panel: AVPlayer pauses on interruption and resumes after.
|
||||
- [ ] On end (with `shouldResume`), mic and conversation resume automatically.
|
||||
|
||||
## 6. Mic permission denied fallback
|
||||
## 5. Mic permission denied fallback
|
||||
|
||||
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
|
||||
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
|
||||
banner shows the mic error; text mode remains fully usable.
|
||||
|
||||
## 7. General device sanity
|
||||
## 6. General device sanity
|
||||
|
||||
- [ ] Connect / hang-up / reconnect matrix (3×) — no crashes, state resets cleanly.
|
||||
- [ ] Change server URL in Settings while idle → session recreates without crash.
|
||||
|
||||
+32
-25
@@ -1,13 +1,20 @@
|
||||
# Project status and handoff
|
||||
|
||||
Last updated: 2026-09-09.
|
||||
Last updated: 2026-09-12.
|
||||
|
||||
## Current direction
|
||||
|
||||
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
||||
a ChatGPT subscription/device-token login. Do not revive that migration because
|
||||
an old saved goal mentions it. Current work is the native iOS UI/UX redesign,
|
||||
restoring the music/AI server, and end-to-end qualification.
|
||||
an old saved goal mentions it. Current work is the native iOS UI/UX redesign
|
||||
and end-to-end qualification.
|
||||
|
||||
**2026-09-12: the embedded music player was removed** by user decision — the
|
||||
feature is gone from the app (Media tab, `Sources/Media/`, `MediaPanelView`,
|
||||
`MediaAPI`, `MediaPlayerController`, `PinnedMediaResourceLoader`, media tests,
|
||||
and the `media_state` control publish). The backend media API still exists and
|
||||
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
|
||||
the iOS app but remains deployed.
|
||||
|
||||
The UI and playback changes are implemented, and the NAS deployment was verified
|
||||
on September 8. Full app qualification is **not complete**: physical-device tests
|
||||
@@ -19,6 +26,8 @@ server health and device availability must be rechecked when resuming.
|
||||
### Native interface and usability
|
||||
|
||||
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
|
||||
(The Music tab was removed again on 2026-09-12 by user decision; the app now has
|
||||
**Talk** and **Activity** tabs.)
|
||||
- Replaced the large animated orb with a compact waveform/status presentation;
|
||||
added a clearer welcome screen, Voice/Type selection before connecting, explicit
|
||||
start/end controls, microphone meter, and recoverable connection errors.
|
||||
@@ -27,7 +36,7 @@ server health and device availability must be rechecked when resuming.
|
||||
- Reworked Settings around voice, input mode, model, and connection; server changes
|
||||
require an explicit validated save. Voice selection persists.
|
||||
- Improved Music library/search navigation, now-playing controls, seek/volume,
|
||||
empty/loading/error states, and validated station-entry sheet.
|
||||
empty/loading/error states, and validated station-entry sheet. (Removed 2026-09-12.)
|
||||
- Improved Activity hierarchy and task detail presentation with an active-task badge.
|
||||
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
|
||||
and larger interaction targets, informed by the Impeccable native-iOS guidance.
|
||||
@@ -35,14 +44,11 @@ server health and device availability must be rechecked when resuming.
|
||||
### Functional fixes
|
||||
|
||||
- Observed nested transcript/state/task objects so asynchronous updates render.
|
||||
- Preserved latest-wins server replacement; clear old media when switching servers.
|
||||
- Sent standalone media controls and reports to the default server room.
|
||||
- Refreshed track metadata as downloads become ready; exposed request/decoder errors.
|
||||
- Loaded artwork through pinned TLS for the NAS self-signed certificate.
|
||||
- Requested cached AAC (`?format=m4a`) for YouTube library tracks on iOS.
|
||||
- Corrected AVFoundation's content type to a UTI rather than an HTTP MIME string.
|
||||
- Changed the pinned media loader to deliver 256 KB byte ranges instead of buffering
|
||||
an entire song before delivering usable audio, addressing a reproduced timeout.
|
||||
- Preserved latest-wins server replacement.
|
||||
(All media-related behaviors — media clearing on server switch, standalone media
|
||||
controls/reports, track metadata refresh, pinned-TLS artwork, cached-AAC requests,
|
||||
UTI content-type fix, bounded-range media loader — were removed with the music
|
||||
player on 2026-09-12.)
|
||||
|
||||
### Tests and tooling
|
||||
|
||||
@@ -50,9 +56,10 @@ server health and device availability must be rechecked when resuming.
|
||||
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
|
||||
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
|
||||
preferences are not reset by those UI-test launch arguments.
|
||||
- Added media-control/error/metadata tests and opt-in real-NAS UI tests.
|
||||
- Added media-control/error/metadata tests and opt-in real-NAS UI tests. (Media tests
|
||||
were removed with the music player on 2026-09-12.)
|
||||
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
|
||||
they initiated even when an assertion fails.
|
||||
they initiated even when an assertion fails. (Music playback test removed 2026-09-12.)
|
||||
|
||||
## Backend deployment and cross-repository dependency
|
||||
|
||||
@@ -87,15 +94,14 @@ Do not copy credentials or `.env` contents into documentation or commits.
|
||||
|
||||
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
|
||||
|
||||
- 68 unit tests passed; five offline UI tests passed (navigation/error recovery,
|
||||
settings persistence/validation, music validation, transcript/task UI, large text).
|
||||
- 49 unit tests passed (after the 2026-09-12 music-player removal; previously 68 with
|
||||
media tests); four offline UI tests passed (navigation/error recovery,
|
||||
settings persistence/validation, transcript/task UI, large text).
|
||||
- Real NAS tests verified Qwen replies, background task completion, reconnect,
|
||||
library loading, advancing playback time, pause, forward-seek control and stop.
|
||||
(Playback evidence predates the music-player removal; the feature no longer exists
|
||||
in the app.)
|
||||
- Qwen/task/reconnect passed three consecutive iterations.
|
||||
- Music initially failed intermittently. Logs captured an AVPlayer timeout while
|
||||
buffering a whole 6 MB response. After bounded-range delivery, three consecutive
|
||||
live playback iterations passed. The earlier isolated agent-join failure remains
|
||||
unexplained; do not claim all intermittent connection issues are resolved.
|
||||
- Backend integration: 16 passed, 7 skipped, 2 deselected.
|
||||
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
|
||||
and byte ranges); deployed range request returned HTTP 206.
|
||||
@@ -115,19 +121,20 @@ Artifacts live on this Mac under the XcodeBuildMCP workspace, not in Git.
|
||||
2. **Build, sign, install and run on that iPhone.** No successful device install is
|
||||
recorded. Check Developer Mode, trust, signing team/profile, and LAN access.
|
||||
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
||||
barge-in, media/voice handoff, Siri/phone interruptions, lock/background behavior,
|
||||
barge-in, Siri/phone interruptions, lock/background behavior,
|
||||
microphone-permission-denied fallback, and network-loss recovery. Some checks
|
||||
require the user's speech/listening or a real interruption; simulator state tests
|
||||
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
||||
4. **Broader regression:** rerun the full suite after the range-loader change; test
|
||||
longer playback, seek near EOF, slow network/cancellation and server replacement.
|
||||
Exercise new downloads, test-owned station add/play/remove, and voice/model
|
||||
selection against the live agent. Do not delete the user's existing library song.
|
||||
4. **Broader regression:** rerun the full suite after the music-player removal;
|
||||
test slow network/cancellation and server replacement. Exercise voice/model
|
||||
selection against the live agent.
|
||||
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
|
||||
and dark mode, large text and small-device layouts; test VoiceOver and focus.
|
||||
The existing large-text navigation test is not a full accessibility audit.
|
||||
6. **Publish the backend compatibility source safely** in the sibling repository
|
||||
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
|
||||
(Lower priority now: the iOS app no longer consumes the media API, but the deployed
|
||||
container still serves it for the web UI.)
|
||||
7. Update this report with results and remaining limitations. Do not mark the whole
|
||||
app qualified until the outstanding end-to-end gates have evidence.
|
||||
|
||||
|
||||
Reference in new issue
Block a user