Archived
140 lines
8.1 KiB
Markdown
140 lines
8.1 KiB
Markdown
# Project status and handoff
|
|
|
|
Last updated: 2026-09-09.
|
|
|
|
## Current direction
|
|
|
|
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
|
a ChatGPT subscription/device-token login. Do not revive that migration because
|
|
an old saved goal mentions it. Current work is the native iOS UI/UX redesign,
|
|
restoring the music/AI server, and end-to-end qualification.
|
|
|
|
The UI and playback changes are implemented, and the NAS deployment was verified
|
|
on September 8. Full app qualification is **not complete**: physical-device tests
|
|
and some broader live scenarios remain. This report summarizes prior evidence;
|
|
server health and device availability must be rechecked when resuming.
|
|
|
|
## Completed work
|
|
|
|
### Native interface and usability
|
|
|
|
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
|
|
- Replaced the large animated orb with a compact waveform/status presentation;
|
|
added a clearer welcome screen, Voice/Type selection before connecting, explicit
|
|
start/end controls, microphone meter, and recoverable connection errors.
|
|
- Improved transcript readability, timestamps, live updates, copy/selection,
|
|
follow-latest scrolling, and explicit Cancel/Clear confirmation.
|
|
- Reworked Settings around voice, input mode, model, and connection; server changes
|
|
require an explicit validated save. Voice selection persists.
|
|
- Improved Music library/search navigation, now-playing controls, seek/volume,
|
|
empty/loading/error states, and validated station-entry sheet.
|
|
- Improved Activity hierarchy and task detail presentation with an active-task badge.
|
|
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
|
|
and larger interaction targets, informed by the Impeccable native-iOS guidance.
|
|
|
|
### Functional fixes
|
|
|
|
- Observed nested transcript/state/task objects so asynchronous updates render.
|
|
- Preserved latest-wins server replacement; clear old media when switching servers.
|
|
- Sent standalone media controls and reports to the default server room.
|
|
- Refreshed track metadata as downloads become ready; exposed request/decoder errors.
|
|
- Loaded artwork through pinned TLS for the NAS self-signed certificate.
|
|
- Requested cached AAC (`?format=m4a`) for YouTube library tracks on iOS.
|
|
- Corrected AVFoundation's content type to a UTI rather than an HTTP MIME string.
|
|
- Changed the pinned media loader to deliver 256 KB byte ranges instead of buffering
|
|
an entire song before delivering usable audio, addressing a reproduced timeout.
|
|
|
|
### Tests and tooling
|
|
|
|
- Installed XcodeBuildMCP and its skills; used it for simulator build/run/test work.
|
|
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
|
|
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
|
|
preferences are not reset by those UI-test launch arguments.
|
|
- Added media-control/error/metadata tests and opt-in real-NAS UI tests.
|
|
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
|
|
they initiated even when an assertion fails.
|
|
|
|
## Backend deployment and cross-repository dependency
|
|
|
|
SSH host: `[email protected]` (Unraid). App URL: `https://192.168.86.2:9178`.
|
|
|
|
- `voice` image: `hope-voice-api-voice:68f4c46-ios-audio`, host networking,
|
|
restart policy `unless-stopped`. Includes LiveKit, voice agent, token service,
|
|
media API, and nginx.
|
|
- LLM: `qwen2.5-vl-3b` through existing OpenWebUI at
|
|
`http://192.168.86.2:3005/api`. Existing Azure speech configuration retained.
|
|
- `voice`, `open-webui`, and `local-vl-models` were present and healthy when the
|
|
user suspected deletion. Do not recreate healthy services unnecessarily.
|
|
- Original library, certificates, and skills were retained under
|
|
`/mnt/user/dev/hope-voice-api`; the dirty remote checkout was not overwritten.
|
|
- Separate deployment source: `/mnt/user/dev/hope-voice-deploy-68f4c46`.
|
|
- Stopped rollback container: `voice-before-ios-audio`.
|
|
- Backend AAC conversion preserves original WebM files and serves cached AAC with
|
|
byte-range support. Browser/raw requests retain their original behavior.
|
|
|
|
**Important:** the backend patch and its test are in a different local repository:
|
|
`/Users/shane/dev/hope-voice-api/web/media_server.py` and
|
|
`/Users/shane/dev/hope-voice-api/tests/test_ios_audio.py`. They were still uncommitted
|
|
at this handoff. Committing/pushing `voice-ios` does **not** publish those files.
|
|
The deployed image contains the patch, but a future backend CI rebuild can lose it
|
|
unless it is committed and deployed through that repository's workflow. Inspect
|
|
that workflow before pushing: it may replace the live container.
|
|
|
|
Launch configuration and rollback details: [DEPLOYMENT.md](DEPLOYMENT.md).
|
|
Do not copy credentials or `.env` contents into documentation or commits.
|
|
|
|
## Verification performed
|
|
|
|
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
|
|
|
|
- 68 unit tests passed; five offline UI tests passed (navigation/error recovery,
|
|
settings persistence/validation, music validation, transcript/task UI, large text).
|
|
- Real NAS tests verified Qwen replies, background task completion, reconnect,
|
|
library loading, advancing playback time, pause, forward-seek control and stop.
|
|
- Qwen/task/reconnect passed three consecutive iterations.
|
|
- Music initially failed intermittently. Logs captured an AVPlayer timeout while
|
|
buffering a whole 6 MB response. After bounded-range delivery, three consecutive
|
|
live playback iterations passed. The earlier isolated agent-join failure remains
|
|
unexplained; do not claim all intermittent connection issues are resolved.
|
|
- Backend integration: 16 passed, 7 skipped, 2 deselected.
|
|
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
|
|
and byte ranges); deployed range request returned HTTP 206.
|
|
- The final bounded-range change was verified by focused playback repetitions,
|
|
not a new full combined regression run.
|
|
|
|
Exact result-bundle names and qualifications: [VERIFICATION.md](VERIFICATION.md).
|
|
Artifacts live on this Mac under the XcodeBuildMCP workspace, not in Git.
|
|
|
|
## What remains, in order
|
|
|
|
1. **Pair the physical iPhone.** Latest check detected it on USB and reported iOS
|
|
26.6.1, device ID `BF93D35E-0853-5470-A125-6EB0DB3F17AB`, state `unpaired`.
|
|
The user had unlocked it but could not see it in Finder. Last guidance was to
|
|
enable iOS devices in Finder's Sidebar settings and accept any accessory/Trust
|
|
prompts. Recheck current state rather than repeating old cable assumptions.
|
|
2. **Build, sign, install and run on that iPhone.** No successful device install is
|
|
recorded. Check Developer Mode, trust, signing team/profile, and LAN access.
|
|
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
|
barge-in, media/voice handoff, Siri/phone interruptions, lock/background behavior,
|
|
microphone-permission-denied fallback, and network-loss recovery. Some checks
|
|
require the user's speech/listening or a real interruption; simulator state tests
|
|
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
|
4. **Broader regression:** rerun the full suite after the range-loader change; test
|
|
longer playback, seek near EOF, slow network/cancellation and server replacement.
|
|
Exercise new downloads, test-owned station add/play/remove, and voice/model
|
|
selection against the live agent. Do not delete the user's existing library song.
|
|
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
|
|
and dark mode, large text and small-device layouts; test VoiceOver and focus.
|
|
The existing large-text navigation test is not a full accessibility audit.
|
|
6. **Publish the backend compatibility source safely** in the sibling repository
|
|
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
|
|
7. Update this report with results and remaining limitations. Do not mark the whole
|
|
app qualified until the outstanding end-to-end gates have evidence.
|
|
|
|
## Repository handoff
|
|
|
|
The iOS work and this report belong to branch `feat/ios-app`, remote `origin`
|
|
(`https://git.srmr.xyz/snowspeeder/voice-ios.git`). The requested handoff commit
|
|
includes the implementation, tests, deployment notes and startup instructions.
|
|
Check Git history/remote status for the final commit and push result.
|