161 lines
9.0 KiB
Markdown
161 lines
9.0 KiB
Markdown
# Project status and handoff
|
||
|
||
Last updated: 2026-09-12.
|
||
|
||
## Current direction
|
||
|
||
Keep **Qwen 2.5**. The user explicitly canceled the earlier request to switch to
|
||
a ChatGPT subscription/device-token login. Do not revive that migration because
|
||
an old saved goal mentions it. Current work is the native iOS UI/UX redesign
|
||
and end-to-end qualification.
|
||
|
||
**2026-09-12: the embedded music player was removed** by user decision — the
|
||
feature is gone from the app (Media tab, `Sources/Media/`, `MediaPanelView`,
|
||
`MediaAPI`, `MediaPlayerController`, `PinnedMediaResourceLoader`, media tests,
|
||
and the `media_state` control publish). The backend media API still exists and
|
||
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
|
||
the iOS app but remains deployed.
|
||
|
||
The **Nearlight** native redesign is implemented across Talk, Activity, Settings,
|
||
pickers, task details, and connection diagnostics. Current simulator qualification
|
||
is green, and GPT-6 Astra's final blocker re-verification returned **PASS**. The
|
||
final source also produced a signed generic-iOS build, and once the phone
|
||
reconnected the final build was installed and launched on the paired iPhone.
|
||
Hardware audio and interruption qualification remains open.
|
||
|
||
## Completed work
|
||
|
||
### Native interface and usability
|
||
|
||
- The app now has native **Talk** and **Activity** tabs sharing one session;
|
||
changing tabs never ends a conversation.
|
||
- Implemented Astra's **Nearlight** direction: an adaptive plum/rosewood semantic
|
||
palette, restrained rounded typography, shared spacing/radius/motion tokens, and
|
||
the compact Lightwell aperture as the conversation state signature.
|
||
- Rebuilt Talk around a clear pre-connection welcome, Voice/Type choice, explicit
|
||
start/end controls, compact active status, microphone meter, persistent Type
|
||
draft, and a dock that keeps primary actions reachable.
|
||
- Reworked the transcript as a readable shared record with speaker/time metadata,
|
||
live partials, copy/selection, explicit destructive confirmation, and measured
|
||
follow-latest behavior. Restored history stays at the top; manual reading,
|
||
selection, and VoiceOver row focus suspend following until **Latest**.
|
||
- Made connection and ready-session microphone failures visible without discarding
|
||
the real error. Permission denial exposes an iPhone Settings recovery action;
|
||
Type mode remains available.
|
||
- Reworked Settings around voice, mode, model, and connection. Server edits are
|
||
validated and explicitly saved; normalization confirmation survives session
|
||
replacement and clears on the next user edit.
|
||
- Improved Activity hierarchy, active-count badge, empty state, state-specific
|
||
rows, and task detail presentation.
|
||
- Removed forced dark appearance; all redesigned surfaces support light/dark mode,
|
||
Dynamic Type, Reduce Motion, increased contrast branches, and 44-point-or-larger
|
||
interaction targets where applicable.
|
||
|
||
### Functional fixes
|
||
|
||
- Observed nested transcript, conversation-state, and task objects so asynchronous
|
||
updates render.
|
||
- Preserved latest-wins server replacement while keeping an open Settings
|
||
navigation stack and its success feedback intact.
|
||
- Preserved Type-mode microphone privacy, pending-message send gating, clean session
|
||
teardown, transcript persistence/deduplication, and interruption ordering.
|
||
- All media-related client behavior was removed with the music player on 2026-09-12.
|
||
|
||
### Tests and tooling
|
||
|
||
- Installed XcodeBuildMCP and its skills; used it for simulator build/run/test work.
|
||
- The current combined simulator run passed **50 tests, 0 failures**, with the
|
||
opt-in live suite skipped once. Five offline UI scenarios cover recovery,
|
||
settings validation/persistence and save feedback, transcript/task flows,
|
||
restored-history reading position, and accessibility-size navigation.
|
||
- A post-focus-change targeted restored-history UI regression passed independently.
|
||
- DEBUG UI fixtures use isolated preferences; ordinary application preferences and
|
||
the NAS are not modified by offline UI runs.
|
||
- GPT-6 Astra completed the design specification and four verification rounds. Its
|
||
final scoped review found B1–B3 resolved, N1/N2 preserved, and no concrete
|
||
remaining release blocker. Accepted visual/adaptive/accessibility polish remains
|
||
documented as debt, not as exhaustive WCAG or VoiceOver qualification.
|
||
|
||
## Backend deployment and cross-repository dependency
|
||
|
||
SSH host: `[email protected]` (Unraid). App URL: `https://192.168.86.2:9178`.
|
||
|
||
- `voice` image: `hope-voice-api-voice:68f4c46-ios-audio`, host networking,
|
||
restart policy `unless-stopped`. Includes LiveKit, voice agent, token service,
|
||
media API, and nginx.
|
||
- LLM: `qwen2.5-vl-3b` through existing OpenWebUI at
|
||
`http://192.168.86.2:3005/api`. Existing Azure speech configuration retained.
|
||
- `voice`, `open-webui`, and `local-vl-models` were present and healthy when the
|
||
user suspected deletion. Do not recreate healthy services unnecessarily.
|
||
- Original library, certificates, and skills were retained under
|
||
`/mnt/user/dev/hope-voice-api`; the dirty remote checkout was not overwritten.
|
||
- Separate deployment source: `/mnt/user/dev/hope-voice-deploy-68f4c46`.
|
||
- Stopped rollback container: `voice-before-ios-audio`.
|
||
- Backend AAC conversion preserves original WebM files and serves cached AAC with
|
||
byte-range support. Browser/raw requests retain their original behavior.
|
||
|
||
**Important:** the backend patch and its test are in a different local repository:
|
||
`/Users/shane/dev/hope-voice-api/web/media_server.py` and
|
||
`/Users/shane/dev/hope-voice-api/tests/test_ios_audio.py`. They were still uncommitted
|
||
at this handoff. Committing/pushing `voice-ios` does **not** publish those files.
|
||
The deployed image contains the patch, but a future backend CI rebuild can lose it
|
||
unless it is committed and deployed through that repository's workflow. Inspect
|
||
that workflow before pushing: it may replace the live container.
|
||
|
||
Launch configuration and rollback details: [DEPLOYMENT.md](DEPLOYMENT.md).
|
||
Do not copy credentials or `.env` contents into documentation or commits.
|
||
|
||
## Verification performed
|
||
|
||
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator
|
||
`AFDA137E-1657-428E-BF01-E5CD7989772C`.
|
||
|
||
- Current combined run: **50 passed, 0 failed, 1 live skip** in 104.9 seconds.
|
||
Result bundle:
|
||
`test_sim_2026-09-12T14-03-09-058Z_pid95846_2ca9da7e.xcresult`.
|
||
- Post-focus-change restored-history regression: **1 passed, 0 failed**.
|
||
Result bundle:
|
||
`test_sim_2026-09-12T14-05-21-070Z_pid98187_bd3e531a.xcresult`.
|
||
- A fresh isolated simulator launch showed restored history top-anchored with
|
||
**Latest** visible; the UI regression then tapped Latest and verified the final
|
||
event became visible.
|
||
- Astra's targeted isolated runtime check verified a padded changed-address save
|
||
stays in Settings, normalizes the field, shows its checkmark and confirmation,
|
||
and clears both only after the next user edit. Final verdict: **PASS**.
|
||
- iPhone 14 Pro Max `BF93D35E-0853-5470-A125-6EB0DB3F17AB` was paired, signed,
|
||
installed, trusted, and launched successfully. The final Nearlight build
|
||
(`/tmp/voice-ios-device-derived/Build/Products/Debug-iphoneos/Voice.app`,
|
||
`dev.shane.voice`) was reinstalled and launched via `devicectl` after the phone
|
||
reconnected. Free provisioning expires after seven days; reinstall to re-sign.
|
||
- Historical live NAS runs verified Qwen replies, background task completion, and
|
||
reconnect. Backend integration remains 16 passed, 7 skipped, 2 deselected.
|
||
|
||
Exact historical qualifications: [VERIFICATION.md](VERIFICATION.md). Artifacts
|
||
live on this Mac under the XcodeBuildMCP workspace, not in Git.
|
||
|
||
## What remains, in order
|
||
|
||
1. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
|
||
barge-in, Siri/phone interruptions, lock/background behavior,
|
||
microphone-permission-denied fallback, and network-loss recovery. These require
|
||
the user's speech/listening or a real interruption; simulator state tests do not
|
||
prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
|
||
2. **Broader live regression:** exercise slow-network cancellation, network-loss
|
||
recovery, repeated server replacement, and voice/model selection against the
|
||
live agent.
|
||
3. **Manual accessibility/visual qualification:** inspect populated/error screens
|
||
in light and dark mode, accessibility text sizes, narrow/landscape layouts, and
|
||
a real VoiceOver focus session. Astra accepted the remaining presentation debt
|
||
for this personal-app release; that is not proof of exhaustive conformance.
|
||
4. **Publish the backend compatibility source safely** in the sibling repository
|
||
and ensure its normal deployment preserves mounts, certificates, and Qwen
|
||
settings. This is lower priority because the iOS app no longer consumes the
|
||
media API, but the deployed container still serves it for the web UI.
|
||
|
||
## Repository handoff
|
||
|
||
The iOS work and this report belong to branch `feat/ios-app`, remote `origin`
|
||
(`https://git.srmr.xyz/snowspeeder/hope-voice-ios.git`). Keep the Nearlight
|
||
implementation, tests, physical-device notes, and this handoff together when
|
||
committing. Check Git history and remote status for the final publish result.
|