fix: close Nearlight release blockers

This commit is contained in:
snowspeeder committed 2026-09-12 10:13:14 -04:00
1 parent 5577162594
commit b7cec80a99
7 files changed
+310 -179

No files matched your search

+1 -1
View File
@@ -12,7 +12,7 @@ ensure the server cert pin is current (see README — re-export after any server
## 1. Mic input with real speech
- [ ] Start → agent joins → mic enables **without** `-4010`.
- [ ] Speak: orb/status transitions to **listening**; mic meter moves with voice level.
- [ ] Speak: Lightwell/status transitions to **listening**; mic meter moves with voice level.
- [ ] Stop speaking, wait >2 s while agent is silent → **thinking** chip appears.
- [ ] Agent replies: state goes to **speaking** (remote audio drives it), TTS audible.
- [ ] User transcript finals arrive in the list with correct role/timestamps.
+95 -76
View File
@@ -16,50 +16,66 @@ and the `media_state` control publish). The backend media API still exists and
is untouched. The backend AAC compatibility patch (below) is now irrelevant to
the iOS app but remains deployed.
The UI and playback changes are implemented, and the NAS deployment was verified
on September 8. Full app qualification is **not complete**: physical-device tests
and some broader live scenarios remain. This report summarizes prior evidence;
server health and device availability must be rechecked when resuming.
The **Nearlight** native redesign is implemented across Talk, Activity, Settings,
pickers, task details, and connection diagnostics. Current simulator qualification
is green, and GPT-6 Astra's final blocker re-verification returned **PASS**. The
final source also produced a signed generic-iOS build. A physical install and
launch succeeded before the last Nearlight corrections; the paired iPhone is
currently unavailable, so install the final build when it reconnects. Hardware
audio and interruption qualification remains open.
## Completed work
### Native interface and usability
- Reorganized the app into native **Talk, Music, Activity** tabs, sharing one session.
(The Music tab was removed again on 2026-09-12 by user decision; the app now has
**Talk** and **Activity** tabs.)
- Replaced the large animated orb with a compact waveform/status presentation;
added a clearer welcome screen, Voice/Type selection before connecting, explicit
start/end controls, microphone meter, and recoverable connection errors.
- Improved transcript readability, timestamps, live updates, copy/selection,
follow-latest scrolling, and explicit Cancel/Clear confirmation.
- Reworked Settings around voice, input mode, model, and connection; server changes
require an explicit validated save. Voice selection persists.
- Improved Music library/search navigation, now-playing controls, seek/volume,
empty/loading/error states, and validated station-entry sheet. (Removed 2026-09-12.)
- Improved Activity hierarchy and task detail presentation with an active-task badge.
- Removed forced dark appearance; adopted adaptive system styling, Dynamic Type,
and larger interaction targets, informed by the Impeccable native-iOS guidance.
- The app now has native **Talk** and **Activity** tabs sharing one session;
changing tabs never ends a conversation.
- Implemented Astra's **Nearlight** direction: an adaptive plum/rosewood semantic
palette, restrained rounded typography, shared spacing/radius/motion tokens, and
the compact Lightwell aperture as the conversation state signature.
- Rebuilt Talk around a clear pre-connection welcome, Voice/Type choice, explicit
start/end controls, compact active status, microphone meter, persistent Type
draft, and a dock that keeps primary actions reachable.
- Reworked the transcript as a readable shared record with speaker/time metadata,
live partials, copy/selection, explicit destructive confirmation, and measured
follow-latest behavior. Restored history stays at the top; manual reading,
selection, and VoiceOver row focus suspend following until **Latest**.
- Made connection and ready-session microphone failures visible without discarding
the real error. Permission denial exposes an iPhone Settings recovery action;
Type mode remains available.
- Reworked Settings around voice, mode, model, and connection. Server edits are
validated and explicitly saved; normalization confirmation survives session
replacement and clears on the next user edit.
- Improved Activity hierarchy, active-count badge, empty state, state-specific
rows, and task detail presentation.
- Removed forced dark appearance; all redesigned surfaces support light/dark mode,
Dynamic Type, Reduce Motion, increased contrast branches, and 44-point-or-larger
interaction targets where applicable.
### Functional fixes
- Observed nested transcript/state/task objects so asynchronous updates render.
- Preserved latest-wins server replacement.
(All media-related behaviors — media clearing on server switch, standalone media
controls/reports, track metadata refresh, pinned-TLS artwork, cached-AAC requests,
UTI content-type fix, bounded-range media loader — were removed with the music
player on 2026-09-12.)
- Observed nested transcript, conversation-state, and task objects so asynchronous
updates render.
- Preserved latest-wins server replacement while keeping an open Settings
navigation stack and its success feedback intact.
- Preserved Type-mode microphone privacy, pending-message send gating, clean session
teardown, transcript persistence/deduplication, and interruption ordering.
- All media-related client behavior was removed with the music player on 2026-09-12.
### Tests and tooling
- Installed XcodeBuildMCP and its skills; used it for simulator build/run/test work.
- Added the `VoiceUITests` target in `project.yml` and regenerated the Xcode project.
- Added isolated DEBUG UI-test preferences and optional fixtures; normal application
preferences are not reset by those UI-test launch arguments.
- Added media-control/error/metadata tests and opt-in real-NAS UI tests. (Media tests
were removed with the music player on 2026-09-12.)
- Live tests retain failure screenshots/UI hierarchy and attempt to stop music
they initiated even when an assertion fails. (Music playback test removed 2026-09-12.)
- The current combined simulator run passed **50 tests, 0 failures**, with the
opt-in live suite skipped once. Five offline UI scenarios cover recovery,
settings validation/persistence and save feedback, transcript/task flows,
restored-history reading position, and accessibility-size navigation.
- A post-focus-change targeted restored-history UI regression passed independently.
- DEBUG UI fixtures use isolated preferences; ordinary application preferences and
the NAS are not modified by offline UI runs.
- GPT-6 Astra completed the design specification and four verification rounds. Its
final scoped review found B1–B3 resolved, N1/N2 preserved, and no concrete
remaining release blocker. Accepted visual/adaptive/accessibility polish remains
documented as debt, not as exhaustive WCAG or VoiceOver qualification.
## Backend deployment and cross-repository dependency
@@ -92,57 +108,60 @@ Do not copy credentials or `.env` contents into documentation or commits.
## Verification performed
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator.
Project `Voice.xcodeproj`, scheme `Voice_iOS`, iPhone 17 Pro simulator
`AFDA137E-1657-428E-BF01-E5CD7989772C`.
- 49 unit tests passed (after the 2026-09-12 music-player removal; previously 68 with
media tests); four offline UI tests passed (navigation/error recovery,
settings persistence/validation, transcript/task UI, large text).
- Real NAS tests verified Qwen replies, background task completion, reconnect,
library loading, advancing playback time, pause, forward-seek control and stop.
(Playback evidence predates the music-player removal; the feature no longer exists
in the app.)
- Qwen/task/reconnect passed three consecutive iterations.
- Backend integration: 16 passed, 7 skipped, 2 deselected.
- Backend AAC regression: one passed (conversion, cache reuse, original preservation,
and byte ranges); deployed range request returned HTTP 206.
- The final bounded-range change was verified by focused playback repetitions,
not a new full combined regression run.
- Current combined run: **50 passed, 0 failed, 1 live skip** in 104.9 seconds.
Result bundle:
`test_sim_2026-09-12T14-03-09-058Z_pid95846_2ca9da7e.xcresult`.
- Post-focus-change restored-history regression: **1 passed, 0 failed**.
Result bundle:
`test_sim_2026-09-12T14-05-21-070Z_pid98187_bd3e531a.xcresult`.
- A fresh isolated simulator launch showed restored history top-anchored with
**Latest** visible; the UI regression then tapped Latest and verified the final
event became visible.
- Astra's targeted isolated runtime check verified a padded changed-address save
stays in Settings, normalizes the field, shows its checkmark and confirmation,
and clears both only after the next user edit. Final verdict: **PASS**.
- iPhone 14 Pro Max `BF93D35E-0853-5470-A125-6EB0DB3F17AB` was paired, signed,
installed, trusted, and launched successfully. The final source subsequently
produced a signed device build at
`/tmp/voice-ios-device-derived/Build/Products/Debug-iphoneos/Voice.app`.
Free provisioning expires after seven days. The phone was unavailable during
the final correction pass, so that final build still needs installing.
- Historical live NAS runs verified Qwen replies, background task completion, and
reconnect. Backend integration remains 16 passed, 7 skipped, 2 deselected.
Exact result-bundle names and qualifications: [VERIFICATION.md](VERIFICATION.md).
Artifacts live on this Mac under the XcodeBuildMCP workspace, not in Git.
Exact historical qualifications: [VERIFICATION.md](VERIFICATION.md). Artifacts
live on this Mac under the XcodeBuildMCP workspace, not in Git.
## What remains, in order
1. ~~Pair the physical iPhone.~~ **Done 2026-09-12.** iPhone 14 Pro Max
(`BF93D35E-0853-5470-A125-6EB0DB3F17AB`) is paired and available.
2. ~~Build, sign, install and run on that iPhone.~~ **Done 2026-09-12.** Signing
team is the free personal team `YJFYRB92NW` (`DEVELOPMENT_TEAM` in
`project.yml`; `H8D4WY66F6` is the certificate's individual-ID suffix, not a
team ID — do not use it). Xcode required a fresh account sign-in, then the
user tapped Trust under Settings → General → VPN & Device Management before
first launch. App installed and launched via `devicectl`. **Limitation:** free
provisioning expires the app after 7 days; reinstall to re-sign.
3. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
1. **Install the final signed build on the paired iPhone** when it reconnects.
The current artifact is
`/tmp/voice-ios-device-derived/Build/Products/Debug-iphoneos/Voice.app`; the
earlier successful install predates the last reading-safety, microphone-notice,
and Settings-confirmation corrections.
2. **Qualify real audio:** microphone capture, TTS audibility, speaker/headphones,
barge-in, Siri/phone interruptions, lock/background behavior,
microphone-permission-denied fallback, and network-loss recovery. Some checks
require the user's speech/listening or a real interruption; simulator state tests
do not prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
4. **Broader regression:** rerun the full suite after the music-player removal;
test slow network/cancellation and server replacement. Exercise voice/model
selection against the live agent.
5. **Final visual/accessibility pass:** inspect all populated/error screens in light
and dark mode, large text and small-device layouts; test VoiceOver and focus.
The existing large-text navigation test is not a full accessibility audit.
6. **Publish the backend compatibility source safely** in the sibling repository
and ensure its normal deployment preserves mounts, certificates and Qwen settings.
(Lower priority now: the iOS app no longer consumes the media API, but the deployed
container still serves it for the web UI.)
7. Update this report with results and remaining limitations. Do not mark the whole
app qualified until the outstanding end-to-end gates have evidence.
microphone-permission-denied fallback, and network-loss recovery. These require
the user's speech/listening or a real interruption; simulator state tests do not
prove them. Follow [DEVICE_TESTING.md](DEVICE_TESTING.md).
3. **Broader live regression:** exercise slow-network cancellation, network-loss
recovery, repeated server replacement, and voice/model selection against the
live agent.
4. **Manual accessibility/visual qualification:** inspect populated/error screens
in light and dark mode, accessibility text sizes, narrow/landscape layouts, and
a real VoiceOver focus session. Astra accepted the remaining presentation debt
for this personal-app release; that is not proof of exhaustive conformance.
5. **Publish the backend compatibility source safely** in the sibling repository
and ensure its normal deployment preserves mounts, certificates, and Qwen
settings. This is lower priority because the iOS app no longer consumes the
media API, but the deployed container still serves it for the web UI.
## Repository handoff
The iOS work and this report belong to branch `feat/ios-app`, remote `origin`
(`https://git.srmr.xyz/snowspeeder/hope-voice-ios.git`). The requested handoff commit
includes the implementation, tests, deployment notes and startup instructions.
Check Git history/remote status for the final commit and push result.
(`https://git.srmr.xyz/snowspeeder/hope-voice-ios.git`). Keep the Nearlight
implementation, tests, physical-device notes, and this handoff together when
committing. Check Git history and remote status for the final publish result.
+74 -65
View File
@@ -1,77 +1,86 @@
# Verification — 2026-09-08
# Verification — 2026-09-12
## Scope and implementation
## Scope
The requested model migration was canceled: Qwen 2.5 remains configured.
The native redesign separates Talk, Music and Activity; simplifies the welcome
and connection controls; supports light/dark appearance and Dynamic Type;
adds recoverable error states; and gives transcript clearing explicit confirmation.
The Impeccable native-iOS guidance informed system controls, hierarchy, readable
spacing and touch targets. Settings, transcripts and task updates are observed
directly so asynchronous changes render promptly.
Qwen 2.5 remains configured; the canceled subscription-login migration was not
revived. The embedded music player was removed by user decision. The current app
is the Astra-designed **Nearlight** native interface with shared Talk and Activity
tabs, adaptive semantic styling, the Lightwell conversation indicator, rebuilt
conversation controls, readable transcripts, actionable failure states, explicit
Settings saves, and revised Activity/detail surfaces.
Music fixes include standalone control requests, current download metadata,
visible failures, pinned artwork, server-change cleanup, cached AAC delivery,
and correct AVFoundation content-type identifiers in the pinned resource loader.
## Current iOS evidence
## Evidence
Project: `Voice.xcodeproj`; scheme: `Voice_iOS`; simulator: iPhone 17 Pro
`AFDA137E-1657-428E-BF01-E5CD7989772C`.
Project: `Voice.xcodeproj`; scheme: `Voice_iOS`; simulator: iPhone 17 Pro.
- Full combined simulator run: **50 passed, 0 failed, 1 live skip** in 104.9
seconds. It includes unit/state tests and five offline UI scenarios.
- The offline UI scenarios cover navigation/offline recovery; persisted voice and
server validation; normalized same-address and changed-address save feedback;
transcript clearing and Activity details; restored-history reading position and
explicit Latest recovery; and accessibility-size navigation.
- A second targeted run after the final VoiceOver eligibility strengthening passed
`testRestoredTranscriptStaysAtTopUntilLatest`: **1 passed, 0 failed**.
- Fresh runtime inspection showed the long restored transcript at its beginning
with the **Latest** affordance visible. The regression tapped Latest and verified
the final row became visible.
- GPT-6 Astra re-read its design specification and prior findings, inspected the
final source, and performed an isolated Settings runtime reproduction. Its
round-four verdict was **PASS**: reading eligibility, ready-session microphone
recovery visibility, and save-confirmation lifetime were resolved; the earlier
selection and pending-Send fixes remained intact.
- Astra's Settings reproduction saved ` https://127.0.0.1:2 ` using isolated
UI-test preferences. Settings stayed open, the field normalized, and the
checkmark plus “Server address saved.” remained visible. The next user edit
cleared both indicators. It did not touch normal preferences or the NAS.
- 68 unit tests passed, including standalone media controls, retry error clearing,
download metadata, transcript persistence/deduplication, session replacement,
and interruption state handling.
- Five offline UI tests passed: navigation/offline recovery, voice persistence and
URL validation, music search failure/station validation, transcript confirmation
and task details, and accessibility-size navigation.
- Two live UI tests passed together against the NAS at 17:56 UTC: real Qwen
response, background task completion, disconnect/reconnect, library playback
with advancing media time, pause, forward seek and stop.
- Backend integration run: 16 passed, 7 skipped, 2 deselected. This is not a claim
that the skipped or slow integration cases were verified.
- AAC conversion regression: one passed, checking original-file preservation,
AAC output, cache reuse and HTTP byte ranges. Deployed range request returned 206.
XcodeBuildMCP result bundles are under
Result bundles under
`~/Library/Developer/XcodeBuildMCP/workspaces/voice-ios-9eb036fdea81/result-bundles/`:
- Unit/offline UI: `test_sim_2026-09-08T11-48-52-164Z_pid87175_79b1a610.xcresult`.
This combined run's two live cases failed after long stalls; its unit/offline
results above passed.
- Latest live pair: `test_sim_2026-09-08T17-56-51-984Z_pid30949_ec504db2.xcresult`.
- Earlier live rerun: `test_sim_2026-09-08T17-46-07-841Z_pid30949_bd804a04.xcresult`
failed agent join and advancing playback. Their intermittent cause has not
been established. The subsequent passing pair did not require a server restart.
- Full run:
`test_sim_2026-09-12T14-03-09-058Z_pid95846_2ca9da7e.xcresult`
- Targeted restored-history run:
`test_sim_2026-09-12T14-05-21-070Z_pid98187_bd3e531a.xcresult`
## Physical-device evidence
iPhone 14 Pro Max `BF93D35E-0853-5470-A125-6EB0DB3F17AB` was paired, built for,
signed with personal team `YJFYRB92NW`, installed, trusted, and launched through
`devicectl`. This proves device signing/install/launch only, not microphone or
speaker behavior. Free provisioning expires after seven days.
The final source also completed a signed generic-iOS build at
`/tmp/voice-ios-device-derived/Build/Products/Debug-iphoneos/Voice.app`. The
device was unavailable to `devicectl`, so this artifact was not installed. The
successful earlier install therefore predates the last Nearlight
reading-safety, microphone-notice, and save-confirmation corrections.
## Historical live and backend evidence
Earlier live NAS runs verified Qwen replies, background task completion,
disconnect/reconnect, and three consecutive Qwen/task/reconnect iterations. Older
music playback evidence is historical only: the player and its tests no longer
exist in the iOS app.
Backend integration previously passed 16 tests with 7 skipped and 2 deselected.
The AAC compatibility regression passed and the deployed byte-range request
returned HTTP 206. That patch remains in a separate backend repository and is not
published by this iOS branch.
## Not yet qualified
Passing simulator tests do not prove audible speaker/headphone output, microphone
capture, barge-in, phone/Siri interruption recovery, background audio, or network
loss recovery on hardware. Follow `DEVICE_TESTING.md` with a physical iPhone on
the LAN. Navigation at large text sizes is tested, not a complete VoiceOver audit.
Live tests use an existing library track; they do not prove new YouTube downloads
or every radio stream format.
- Audible speaker/headphone output, physical microphone capture, barge-in,
phone/Siri interruption recovery, lock/background behavior, and live
network-loss recovery.
- A real microphone-permission-denied session. The visibility/recovery branches
are source-reviewed, but no permission was changed during final verification.
- A full VoiceOver audio/focus session, exhaustive contrast audit, every Dynamic
Type/layout combination, or strict conformance to every Nearlight design detail.
Astra explicitly accepted the remaining visual/adaptive/presentation debt for
this personal release.
- The final corrected binary on the physical iPhone while it remains unavailable.
## Repeatability follow-up — 18:04 UTC
Three consecutive Qwen/task/reconnect iterations passed. Music passed twice and
failed on the third iteration; runtime logs captured AVPlayer's request timeout
while the TLS loader buffered a whole 6 MB response. The loader now delivers
bounded 256 KB ranges to AVPlayer, retaining cancellation and EOF handling.
After that change, **all three consecutive live playback iterations passed**
in about 13 seconds each. This addresses the observed music buffering failure;
the earlier isolated agent-join failure has not been reproduced or explained.
- Reproduction bundle: `test_sim_2026-09-08T18-00-03-993Z_pid30949_49da9d2c.xcresult`.
- Post-fix playback bundle: `test_sim_2026-09-08T18-04-30-014Z_pid30949_df323119.xcresult`.
- Live tests now attach failure screenshots and UI hierarchy, and attempt to stop
test-initiated music during teardown even after an assertion failure.
The initial hardware check found an unavailable iPhone. A later USB check detected
the phone running iOS 26.6.1, explicitly marked **unpaired** by XcodeBuildMCP.
No successful device install or hardware qualification is recorded. See
`PROJECT_STATUS.md` for the current pairing blocker and next steps.
Deployment and rollback are recorded in `DEPLOYMENT.md`. The NAS image includes
the compatibility patch; the sibling backend source still requires its own commit
and safe deployment workflow. See `PROJECT_STATUS.md` for repository handoff scope.
Follow [DEVICE_TESTING.md](DEVICE_TESTING.md) for hardware qualification and
[PROJECT_STATUS.md](PROJECT_STATUS.md) for current priorities and deployment
dependencies.