Document engineering learnings and refresh handoff guidance

This commit is contained in:
snowspeeder committed 2026-09-19 14:40:24 -04:00
1 parent 85169d651c
commit 9bdaea2408
7 files changed
+135 -10

No files matched your search

+8 -2
View File
@@ -1,5 +1,9 @@
# NAS deployment — 2026-09-08
Historical deployment record, not a current health assertion. The iOS music player
was removed September 12; the media/AAC details below concern the retained backend,
not a current iOS feature. Reinspect server state before making deployment changes.
The `voice` container on `192.168.86.2` hosts LiveKit, the voice agent,
token service, music API and nginx. The app connects to
`https://192.168.86.2:9178`. Docker uses host networking and an
@@ -34,5 +38,7 @@ containers or the music library; it does not include iOS AAC conversion.
`scripts/deploy-backend.sh` records the launch configuration and refuses to
replace an existing `voice` container. Run it on the NAS only after building
the named image and explicitly handling the existing container. It reads
credentials on the NAS and never prints them. No source changes were pushed;
future CI deployments must include the compatibility patch to preserve it.
credentials on the NAS and never prints them. The backend compatibility source was
not pushed during that deployment; recheck the sibling repository before acting.
An iOS push does not publish it. Future backend CI deployments must include the
compatibility patch to preserve it.
+12 -5
View File
@@ -6,8 +6,15 @@ iPhone on the LAN** (`https://192.168.86.2:9178`) because mic input, WebRTC capt
and audio session behavior don't work in this simulator environment (CoreAudio `-4010`,
0 input channels).
Setup: build with Xcode for a physical device (or `xcodebuild ... -destination`), install,
ensure the server cert pin is current (see README — re-export after any server restart).
Setup: recheck device availability and signing, then build/install with Xcode or
the installed XcodeBuildMCP skill workflow. The phone was paired and the Nearlight
build launched successfully on September 12; that does not prove today's availability
or a still-valid provisioning profile. Verify the server certificate pin after an
actual certificate rotation, not automatically after every container restart.
For each check, record date, source revision, device/iOS version, backend/model,
observed result and supporting logs. Leave unchecked items unqualified; installing
the app or passing a simulator state test is not equivalent to hearing a reply.
## 1. Mic input with real speech
@@ -21,7 +28,7 @@ ensure the server cert pin is current (see README — re-export after any server
- [ ] While the agent is speaking, start talking → conversation continues without a
"barge-in" glitch; state returns to listening/thinking as expected.
- [ ] Text mode: barge-in hint line visible while in text mode.
- [ ] Switching to Type leaves microphone capture off and text messaging usable.
## 3. Wake lock (screen off)
@@ -39,8 +46,8 @@ ensure the server cert pin is current (see README — re-export after any server
## 5. Mic permission denied fallback
- [ ] Deny microphone permission in iOS Settings for the app, relaunch.
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); orange error
banner shows the mic error; text mode remains fully usable.
- [ ] Start → session reaches **ready** despite mic failure (non-fatal); a visible
error explains the problem and offers iPhone Settings recovery; Type remains usable.
## 6. General device sanity
+89
View File
@@ -0,0 +1,89 @@
# Engineering learnings
Updated 2026-09-19 from repository source, recorded test evidence, and prior debugging.
This is guidance, not a claim that hardware or server checks were rerun today.
Read [PROJECT_STATUS.md](PROJECT_STATUS.md) for the current product scope.
## Preserve behavior across UI changes
- Observe nested observable objects directly where their data is rendered. Watching
only the parent session does not guarantee transcript/state/task updates redraw.
- Keep one conversation session across tabs. Server replacement must remain
latest-wins and reject late callbacks from a torn-down session.
- Keep Settings presentation outside the replaced session's identity. Otherwise
saving a changed server can destroy the navigation stack and its success feedback.
- Treat normalization as a programmatic edit: trimming a saved URL must not clear
its own confirmation. Clear confirmation on the next genuine user edit instead.
- A restored transcript is reading history, not a request to jump to the bottom.
User scrolling, text selection, and VoiceOver row focus suspend auto-follow;
explicit **Latest** restores it. Test the reading position, not just row existence.
- Gate Send on transport readiness, reconnect state and pending sends. Retain an
unsent draft on failure. Type mode must not enable microphone capture.
- A ready room can still have a microphone failure. Show the actual error and its
recovery action without disabling the usable text path.
These are regression invariants of the current Nearlight implementation, not a
reason to restore the removed music UI or the canceled ChatGPT migration.
## Match evidence to the claim
- A token/TLS check is not an agent-join test; Docker health is not a Qwen-response
test; app installation is not proof of microphone capture or audible output.
- Offline UI fixtures must use isolated preferences and deterministic local failure
endpoints. Never reset production preferences to make a UI test pass.
- Keep live tests opt-in. Record which model/server was used, what was actually
asserted, skips, result bundles, and any cleanup of test-created state.
- Test repeated lifecycle sequences, not just one successful launch. Repeat runs
exposed an intermittent playback timeout that a single green run missed.
- After an interrupted tool call, inspect the existing process/result before
restarting. A lost tool handle does not imply the underlying test stopped.
- When UI snapshots expose no targets, cross-check screenshots and XCTest results;
this occurred despite a rendered app and was not sufficient evidence of a crash.
- Keep failed-run evidence. A later pass does not explain the earlier failure.
Source review and automated large-text checks also do not replace manual VoiceOver
or real-device speech/interruption testing.
## Device connection and signing
Diagnose the layers separately: USB detection → pairing/trust → developer/signing
readiness → install/launch → live voice behavior. An unlocked phone may still be
unpaired. Once USB detects it, investigate pairing rather than repeatedly blaming
the cable. Query fresh device details; do not reuse an old availability report.
The phone was eventually paired and the final Nearlight build installed/launched
on September 12. That supersedes the earlier Finder/pairing blocker. Recheck its
current availability and provisioning before the next run; the recorded personal
provisioning was short-lived. Do not reset privacy settings or erase a device as a
routine diagnostic step.
## Project generation and assets
`project.yml` is the source for `Voice.xcodeproj`; the scheme is `Voice_iOS`, bundle
ID `dev.shane.voice`. Adding an icon file alone is insufficient: the asset catalog
must also be included in the resource build phase. The September 12 follow-up
commits added the Lightwell icon and regenerated the project to include it.
Regenerate deliberately and review generated diffs alongside source changes.
## NAS deployment and historical audio findings
- Inspect actual containers, images, mounts and health before recreating anything.
A user concern about deletion is a reason to check, not proof of deletion.
- Preserve the persistent certificate. A container restart need not rotate it;
update the app pin only after verifying an actual certificate change.
- Stage clean deployment source separately from a dirty remote checkout. Retain
library/configuration mounts and a recoverable previous container.
- Record the deployed image and source patch. Cross-repository changes must be
published separately, with any push-triggered deployment reviewed first.
- Do not print environment files, API keys, or full credential-bearing configuration.
Historical only—the embedded player no longer exists in this app:
- WebM/Opus library downloads needed cached AAC for the iOS playback path tested.
Preserve originals and verify byte ranges/cache behavior separately from conversion.
- AVFoundation's resource-loader content type required a UTI, not an HTTP MIME type.
- Buffering an entire 6 MB response before returning audio reproduced an AVPlayer
timeout. Bounded 256 KB range delivery passed three subsequent live repetitions.
These findings do not establish support for every radio format or indefinite stream.
See [DEPLOYMENT.md](DEPLOYMENT.md) and [VERIFICATION.md](VERIFICATION.md) for recorded
deployment and test history. Do not reintroduce the client player to preserve history.
+12 -2
View File
@@ -1,6 +1,10 @@
# Project status and handoff
Last updated: 2026-09-12.
Last updated: 2026-09-19 (documentation review; no new runtime qualification).
Reusable implementation, testing, device and deployment lessons are in
[ENGINEERING_LEARNINGS.md](ENGINEERING_LEARNINGS.md). September 12 test/install
results below are historical evidence, not a September 19 health check.
## Current direction
@@ -63,6 +67,10 @@ Hardware audio and interruption qualification remains open.
### Tests and tooling
- Follow-up commits added the Lightwell app icon (`7ac8a1c`) and regenerated the
project to include its asset-catalog resource phase (`85169d6`). The qualification
results below predate those follow-ups; do not describe them as a fresh full run
of the current HEAD.
- Installed XcodeBuildMCP and its skills; used it for simulator build/run/test work.
- The current combined simulator run passed **50 tests, 0 failures**, with the
opt-in live suite skipped once. Five offline UI scenarios cover recovery,
@@ -97,7 +105,9 @@ SSH host: `[email protected]` (Unraid). App URL: `https://192.168.86.2:9178`.
**Important:** the backend patch and its test are in a different local repository:
`/Users/shane/dev/hope-voice-api/web/media_server.py` and
`/Users/shane/dev/hope-voice-api/tests/test_ios_audio.py`. They were still uncommitted
at this handoff. Committing/pushing `voice-ios` does **not** publish those files.
at the earlier deployment handoff; their current sibling-repository status has not
been rechecked in this documentation update. Committing/pushing the iOS repository
does **not** publish those files.
The deployed image contains the patch, but a future backend CI rebuild can lose it
unless it is committed and deployed through that repository's workflow. Inspect
that workflow before pushing: it may replace the live container.
+7 -1
View File
@@ -1,5 +1,9 @@
# Verification — 2026-09-12
Documentation reviewed 2026-09-19; no tests rerun for that update. These results
predate the subsequent app-icon and generated-resource-phase commits. See
[ENGINEERING_LEARNINGS.md](ENGINEERING_LEARNINGS.md) for reusable lessons.
## Scope
Qwen 2.5 remains configured; the canceled subscription-login migration was not
@@ -78,7 +82,9 @@ published by this iOS branch.
Type/layout combination, or strict conformance to every Nearlight design detail.
Astra explicitly accepted the remaining visual/adaptive/presentation debt for
this personal release.
- The final corrected binary on the physical iPhone while it remains unavailable.
- A fresh build/install and regression run after the subsequent app-icon/resource
commits. The September 12 Nearlight install succeeded; current phone availability
and provisioning have not been rechecked here.
Follow [DEVICE_TESTING.md](DEVICE_TESTING.md) for hardware qualification and
[PROJECT_STATUS.md](PROJECT_STATUS.md) for current priorities and deployment