hope-voice-api/IMPLEMENTATION_PLAN.md

Multi-user Hope Voice + per-user Hermes

Objective

Authenticated users talk to their own Hermes agent. Hope owns speech, transport, accounts and progress presentation; Hermes owns reasoning, tools, memory and task execution. The upstream is the supplied local OpenAI Responses endpoint at http://192.168.86.5:2099/v1, model auto (verify actual served model rather than assume a spelling/model ID), with thinking disabled. Credentials are runtime secrets only.

Architecture

• Browser/iOS → authenticated Hope account API → account-scoped LiveKit token.
• Browser/iOS → LiveKit → Azure STT → per-user Hermes API → Azure TTS.
• Dedicated trusted account/provisioning control plane on Unraid owns SQLite account/session records and manages only Docker containers labelled as Hope-managed. The speech container and Hermes containers never receive the Docker socket.
• Each account has a deterministic hope-hermes-<slug>-<id> container, independent private persistent home/workspace, independent API credential, restricted runtime privileges/resource limits, and loopback-only API publication.
• Account IDs and credentials come from trusted server-side records, never browser-selected room names or URLs.

Implementation sequence

1. Verify Hermes's upstream Responses transport, thinking controls, container image/launch settings, streaming progress, session identity and cancellation contracts. Pin the verified Hermes revision/image.
2. Implement persisted accounts with password hashing, administrator/user roles, expiring revocable sessions, login/logout, authenticated /token, admin create/list/reset/delete, login throttling and origin/CSRF controls. No anonymous default-user fallback.
3. Implement durable provisioning states and reconciliation: creating → provisioning → ready/error; deleting → container removed → account deleted. Recover after restart; idempotent retries; verify ownership labels before removal. Preserve user data separately by default, revoke login/session access immediately on deletion, stop active conversations. Never delete existing shared deployment data.
4. Replace agent-side weather/dispatch/skills/memory execution with an asynchronous Hermes bridge. Preserve STT/TTS, safe interruptions, voice/text mode, transcripts. Keep per-room conversation identifiers inside each user's Hermes. Do not emit hidden reasoning or expose provider keys. Provide a visible progress event and a single spoken wait notice when tool execution/latency signals longer work; deliver the eventual answer and explicit errors.
5. Add browser login, account identity, logout, admin account management and provisioning status. Remove client model selection and obsolete local task controls; replace activity presentation with Hermes progress.
6. Add iOS login/logout, Keychain credentials, authenticated token requests, user-scoped history, Hermes progress and server-managed-model presentation. Qualify secure signaling compatibility rather than silently claim that HTTPS bootstrap protects plaintext signaling.
7. Add offline behavioral tests for authorization, isolation, provisioning/deletion/restart failures, SSE parsing, cancellation, delayed progress and final responses, plus client tests.
8. Build/deploy from a clean staged source, preserving existing dirty Unraid checkout, certificate mounts, music and unrelated containers. Retain rollback configuration/image. Update CI to deploy the control plane and persistent account state without embedded credentials.
9. Create John Doe via the real administrative API/UI. Verify its named running Hermes container, authenticated login/token, upstream non-thinking behavior, real reply and long-task completion. Verify a separate disposable account's create/delete lifecycle and denial of cross-account access. Document exact receipts and remaining physical-device qualifications.

Acceptance criteria

• Anonymous callers cannot mint LiveKit tokens; ordinary users cannot administer accounts or select another user's Hermes.
• John Doe has a running named dedicated container on Unraid and private persistent state.
• Creating/deleting an account through administration automatically creates/removes its owned container; operations survive retries/restarts.
• Hermes receives real upstream Responses requests configured with thinking disabled; Hope has no duplicate reasoning/tool execution loop.
• A slow/task response provides understandable wait feedback and ultimately produces an answer or explicit failure.
• Tests, live deployment checks and hardware-only limitations are clearly distinguished.

Safety/migration decisions

• Preserve existing Shane profile/memory and current music deployment; do not copy one user's private Hermes home into another's.
• Bootstrap administrator/test credentials are generated and stored in ignored protected runtime files, not commits or logs.
• User deletion removes the container and revokes access; retained private data is not recursively purged without an explicit, separately guarded erase action.
• No push until automation is updated: current backend pushes automatically replace the running voice container.
• Physical microphone/TTS listening and iPhone interruption qualification may require the user's participation; API/UI evidence does not substitute for it.