feat: provision private Hermes assistants for authenticated Hope accounts
This commit is contained in:
1 parent
da65da68b3
commit
2aedea84b5
33 files changed
+3063
-1743
No files matched your search
+17
-32
@@ -1,39 +1,24 @@
|
||||
# Hope test suite
|
||||
# Hope multi-user qualification
|
||||
|
||||
End-to-end tests that drive the **actual running container** — no mocks.
|
||||
They run on the host and hit the live web UI, token endpoint, LLM API, MCP
|
||||
tool servers, and agent process.
|
||||
## Offline regression suite (no NAS or model calls)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The container is up: `docker compose up -d` (and healthy)
|
||||
- Host deps installed: `pip install -r tests/requirements.txt`
|
||||
(or `uv pip install -r tests/requirements.txt`)
|
||||
- The LLM at `LLM_BASE_URL` (default `http://192.168.86.2:3005/api`, OpenWebUI) is reachable
|
||||
- Internet access for the weather tool tests (wttr.in)
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
cd ~/dev/hope-voice-api
|
||||
python -m pytest tests/ -v # everything
|
||||
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
||||
```sh
|
||||
python -m venv .venv-tests
|
||||
.venv-tests/bin/python -m pip install pytest fastapi httpx
|
||||
.venv-tests/bin/python -m pytest --noconftest tests/unit/test_accounts.py tests/unit/test_provisioner.py tests/unit/test_token_bootstrap.py -q
|
||||
agent/.venv/bin/python -m unittest discover -s tests/unit -p test_hermes_llm.py -v
|
||||
```
|
||||
|
||||
## What each file covers
|
||||
Covers authorization, revocation, CSRF, lifecycle/restart failures, Docker ownership, authenticated server-derived room routing, SSE framing, content-only output, cancellation, redacted errors, and wait/final-answer callbacks.
|
||||
|
||||
| File | Target | Notes |
|
||||
|------|--------|-------|
|
||||
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
||||
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
||||
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
||||
| `test_media.py` | external Hope Music boundary + media MCP client | LAN service health, stable room scope, split regression guards, local-container MCP probes |
|
||||
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
||||
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
||||
## Legacy tests
|
||||
|
||||
## Notes
|
||||
Top-level `test_*.py` and `conftest.py` describe the former OpenWebUI/MCP architecture and default to live Unraid. **Do not run them as the migrated regression suite.** Retained only as historical references: weather/dispatch/local memory and anonymous-token assumptions no longer apply. Hope Music remains an independent deployment.
|
||||
|
||||
- LLM and MCP tests are marked `@pytest.mark.slow` (1–30s each).
|
||||
- The token endpoint is **POST** `/token` → `{"token": "<JWT>"}`.
|
||||
- Process checks use `/proc/*/cmdline` because the container image has no
|
||||
`ps`, and the supervisorctl unix socket is not exposed in this build.
|
||||
## Live qualification
|
||||
|
||||
Explicitly opt in with administrator-created test accounts. Never put passwords or bearer tokens in test source/argv. Confirm anonymous denial, browser login/admin creation, ready private containers, direct Hermes content, tool progress and final reply, normal-user administration denial, account deletion/container removal/session revocation, logout room revocation, and account-scoped history isolation.
|
||||
|
||||
API/UI/stream evidence is not physical microphone or iPhone audio/interruption qualification. Recorded results and limitations are in `QUALIFICATION.md`.
|
||||
|
||||
Keep credentials in protected ignored runtime files; never print response tokens or Docker environment contents.
|
||||
Reference in new issue
Block a user