fix: ChatContext is not a sequence — use .items for len/list; add test suite

Bug fix:
- GemmaLLM.chat(): len(chat_ctx) → len(chat_ctx.items)
- GemmaLLM._compact_context(): list(chat_ctx) → list(chat_ctx.items)
  ChatContext in livekit-agents 1.7 is not iterable or sized directly;
  it exposes an .items list. The TypeError was silently killing all LLM
  responses (agent heard user but never spoke back).

Test suite (tests/):
- test_web_ui.py: static assets, token endpoint, LiveKit WS proxy
- test_llm_api.py: completion, tool calling, thinking-disabled latency, streaming
- test_mcp_tools.py: weather (Fahrenheit), time, memory, skills — all via
  docker exec + in-container MCP client
- test_agent_integration.py: process alive, worker registered, supervisord,
  container health, LiveKit API
- test_compaction.py: size-triggered compaction, system prompt preservation,
  short-context no-op
- conftest.py: shared fixtures (HTTPS client, token, LLM, docker exec helpers)
- Run with: .venv-tests/bin/python -m pytest tests/ -v
This commit is contained in:
Shane
2026-08-22 17:09:29 -04:00
parent 938629df83
commit a1d59580f7
10 changed files with 658 additions and 4 deletions
+38
View File
@@ -0,0 +1,38 @@
# Hope test suite
End-to-end tests that drive the **actual running container** — no mocks.
They run on the host and hit the live web UI, token endpoint, LLM API, MCP
tool servers, and agent process.
## Prerequisites
- The container is up: `docker compose up -d` (and healthy)
- Host deps installed: `pip install -r tests/requirements.txt`
(or `uv pip install -r tests/requirements.txt`)
- The LLM at `GEMMA_BASE_URL` (default `http://192.168.86.2:8023/v1`) is reachable
- Internet access for the weather tool tests (wttr.in)
## Run
```bash
cd /home/shane/dev/voice
python -m pytest tests/ -v # everything
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
```
## What each file covers
| File | Target | Notes |
|------|--------|-------|
| `test_web_ui.py` | nginx HTTPS UI (:8090) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
| `test_llm_api.py` | Gemma LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather (Fahrenheit check), time, memory save/recall/list, skills save/recall/list — probe files are cleaned up |
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
| `test_compaction.py` | `GemmaLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
## Notes
- LLM and MCP tests are marked `@pytest.mark.slow` (130s each).
- The token endpoint is **POST** `/token``{"token": "<JWT>"}`.
- Process checks use `/proc/*/cmdline` because the container image has no
`ps`, and the supervisorctl unix socket is not exposed in this build.