Move the media server, downloader, library volume, and browser player to the
standalone hope-media-music product. Keep Hope's music tools as a thin HTTP
client of the LAN service on port 8094.
Use a stable voice-room player scope instead of ephemeral LiveKit room names.
The agent now owns the conversation handshake: fresh reported playback mutes
its input, joining pauses music, and the last participant leaving resumes only
music it paused. Peer failures remain non-fatal.
Also move the voice deploy workflow onto the active Unraid Gitea runner and
remove yt-dlp/ffmpeg/media health and volume dependencies from the image.
Small local models (qwen2.5-vl-3b, gemma-4-e4b) narrate actions ('Sure,
I'll pause the music') without emitting the tool call once the full
system prompt and 26-tool set are in context. Measured: qwen emitted
tools in 2/8 realistic intents, gemma 5/8; single-tool tests pass 100%,
so this is context-induced drift, not plumbing.
- agent/intent_router.py: deterministic forced-call router for
high-frequency intents — media controls, now_playing/list_library,
'remember that ...' memory saves, explicit YouTube Music searches,
and personal-fact recall (memory_recall/search_notes/skill_recall,
excluded for weather/time questions). Fires only when the tool is
available; guards against how-do-I questions.
- agent.py: generalize _ForcedToolCallStream to any tool; route through
intent_router after the dispatch check. System prompt adds a hard
'call, don't narrate' rule and reconditions 'answer directly'.
- tests/test_intent_router.py: 30 unit tests pinning router behavior.
Verified against the live LLM with the full fidelity harness: combined
forced-or-emitted coverage 8/8 on qwen2.5-vl-3b (was 2/8) and 8/8 on
gemma-4-e4b (was 5/8). Full suite: 79 passed.
- web/media_server.py: FastAPI service owning the permanent library
(library.json + ./media volume), yt-dlp YouTube Music search, queued
download jobs with mp3 transcode (ffmpeg), per-room player desired
state, and an HTTPS proxy for external live-stream URLs.
- agent/media_mcp.py: Hope tools — search_music, play_song,
offer_song_choices, pause/resume/stop/skip/seek, now_playing,
list_library, add_stream.
- agent/agent.py: media toolset wiring + conversation rule (music
playing disables the agent mic; re-engaging pauses music).
- web UI: slide-out Music panel (search, suggestions, library, live
stream add, play/pause, ±10s, seek, volume) with a reconciling poller
and media_state data-channel reports.
- token server returns the minted room name; nginx serves /media/files
as static (native Range) and proxies /media/api to the media server.
- supervisord runs media-server; compose mounts ./media and adds
ffmpeg to the image.
- Remove memory_mcp.py (stdio markdown-file memory)
- Add Cognee MCP as remote Streamable HTTP toolset at 192.168.86.2:8003/mcp
- Filter to only remember/recall/forget tools via allowed_tools
- Update system prompt: Memory section moved to top priority with
explicit 'call recall FIRST' instructions and examples
- Add COGNEE_MCP_URL env var to docker-compose
- Remove /memory volume mount (no longer needed)
- Rewrite memory tests to use Cognee HTTP client fixture
- 28/28 tests passing
- agent/task_registry.py: file-based JSONL event registry (cross-process)
- agent/task_worker.py: autonomous LLM loop with weather/time/memory/web tools
- agent/dispatch_mcp.py: MCP tool exposing dispatch_task to the main agent
- agent/agent.py: registers dispatch toolset, polls task events → room data
- web: slide-out task panel (FAB button + badge), live step streaming via
data channel topic 'tasks', status dots (running/completed/failed)
- Dockerfile: copies new task_*.py and dispatch_mcp.py files
The dispatch MCP runs in its own process; events flow through
/tmp/tasks/events.jsonl which the main agent tails every second and
forwards to the browser. Tasks run up to 10 LLM iterations with tool calls.
Compaction:
- GemmaLLM.chat() truncates ChatContext to last 30 items (~15 turns)
before sending to LLM, preventing context window overflow on long
conversations. Preserves system prompt and removes orphaned tool calls.
Skills:
- agent/skills_mcp.py: MCP server with skill_save, skill_recall,
skill_list, skill_update tools backed by .md files in /skills
- skills/ dir bind-mounted into container, git-trackable
- System prompt instructs Hope to save repeatable procedures as skills
and recall them before performing tasks she's done before
- Distinct from memory (facts) — skills are learned *procedures*
- agent/memory_mcp.py: MCP server with memory_recall, memory_save, memory_list
tools backed by .md files in /memory (simple keyword matching for v1)
- memory/ dir bind-mounted into container, persists across rebuilds,
easily backed up via git
- System prompt instructs Hope to recall on past references and save
personal info/preferences naturally without announcing it
- Dockerfile: copy memory_mcp.py; compose: ./memory:/memory volume
- preemptive_generation was incompatible with MCP tool calls: it starts
generating before the turn finalizes, breaking the tool execution loop
(agent said 'let me check' then hung forever)
- Add get_time(location?) to weather_mcp.py for date/time queries
- Update system prompt with Weather & Time section
- agent/web_mcp.py: stdio MCP server exposing web_search and web_scrape,
backed by the self-hosted Firecrawl stack on xNAS (no API key needed)
- agent.py: Agent now attaches mcp_servers built from config; EXTRA_MCP_SERVERS
env var allows adding arbitrary HTTP/SSE MCP servers as JSON
- Dockerfile: installs livekit-agents[mcp], copies web_mcp.py
- .env.example: WEB_MCP_ENABLED, FIRECRAWL_BASE, EXTRA_MCP_SERVERS documented
- nginx serves the UI over HTTPS on 8090 with a self-signed cert
(browsers require a secure context for microphone access)
- added /token endpoint (tiny Python HTTP server) that signs LiveKit
JWTs server-side, keeping the API secret out of the browser
- app.js now fetches a signed token from /token and uses wss:// when
the page is served over HTTPS
- supervisord runs the token-server as a fourth process
snowspeeder
Shane