Hope often starts an already-downloaded song from a voice command, with no
click on the player. Two things could leave it paused at 0:
- The <audio> element sits outside the LiveKit room's audio-unlock flow, so
autoplay policy rejected the poll-driven play(). Capture the first page
gesture (pointerdown/keydown/touchstart) and play the current src inside it
to whitelist the element for autoplay.
- Player state is scoped per LiveKit room, but mediaScope() could return a
stale shell room after a reconnect. Derive it from the connected room.name
so the browser polls exactly the room the agent joined, and a "play X"
request can never land in a scope the browser isn't watching.
Add an in-container regression test (test_media_play_resolves_cached_ready_track)
pinning the server-side resolve: an already-ready track plays without a
re-download and serves a stable audio_url.
The LED wall square switches to the track's cover art whenever a song is
loaded outside a conversation, and returns to the pulsing LED wall while
speaking to the agent (music and conversation are mutually exclusive, so
starting a talk hides the art and ending it restores it).
- Swap the CSS orb for a 21x21 LED dot matrix on a glass panel, matching
the livekit.com LED-array hero visual.
- Pulse driven by the agent's rendered audio: an expanding radial bloom
brightens and widens with remote-audio level, dimmer when listening on
the mic, faint static matrix when idle, and a subtle warm breathing
state while thinking.
- State-tinted palettes: cool azure when speaking, blue when listening,
amber when thinking, dim slate when idle.
- Drop the unused orb CSS-variable pulse plumbing and the presence-breathe
/ orbit-turn keyframes; the wall renders per-frame from the analyser.
- The play/pause button never swapped icons: app.js set the .hidden
property on <svg> elements, which does not reflect to the hidden
attribute (that IDL attribute is on HTMLElement, not SVGElement), so the
CSS [hidden] rule never applied and both glyphs overlapped. Toggle a
.playing class on the button and show/hide the icons via CSS instead.
- The Start conversation and mute buttons were pushed off-screen as the
transcript grew: the workspace grid had no template-rows, so row 1
auto-sized to the tall transcript and stretched the voice-console
beyond the clipped 700px box. Pin the row with grid-template-rows:
minmax(0,1fr) and give both grid columns min-height:0 so the transcript
(and the inner #messages) scroll instead of expanding the layout.
- Stream downloads as they grow: raw best-audio (no mp3 transcode) served
chunked from the stream endpoint mid-download, then with native Range
once complete. Songs play within seconds of hitting Play.
- Fix searched songs never starting: reconcileMedia now wires the stream
URL when a track that was preparing gains one; audio_url is stable for
the whole lifecycle so the player never reloads mid-playback.
- Progress bars now parse yt-dlp stderr (stdout never carried [download]).
- Prefer downloaded songs: /media/api/search merges ready library matches
ahead of YouTube results; search_music/play_song tooling and the agent
prompt tell Hope to always pick in_library=true.
- Mute button turns solid red while the mic is muted.
The Unraid deployment's /media/api requests were answered by an
Apache/YOURLS stack squatting on 8092, surfacing as UI 503s; the CI
healthcheck (curl without -f) counted that 503 as healthy, so run #15
deployed green against a broken media server.
- media server port 8092 -> 8093 everywhere (server, MCP, agent, nginx,
compose, CI, .env.example, docs, tests).
- healthchecks use curl -f in compose and rebuild.yml so HTTP error
statuses fail deployment.
- tests default to the Unraid deployment (VOICE_WEB_URL=
https://192.168.86.2:9178); token goes through nginx (loopback :8091
is unreachable off-host); bind-mount-coupled tests skip on remote
targets. AGENTS.md documents the convention and both failure modes.
- yt-dlp stderr now goes to a temp file so a chatty stderr can never
deadlock the stdout progress loop.
- report/search/seek numeric parsing returns 400 instead of 500.
- now-playing card reflects failed/preparing download states.
- media runtime data (library.json, jobs/) is gitignored; only
.gitkeep is tracked.
Document the media server process, /media/ port map, library layout on
the ./media bind mount, Hope's media MCP toolset, the music/mic
mutual-exclusion rule, and yt-dlp download/proxy conventions.
- web/media_server.py: FastAPI service owning the permanent library
(library.json + ./media volume), yt-dlp YouTube Music search, queued
download jobs with mp3 transcode (ffmpeg), per-room player desired
state, and an HTTPS proxy for external live-stream URLs.
- agent/media_mcp.py: Hope tools — search_music, play_song,
offer_song_choices, pause/resume/stop/skip/seek, now_playing,
list_library, add_stream.
- agent/agent.py: media toolset wiring + conversation rule (music
playing disables the agent mic; re-engaging pauses music).
- web UI: slide-out Music panel (search, suggestions, library, live
stream add, play/pause, ±10s, seek, volume) with a reconciling poller
and media_state data-channel reports.
- token server returns the minted room name; nginx serves /media/files
as static (native Range) and proxies /media/api to the media server.
- supervisord runs media-server; compose mounts ./media and adds
ffmpeg to the image.
- Move task_dir assignment before VoiceAssistant construction in
handle_job; UnboundLocalError was crashing every agent job before it
joined the room, leaving the UI stuck on 'Opening a secure room'
- Make settings sheet full-window height (100dvh) to match tasks panel
- Point the agent at OpenWebUI's OpenAI-compatible API (LLM_BASE_URL,
LLM_API_KEY, LLM_MODEL) with GEMMA_* env fallbacks
- Add list_available_models() fetching {base_url}/models, filtering
non-chat ids (embedding/reranker)
- Rename GemmaLLM -> HopeLLM with update_options(model=...) for runtime
model switching (mutates _opts.model) and instance-based compaction
- Data channel: agent publishes {type: models} after connect and on
get_models; set_model switches live and acks model_changed
- Web UI: Brain section in the settings sheet with a model picker that
persists the choice in localStorage
- Update tests, CI workflow env, and docs for the new endpoint
- token_server: generate voice-{uuid8} room name per request so each
browser session creates a fresh room and triggers agent dispatch
- token claim: add jobType JT_ROOM to roomConfig.agents
- livekit.yaml: revert empty_timeout to default (300s)
- agent.py: system prompt now mandates dispatch_task for all non-trivial
tasks (not just research); removed inline web_search instructions;
added tool_call logging in _ReasoningFallbackWrapper
- agent/task_registry.py: file-based JSONL event registry (cross-process)
- agent/task_worker.py: autonomous LLM loop with weather/time/memory/web tools
- agent/dispatch_mcp.py: MCP tool exposing dispatch_task to the main agent
- agent/agent.py: registers dispatch toolset, polls task events → room data
- web: slide-out task panel (FAB button + badge), live step streaming via
data channel topic 'tasks', status dots (running/completed/failed)
- Dockerfile: copies new task_*.py and dispatch_mcp.py files
The dispatch MCP runs in its own process; events flow through
/tmp/tasks/events.jsonl which the main agent tails every second and
forwards to the browser. Tasks run up to 10 LLM iterations with tool calls.
- lk.transcription handler now uses addPartial() (in-place update)
instead of addMessage() (new row per word)
- addMessage() clears stale partials before rendering the final
- agent/web_mcp.py: stdio MCP server exposing web_search and web_scrape,
backed by the self-hosted Firecrawl stack on xNAS (no API key needed)
- agent.py: Agent now attaches mcp_servers built from config; EXTRA_MCP_SERVERS
env var allows adding arbitrary HTTP/SSE MCP servers as JSON
- Dockerfile: installs livekit-agents[mcp], copies web_mcp.py
- .env.example: WEB_MCP_ENABLED, FIRECRAWL_BASE, EXTRA_MCP_SERVERS documented
- nginx serves the UI over HTTPS on 8090 with a self-signed cert
(browsers require a secure context for microphone access)
- added /token endpoint (tiny Python HTTP server) that signs LiveKit
JWTs server-side, keeping the API secret out of the browser
- app.js now fetches a signed token from /token and uses wss:// when
the page is served over HTTPS
- supervisord runs the token-server as a fourth process
snowspeeder
Shane