refactor: split music streaming into Hope Music
Rebuild & Deploy / rebuild (push) Successful in 2m7s

Move the media server, downloader, library volume, and browser player to the
standalone hope-media-music product. Keep Hope's music tools as a thin HTTP
client of the LAN service on port 8094.

Use a stable voice-room player scope instead of ephemeral LiveKit room names.
The agent now owns the conversation handshake: fresh reported playback mutes
its input, joining pauses music, and the last participant leaving resumes only
music it paused. Peer failures remain non-fatal.

Also move the voice deploy workflow onto the active Unraid Gitea runner and
remove yt-dlp/ffmpeg/media health and volume dependencies from the image.
This commit is contained in:
snowspeeder authored and Shane Maynard committed 2026-09-12 06:35:15 -04:00
1 parent d4ea4fc984
commit 436b3fc8b1
23 files changed
+349 -2511

No files matched your search

+10 -12
View File
@@ -29,18 +29,16 @@ FIRECRAWL_BASE=http://192.168.86.2:3002
# Per-user memory profiles and Bearer tokens live in config/users.json. # Per-user memory profiles and Bearer tokens live in config/users.json.
VOICE_DEFAULT_USER=shane VOICE_DEFAULT_USER=shane
# ── Media player (music library + YouTube Music downloads) ───────────────── # ── Hope Music (separate LAN service) ───────────────────────────────────────
# The permanent library lives in ./media (bind-mounted to /data/media): # The voice agent keeps the music MCP tools, but the library, downloads,
# media/files/ transcoded mp3 tracks # streaming, and browser player live in the hope-media-music product.
# media/thumbs/ cover art MUSIC_BASE_URL=http://192.168.86.2:8094
# media/jobs/ download job status # Stable player scope shared with the standalone Hope Music page. Do not use
# MEDIA_SERVER_PORT=8093 (8092 is taken by YOURLS/Apache on xNAS) # the ephemeral LiveKit room name minted for each conversation.
# MUSIC_ROOM=voice-room
# Optional: export cookies.txt from a browser logged into YouTube and drop it # Player reports are refreshed every few seconds. Older "playing" reports are
# in ./media/cookies.txt to avoid "confirm you're not a bot" download blocks. # treated as stale so a closed player cannot leave Hope's mic disabled.
# Anonymous access works from most residential IPs; cookies are the reliable MUSIC_REPORT_TTL=30
# path.
# YTDLP_COOKIES=/data/media/cookies.txt
# Extra MCP servers (optional). JSON list of {url, transport} objects. # Extra MCP servers (optional). JSON list of {url, transport} objects.
# transport: "sse" | "streamable_http" | omit for auto-detect # transport: "sse" | "streamable_http" | omit for auto-detect
+4 -6
View File
@@ -8,7 +8,7 @@ on:
jobs: jobs:
rebuild: rebuild:
runs-on: runner-voice runs-on: unraid
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -26,7 +26,7 @@ jobs:
--name voice \ --name voice \
--network host \ --network host \
--restart unless-stopped \ --restart unless-stopped \
--health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null' \ --health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null' \
--health-interval=15s \ --health-interval=15s \
--health-timeout=5s \ --health-timeout=5s \
--health-retries=3 \ --health-retries=3 \
@@ -34,7 +34,6 @@ jobs:
-v /mnt/user/dev/hope-voice-api/certs:/etc/voice/certs \ -v /mnt/user/dev/hope-voice-api/certs:/etc/voice/certs \
-v /mnt/user/dev/hope-voice-api/skills:/skills \ -v /mnt/user/dev/hope-voice-api/skills:/skills \
-v /mnt/user/dev/hope-voice-api/livekit.yaml:/etc/livekit.yaml:ro \ -v /mnt/user/dev/hope-voice-api/livekit.yaml:/etc/livekit.yaml:ro \
-v /mnt/user/dev/hope-voice-api/media:/data/media \
-e LIVEKIT_URL=http://localhost:7880 \ -e LIVEKIT_URL=http://localhost:7880 \
-e LIVEKIT_API_KEY=devkey \ -e LIVEKIT_API_KEY=devkey \
-e LIVEKIT_API_SECRET=hopevoice-devsecret-2026-lan-only \ -e LIVEKIT_API_SECRET=hopevoice-devsecret-2026-lan-only \
@@ -49,9 +48,8 @@ jobs:
-e WEB_PORT=9178 \ -e WEB_PORT=9178 \
-e VOICE_USERS_FILE=/etc/hope/users.json \ -e VOICE_USERS_FILE=/etc/hope/users.json \
-e VOICE_DEFAULT_USER=shane \ -e VOICE_DEFAULT_USER=shane \
-e MEDIA_DIR=/data/media \ -e MUSIC_BASE_URL=http://192.168.86.2:8094 \
-e MEDIA_SERVER_PORT=8093 \ -e MUSIC_ROOM=voice-room \
-e YTDLP_COOKIES=/data/media/cookies.txt \
hope-voice-api-voice:latest hope-voice-api-voice:latest
- name: Verify deployment - name: Verify deployment
-4
View File
@@ -8,7 +8,3 @@ certs/
.venv-tests/ .venv-tests/
__pycache__/ __pycache__/
.pytest_cache/ .pytest_cache/
# Permanent music library (media server runtime data)
/media/*
!/media/.gitkeep
+57 -75
View File
@@ -9,16 +9,13 @@ One Docker container runs four processes via supervisord:
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media) 1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS 2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
3. **Web frontend** — static HTML served by nginx (port 9178 by default) 3. **Web frontend** — static HTML served by nginx (port 9178 by default)
4. **Media server** — FastAPI music library/player coordinator (port 8093 internal, proxied at /media/api/) 4. **Token server** — authenticated LiveKit bootstrap endpoint (port 8091 internal, proxied at `/token`)
``` ```
Browser ──WebRTC──► LiveKit Server ──audio──► Agent Browser ──WebRTC──► LiveKit Server ──audio──► Agent
▲ │ ▲ │
└────────── audio ◄────────────────────────────┘ └────────── audio ◄────────────────────────────┘
Azure STT → LLM (OpenWebUI) → Azure TTS Azure STT → LLM (OpenWebUI) → Azure TTS
Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media
└────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘
``` ```
## Quick Start ## Quick Start
@@ -41,10 +38,10 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp`
| Port | Protocol | Service | Access | | Port | Protocol | Service | Access |
|-------|----------|----------------------|--------------| |-------|----------|----------------------|--------------|
| 8091 | TCP | Token server | internal only (proxied via nginx at /token) |
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) | | 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN | | 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
| 7882 | UDP | LiveKit RTC media (muxed) | LAN | | 7882 | UDP | LiveKit RTC media (muxed) | LAN |
| 8093 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) |
| 9178 | TCP | Web frontend (HTTPS) | LAN | | 9178 | TCP | Web frontend (HTTPS) | LAN |
## Configuration ## Configuration
@@ -58,36 +55,32 @@ Key variables:
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI) - `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI) - `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`) - `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads - `MUSIC_BASE_URL` — standalone Hope Music LAN API (default `http://192.168.86.2:8094`)
- `MUSIC_ROOM` — stable player scope shared with Hope Music (default `voice-room`)
## Media Player ## Hope Music Integration
Hope can play music on the web UI's media player: YouTube Music search → Music is a separate product and deployment: `snowspeeder/hope-media-music`. It
yt-dlp raw best-audio download (webm/m4a, no transcode) → progressive owns the library, YouTube Music search/download pipeline, stream proxy, and
streaming from the media server while the file grows → finished file served the standalone browser player at `https://192.168.86.2:9179`. This repo keeps
with native HTTP Range for seeking. Songs start playing within seconds of the only the agent-side control client.
download starting and remain in the library for instant offline replay.
Live audio stream URLs (internet radio) are added to the library and proxied
over HTTPS by the media server (mixed-content safe). Search results put ready
library matches first so Hope prefers a downloaded copy over a fresh lookup.
- The permanent library lives in `./media` (bind-mounted to `/data/media`): - Hope's tools live in `agent/media_mcp.py` (MCP, id `media`):
`files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
- `web/media_server.py` owns the library, search, download jobs (one at a
time), and per-room player desired state. The browser is the actual player:
it polls `GET /media/api/state?room=…`, reconciles, and reports real
playback back via `POST /media/api/report`.
- Hope's tools live in `agent/media_mcp.py` (MCP, id "media"):
`search_music`, `play_song`, `offer_song_choices`, `pause_music`, `search_music`, `play_song`, `offer_song_choices`, `pause_music`,
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`, `resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
`list_library`, `add_stream`. The MCP process is spawned per session with `list_library`, `add_stream`. They call `MUSIC_BASE_URL` over HTTP.
`MEDIA_ROOM` set to the LiveKit room name, scoping player state per room. - Music uses the stable `MUSIC_ROOM` scope (`voice-room` by default), not the
- **Music and conversation are mutually exclusive.** When a track plays, the ephemeral LiveKit room minted for each conversation. The standalone player
UI reports `{type: "media_state", state: "playing"}` on data-channel topic defaults to that same scope, so voice commands control the open music page
"media-control" and the agent disables its mic (`_apply_listening` in and playback survives across voice sessions.
agent.py). Engaging Hope (Start conversation / text message) pauses the - **Music and conversation remain mutually exclusive.** The agent polls Hope
music; ending the conversation resumes what was playing. Voice commands Music's fresh browser-reported state. Actual playback disables the agent's
(pause/skip/…) work whenever the conversation is active. mic; joining a conversation pauses music, and the last participant leaving
resumes only music that the agent paused.
- A stale `playing` report (default: thirty seconds) is treated as stopped so
closing the music page cannot leave Hope's mic disabled.
- Hope Music is a peer service: an outage logs a warning but never takes down
the voice conversation. Its health and E2E tests belong in its own repo.
## Voice Selection ## Voice Selection
@@ -153,8 +146,7 @@ identical to a failure). Isolated single-tool probes will pass and mislead.
via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit
and the one CI (rebuild.yml) deploys on every push to master. The local dev and the one CI (rebuild.yml) deploys on every push to master. The local dev
compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178` compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178`
to target it instead. Tests that can only pass on the deployment host to target it instead. Tests that probe the local container through
(repo `./media` bind mount) or that probe the local container via
`docker exec voice` skip automatically on remote targets. `docker exec voice` skip automatically on remote targets.
```bash ```bash
@@ -164,8 +156,8 @@ docker compose ps
# Run the suite against the Unraid deployment (default) # Run the suite against the Unraid deployment (default)
python -m pytest tests/ -v -m "not slow" python -m pytest tests/ -v -m "not slow"
# Real end-to-end YouTube Music download (network-heavy, gated) # Verify the external Hope Music boundary + client configuration
HOPE_RUN_YTDLP_DOWNLOAD=1 python -m pytest tests/test_media.py -m slow python -m pytest tests/test_media.py -v
# Check all logs (single service: "voice") # Check all logs (single service: "voice")
docker compose logs -f voice docker compose logs -f voice
@@ -184,7 +176,7 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
## File Layout ## File Layout
``` ```
~/dev/voice/ ~/dev/hope-voice-api/
├── AGENTS.md ← you are here ├── AGENTS.md ← you are here
├── .env.example ← config template (copy to .env) ├── .env.example ← config template (copy to .env)
├── .gitignore ├── .gitignore
@@ -192,24 +184,18 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
├── Dockerfile ← multi-stage build ├── Dockerfile ← multi-stage build
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start ├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
├── livekit.yaml ← LiveKit server config ├── livekit.yaml ← LiveKit server config
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy + /media/ (static + API) ├── nginx.conf ← HTTPS UI + /livekit/ WS proxy
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server, media-server) ├── supervisord.conf ← process manager (livekit, agent, nginx, token-server)
├── certs/ ← persisted self-signed cert (volume mount) ├── certs/ ← persisted self-signed cert (volume mount)
├── media/ ← permanent music library (bind-mounted to /data/media)
│ ├── files/ ← raw audio tracks (webm/m4a; media-server streamed, Range when complete)
│ ├── thumbs/ ← cover art
│ ├── jobs/ ← download job status snapshots
│ └── library.json ← library metadata (permanent)
├── agent/ ├── agent/
│ ├── agent.py ← LiveKit Agents voice pipeline │ ├── agent.py ← LiveKit Agents voice pipeline + Hope Music handshake
│ ├── media_mcp.py ← Hope's music tools (YouTube Music via media server) │ ├── media_mcp.py ← thin HTTP client for the separate Hope Music service
│ ├── pyproject.toml ← Python deps (uv) │ ├── pyproject.toml ← Python deps (uv)
│ └── uv.lock ← locked dependency versions │ └── uv.lock ← locked dependency versions
└── web/ └── web/
├── index.html ← single-page voice UI (+ music panel) ├── index.html ← voice-only single-page UI
├── app.js ← LiveKit client logic (+ player reconciliation) ├── app.js ← LiveKit client logic (no music player)
├── media_server.py ← FastAPI library/search/download/player state ├── token_server.py ← LiveKit token endpoint (mints ephemeral room names)
├── token_server.py ← LiveKit token endpoint (mints room name)
├── manifest.json ← PWA manifest ├── manifest.json ← PWA manifest
├── favicon.svg ← site icon ├── favicon.svg ← site icon
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN) ├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
@@ -218,9 +204,9 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
## Conventions ## Conventions
- **Single container.** All services (LiveKit, agent, web, token endpoint) run in one Docker container via supervisord. No multi-service compose. - **Single voice container.** LiveKit, the agent, nginx, and token endpoint run in one Docker container via supervisord. Hope Music is deliberately a separate product/container.
- **No published UDP ports in compose.** LiveKit binds its media ports directly on the host network (`network_mode: host`). This avoids the docker-proxy process explosion that hit hope-webui. - **No published UDP ports in compose.** LiveKit binds its media ports directly on the host network (`network_mode: host`). This avoids the docker-proxy process explosion that hit hope-webui.
- **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server now pins the room name to "voice-room" server-side and no longer accepts arbitrary room names. - **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server ignores browser-supplied room names and mints `voice-{user}-{uuid8}` server-side.
- **Interruption mode must be "vad".** `interruption={"mode": "adaptive"}` requires the LiveKit Cloud barge-in service (agent-gateway.livekit.cloud) and spams 401 retries on self-hosted setups. - **Interruption mode must be "vad".** `interruption={"mode": "adaptive"}` requires the LiveKit Cloud barge-in service (agent-gateway.livekit.cloud) and spams 401 retries on self-hosted setups.
- **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`. - **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`.
- **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access. - **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access.
@@ -231,31 +217,27 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing. - **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage. - **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files. - **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
- **Music downloads are raw audio, streamed as they grow.** yt-dlp fetches - **Hope Music stays external.** `hope-media-music` owns yt-dlp, ffmpeg, the
bestaudio directly (`-f bestaudio/best`, no `-x`/transcode) into library volume, streaming endpoints, and the player UI. Do not add those
`/data/media/files/{video_id}.{ext}`; the stream endpoint dependencies or files back to this image. This repo owns only the MCP/HTTP
(`GET /media/api/stream/{id}`) serves the growing `.part` file chunked client and the conversation/music handshake.
(no Content-Length) while the download runs, then serves the finished file - **Music scope is stable.** LiveKit room names are intentionally ephemeral;
with native Range for seeking. `audio_url` is that endpoint for the whole `MUSIC_ROOM` defaults to `voice-room`, which is also the standalone player's
lifecycle — a stable URL means the browser never reloads mid-playback. The default. Never derive the music scope from `ctx.room.name`.
mp3 transcode was dropped: it delayed playback by the whole download+encode, - **Playback truth comes from fresh reports.** The standalone player reports
and modern browsers play webm/opus and m4a everywhere. Never point the real state every few seconds. The agent polls it and accepts `playing` only
player at YouTube directly. while the report timestamp is fresh (`MUSIC_REPORT_TTL`, default thirty
- **Search prefers the library.** `/media/api/search` merges ready library seconds), preventing a closed player from silencing the mic forever.
matches (title token/substring match) in front of the yt-dlp results, and - **Peer failure is non-fatal.** Hope Music being unavailable must not stop the
the search_music tool docstring + system prompt tell Hope to always pick voice agent or fail the voice container healthcheck. Music calls log and
`in_library=true`. Deterministic preference lives in the server merge, not degrade gracefully.
in prompt hopes. - **Never hard-require optional container env in supervisord.** `%(ENV_X)s`
- **Progress lines come from yt-dlp stderr.** `[download] N%` goes to stderr; expansion aborts the whole config when a variable is missing. Settings with
only `--print-json` metadata goes to stdout. The download worker drains safe in-code defaults (including Hope Music) should inherit the container
stderr in a thread (deadlock-safe) and reads stdout for the JSON — parsing environment instead of being listed there.
progress from stdout silently never matched. - **Healthchecks must use `curl -f`.** Without it, any HTTP response — including
- **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`. a 503 from a squatter — counts as healthy. Voice health covers only LiveKit
- **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`. and its HTTPS UI; Hope Music has its own independent healthcheck.
- **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.
- **Never hard-require container env in supervisord.** `%(ENV_X)s` expansion aborts the whole supervisord config when the var is missing, taking every program down (this silently killed the run-151 deploy). Programs with safe in-code defaults (media server) must inherit the container env instead of listing it.
- **Healthchecks must use `curl -f`.** Without it, any HTTP response — including a 503 from a squatter — counts as healthy. Run #15 deployed "healthy" against a broken media port this way.
- **Port 8092 is taken on xNAS.** An Apache/YOURLS stack occupies 8092 on the Unraid host; the media server uses 8093. Before picking a new internal port, probe the Unraid deployment host, not just the dev box.
## Git ## Git
+4 -8
View File
@@ -24,10 +24,10 @@ ENV PYTHONUNBUFFERED=1
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs) # Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
# Install LiveKit server binary + supervisord + nginx + ffmpeg (yt-dlp audio # Install LiveKit server runtime dependencies, supervisord, and nginx.
# extraction/transcoding for the media player) # Music download/transcode dependencies live in hope-media-music now.
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates supervisor nginx libasound2 iproute2 ffmpeg \ curl ca-certificates supervisor nginx libasound2 iproute2 \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
# Download LiveKit server (latest stable) # Download LiveKit server (latest stable)
@@ -52,7 +52,7 @@ COPY agent/task_worker.py /opt/voice-agent/task_worker.py
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py
# Copy web frontend + token endpoint + media server # Copy the voice frontend + token endpoint. Hope Music is a separate service.
COPY web/index.html /var/www/voice/ COPY web/index.html /var/www/voice/
COPY web/app.js /var/www/voice/ COPY web/app.js /var/www/voice/
COPY web/style.css /var/www/voice/ COPY web/style.css /var/www/voice/
@@ -60,10 +60,6 @@ COPY web/livekit-client.umd.js /var/www/voice/
COPY web/manifest.json /var/www/voice/ COPY web/manifest.json /var/www/voice/
COPY web/favicon.svg /var/www/voice/ COPY web/favicon.svg /var/www/voice/
COPY web/token_server.py /opt/voice/token_server.py COPY web/token_server.py /opt/voice/token_server.py
COPY web/media_server.py /opt/voice/media_server.py
# Permanent media library mount point (bind-mounted from ./media)
RUN mkdir -p /data/media/files /data/media/thumbs /data/media/jobs
# Config files # Config files
COPY livekit.yaml /etc/livekit.yaml COPY livekit.yaml /etc/livekit.yaml
+144 -31
View File
@@ -11,6 +11,7 @@ import logging
import os import os
import re import re
import textwrap import textwrap
import time
import uuid import uuid
from dotenv import load_dotenv from dotenv import load_dotenv
@@ -58,6 +59,18 @@ WEB_MCP_ENABLED = os.environ.get("WEB_MCP_ENABLED", "true").lower() in ("1", "tr
FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002") FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills") SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
VOICE_USERS_FILE = os.environ.get("VOICE_USERS_FILE", "/etc/hope/users.json") VOICE_USERS_FILE = os.environ.get("VOICE_USERS_FILE", "/etc/hope/users.json")
# Hope Music is a separate LAN service. The agent is its control surface; the
# standalone Hope Music page is the browser audio player.
MUSIC_BASE_URL = os.environ.get(
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
).rstrip("/")
# This is deliberately stable rather than the short-lived LiveKit room name.
# The Hope Music page defaults to the same scope, so playback survives across
# voice sessions and both products address one virtual player.
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
# The player reports every few seconds while audio is active. A stale playing
# report must never leave Hope's microphone disabled after the page closes.
MUSIC_REPORT_TTL = float(os.environ.get("MUSIC_REPORT_TTL", "30"))
def load_user_profile(user_id: str) -> dict: def load_user_profile(user_id: str) -> dict:
@@ -208,15 +221,15 @@ SYSTEM_PROMPT = textwrap.dedent("""\
stalled); task_status can also be called to check up on a task. Use stalled); task_status can also be called to check up on a task. Use
list_tasks to see every task you have dispatched. list_tasks to see every task you have dispatched.
# Music (media player in the web UI) # Music (standalone Hope Music player)
You can play music on the user's web player. For any music request, call You can control the user's separate Hope Music web player. For any music
search_music first, then: request, call search_music first, then:
- One clearly-right result → play_song with its video_id. Announce it in - One clearly-right result → play_song with its video_id. Announce it in
one short sentence, then stop talking — playback pauses the one short sentence, then stop talking — playback pauses the
conversation until the user comes back to you. conversation until the user comes back to you.
- Ambiguous results → offer_song_choices with the top 2-3 options, name - Ambiguous results → offer_song_choices with the top 2-3 options, name
them briefly, and let the user pick by voice or in the player panel. them briefly, and let the user pick by voice or in Hope Music. Never
Never start playback on a guess. start playback on a guess.
Prefer what is already saved: search_music puts downloaded library Prefer what is already saved: search_music puts downloaded library
matches first (in_library=true) — always pick one of those over a fresh matches first (in_library=true) — always pick one of those over a fresh
YouTube result, and list_library shows the whole saved collection which YouTube result, and list_library shows the whole saved collection which
@@ -693,9 +706,9 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
) )
logger.info("Dispatch MCP toolset enabled (%s)", task_dir) logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
# Media tools talk to the local media server; the room name scopes player # Media tools call the separate hope-media-music service over HTTP. A
# state so concurrent sessions don't fight over one virtual player. # stable scope lets the standalone player and every voice session address
media_server_url = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093") # the same virtual player.
media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py") media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py")
toolsets.append( toolsets.append(
mcp.MCPToolset( mcp.MCPToolset(
@@ -705,14 +718,14 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
args=[media_mcp_script], args=[media_mcp_script],
env={ env={
**os.environ, **os.environ,
"MEDIA_SERVER_URL": media_server_url, "MUSIC_BASE_URL": MUSIC_BASE_URL,
"MEDIA_ROOM": os.path.basename(task_dir.rstrip("/")) or "default", "MUSIC_ROOM": MUSIC_ROOM,
}, },
client_session_timeout_seconds=60, client_session_timeout_seconds=60,
), ),
) )
) )
logger.info("Media MCP toolset enabled (%s)", media_server_url) logger.info("Media MCP enabled (%s, room %s)", MUSIC_BASE_URL, MUSIC_ROOM)
extra = os.environ.get("EXTRA_MCP_SERVERS", "") extra = os.environ.get("EXTRA_MCP_SERVERS", "")
if extra: if extra:
@@ -828,13 +841,12 @@ async def handle_job(ctx: JobContext) -> None:
except Exception as e: # noqa: BLE001 except Exception as e: # noqa: BLE001
logger.warning("failed to publish model list: %s", e) logger.warning("failed to publish model list: %s", e)
# Listen for conversation controls (voice selection and voice/text mode). # Listen for conversation controls (voice selection, voice/text mode, and
# Defined here so it captures this job's session via closure. # model switching). Music state comes from the separate service, not from
# Music and conversation are mutually exclusive by design: while a track # the voice UI's data channel.
# plays the agent's ears are off; re-engaging the conversation pauses the
# music, and ending the conversation lets the user resume it.
session_mode = {"value": "voice"} # UI voice/text switch session_mode = {"value": "voice"} # UI voice/text switch
media_playback = {"value": "stopped"} # UI-reported player state media_playback = {"value": "stopped"} # fresh service-reported state
music_paused_by_agent = {"value": False}
def _apply_listening() -> None: def _apply_listening() -> None:
listening = ( listening = (
@@ -843,9 +855,108 @@ async def handle_job(ctx: JobContext) -> None:
) )
session.input.set_audio_enabled(listening) session.input.set_audio_enabled(listening)
async def _music_request(
method: str, path: str, *, warn: bool = True, **kwargs
) -> dict | None:
import httpx
url = f"{MUSIC_BASE_URL}/media/api{path}"
try:
async with httpx.AsyncClient(timeout=10) as http:
response = await http.request(method, url, **kwargs)
response.raise_for_status()
return response.json()
except Exception as exc: # noqa: BLE001
# Hope Music is a peer service: it being unavailable must never
# take the conversation down with it.
if warn:
logger.warning("Music service %s %s failed: %s", method, path, exc)
return None
async def _music_snapshot(*, warn: bool = True) -> dict | None:
return await _music_request(
"GET", "/state", warn=warn, params={"room": MUSIC_ROOM}
)
def _music_is_playing(snapshot: dict | None) -> bool:
"""Return true only for fresh, browser-reported playback."""
reported = (snapshot or {}).get("reported") or {}
if reported.get("state") != "playing":
return False
try:
age = time.time() - float(reported.get("ts") or 0)
except (TypeError, ValueError):
return False
return 0 <= age < MUSIC_REPORT_TTL
async def _pause_music_for_conversation() -> None:
"""Pause actual playback when the user engages Hope."""
if music_paused_by_agent["value"]:
return
if not _music_is_playing(await _music_snapshot()):
return
result = await _music_request(
"POST", "/control",
json={"room": MUSIC_ROOM, "action": "pause"},
)
if result is not None:
music_paused_by_agent["value"] = True
logger.info("Paused Hope Music for the conversation (%s)", MUSIC_ROOM)
async def _resume_music_after_conversation() -> None:
if not music_paused_by_agent["value"]:
return
result = await _music_request(
"POST", "/control",
json={"room": MUSIC_ROOM, "action": "resume"},
)
if result is not None:
music_paused_by_agent["value"] = False
logger.info("Resumed Hope Music after the conversation (%s)", MUSIC_ROOM)
async def _poll_music_state() -> None:
while True:
try:
await asyncio.sleep(2)
# Jobs deliberately survive page reloads. Do not keep making
# network requests after the room has no human participant.
if not ctx.room.remote_participants:
continue
snapshot = await _music_snapshot(warn=False)
reported = (snapshot or {}).get("reported") or {}
state = reported.get("state") or "stopped"
if state == "playing" and not _music_is_playing(snapshot):
state = "stopped"
if state not in ("playing", "paused", "stopped"):
state = "stopped"
if state != media_playback["value"]:
logger.info("Hope Music playback -> %s", state)
media_playback["value"] = state
_apply_listening()
except asyncio.CancelledError:
raise
except Exception: # noqa: BLE001
logger.exception("Hope Music state poller failed")
def _on_participant_joined(*_args) -> None:
"""A returning participant starts a new conversation handshake."""
asyncio.create_task(_pause_music_for_conversation())
def _on_participant_left(*_args) -> None:
"""When the last human leaves, hand playback back to Hope Music."""
async def _deferred() -> None:
# The event can fire before the participant is removed from the
# room object; re-check the roster after it settles.
await asyncio.sleep(0.5)
if not ctx.room.remote_participants:
await _resume_music_after_conversation()
asyncio.create_task(_deferred())
def _on_room_data(packet) -> None: def _on_room_data(packet) -> None:
topic = getattr(packet, "topic", None) topic = getattr(packet, "topic", None)
if topic and topic not in ("voice-control", "conversation-control", "media-control"): if topic and topic not in ("voice-control", "conversation-control"):
return return
try: try:
msg = json.loads(packet.data.decode("utf-8")) msg = json.loads(packet.data.decode("utf-8"))
@@ -865,17 +976,6 @@ async def handle_job(ctx: JobContext) -> None:
session.output.set_audio_enabled(voice_enabled) session.output.set_audio_enabled(voice_enabled)
_apply_listening() _apply_listening()
return return
if msg.get("type") == "media_state":
state = msg.get("state")
if state not in ("playing", "paused", "stopped"):
return
logger.info("Media playback -> %s (agent listening: %s)",
state, session_mode["value"] == "voice" and state != "playing")
media_playback["value"] = state
_apply_listening()
return
if msg.get("type") == "set_voice": if msg.get("type") == "set_voice":
voice = msg.get("voice", "") voice = msg.get("voice", "")
if not voice: if not voice:
@@ -913,6 +1013,8 @@ async def handle_job(ctx: JobContext) -> None:
logger.warning("Failed to update model: %s", e) logger.warning("Failed to update model: %s", e)
ctx.room.on("data_received", _on_room_data) ctx.room.on("data_received", _on_room_data)
ctx.room.on("participant_connected", _on_participant_joined)
ctx.room.on("participant_disconnected", _on_participant_left)
# Publish user/agent transcripts to the room so the web UI can render them. # Publish user/agent transcripts to the room so the web UI can render them.
async def publish_transcript(role: str, text: str) -> None: async def publish_transcript(role: str, text: str) -> None:
@@ -933,9 +1035,9 @@ async def handle_job(ctx: JobContext) -> None:
role = getattr(msg, "role", None) role = getattr(msg, "role", None)
text = getattr(msg, "text_content", None) text = getattr(msg, "text_content", None)
if role == "user": if role == "user":
asyncio.create_task(publish_transcript("user", text)) asyncio.create_task(publish_transcript("user", text or ""))
elif role == "assistant": elif role == "assistant":
asyncio.create_task(publish_transcript("agent", text)) asyncio.create_task(publish_transcript("agent", text or ""))
@session.on("user_input_transcribed") @session.on("user_input_transcribed")
def _on_user_transcribed(ev) -> None: def _on_user_transcribed(ev) -> None:
@@ -1017,6 +1119,17 @@ async def handle_job(ctx: JobContext) -> None:
) )
await ctx.connect() await ctx.connect()
await _pause_music_for_conversation()
music_poll_task = asyncio.create_task(
_poll_music_state(), name=f"music-poller-{ctx.room.name}"
)
async def _stop_music_poller(*_args) -> None:
music_poll_task.cancel()
await asyncio.gather(music_poll_task, return_exceptions=True)
await _resume_music_after_conversation()
ctx.add_shutdown_callback(_stop_music_poller)
# The participant can appear in the browser slightly before AgentSession is # The participant can appear in the browser slightly before AgentSession is
# ready to accept lk.chat streams. Explicitly signal readiness so the first # ready to accept lk.chat streams. Explicitly signal readiness so the first
# typed message is never raced against session startup. # typed message is never raced against session startup.
+24 -23
View File
@@ -1,18 +1,17 @@
"""Media MCP server — lets Hope find and control music by voice. """Media MCP client — lets Hope control the standalone Hope Music service.
Tools talk to the local media server (web/media_server.py) over HTTP, which Tools call the separate ``hope-media-music`` LAN service over HTTP. That
owns the library, YouTube Music search/download jobs and per-room player service owns the library, YouTube Music search/download jobs, stream proxy,
state. The browser is the actual player: it reconciles against the media and per-room player state. Its standalone browser page is the actual audio
server and reports playback back, so voice controls and the web panel stay player: it reconciles against the service and reports real playback.
in sync.
Per-session environment (set by the agent when spawning this MCP): Environment (set by the voice agent when spawning this MCP):
- MEDIA_SERVER_URL e.g. http://127.0.0.1:8093 - MUSIC_BASE_URL e.g. http://192.168.86.2:8094
- MEDIA_ROOM LiveKit room name this session controls - MUSIC_ROOM stable player scope shared with the Hope Music page
The "stop listening while music plays" rule lives in the agent itself: the The voice agent polls the same service state to keep music and conversation
UI reports media_state over the data channel and the agent disables its mic mutually exclusive; there is no music server or music player in the voice
while a track is playing. container anymore.
""" """
from __future__ import annotations from __future__ import annotations
@@ -22,20 +21,22 @@ import os
import httpx import httpx
from mcp.server.fastmcp import FastMCP from mcp.server.fastmcp import FastMCP
MEDIA_SERVER_URL = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093").rstrip("/") MUSIC_BASE_URL = os.environ.get(
MEDIA_ROOM = os.environ.get("MEDIA_ROOM", "default") "MUSIC_BASE_URL", "http://192.168.86.2:8094"
).rstrip("/")
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
mcp = FastMCP("media") mcp = FastMCP("media")
def _media_get(path: str, params: dict | None = None) -> dict: def _media_get(path: str, params: dict | None = None) -> dict:
resp = httpx.get(f"{MEDIA_SERVER_URL}{path}", params=params, timeout=10) resp = httpx.get(f"{MUSIC_BASE_URL}{path}", params=params, timeout=10)
resp.raise_for_status() resp.raise_for_status()
return resp.json() return resp.json()
def _media_post(path: str, body: dict) -> dict: def _media_post(path: str, body: dict) -> dict:
resp = httpx.post(f"{MEDIA_SERVER_URL}{path}", json=body, timeout=30) resp = httpx.post(f"{MUSIC_BASE_URL}{path}", json=body, timeout=30)
if resp.status_code >= 400: if resp.status_code >= 400:
try: try:
detail = resp.json().get("error", resp.text) detail = resp.json().get("error", resp.text)
@@ -46,7 +47,7 @@ def _media_post(path: str, body: dict) -> dict:
def _control(action: str, **extra) -> str: def _control(action: str, **extra) -> str:
data = _media_post("/media/api/control", {"room": MEDIA_ROOM, "action": action, **extra}) data = _media_post("/media/api/control", {"room": MUSIC_ROOM, "action": action, **extra})
if "error" in data: if "error" in data:
return f"Could not {action} the music: {data['error']}" return f"Could not {action} the music: {data['error']}"
track = data.get("track") or {} track = data.get("track") or {}
@@ -87,14 +88,14 @@ def search_music(query: str, limit: int = 5) -> str:
@mcp.tool() @mcp.tool()
def play_song(video_id: str, title: str = "", channel: str = "") -> str: def play_song(video_id: str, title: str = "", channel: str = "") -> str:
"""Play a song in the user's web player. Pass the video_id from """Play a song in the user's standalone Hope Music player. Pass the video_id from
search_music — for an in_library result this plays the saved copy with no search_music — for an in_library result this plays the saved copy with no
download; a new song downloads and starts playing automatically while it download; a new song downloads and starts playing automatically while it
arrives. After calling this, tell the user what is playing in one short arrives. After calling this, tell the user what is playing in one short
sentence and stop talking — the player takes over and the conversation sentence and stop talking — the player takes over and the conversation
pauses until they return.""" pauses until they return."""
data = _media_post("/media/api/play", { data = _media_post("/media/api/play", {
"room": MEDIA_ROOM, "video_id": video_id, "title": title, "channel": channel, "room": MUSIC_ROOM, "video_id": video_id, "title": title, "channel": channel,
}) })
if "error" in data: if "error" in data:
return f"Could not start playback: {data['error']}" return f"Could not start playback: {data['error']}"
@@ -114,12 +115,12 @@ def play_song(video_id: str, title: str = "", channel: str = "") -> str:
def offer_song_choices(choices: list[dict]) -> str: def offer_song_choices(choices: list[dict]) -> str:
"""Offer the user 2-4 song options when a search is ambiguous. Pass a list """Offer the user 2-4 song options when a search is ambiguous. Pass a list
like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}]. like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}].
The choices appear as clickable cards in the music panel; the user can The choices appear as clickable cards in Hope Music; the user can
also just say which one they want. Briefly name the options aloud, then also just say which one they want. Briefly name the options aloud, then
wait for their pick.""" wait for their pick."""
if not choices: if not choices:
return "No choices provided." return "No choices provided."
data = _media_post("/media/api/choices", {"room": MEDIA_ROOM, "choices": choices}) data = _media_post("/media/api/choices", {"room": MUSIC_ROOM, "choices": choices})
if "error" in data: if "error" in data:
return f"Could not present choices: {data['error']}" return f"Could not present choices: {data['error']}"
names = [ names = [
@@ -162,7 +163,7 @@ def seek_music(delta_seconds: int) -> str:
jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips
20 seconds forward).""" 20 seconds forward)."""
try: try:
reported = (_media_get("/media/api/state", {"room": MEDIA_ROOM}).get("reported") or {}) reported = (_media_get("/media/api/state", {"room": MUSIC_ROOM}).get("reported") or {})
position = float(reported.get("position") or 0.0) position = float(reported.get("position") or 0.0)
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
position = 0.0 position = 0.0
@@ -176,7 +177,7 @@ def now_playing() -> str:
its playback position. Use when the user asks what's playing or what this its playback position. Use when the user asks what's playing or what this
song is.""" song is."""
try: try:
st = _media_get("/media/api/state", {"room": MEDIA_ROOM}) st = _media_get("/media/api/state", {"room": MUSIC_ROOM})
except Exception as exc: # noqa: BLE001 except Exception as exc: # noqa: BLE001
return f"Media server unreachable: {exc}" return f"Media server unreachable: {exc}"
track = st.get("track") or {} track = st.get("track") or {}
-1
View File
@@ -16,7 +16,6 @@ dependencies = [
"livekit-plugins-openai~=1.7", "livekit-plugins-openai~=1.7",
"python-dotenv", "python-dotenv",
"uvicorn>=0.52.4", "uvicorn>=0.52.4",
"yt-dlp>=2026.8.19",
] ]
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
Generated
+1 -12
View File
@@ -664,7 +664,7 @@ name = "exceptiongroup"
version = "1.3.1" version = "1.3.1"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "typing-extensions" }, { name = "typing-extensions", marker = "python_full_version < '3.11'" },
] ]
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" } sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
wheels = [ wheels = [
@@ -2767,7 +2767,6 @@ dependencies = [
{ name = "livekit-plugins-openai" }, { name = "livekit-plugins-openai" },
{ name = "python-dotenv" }, { name = "python-dotenv" },
{ name = "uvicorn" }, { name = "uvicorn" },
{ name = "yt-dlp" },
] ]
[package.metadata] [package.metadata]
@@ -2779,7 +2778,6 @@ requires-dist = [
{ name = "livekit-plugins-openai", specifier = "~=1.7" }, { name = "livekit-plugins-openai", specifier = "~=1.7" },
{ name = "python-dotenv" }, { name = "python-dotenv" },
{ name = "uvicorn", specifier = ">=0.52.4" }, { name = "uvicorn", specifier = ">=0.52.4" },
{ name = "yt-dlp", specifier = ">=2026.8.19" },
] ]
[[package]] [[package]]
@@ -3062,12 +3060,3 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" }, { url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" }, { url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
] ]
[[package]]
name = "yt-dlp"
version = "2026.8.19"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1e/e0/832fa4ca334b766a06933a196066edc3dba37cdb6f14cd98d59bcc69a4b4/yt_dlp-2026.8.19.tar.gz", hash = "sha256:9e213e48cea35c66b378e4447903f118f6392a5fa380a2b6d7070ec86f4e0af1", size = 3052025, upload-time = "2026-08-19T23:48:59.291Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/69/b2/8cd1613f56eed7ceb64fbd4df3f1c01246bfb098e6f398228bafda22b80b/yt_dlp-2026.8.19-py3-none-any.whl", hash = "sha256:1d57897e94c6665a0a6f9bc54b34e584284e32c034ffab3a7df25d8f7b24eedf", size = 3185533, upload-time = "2026-08-19T23:48:56.925Z" },
]
-1
View File
@@ -10,4 +10,3 @@ Requires-Dist: livekit-plugins-azure~=1.7
Requires-Dist: livekit-plugins-openai~=1.7 Requires-Dist: livekit-plugins-openai~=1.7
Requires-Dist: python-dotenv Requires-Dist: python-dotenv
Requires-Dist: uvicorn>=0.52.4 Requires-Dist: uvicorn>=0.52.4
Requires-Dist: yt-dlp>=2026.8.19
-1
View File
@@ -5,4 +5,3 @@ livekit-plugins-azure~=1.7
livekit-plugins-openai~=1.7 livekit-plugins-openai~=1.7
python-dotenv python-dotenv
uvicorn>=0.52.4 uvicorn>=0.52.4
yt-dlp>=2026.8.19
+3 -5
View File
@@ -18,16 +18,14 @@ services:
WEB_PORT: "${WEB_PORT:-9178}" WEB_PORT: "${WEB_PORT:-9178}"
VOICE_USERS_FILE: "/etc/hope/users.json" VOICE_USERS_FILE: "/etc/hope/users.json"
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}" VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
MEDIA_DIR: "/data/media" MUSIC_BASE_URL: "${MUSIC_BASE_URL:-http://192.168.86.2:8094}"
MEDIA_SERVER_PORT: "${MEDIA_SERVER_PORT:-8093}" MUSIC_ROOM: "${MUSIC_ROOM:-voice-room}"
YTDLP_COOKIES: "${YTDLP_COOKIES:-/data/media/cookies.txt}"
volumes: volumes:
- ./livekit.yaml:/etc/livekit.yaml:ro - ./livekit.yaml:/etc/livekit.yaml:ro
- ./certs:/etc/voice/certs - ./certs:/etc/voice/certs
- ./skills:/skills - ./skills:/skills
- ./media:/data/media
healthcheck: healthcheck:
test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:$${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null"] test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null"]
interval: 15s interval: 15s
timeout: 5s timeout: 5s
retries: 3 retries: 3
View File
Whitespace-only changes.
-21
View File
@@ -22,27 +22,6 @@ server {
proxy_read_timeout 3600s; proxy_read_timeout 3600s;
proxy_send_timeout 3600s; proxy_send_timeout 3600s;
} }
# Media server API (library, search, player state) and the proxy for
# external live-stream URLs (http:// sources would be blocked as mixed
# content in the browser, so they are re-served over HTTPS from here).
location /media/api/ {
proxy_pass http://127.0.0.1:8093/media/api/;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
# Downloaded tracks are plain static files — nginx serves them with
# native HTTP Range support, which gives the web player seeking for free.
location /media/files/ {
alias /data/media/files/;
}
location /media/thumbs/ {
alias /data/media/thumbs/;
}
location / { location / {
try_files $uri $uri/ =404; try_files $uri $uri/ =404;
} }
-15
View File
@@ -56,18 +56,3 @@ environment=
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s", LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s", VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s" VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
[program:media-server]
; Runs as root because the /data/media bind mount is a host (Unraid) share
; whose uid does not match the container's voiceuser.
; MEDIA_DIR / MEDIA_SERVER_PORT / YTDLP_COOKIES are inherited from the
; container environment (media_server.py has safe defaults for all three) —
; do NOT list them via %(ENV_x)s here: a missing var aborts supervisord's
; config expansion and takes every program down with it.
command=/opt/voice-agent/.venv/bin/python /opt/voice/media_server.py
user=root
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
redirect_stderr=true
+2 -2
View File
@@ -15,7 +15,7 @@ tool servers, and agent process.
## Run ## Run
```bash ```bash
cd /home/shane/dev/voice cd ~/dev/hope-voice-api
python -m pytest tests/ -v # everything python -m pytest tests/ -v # everything
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
``` ```
@@ -27,7 +27,7 @@ python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy | | `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming | | `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up | | `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
| `test_media.py` | media server API + player UI + media MCP | library/player state flow, Range proxy, static mp3 Range, live YT Music search (`-m slow`), gated full download (`HOPE_RUN_YTDLP_DOWNLOAD=1`) | | `test_media.py` | external Hope Music boundary + media MCP client | LAN service health, stable room scope, split regression guards, local-container MCP probes |
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API | | `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched | | `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
+1 -2
View File
@@ -27,8 +27,7 @@ WEB_BASE_URL = os.environ.get("VOICE_WEB_URL", "https://192.168.86.2:9178").rstr
TOKEN_URL = os.environ.get( TOKEN_URL = os.environ.get(
"VOICE_TOKEN_URL", f"{WEB_BASE_URL}/token" "VOICE_TOKEN_URL", f"{WEB_BASE_URL}/token"
) )
# Filesystem-coupled tests (repo ./media bind mount) only make sense when # docker-exec integration probes only make sense when the test target is local.
# the test host IS the deployment host.
IS_LOCAL_DEPLOYMENT = urlparse(WEB_BASE_URL).hostname in ("localhost", "127.0.0.1") IS_LOCAL_DEPLOYMENT = urlparse(WEB_BASE_URL).hostname in ("localhost", "127.0.0.1")
LLM_BASE_URL = ( LLM_BASE_URL = (
os.environ.get("LLM_BASE_URL") os.environ.get("LLM_BASE_URL")
+96 -527
View File
@@ -1,484 +1,112 @@
"""Tests for the media player subsystem. """Voice-side integration tests for the standalone Hope Music service.
Covers the media server API (via nginx HTTPS and direct), the stream proxy The library, downloads, stream proxy, web player, and their exhaustive tests
with HTTP Range passthrough, static file serving of the ./media volume, the live in ``hope-media-music`` now. This module guards the remaining product
web player markup, and Hope's media MCP toolset. Download tests are gated boundary: the voice repo contains only the MCP/HTTP client, uses one stable
behind HOPE_RUN_YTDLP_DOWNLOAD=1 (network-heavy, occasionally flaky). player scope, and can reach the LAN service.
""" """
import json
import os import os
import subprocess import socket
import threading import threading
import time
import uuid
from http.server import BaseHTTPRequestHandler, HTTPServer from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
from urllib.parse import urlparse from urllib.parse import urlparse
import httpx import httpx
import pytest import pytest
from conftest import ( from conftest import IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
TOKEN_URL, docker_exec, docker_exec_python, IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
ROOT = Path(__file__).parents[1]
MUSIC_BASE_URL = os.environ.get(
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
).rstrip("/")
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
MEDIA_API = "/media/api"
def test_external_music_service_health_and_library():
"""The peer service used by the deployed agent is reachable over the LAN."""
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as client:
health = client.get(f"{MEDIA_API}/healthz")
assert health.status_code == 200
assert health.json()["ok"] is True
library = client.get(f"{MEDIA_API}/library")
assert library.status_code == 200
assert isinstance(library.json()["tracks"], list)
def test_voice_repo_contains_only_the_music_client():
"""Regression guard for the product split: backend/player cannot creep back."""
agent = (ROOT / "agent" / "agent.py").read_text()
client = (ROOT / "agent" / "media_mcp.py").read_text()
web_js = (ROOT / "web" / "app.js").read_text()
web_html = (ROOT / "web" / "index.html").read_text()
assert not (ROOT / "web" / "media_server.py").exists()
assert not (ROOT / "media").exists()
assert "MUSIC_BASE_URL" in agent and "MUSIC_ROOM" in agent
assert "MUSIC_BASE_URL" in client and "MUSIC_ROOM" in client
assert "MEDIA_SERVER_URL" not in agent + client
assert "media-control" not in agent + web_js
assert "media_state" not in agent + web_js
assert "/media/api" not in web_js
assert 'id="mediaPanel"' not in web_html
assert 'id="mediaAudio"' not in web_html
def test_voice_uses_a_stable_music_scope():
"""The ephemeral LiveKit room must not be the standalone player's scope."""
assert MUSIC_ROOM == "voice-room" or os.environ.get("MUSIC_ROOM")
source = (ROOT / "agent" / "agent.py").read_text()
assert 'MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")' in source
assert '"MUSIC_ROOM": MUSIC_ROOM' in source
# These exercise the actual MCP subprocess inside a local voice container. The
# default suite targets Unraid remotely, where docker-exec intentionally skips.
needs_local_container = pytest.mark.skipif(
not IS_LOCAL_DEPLOYMENT,
reason="docker-exec MCP probes require a local voice container",
) )
MEDIA_API = "/media/api" MEDIA_MCP_ENV = {
TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}" "MUSIC_BASE_URL": MUSIC_BASE_URL,
"MUSIC_ROOM": "mcp-test-room",
}
# ── local upstream for stream-proxy tests ───────────────────────────────────
def _serve_fake_audio(payload: bytes): def _serve_fake_audio(payload: bytes):
"""Tiny Range-capable HTTP server on an ephemeral port. """Tiny Range-capable upstream reachable by a host-networked container."""
Binds 0.0.0.0 and advertises this host's LAN IP so the deployment host's
media server (which may be a different machine — Unraid) can reach it
over the LAN when fetching external stream URLs.
"""
import socket
class Handler(BaseHTTPRequestHandler): class Handler(BaseHTTPRequestHandler):
def do_GET(self): def do_GET(self):
rng = self.headers.get("Range")
if rng and rng.startswith("bytes="):
start_s, _, end_s = rng[6:].partition("-")
start = int(start_s or 0)
end = int(end_s) if end_s else len(payload) - 1
end = min(end, len(payload) - 1)
body = payload[start : end + 1]
self.send_response(206)
self.send_header(
"Content-Range", f"bytes {start}-{end}/{len(payload)}"
)
else:
body = payload
self.send_response(200) self.send_response(200)
self.send_header("Content-Type", "audio/mpeg") self.send_header("Content-Type", "audio/mpeg")
self.send_header("Content-Length", str(len(body))) self.send_header("Content-Length", str(len(payload)))
self.send_header("Accept-Ranges", "bytes")
self.end_headers() self.end_headers()
self.wfile.write(body) self.wfile.write(payload)
def log_message(self, *args): def log_message(self, format, *args): # noqa: A002
pass pass
server = HTTPServer(("0.0.0.0", 0), Handler) server = HTTPServer(("0.0.0.0", 0), Handler)
port = server.server_address[1] port = server.server_address[1]
# The local IP toward the deployment host (works across the LAN).
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try: try:
probe.connect((urlparse(WEB_BASE_URL).hostname or "192.168.86.2", 9178)) target = urlparse(WEB_BASE_URL).hostname or "192.168.86.2"
lan_ip = probe.getsockname()[0] probe.connect((target, urlparse(WEB_BASE_URL).port or 9178))
host = probe.getsockname()[0]
except OSError: except OSError:
lan_ip = "127.0.0.1" host = "127.0.0.1"
finally: finally:
probe.close() probe.close()
threading.Thread(target=server.serve_forever, daemon=True).start() threading.Thread(target=server.serve_forever, daemon=True).start()
return server, f"http://{lan_ip}:{port}/audio.mp3" return server, f"http://{host}:{port}/audio.mp3"
# ── health + basic API shape ────────────────────────────────────────────────
def test_media_server_health_via_nginx(client):
resp = client.get(f"{MEDIA_API}/healthz")
assert resp.status_code == 200
assert resp.json()["ok"] is True
def test_media_library_endpoint(client):
resp = client.get(f"{MEDIA_API}/library")
assert resp.status_code == 200
data = resp.json()
assert isinstance(data["tracks"], list)
assert data["version"] >= 1
for track in data["tracks"]:
assert {"id", "kind", "title", "status"} <= set(track)
def test_media_search_requires_query(client):
resp = client.post(f"{MEDIA_API}/search", json={"query": ""})
assert resp.status_code == 400
def test_media_play_requires_target(client):
resp = client.post(f"{MEDIA_API}/play", json={"room": TEST_ROOM})
assert resp.status_code == 400
def test_media_unknown_stream_404(client):
resp = client.get(f"{MEDIA_API}/stream/does-not-exist")
assert resp.status_code == 404
# ── player state flow (uses a live stream track as the fixture) ─────────────
def test_media_player_state_flow(client):
"""add stream -> play -> state -> report -> pause -> seek hint -> stop."""
upstream, url = _serve_fake_audio(b"\x00" * 64)
try:
added = client.post(
f"{MEDIA_API}/streams", json={"name": "Test FM", "url": url}
)
assert added.status_code == 200
track = added.json()["track"]
track_id = track["id"]
assert track["kind"] == "stream"
assert track["status"] == "ready"
assert track["audio_url"].startswith(f"{MEDIA_API}/stream/")
played = client.post(
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
).json()
assert played["desired"] == "playing"
assert played["track"]["id"] == track_id
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
assert state["track"]["id"] == track_id
client.post(
f"{MEDIA_API}/report",
json={"room": TEST_ROOM, "track_id": track_id,
"state": "playing", "position": 12.5},
)
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
assert abs(state["reported"]["position"] - 12.5) < 0.01
client.post(f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "pause"})
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["desired"] == "paused"
client.post(
f"{MEDIA_API}/control",
json={"room": TEST_ROOM, "action": "seek", "position": 42},
)
hint = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["seek_hint"]
assert hint["position"] == 42
# start_paused variant: scope transfer without starting playback
transfer = client.post(
f"{MEDIA_API}/play",
json={"room": TEST_ROOM + "-b", "track_id": track_id, "start_paused": True},
).json()
assert transfer["desired"] == "paused"
# playing clears any open suggestion choices
client.post(
f"{MEDIA_API}/choices",
json={"room": TEST_ROOM, "choices": [{"video_id": "x", "title": "X"}]},
)
client.post(
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
)
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["choices"] == []
stopped = client.post(
f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "stop"}
).json()
assert stopped["desired"] == "stopped"
finally:
upstream.shutdown()
# Remove the test stream from the permanent library.
try:
client.delete(f"{MEDIA_API}/tracks/{track_id}")
except (httpx.HTTPError, UnboundLocalError):
pass
def test_media_stream_proxy_and_range(client):
"""External audio URLs must be reachable same-origin (HTTPS) and support
Range passthrough so the browser can seek progressive streams.
Local deployments test against a controlled fake upstream; remote
deployments (Unraid) cannot reach the test host's ephemeral ports, so a
public reference mp3 is used and Range semantics are asserted without
byte equality."""
if IS_LOCAL_DEPLOYMENT:
payload = b"0123456789" * 100
upstream, url = _serve_fake_audio(payload)
track_id = None
try:
track_id = client.post(
f"{MEDIA_API}/streams", json={"name": "Range FM", "url": url}
).json()["track"]["id"]
full = client.get(f"{MEDIA_API}/stream/{track_id}")
assert full.status_code == 200
assert full.content == payload
ranged = client.get(
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=100-199"}
)
assert ranged.status_code == 206
assert ranged.content == payload[100:200]
assert ranged.headers["content-range"] == f"bytes 100-199/{len(payload)}"
finally:
upstream.shutdown()
if track_id:
try:
client.delete(f"{MEDIA_API}/tracks/{track_id}")
except httpx.HTTPError:
pass
return
# Remote deployment: public reference audio (SoundHelix).
url = "https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3"
track_id = None
try:
track_id = client.post(
f"{MEDIA_API}/streams", json={"name": "Range FM public", "url": url}
).json()["track"]["id"]
full = client.get(f"{MEDIA_API}/stream/{track_id}")
assert full.status_code == 200
assert full.headers.get("content-type", "").startswith("audio/")
assert len(full.content) > 100_000
ranged = client.get(
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=0-1023"}
)
assert ranged.status_code == 206
assert len(ranged.content) == 1024
assert (ranged.headers.get("content-range") or "").startswith("bytes 0-1023/")
finally:
if track_id:
try:
client.delete(f"{MEDIA_API}/tracks/{track_id}")
except httpx.HTTPError:
pass
def test_media_files_static_range_support(client):
"""Downloaded tracks are nginx static files with native Range support.
Writes a probe file into the repo's ./media bind mount, so it only runs
when the test host IS the deployment host (local dev deployment)."""
if not IS_LOCAL_DEPLOYMENT:
pytest.skip("writes to the repo ./media bind mount — local deployments only")
import pathlib
marker = uuid.uuid4().hex[:8]
host_media = pathlib.Path(__file__).parents[1] / "media" / "files"
host_media.mkdir(parents=True, exist_ok=True)
fname = f"test-{marker}.mp3"
payload = b"IMHEADDATA" + b"\x01" * 2048
(host_media / fname).write_bytes(payload)
try:
full = client.get(f"/media/files/{fname}")
assert full.status_code == 200
assert full.content == payload
ranged = client.get(f"/media/files/{fname}", headers={"Range": "bytes=0-9"})
assert ranged.status_code == 206
assert ranged.content == b"IMHEADDATA"
finally:
(host_media / fname).unlink(missing_ok=True)
def test_media_library_persists_in_volume(client):
"""The library JSON is written to the ./media bind mount (permanent).
Reads the repo's ./media directory — local deployments only."""
if not IS_LOCAL_DEPLOYMENT:
pytest.skip("reads the repo ./media bind mount — local deployments only")
import pathlib
upstream, url = _serve_fake_audio(b"\x00" * 16)
track_id = None
try:
track_id = client.post(
f"{MEDIA_API}/streams", json={"name": "Persist FM", "url": url}
).json()["track"]["id"]
library_file = pathlib.Path(__file__).parents[1] / "media" / "library.json"
data = json.loads(library_file.read_text())
assert track_id in data["tracks"]
finally:
upstream.shutdown()
if track_id:
try:
client.delete(f"{MEDIA_API}/tracks/{track_id}")
except httpx.HTTPError:
pass
# ── web player UI ───────────────────────────────────────────────────────────
def test_ui_media_panel_serves(client):
resp = client.get("/")
body = resp.text
assert 'id="mediaBtn"' in body
assert 'id="mediaPanel"' in body
assert 'id="playPauseBtn"' in body
assert 'id="seekBar"' in body
assert 'id="mediaAudio"' in body
def test_ui_media_logic_serves(client):
resp = client.get("/app.js")
body = resp.text
assert "reconcileMedia" in body
assert "media-control" in body # data-channel topic for the agent
assert "media_state" in body # playback state reports
assert "/media/api/state" in body # reconciliation polling
def test_ui_media_styles_serve(client):
resp = client.get("/style.css")
assert ".media-panel" in resp.text
assert ".media-row" in resp.text
def test_token_response_includes_room():
resp = httpx.post(TOKEN_URL, json={}, timeout=10, verify=False)
assert resp.status_code == 200
assert resp.json()["room"].startswith("voice-")
# ── Hope's media MCP toolset ────────────────────────────────────────────────
# These spawn the MCP server inside the LOCAL dev container (docker exec),
# whose media server is the local one — they can only be consistent when
# the test target IS the local deployment.
MEDIA_MCP_ENV = {
"MEDIA_SERVER_URL": "http://127.0.0.1:8093",
"MEDIA_ROOM": "mcp-test-room",
}
needs_local_container = pytest.mark.skipif(
not IS_LOCAL_DEPLOYMENT,
reason="docker-exec MCP probes run against the local dev container",
)
# ── media server internals (in-process, inside the container) ───────────────
@needs_local_container
def test_media_search_prefers_library_matches(client):
"""Search returns ready library matches ahead of YouTube results, so
Hope prefers an already-downloaded copy. Runs the endpoint in-process
with _run_search stubbed — fully deterministic, no network."""
code = '''
import sys
sys.path.insert(0, "/opt/voice")
import media_server as ms
from fastapi.testclient import TestClient
ms._library["tracks"]["libpref01"] = {
"id": "libpref01", "kind": "youtube", "video_id": "libpref01",
"title": "Preference Probe Song", "channel": "Probe Artist",
"duration": 120, "status": "ready", "ext": "webm",
"error": None, "added_at": 1.0,
}
ms._run_search = lambda q, limit: [
{"video_id": "remote01", "title": "Preference Probe Song (cover)",
"channel": "", "duration": 99, "view_count": 1,
"in_library": False, "track_id": None, "library_status": None},
]
c = TestClient(ms.app)
r = c.post("/media/api/search", json={"query": "preference probe", "limit": 5})
assert r.status_code == 200, r.text
results = r.json()["results"]
assert results, "no results"
assert results[0]["video_id"] == "libpref01", results
assert results[0]["in_library"] is True
assert any(x["video_id"] == "remote01" for x in results), results
print("OK")
'''
proc = docker_exec_python(code, timeout=60, check=False)
assert proc.returncode == 0, proc.stderr[-800:]
assert "OK" in proc.stdout
@needs_local_container
def test_media_stream_endpoint_serves_ready_track(client):
"""GET /media/api/stream/{id} for a finished youtube track serves the
file bytes with native Range support (progressive-playback endpoint)."""
code = '''
import sys
sys.path.insert(0, "/opt/voice")
import media_server as ms
from fastapi.testclient import TestClient
probe = ms.FILES_DIR / "probetrk.mp3"
probe.write_bytes(b"PROBEDATA" * 100)
ms._library["tracks"]["probetrk"] = {
"id": "probetrk", "kind": "youtube", "video_id": "probetrk",
"title": "Probe", "channel": "", "duration": 1,
"status": "ready", "ext": "mp3", "error": None, "added_at": 1.0,
}
try:
c = TestClient(ms.app)
full = c.get("/media/api/stream/probetrk")
assert full.status_code == 200, full.status_code
assert full.content == b"PROBEDATA" * 100
part = c.get("/media/api/stream/probetrk", headers={"Range": "bytes=0-8"})
assert part.status_code == 206, part.status_code
assert part.content == b"PROBEDATA"
assert (part.headers.get("content-range") or "").startswith("bytes 0-8/")
# track dict exposes a stable stream URL while downloading and ready
d = ms._track_dict(ms._library["tracks"]["probetrk"])
assert d["audio_url"] == "/media/api/stream/probetrk", d
print("OK")
finally:
probe.unlink(missing_ok=True)
ms._library["tracks"].pop("probetrk", None)
'''
proc = docker_exec_python(code, timeout=60, check=False)
assert proc.returncode == 0, proc.stderr[-800:]
assert "OK" in proc.stdout
@needs_local_container
def test_media_play_resolves_cached_ready_track(client):
"""Playing an ALREADY-cached song must resolve to the on-disk ready track
(no redownload) and hand the browser a playable stream URL, so a cached
pick starts instantly rather than re-fetching from YouTube.
Regression: a cached track could be left as paused with currentTime 0 if
the room the agent played into diverged from the room the browser polled,
or if autoplay was blocked. This pins the server-side resolve + track
dict so a ready file is always returned with a stable audio_url."""
code = '''
import sys
sys.path.insert(0, "/opt/voice")
import media_server as ms
from fastapi.testclient import TestClient
probe = ms.FILES_DIR / "cachetrk.webm"
probe.write_bytes(b"CACHEDBYTES" * 40)
ms._library["tracks"]["cachetrk"] = {
"id": "cachetrk", "kind": "youtube", "video_id": "cachetrk",
"title": "Cached Song", "channel": "Artist", "duration": 200,
"status": "ready", "ext": "webm", "error": None, "added_at": 1.0,
}
ms._start_download = lambda *a, **k: (_ for _ in ()).throw(
AssertionError("must not re-download an already cached track")
)
try:
c = TestClient(ms.app)
# play by video_id (as Hope's play_song does) must reuse the ready track
r = c.post("/media/api/play", json={
"room": "cached-test-room", "video_id": "cachetrk",
"title": "Cached Song", "channel": "Artist",
}, timeout=10)
assert r.status_code == 200, r.text
body = r.json()
assert body["desired"] == "playing", body
track = body["track"]
assert track["status"] == "ready", track
assert track["audio_url"] == "/media/api/stream/cachetrk", track
# the stream endpoint serves the cached bytes (browser plays it directly)
data = c.get("/media/api/stream/cachetrk")
assert data.status_code == 200
assert data.content == b"CACHEDBYTES" * 40
print("OK")
finally:
probe.unlink(missing_ok=True)
ms._library["tracks"].pop("cachetrk", None)
'''
proc = docker_exec_python(code, timeout=60, check=False)
assert proc.returncode == 0, proc.stderr[-800:]
assert "OK" in proc.stdout
@needs_local_container @needs_local_container
def test_mcp_media_list_library(mcp_client_factory): def test_mcp_media_list_library(mcp_client_factory):
@@ -490,8 +118,9 @@ def test_mcp_media_list_library(mcp_client_factory):
@needs_local_container @needs_local_container
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client): def test_mcp_media_add_stream_and_now_playing(mcp_client_factory):
upstream, url = _serve_fake_audio(b"\x00" * 16) upstream, url = _serve_fake_audio(b"\x00" * 16)
track_id = None
try: try:
text, is_error = mcp_client_factory( text, is_error = mcp_client_factory(
"media_mcp.py", "add_stream", "media_mcp.py", "add_stream",
@@ -500,99 +129,39 @@ def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
assert not is_error, text assert not is_error, text
assert "Added" in text assert "Added" in text
library = client.get(f"{MEDIA_API}/library").json()["tracks"] with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
library = music.get(f"{MEDIA_API}/library").json()["tracks"]
probe = next(t for t in library if t["title"] == "MCP Test Radio") probe = next(t for t in library if t["title"] == "MCP Test Radio")
try: track_id = probe["id"]
text, is_error = mcp_client_factory( music.post(
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV f"{MEDIA_API}/play",
) json={"room": "mcp-test-room", "track_id": track_id},
assert not is_error ).raise_for_status()
# Nothing loaded in mcp-test-room: the tool must say so plainly.
assert "Nothing is loaded" in text
client.post(
f"{MEDIA_API}/play", json={"room": "mcp-test-room", "track_id": probe["id"]}
)
text, is_error = mcp_client_factory( text, is_error = mcp_client_factory(
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV "media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
) )
assert not is_error assert not is_error
assert "MCP Test Radio" in text assert "MCP Test Radio" in text
finally:
client.delete(f"{MEDIA_API}/tracks/{probe['id']}")
finally: finally:
upstream.shutdown() upstream.shutdown()
if track_id:
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
music.delete(f"{MEDIA_API}/tracks/{track_id}")
@needs_local_container @needs_local_container
def test_mcp_media_resume_without_load(mcp_client_factory, client): def test_mcp_media_resume_without_load(mcp_client_factory):
"""Errors from the media server reach the LLM as tool text (not MCP room = "mcp-empty-room"
errors) so Hope can explain them naturally.""" with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
client.post( music.post(
f"{MEDIA_API}/control", json={"room": "mcp-empty-room", "action": "stop"} f"{MEDIA_API}/control",
) json={"room": room, "action": "stop"},
).raise_for_status()
text, is_error = mcp_client_factory( text, is_error = mcp_client_factory(
"media_mcp.py", "resume_music", {}, env={ "media_mcp.py", "resume_music", {},
"MEDIA_SERVER_URL": "http://127.0.0.1:8093", env={"MUSIC_BASE_URL": MUSIC_BASE_URL, "MUSIC_ROOM": room},
"MEDIA_ROOM": "mcp-empty-room",
}
) )
assert not is_error assert not is_error
assert "nothing loaded" in text.lower() assert "nothing loaded" in text.lower()
# ── live YouTube Music (network-dependent) ──────────────────────────────────
@pytest.mark.slow
def test_media_search_live(client):
resp = client.post(
f"{MEDIA_API}/search",
json={"query": "blind guardian timeless miracle", "limit": 3},
)
assert resp.status_code == 200
results = resp.json()["results"]
assert len(results) >= 1
for r in results:
assert r["video_id"]
assert r["title"]
@pytest.mark.slow
def test_media_download_and_stream_end_to_end(client):
"""Full pipeline: search -> play -> yt-dlp download -> playable stream.
The audio_url stays on /media/api/stream/ for the whole lifecycle; once
ready it serves the finished file with Range support. Gated: run with
HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")
search = client.post(
f"{MEDIA_API}/search", json={"query": "kingdom hearts dearly beloved", "limit": 3}
)
results = search.json()["results"]
assert results, "search returned nothing"
video = results[0]
played = client.post(
f"{MEDIA_API}/play",
json={"room": TEST_ROOM, "video_id": video["video_id"],
"title": video["title"], "channel": video["channel"]},
).json()
track_id = played["track"]["id"]
try:
deadline = time.time() + 240
track = played["track"]
while time.time() < deadline:
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
track = next(t for t in library if t["id"] == track_id)
if track["status"] in ("ready", "failed"):
break
time.sleep(3)
assert track["status"] == "ready", f"download failed: {track.get('error')}"
stream = client.get(track["audio_url"], headers={"Range": "bytes=0-1023"})
assert stream.status_code in (200, 206)
assert len(stream.content) >= 1
finally:
client.delete(f"{MEDIA_API}/tracks/{track_id}")
-568
View File
@@ -269,18 +269,6 @@ const LED_GROW = 1.4; // how much the dots grow toward the center
let ledDots = []; let ledDots = [];
let ledWallTime = 0; let ledWallTime = 0;
// Decides whether the LED wall square should show the track's album art.
// Music and conversation are mutually exclusive: while a conversation is
// active the music is paused, so the wall pulses (agent audio) and any loaded
// track's cover art returns once the talk ends.
function updateAvatarMedia() {
const inConversation = !!(room && room.localParticipant);
const showArt = !inConversation && !!mediaCurrentTrackId && !!mediaCurrentArtUrl;
ledWallEl.classList.toggle("show-art", showArt);
if (showArt && ledArtEl.getAttribute("src") !== mediaCurrentArtUrl) {
ledArtEl.src = mediaCurrentArtUrl;
}
}
function buildLedWall() { function buildLedWall() {
const inner = Math.floor(LED_COLS / 2); const inner = Math.floor(LED_COLS / 2);
@@ -781,7 +769,6 @@ document.addEventListener("keydown", (event) => {
if (event.key !== "Escape") return; if (event.key !== "Escape") return;
if (settingsSheet.classList.contains("open")) closeSheet(); if (settingsSheet.classList.contains("open")) closeSheet();
if (tasksPanel.classList.contains("open")) setTasksOpen(false); if (tasksPanel.classList.contains("open")) setTasksOpen(false);
if (mediaPanel.classList.contains("open")) setMediaOpen(false);
}); });
// ── Wake lock ─────────────────────────────────────────────────────────────── // ── Wake lock ───────────────────────────────────────────────────────────────
async function acquireWakeLock() { async function acquireWakeLock() {
@@ -905,8 +892,6 @@ function createRoom(token) {
stopBtn.disabled = true; stopBtn.disabled = true;
agentParticipant = null; agentParticipant = null;
agentReady = false; agentReady = false;
// Leaving the conversation hands the room back to the music player.
resumePlaybackAfterTalk();
}); });
return newRoom.connect(LIVEKIT_URL, token, { return newRoom.connect(LIVEKIT_URL, token, {
@@ -965,7 +950,6 @@ async function connectConversation() {
// Get a signed access token from the token endpoint. // Get a signed access token from the token endpoint.
const tokenData = await fetchToken(ROOM_NAME); const tokenData = await fetchToken(ROOM_NAME);
await enterMediaScope(tokenData.room);
room = await createRoom(tokenData.token); room = await createRoom(tokenData.token);
// The agent is dispatched when we join. If it misses the dispatch // The agent is dispatched when we join. If it misses the dispatch
@@ -979,7 +963,6 @@ async function connectConversation() {
agentReady = false; agentReady = false;
await stale.disconnect(); await stale.disconnect();
const freshTokenData = await fetchToken(ROOM_NAME); const freshTokenData = await fetchToken(ROOM_NAME);
await enterMediaScope(freshTokenData.room);
room = await createRoom(freshTokenData.token); room = await createRoom(freshTokenData.token);
if (!(await waitForAgent(15000))) { if (!(await waitForAgent(15000))) {
throw new Error("Assistant did not join — reload the page and try again"); throw new Error("Assistant did not join — reload the page and try again");
@@ -1011,7 +994,6 @@ async function connectConversation() {
muteBtn.disabled = conversationMode !== "voice"; muteBtn.disabled = conversationMode !== "voice";
updateMuteIcon(); updateMuteIcon();
setState("listening"); setState("listening");
updateAvatarMedia();
return room; return room;
} catch (err) { } catch (err) {
console.error("Connection failed:", err); console.error("Connection failed:", err);
@@ -1093,9 +1075,6 @@ stopBtn.addEventListener("click", () => {
muteBtn.disabled = true; muteBtn.disabled = true;
micMuted = false; micMuted = false;
updateMuteIcon(); updateMuteIcon();
// Ending the conversation resumes music that was paused for the talk.
resumePlaybackAfterTalk();
updateAvatarMedia();
}); });
// ── Mic mute toggle ───────────────────────────────────────────────────────── // ── Mic mute toggle ─────────────────────────────────────────────────────────
@@ -1162,551 +1141,4 @@ function sendModelMessage(model, targetRoom) {
}); });
} }
// ── Media player ────────────────────────────────────────────────────────────
// The browser <audio> element is the player. The media server owns library +
// per-room desired state; this UI reconciles against GET /media/api/state
// (polling) and reports real playback back. Hope drives the same state via
// her media tools, so voice commands and panel clicks stay in sync.
//
// Conversation rule (mirrored in agent.py): while a track plays the agent's
// mic is off. Connecting to Hope pauses music; ending the conversation
// resumes what was playing.
const mediaBtn = document.getElementById("mediaBtn");
const mediaPanel = document.getElementById("mediaPanel");
const closeMediaBtn = document.getElementById("closeMediaBtn");
const mediaBadge = document.getElementById("mediaBadge");
const mediaArt = document.getElementById("mediaArt");
const mediaArtPlaceholder = document.getElementById("mediaArtPlaceholder");
const mediaTitleEl = document.getElementById("mediaTitle");
const mediaChannelEl = document.getElementById("mediaChannel");
const mediaProgressWrap = document.getElementById("mediaProgressWrap");
const mediaProgressFill = document.getElementById("mediaProgressFill");
const mediaProgressLabel = document.getElementById("mediaProgressLabel");
const rewBtn = document.getElementById("rewBtn");
const fwdBtn = document.getElementById("fwdBtn");
const playPauseBtn = document.getElementById("playPauseBtn");
const playIcon = document.getElementById("playIcon");
const pauseIcon = document.getElementById("pauseIcon");
const stopMediaBtn = document.getElementById("stopMediaBtn");
const seekBar = document.getElementById("seekBar");
const mediaCurEl = document.getElementById("mediaCur");
const mediaDurEl = document.getElementById("mediaDur");
const volumeBar = document.getElementById("volumeBar");
const mediaSearchForm = document.getElementById("mediaSearchForm");
const mediaSearchInput = document.getElementById("mediaSearchInput");
const mediaResultsEl = document.getElementById("mediaResults");
const mediaChoicesWrap = document.getElementById("mediaChoicesWrap");
const mediaChoicesEl = document.getElementById("mediaChoices");
const mediaLibraryEl = document.getElementById("mediaLibrary");
const streamForm = document.getElementById("streamForm");
const streamNameInput = document.getElementById("streamName");
const streamUrlInput = document.getElementById("streamUrl");
const mediaAudio = document.getElementById("mediaAudio");
const ledArtEl = document.getElementById("ledArt");
let mediaRoom = null; // LiveKit room name (player state scope)
let mediaCurrentTrackId = null; // track id loaded into the audio element
let mediaCurrentArtUrl = null; // current track's cover art
let mediaAppliedSeekTs = 0; // last applied seek_hint timestamp
let mediaWasPlayingBeforeTalk = false;
let mediaPollTimer = null;
let mediaLastReport = 0;
function mediaScope() {
// The LiveKit room is single-source of truth for the player scope. If a
// room is connected, poll the room the agent actually joined (room.name),
// so a "play X" request from the agent can never land in a stale scope.
if (room && room.name) return room.name;
return mediaRoom || "default";
}
async function mediaPost(path, body) {
const resp = await fetch(`/media/api${path}`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ room: mediaScope(), ...body }),
});
if (!resp.ok) {
const detail = await resp.json().catch(() => ({}));
throw new Error(detail.error || `media API ${path} -> ${resp.status}`);
}
return resp.json();
}
// Called just before joining the LiveKit room: carry whatever the audio
// element holds into the room-scoped player state, paused (talking pauses
// music). resumePlaybackAfterTalk() reverses this when the talk ends.
async function enterMediaScope(newRoom) {
if (!newRoom || newRoom === mediaRoom) return;
const wasPlaying = !!mediaCurrentTrackId && !mediaAudio.paused;
mediaWasPlayingBeforeTalk = wasPlaying;
const carriedTrack = mediaCurrentTrackId;
mediaRoom = newRoom;
if (carriedTrack) {
try {
await mediaPost("/play", { room: mediaRoom, track_id: carriedTrack, start_paused: true });
} catch (e) { console.warn("media scope transfer failed:", e); }
mediaAudio.pause();
}
}
async function resumePlaybackAfterTalk() {
if (!mediaWasPlayingBeforeTalk) return;
mediaWasPlayingBeforeTalk = false;
try { await mediaPost("/control", { action: "resume" }); } catch (e) { /* poll retries */ }
}
function publishMediaState(state) {
if (!room || !room.localParticipant) return;
const payload = new TextEncoder().encode(
JSON.stringify({ type: "media_state", state, track_id: mediaCurrentTrackId })
);
room.localParticipant.publishData(payload, { topic: "media-control", reliable: true });
}
function reportMediaPlayback(state) {
const body = {
state,
track_id: mediaCurrentTrackId,
position: isFinite(mediaAudio.currentTime) ? mediaAudio.currentTime : 0,
};
if (isFinite(mediaAudio.duration)) body.duration = mediaAudio.duration;
mediaPost("/report", body).catch(() => {});
}
// ── Polling + reconciliation ────────────────────────────────────────────────
function startMediaPolling() {
if (mediaPollTimer) return;
mediaPollTimer = setInterval(() => {
pollMediaState();
}, 1500);
}
async function pollMediaState() {
let s;
try {
const resp = await fetch(`/media/api/state?room=${encodeURIComponent(mediaScope())}`);
if (!resp.ok) return;
s = await resp.json();
} catch (e) {
return; // media server briefly unavailable; try again next tick
}
reconcileMedia(s);
}
function reconcileMedia(s) {
renderMediaChoices(s.choices || []);
updateMediaBadge(s);
const track = s.track;
const pendingJob = (s.pending || []).find(j => !track || j.track_id === track.id);
updateMediaProgress(track, pendingJob);
if (track && track.id !== mediaCurrentTrackId) {
loadMediaTrack(track);
} else if (track && track.audio_url && mediaAudio.getAttribute("src") !== track.audio_url) {
// The track was loaded before it had a stream (still downloading) and
// now has one — wire it up so playback can start without a reload.
mediaAudio.src = track.audio_url;
mediaAudio.load();
updateMediaNowCard(track);
}
if (!track && mediaCurrentTrackId && s.desired === "stopped") clearMediaAudio();
applyDesiredMediaState(s, track);
if (s.seek_hint && s.seek_hint.ts !== mediaAppliedSeekTs && isFinite(mediaAudio.duration)) {
mediaAppliedSeekTs = s.seek_hint.ts;
const target = Math.min(s.seek_hint.position, mediaAudio.duration);
if (Number.isFinite(target)) {
mediaAudio.currentTime = Math.max(0, target);
updateSeekUI();
}
}
}
function applyDesiredMediaState(s, track) {
if (s.desired === "playing") {
// audio_url presence means a playable stream exists — including a
// still-growing download, which plays progressively.
if (track && track.audio_url && mediaAudio.src && mediaAudio.paused) {
mediaAudio.play().catch((e) => console.warn("playback blocked:", e));
}
} else if (!mediaAudio.paused) {
mediaAudio.pause();
}
}
function loadMediaTrack(track) {
mediaCurrentTrackId = track.id;
updateMediaNowCard(track);
if (track.audio_url) {
if (mediaAudio.getAttribute("src") !== track.audio_url) {
mediaAudio.src = track.audio_url;
mediaAudio.load();
}
} else {
// Downloading or failed: no stream yet. reconcileMedia wires the
// stream URL in via polling as soon as one is available.
mediaAudio.removeAttribute("src");
mediaAudio.load();
updateMediaControls();
}
}
function clearMediaAudio() {
mediaCurrentTrackId = null;
mediaCurrentArtUrl = null;
mediaAudio.pause();
mediaAudio.removeAttribute("src");
mediaAudio.load();
mediaTitleEl.textContent = "Nothing loaded";
mediaChannelEl.textContent = "";
mediaArt.hidden = true;
mediaArtPlaceholder.hidden = false;
updateMediaControls();
updateAvatarMedia();
}
function updateMediaNowCard(track) {
mediaTitleEl.textContent = track.title || track.id;
mediaCurrentArtUrl = track.art_url || null;
if (track.status === "failed") {
mediaChannelEl.textContent = "Download failed — pick another result or try again.";
} else if (track.status === "downloading") {
// Progressive streaming: the track plays while the download runs.
mediaChannelEl.textContent = track.channel
? `${track.channel} · downloading` : "Downloading…";
} else {
mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
}
if (track.art_url) {
mediaArt.src = track.art_url;
mediaArt.hidden = false;
mediaArtPlaceholder.hidden = true;
} else {
mediaArt.hidden = true;
mediaArtPlaceholder.hidden = false;
}
updateAvatarMedia();
}
function updateMediaProgress(track, job) {
if (!job) {
mediaProgressWrap.hidden = true;
return;
}
mediaProgressWrap.hidden = false;
const pct = Math.round(job.progress || 0);
mediaProgressFill.style.width = `${pct}%`;
const stage = job.stage === "queued" ? "Queued…" : "Downloading…";
mediaProgressLabel.textContent = `${stage} ${pct}%`;
if (track) updateMediaNowCard(track);
}
function updateMediaBadge(s) {
const pending = (s.pending || []).length;
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
if (pending > 0) {
mediaBadge.hidden = false;
mediaBadge.textContent = String(pending);
} else if (playing) {
mediaBadge.hidden = false;
mediaBadge.innerHTML = "&#9835;";
} else {
mediaBadge.hidden = true;
}
}
// ── Playback controls ───────────────────────────────────────────────────────
function updateMediaControls() {
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
playPauseBtn.classList.toggle("playing", playing);
playPauseBtn.setAttribute("aria-label", playing ? "Pause" : "Play");
updateMediaBadge({ pending: [] });
}
function fmtTime(sec) {
if (!isFinite(sec) || sec < 0) return "0:00";
const s = Math.floor(sec % 60);
const m = Math.floor((sec / 60) % 60);
const h = Math.floor(sec / 3600);
return h ? `${h}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}` : `${m}:${String(s).padStart(2, "0")}`;
}
function updateSeekUI() {
if (!isFinite(mediaAudio.duration) || mediaAudio.duration <= 0) {
seekBar.value = 0;
mediaDurEl.textContent = "0:00";
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
return;
}
seekBar.max = Math.floor(mediaAudio.duration);
seekBar.value = Math.floor(mediaAudio.currentTime);
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
mediaDurEl.textContent = fmtTime(mediaAudio.duration);
}
playPauseBtn.addEventListener("click", () => {
if (!mediaCurrentTrackId || !mediaAudio.src) return;
if (mediaAudio.paused) {
mediaAudio.play().catch((e) => console.warn("play failed:", e));
mediaPost("/control", { action: "resume" }).catch(() => {});
} else {
mediaAudio.pause();
mediaPost("/control", { action: "pause" }).catch(() => {});
}
});
rewBtn.addEventListener("click", () => {
if (!mediaAudio.src) return;
mediaAudio.currentTime = Math.max(0, mediaAudio.currentTime - 10);
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
});
fwdBtn.addEventListener("click", () => {
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
mediaAudio.currentTime = Math.min(mediaAudio.duration - 0.5, mediaAudio.currentTime + 10);
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
});
stopMediaBtn.addEventListener("click", () => {
clearMediaAudio();
mediaPost("/control", { action: "stop" }).catch(() => {});
publishMediaState("stopped");
});
seekBar.addEventListener("change", () => {
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
mediaAudio.currentTime = Number(seekBar.value);
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
});
volumeBar.addEventListener("input", () => {
mediaAudio.volume = Number(volumeBar.value) / 100;
try { localStorage.setItem("media_volume", volumeBar.value); } catch (e) { /* ok */ }
});
mediaAudio.volume = Number(localStorage.getItem("media_volume") ?? 90) / 100;
volumeBar.value = Math.round(mediaAudio.volume * 100);
// Browsers block audio until a user gestured on the page. Hope often starts a
// cached song from a poll (voice command, no click on the player), so the
// <audio> element sits outside the LiveKit room's audio-unlock flow and its
// play() is rejected. Capture the first gesture and unlock the element.
let mediaAudioUnlocked = false;
function unlockMediaAudio() {
if (mediaAudioUnlocked) return;
mediaAudioUnlocked = true;
removeEventListener("pointerdown", unlockMediaAudio, true);
removeEventListener("keydown", unlockMediaAudio, true);
removeEventListener("touchstart", unlockMediaAudio, true);
// Play the current src (if any) inside the gesture to whitelist the
// element for autoplay, so poll-driven play() later isn't blocked.
if (mediaAudio.getAttribute("src") && mediaAudio.paused) {
const p = mediaAudio.play();
if (p) p.catch(() => {});
}
}
addEventListener("pointerdown", unlockMediaAudio, true);
addEventListener("keydown", unlockMediaAudio, true);
addEventListener("touchstart", unlockMediaAudio, true);
mediaAudio.addEventListener("play", () => {
updateMediaControls();
reportMediaPlayback("playing");
publishMediaState("playing");
});
mediaAudio.addEventListener("pause", () => {
updateMediaControls();
if (mediaCurrentTrackId) {
reportMediaPlayback("paused");
publishMediaState("paused");
}
});
mediaAudio.addEventListener("ended", () => {
updateMediaControls();
reportMediaPlayback("stopped");
publishMediaState("stopped");
mediaPost("/control", { action: "stop" }).catch(() => {});
});
mediaAudio.addEventListener("timeupdate", () => {
updateSeekUI();
const now = Date.now();
if (!mediaAudio.paused && now - mediaLastReport > 5000) {
mediaLastReport = now;
reportMediaPlayback("playing");
}
});
mediaAudio.addEventListener("durationchange", updateSeekUI);
mediaAudio.addEventListener("error", () => {
// An empty src (track still downloading) also fires this — only surface
// real playback failures.
if (mediaCurrentTrackId && mediaAudio.getAttribute("src")) {
mediaChannelEl.textContent = "Playback error — the track may still be preparing.";
}
});
// ── Search / choices / library ──────────────────────────────────────────────
function fmtListDuration(sec) {
const s = Number(sec);
return isFinite(s) && s > 0 ? fmtTime(s) : "";
}
function mediaRowActions() {
const play = document.createElement("button");
play.className = "btn btn-secondary media-row-play";
play.type = "button";
play.textContent = "Play";
return play;
}
function renderMediaResults(results) {
mediaResultsEl.innerHTML = "";
if (!results || results.length === 0) {
mediaResultsEl.innerHTML = '<div class="media-empty">No results.</div>';
return;
}
for (const r of results) {
const row = document.createElement("div");
row.className = "media-row";
const info = document.createElement("div");
info.className = "media-row-info";
const dur = fmtListDuration(r.duration);
info.innerHTML = `<strong>${escapeHtml(r.title)}</strong>` +
`<span>${escapeHtml(r.channel || "")}${dur ? " · " + dur : ""}` +
`${r.in_library ? " · in library" : ""}</span>`;
const play = mediaRowActions();
play.addEventListener("click", () => playVideoId(r));
row.appendChild(info);
row.appendChild(play);
mediaResultsEl.appendChild(row);
}
}
async function playVideoId(r) {
try {
await mediaPost("/play", {
video_id: r.video_id,
title: r.title,
channel: r.channel || "",
});
} catch (e) {
mediaChannelEl.textContent = `Could not start: ${e.message}`;
}
}
mediaSearchForm.addEventListener("submit", async (event) => {
event.preventDefault();
const query = mediaSearchInput.value.trim();
if (!query) return;
mediaResultsEl.innerHTML = '<div class="media-empty">Searching YouTube Music…</div>';
try {
const data = await mediaPost("/search", { query, limit: 6 });
renderMediaResults(data.results);
} catch (e) {
mediaResultsEl.innerHTML = `<div class="media-empty">Search failed: ${escapeHtml(e.message)}</div>`;
}
});
function renderMediaChoices(choices) {
mediaChoicesWrap.hidden = choices.length === 0;
mediaChoicesEl.innerHTML = "";
for (const c of choices) {
const row = document.createElement("div");
row.className = "media-row media-choice";
const info = document.createElement("div");
info.className = "media-row-info";
const dur = fmtListDuration(c.duration);
info.innerHTML = `<strong>${escapeHtml(c.title)}</strong>` +
`<span>${escapeHtml(c.channel || "")}${dur ? " · " + dur : ""}</span>`;
const play = mediaRowActions();
play.addEventListener("click", () => playVideoId(c));
row.appendChild(info);
row.appendChild(play);
mediaChoicesEl.appendChild(row);
}
}
async function refreshLibrary() {
let data;
try {
data = await fetch("/media/api/library").then((r) => r.json());
} catch (e) {
return;
}
mediaLibraryEl.innerHTML = "";
const tracks = data.tracks || [];
if (tracks.length === 0) {
mediaLibraryEl.innerHTML = '<div class="media-empty">Downloaded songs and streams live here.</div>';
return;
}
for (const t of tracks) {
const row = document.createElement("div");
row.className = "media-row";
const info = document.createElement("div");
info.className = "media-row-info";
const dur = fmtListDuration(t.duration);
const statusTag = t.status === "ready" ? "" : ` · ${t.status}`;
info.innerHTML = `<strong>${escapeHtml(t.title)}</strong>` +
`<span>${escapeHtml(t.channel || (t.kind === "stream" ? "Live stream" : ""))}` +
`${dur ? " · " + dur : ""}${statusTag}</span>`;
const play = mediaRowActions();
play.disabled = t.status !== "ready";
play.addEventListener("click", () => {
mediaPost("/play", { track_id: t.id }).catch((e) => {
mediaChannelEl.textContent = `Could not play: ${e.message}`;
});
});
const del = document.createElement("button");
del.className = "btn btn-icon media-row-delete";
del.type = "button";
del.setAttribute("aria-label", `Remove ${t.title}`);
del.title = "Remove from library";
del.innerHTML = '<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>';
del.addEventListener("click", async () => {
try {
await fetch(`/media/api/tracks/${encodeURIComponent(t.id)}`, { method: "DELETE" });
if (t.id === mediaCurrentTrackId) clearMediaAudio();
refreshLibrary();
} catch (e) { /* leave the row; next refresh decides */ }
});
row.appendChild(info);
row.appendChild(play);
row.appendChild(del);
mediaLibraryEl.appendChild(row);
}
}
streamForm.addEventListener("submit", async (event) => {
event.preventDefault();
const name = streamNameInput.value.trim() || streamUrlInput.value.trim();
const url = streamUrlInput.value.trim();
if (!url) return;
try {
await mediaPost("/streams", { name, url });
streamNameInput.value = "";
streamUrlInput.value = "";
refreshLibrary();
} catch (e) {
mediaChannelEl.textContent = `Could not add stream: ${e.message}`;
}
});
// ── Panel open/close ────────────────────────────────────────────────────────
function setMediaOpen(open) {
mediaPanel.classList.toggle("open", open);
mediaBtn.setAttribute("aria-expanded", String(open));
if (open) {
refreshLibrary();
pollMediaState();
}
}
mediaBtn.setAttribute("aria-expanded", "false");
mediaBtn.addEventListener("click", () => setMediaOpen(!mediaPanel.classList.contains("open")));
closeMediaBtn.addEventListener("click", () => setMediaOpen(false));
startMediaPolling();
updateMediaControls();
buildLedWall(); buildLedWall();
-85
View File
@@ -42,11 +42,6 @@
<span>Tasks</span> <span>Tasks</span>
<span class="tasks-badge" id="tasksBadge" hidden>0</span> <span class="tasks-badge" id="tasksBadge" hidden>0</span>
</button> </button>
<button id="mediaBtn" class="btn btn-secondary tasks-trigger" aria-label="Music player" title="Music player">
<svg viewBox="0 0 24 24" width="17" height="17" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
<span>Music</span>
<span class="tasks-badge" id="mediaBadge" hidden>&#9835;</span>
</button>
</div> </div>
</header> </header>
@@ -59,7 +54,6 @@
<div class="orb-stage"> <div class="orb-stage">
<div class="led-wall" id="ledWall" data-state="idle" aria-hidden="true"> <div class="led-wall" id="ledWall" data-state="idle" aria-hidden="true">
<img id="ledArt" class="led-art" alt="">
</div> </div>
<div class="orb-labels" aria-live="polite"> <div class="orb-labels" aria-live="polite">
<span class="state-label" id="stateLabel">Ready when you are</span> <span class="state-label" id="stateLabel">Ready when you are</span>
@@ -151,85 +145,6 @@
</div> </div>
</div> </div>
<div class="tasks-panel media-panel" id="mediaPanel" aria-label="Music player">
<div class="tasks-header">
<div>
<h2>Music</h2>
<p>YouTube Music downloads and live streams, streamed to you.</p>
</div>
<button id="closeMediaBtn" class="btn btn-icon" aria-label="Close music player">
<svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>
</button>
</div>
<div class="media-now">
<img id="mediaArt" class="media-art" alt="" hidden>
<div class="media-art-placeholder" id="mediaArtPlaceholder" aria-hidden="true">
<svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
</div>
<div class="media-now-info">
<strong id="mediaTitle">Nothing loaded</strong>
<span id="mediaChannel"></span>
<div class="media-progress" id="mediaProgressWrap" hidden>
<div class="media-progress-fill" id="mediaProgressFill"></div>
<span class="media-progress-label" id="mediaProgressLabel">Fetching&hellip;</span>
</div>
</div>
</div>
<div class="media-controls" role="group" aria-label="Playback controls">
<button id="rewBtn" class="btn btn-icon" aria-label="Back 10 seconds" title="Back 10s">
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 17a5 5 0 1 0-5-5"></path><path d="M11 17H6v-5"></path><path d="m19 5-4 4h-3"></path><path d="m15 9 4-4"></path></svg>
</button>
<button id="playPauseBtn" class="btn btn-icon media-play" aria-label="Play or pause" title="Play / pause">
<svg id="playIcon" class="play-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M8 5v14l11-7Z"></path></svg>
<svg id="pauseIcon" class="pause-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M7 5h4v14H7zM13 5h4v14h-4z"></path></svg>
</button>
<button id="fwdBtn" class="btn btn-icon" aria-label="Forward 10 seconds" title="Forward 10s">
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M13 17a5 5 0 1 1 5-5"></path><path d="M13 17h5v-5"></path><path d="m5 5 4 4h3"></path><path d="m9 9 4-4"></path></svg>
</button>
<button id="stopMediaBtn" class="btn btn-icon" aria-label="Stop" title="Stop">
<svg viewBox="0 0 24 24" width="18" height="18" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"></rect></svg>
</button>
</div>
<div class="media-seek">
<span class="media-time" id="mediaCur">0:00</span>
<input type="range" id="seekBar" min="0" max="1000" value="0" step="1" aria-label="Seek position">
<span class="media-time" id="mediaDur">0:00</span>
</div>
<div class="media-volume">
<svg viewBox="0 0 24 24" width="15" height="15" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 5 6 9H2v6h4l5 4V5Z"></path><path d="M15.5 8.5a5 5 0 0 1 0 7"></path></svg>
<input type="range" id="volumeBar" min="0" max="100" value="90" aria-label="Volume">
</div>
<h3 class="sheet-section-title">Search YouTube Music</h3>
<form class="media-search" id="mediaSearchForm">
<input id="mediaSearchInput" type="search" placeholder="Song, artist, lyrics&hellip;" autocomplete="off">
<button class="btn btn-secondary" type="submit">Find</button>
</form>
<div class="media-list" id="mediaResults"></div>
<div id="mediaChoicesWrap" hidden>
<h3 class="sheet-section-title">Hope&rsquo;s suggestions</h3>
<div class="media-list" id="mediaChoices"></div>
</div>
<h3 class="sheet-section-title">Library</h3>
<div class="media-list" id="mediaLibrary">
<div class="media-empty">Downloaded songs and streams live here.</div>
</div>
<h3 class="sheet-section-title">Add live stream</h3>
<form class="media-search media-stream-form" id="streamForm">
<input id="streamName" type="text" placeholder="Station name" autocomplete="off">
<input id="streamUrl" type="url" placeholder="https://&hellip; audio stream URL" required>
<button class="btn btn-secondary" type="submit">Add</button>
</form>
</div>
<audio id="mediaAudio" preload="metadata"></audio>
<script src="livekit-client.umd.js"></script> <script src="livekit-client.umd.js"></script>
<script src="app.js"></script> <script src="app.js"></script>
</body> </body>
-810
View File
@@ -1,810 +0,0 @@
#!/usr/bin/env python3
"""Media server for the Hope voice assistant.
A small FastAPI service that owns the permanent music library and player
coordination:
- Library: downloaded YouTube Music tracks (raw best-audio files, saved by
yt-dlp with no transcode) and user-added live audio stream URLs, persisted
in library.json.
- Search: yt-dlp ``ytsearch`` (YouTube Music pages share video ids), used by
Hope's media MCP toolset and by the web UI search box. Ready library tracks
that match the query are returned first so Hope prefers a downloaded copy.
- Download jobs: one-at-a-time worker thread; progress is parsed from
yt-dlp stderr and surfaced through GET /media/api/state. Playback starts
while the file is still growing (progressive streaming), and the finished
file stays in the library for instant replay.
- Player state: per-room desired state (loaded track, playing/paused/stopped,
seek hints). The browser is the actual player and reconciles against this;
it reports real playback state back so Hope's voice tools can answer
"what's playing".
Security posture matches the token server: bind 127.0.0.1 only, reachability
exclusively through the nginx HTTPS proxy (/media/api/). Growing downloads
are streamed by this service (chunked, no Content-Length); finished files are
served here too with native Range support for seeking. Live stream URLs are
proxied (mixed-content: an http:// radio URL would be blocked by the browser,
so it is fetched server-side and re-served over HTTPS).
"""
from __future__ import annotations
import asyncio
import json
import os
import re
import subprocess
import sys
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from typing import Any
import httpx
import uvicorn
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/data/media"))
FILES_DIR = MEDIA_DIR / "files"
THUMBS_DIR = MEDIA_DIR / "thumbs"
JOBS_DIR = MEDIA_DIR / "jobs"
LIBRARY_FILE = MEDIA_DIR / "library.json"
COOKIES_FILE = os.environ.get("YTDLP_COOKIES", "")
MEDIA_HOST = os.environ.get("MEDIA_SERVER_HOST", "127.0.0.1")
# 8093, NOT 8092: 8092 is occupied by an Apache/YOURLS stack on the Unraid
# deployment host (xNAS) — a bind conflict there looks like a healthy
# deployment because the healthcheck curl (without -f) accepts any response.
MEDIA_PORT = int(os.environ.get("MEDIA_SERVER_PORT", "8093"))
DOWNLOAD_TIMEOUT = int(os.environ.get("MEDIA_DOWNLOAD_TIMEOUT", "600"))
SEARCH_TIMEOUT = int(os.environ.get("MEDIA_SEARCH_TIMEOUT", "45"))
app = FastAPI(title="hope-media", docs_url=None, redoc_url=None, openapi_url=None)
_lock = threading.Lock() # guards library + rooms below
_library: dict[str, Any] = {"tracks": {}, "version": 1}
_rooms: dict[str, dict[str, Any]] = {}
_jobs: dict[str, dict[str, Any]] = {}
_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="media-dl")
for _d in (FILES_DIR, THUMBS_DIR, JOBS_DIR):
_d.mkdir(parents=True, exist_ok=True)
# The bind mount may be inspected/cleaned from the host by a non-root
# user (Unraid share convention) — keep the dirs world-traversable.
try:
os.chmod(_d, 0o777)
except OSError:
pass
# ── Library store ───────────────────────────────────────────────────────────
def _load_library() -> None:
if LIBRARY_FILE.exists():
try:
data = json.loads(LIBRARY_FILE.read_text(encoding="utf-8"))
tracks = data.get("tracks", {})
if isinstance(tracks, dict):
_library["tracks"] = tracks
_library["version"] = int(data.get("version", 1))
except (json.JSONDecodeError, OSError):
# Corrupt library: keep the broken file aside, start fresh.
try:
LIBRARY_FILE.replace(LIBRARY_FILE.with_suffix(".json.broken"))
except OSError:
pass
def _save_library() -> None:
tmp = LIBRARY_FILE.with_suffix(".json.tmp")
tmp.write_text(json.dumps(_library), encoding="utf-8")
tmp.replace(LIBRARY_FILE)
def _bump_version() -> int:
_library["version"] = int(_library.get("version", 1)) + 1
return _library["version"]
def _track_dict(track: dict) -> dict:
out = dict(track)
if track["kind"] == "youtube":
# The stream endpoint serves the growing file during download and the
# finished file afterwards; the URL is stable across the transition
# so the browser never has to reload mid-playback.
if track.get("status") in ("downloading", "ready"):
out["audio_url"] = f"/media/api/stream/{track['id']}"
out["art_url"] = (
f"/media/thumbs/{track['id']}.jpg"
if (THUMBS_DIR / f"{track['id']}.jpg").exists()
else None
)
else:
out["audio_url"] = f"/media/api/stream/{track['id']}"
return out
def _get_track(track_id: str) -> dict | None:
return _library["tracks"].get(track_id)
def _upsert_track(track: dict) -> None:
_library["tracks"][track["id"]] = track
_bump_version()
_save_library()
_load_library()
# ── Player state (per room) ─────────────────────────────────────────────────
def _room_state(room: str) -> dict:
return _rooms.setdefault(
room,
{"track_id": None, "desired": "stopped", "reported": None,
"seek_hint": None, "choices": []},
)
def _room_snapshot(room: str) -> dict:
st = _room_state(room)
track = _get_track(st["track_id"]) if st["track_id"] else None
pending = [
{
"job_id": j["id"],
"track_id": j["track_id"],
"stage": j["stage"],
"progress": j["progress"],
"error": j.get("error"),
}
for j in _jobs.values()
if j["stage"] in ("queued", "downloading", "transcoding")
]
return {
"room": room,
"track": _track_dict(track) if track else None,
"desired": st["desired"],
"reported": st["reported"],
"seek_hint": st["seek_hint"],
"choices": st["choices"],
"pending": pending,
"version": _library["version"],
}
# ── yt-dlp helpers ──────────────────────────────────────────────────────────
def _ytdlp_base_args() -> list[str]:
args = [sys.executable, "-m", "yt_dlp", "--no-playlist", "--no-warnings"]
if COOKIES_FILE and Path(COOKIES_FILE).exists():
args += ["--cookies", COOKIES_FILE]
return args
_PROGRESS_RE = re.compile(r"\[download\]\s+([\d.]+)%")
# Audio containers yt-dlp can produce for a raw best-audio download.
_AUDIO_EXTS = {
".webm", ".opus", ".ogg", ".m4a", ".mp4", ".m4b", ".aac",
".mp3", ".mka", ".flac", ".wav",
}
_AUDIO_MIME = {
".webm": "audio/webm",
".opus": "audio/opus",
".ogg": "audio/ogg",
".m4a": "audio/mp4",
".mp4": "audio/mp4",
".m4b": "audio/mp4",
".aac": "audio/aac",
".mp3": "audio/mpeg",
".mka": "audio/x-matroska",
".flac": "audio/flac",
".wav": "audio/wav",
}
def _media_type_for(name: str) -> str:
if name.endswith(".part"):
name = name[:-5]
return _AUDIO_MIME.get(Path(name).suffix.lower(), "audio/mpeg")
def _youtube_media_file(track_id: str) -> Path | None:
"""The on-disk audio file for a youtube track, if any.
During an active download yt-dlp writes ``{id}.{ext}.part`` and renames it
to ``{id}.{ext}`` on completion — return whichever currently exists (the
partial file wins, thumbnails and other sidecars are ignored).
"""
complete: Path | None = None
for p in FILES_DIR.glob(f"{track_id}.*"):
suffix = p.suffix.lower()
if suffix == ".part":
if p.stat().st_size > 0:
return p
elif suffix in _AUDIO_EXTS:
if complete is None or p.name < complete.name:
complete = p
return complete
async def _wait_for_media_file(track_id: str, timeout: float = 15.0) -> Path | None:
"""Wait for yt-dlp to create the output file after a play request."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
path = _youtube_media_file(track_id)
if path is not None:
return path
await asyncio.sleep(0.5)
return None
async def _growing_file(track_id: str, job: dict, chunk_size: int = 64 * 1024):
"""Yield bytes from a file that yt-dlp is still writing.
The response stays open with no Content-Length until the download
finishes, so the browser plays the track as it grows. The open descriptor
keeps reading the old inode across yt-dlp's .part -> final rename, and the
path is re-resolved if the file disappears between resolution and open.
"""
fd = -1
pos = 0
stall = 0
try:
while True:
if fd == -1:
path = _youtube_media_file(track_id)
if path is not None:
try:
fd = os.open(path, os.O_RDONLY)
except FileNotFoundError:
fd = -1 # renamed between glob and open — retry
if fd != -1:
data = await asyncio.to_thread(os.pread, fd, chunk_size, pos)
if data:
pos += len(data)
stall = 0
yield data
continue
with _lock:
active = job.get("stage") == "downloading"
if not active:
break
stall += 1
if stall > 60: # ~30s with no new bytes — stop waiting
break
await asyncio.sleep(0.5)
finally:
if fd != -1:
os.close(fd)
def _run_search(query: str, limit: int) -> list[dict]:
"""Flat ytsearch lookup — cheap, no download, works anonymously."""
cmd = _ytdlp_base_args() + [
"--flat-playlist", "--dump-json",
f"ytsearch{limit}:{query}",
]
try:
proc = subprocess.run(
cmd, capture_output=True, text=True, timeout=SEARCH_TIMEOUT, check=False
)
except subprocess.TimeoutExpired:
raise HTTPException(504, "YouTube Music search timed out")
if proc.returncode != 0 and not proc.stdout.strip():
detail = (proc.stderr or "").strip().splitlines()[-1:] or ["unknown error"]
raise HTTPException(502, f"Search failed: {detail[0][:300]}")
results = []
for line in proc.stdout.splitlines():
line = line.strip()
if not line.startswith("{"):
continue
try:
entry = json.loads(line)
except json.JSONDecodeError:
continue
video_id = entry.get("id")
if not video_id:
continue
existing = _get_track(video_id)
results.append({
"video_id": video_id,
"title": entry.get("title") or video_id,
"channel": entry.get("uploader") or entry.get("channel") or "",
"duration": entry.get("duration"),
"view_count": entry.get("view_count"),
"in_library": bool(existing),
"track_id": existing["id"] if existing else None,
"library_status": existing["status"] if existing else None,
})
return results
def _job_dict(job: dict) -> dict:
return {k: job.get(k) for k in
("id", "track_id", "stage", "progress", "error", "started_at")}
def _persist_job(job: dict) -> None:
try:
(JOBS_DIR / f"{job['id']}.json").write_text(
json.dumps(_job_dict(job)), encoding="utf-8"
)
except OSError:
pass
def _fail_job(job: dict, track: dict, message: str) -> None:
job["stage"] = "error"
job["error"] = message[:500]
_persist_job(job)
with _lock:
track["status"] = "failed"
track["error"] = message[:500]
_bump_version()
_save_library()
def _download_worker(job: dict, track: dict, video_id: str) -> None:
"""Runs in the single-worker executor: raw best-audio download.
No mp3 transcode — the audio container is streamed straight into the
library so playback can begin while the file is still growing (the stream
endpoint serves the partial file), and the finished file stays playable
offline. Thumbnail handling is unchanged.
"""
cmd = _ytdlp_base_args() + [
"-f", "bestaudio/best",
"--newline", "--progress",
"--print-json",
"--write-thumbnail", "--convert-thumbnails", "jpg",
"--retries", "3", "--fragment-retries", "3",
"-P", str(FILES_DIR),
"-o", f"{track['id']}.%(ext)s",
f"https://www.youtube.com/watch?v={video_id}",
]
job["stage"] = "downloading"
info: dict = {}
# Progress lines go to stderr, the metadata JSON to stdout. Drain stderr
# in a thread so the stdout read loop can never deadlock the pipe; the
# tail is kept for the failure detail.
stderr_tail: list[str] = []
proc = None
def _drain_stderr() -> None:
assert proc is not None and proc.stderr is not None
for line in proc.stderr:
stderr_tail.append(line)
if len(stderr_tail) > 200:
stderr_tail.pop(0)
m = _PROGRESS_RE.search(line)
if m:
job["progress"] = min(99.0, float(m.group(1)))
_persist_job(job)
try:
proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, errors="replace",
)
threading.Thread(target=_drain_stderr, daemon=True).start()
deadline = time.monotonic() + DOWNLOAD_TIMEOUT
assert proc.stdout is not None
for line in proc.stdout:
if time.monotonic() > deadline:
proc.kill()
proc.wait(timeout=30)
raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
line = line.strip()
if line.startswith("{"):
try:
info = json.loads(line)
except json.JSONDecodeError:
pass
proc.wait(timeout=30)
if proc.returncode != 0:
detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
raise RuntimeError(detail.strip())
media = _youtube_media_file(track["id"])
if media is None:
raise RuntimeError("yt-dlp finished but no audio file was produced")
except Exception as exc: # noqa: BLE001 — any failure marks the track failed
_fail_job(job, track, str(exc))
return
thumb_src = FILES_DIR / f"{track['id']}.jpg"
try:
if thumb_src.exists():
thumb_src.replace(THUMBS_DIR / f"{track['id']}.jpg")
except OSError:
pass
job["stage"] = "ready"
job["progress"] = 100.0
_persist_job(job)
with _lock:
track["status"] = "ready"
track["error"] = None
track["ext"] = media.suffix.lstrip(".").lower()
if info:
if info.get("title"):
track["title"] = info["title"]
if info.get("uploader") or info.get("channel"):
track["channel"] = info.get("uploader") or info.get("channel")
if info.get("duration"):
track["duration"] = round(float(info["duration"]))
_bump_version()
_save_library()
def _start_download(track: dict, video_id: str) -> dict:
job = {
"id": uuid.uuid4().hex[:12],
"track_id": track["id"],
"stage": "queued",
"progress": 0.0,
"error": None,
"started_at": time.time(),
}
_jobs[job["id"]] = job
_persist_job(job)
_executor.submit(_download_worker, job, track, video_id)
return job
# ── API: health, library ────────────────────────────────────────────────────
@app.get("/media/api/healthz")
def healthz() -> dict:
return {"ok": True, "version": _library["version"]}
@app.get("/media/api/library")
def get_library() -> dict:
with _lock:
tracks = sorted(
(_track_dict(t) for t in _library["tracks"].values()),
key=lambda t: t.get("added_at", 0),
reverse=True,
)
return {"tracks": tracks, "version": _library["version"]}
def _delete_track_files(track: dict) -> None:
if track["kind"] != "youtube":
return
paths = [
FILES_DIR / f"{track['id']}.{ext}" for ext in
(s.lstrip(".") for s in _AUDIO_EXTS)
] + [FILES_DIR / f"{track['id']}.jpg", THUMBS_DIR / f"{track['id']}.jpg"]
for path in paths:
try:
path.unlink(missing_ok=True)
except OSError:
pass
@app.delete("/media/api/tracks/{track_id}")
def delete_track(track_id: str) -> dict:
with _lock:
track = _get_track(track_id)
if not track:
raise HTTPException(404, "unknown track")
del _library["tracks"][track_id]
_bump_version()
_save_library()
for room, st in _rooms.items():
if st["track_id"] == track_id:
st["track_id"] = None
st["desired"] = "stopped"
_delete_track_files(track)
return {"ok": True}
# ── API: search + play + control ────────────────────────────────────────────
def _library_search(query: str, limit: int) -> list[dict]:
"""Ready library tracks whose title matches the query.
These are merged in front of the YouTube results so Hope prefers an
already-downloaded copy over fetching a new one.
"""
q = query.strip().lower()
tokens = {t for t in re.split(r"\W+", q) if len(t) > 2}
hits: list[dict] = []
if not q:
return hits
for track in _library["tracks"].values():
if track.get("kind") != "youtube" or track.get("status") != "ready":
continue
title = (track.get("title") or "").strip().lower()
if not title:
continue
title_tokens = {t for t in re.split(r"\W+", title) if t}
if not (q in title or title in q or (tokens and tokens <= title_tokens)):
continue
hits.append({
"video_id": track["id"],
"track_id": track["id"],
"title": track.get("title") or track["id"],
"channel": track.get("channel") or "",
"duration": track.get("duration"),
"view_count": None,
"in_library": True,
"library_status": "ready",
})
return hits[:limit]
@app.post("/media/api/search")
async def search(request: Request) -> dict:
body = await request.json()
query = str(body.get("query", "")).strip()
if not query:
raise HTTPException(400, "query is required")
try:
limit = max(1, min(int(body.get("limit", 6)), 15))
except (TypeError, ValueError):
raise HTTPException(400, "limit must be an integer")
local = _library_search(query, limit)
remote = _run_search(query, limit)
seen = {r["video_id"] for r in local}
merged = local + [r for r in remote if r["video_id"] not in seen]
return {"query": query, "results": merged[:limit]}
def _resolve_play_target(body: dict) -> dict:
"""Find or create the track a play request refers to.
Accepts video_id (YouTube), track_id (existing library entry) or a raw
url+name (live stream). Returns the library track dict.
"""
video_id = str(body.get("video_id") or "").strip()
track_id = str(body.get("track_id") or "").strip()
url = str(body.get("url") or "").strip()
if video_id:
existing = _get_track(video_id)
if existing:
return existing
track = {
"id": video_id,
"kind": "youtube",
"video_id": video_id,
"title": str(body.get("title") or video_id),
"channel": str(body.get("channel") or ""),
"duration": body.get("duration"),
"status": "downloading",
"error": None,
"added_at": time.time(),
}
_upsert_track(track)
_start_download(track, video_id)
return track
if track_id:
track = _get_track(track_id)
if not track:
raise HTTPException(404, "unknown track")
return track
if url:
if not re.match(r"^https?://", url):
raise HTTPException(400, "url must be http(s)")
track_id = "s" + uuid.uuid4().hex[:10]
track = {
"id": track_id,
"kind": "stream",
"video_id": None,
"title": str(body.get("name") or url),
"channel": str(body.get("channel") or ""),
"duration": None,
"status": "ready",
"error": None,
"url": url,
"added_at": time.time(),
}
_upsert_track(track)
return track
raise HTTPException(400, "one of video_id, track_id or url is required")
@app.post("/media/api/play")
async def play(request: Request) -> dict:
body = await request.json()
room = str(body.get("room") or "default")
track = _resolve_play_target(body)
with _lock:
st = _room_state(room)
st["track_id"] = track["id"]
# start_paused carries a track into a new scope without starting it —
# the UI uses this when a conversation begins (talking pauses music).
st["desired"] = "paused" if body.get("start_paused") else "playing"
st["choices"] = [] # a pick resolves any open suggestion list
st["seek_hint"] = None
snapshot = _room_snapshot(room)
return {"ok": True, **snapshot}
@app.post("/media/api/control")
async def control(request: Request) -> dict:
body = await request.json()
room = str(body.get("room") or "default")
action = str(body.get("action") or "")
with _lock:
st = _room_state(room)
if action == "pause":
st["desired"] = "paused"
elif action == "resume":
if not st["track_id"]:
raise HTTPException(409, "nothing loaded")
st["desired"] = "playing"
elif action == "stop":
st["desired"] = "stopped"
elif action == "seek":
try:
position = max(0.0, float(body.get("position", 0)))
except (TypeError, ValueError):
raise HTTPException(400, "position must be a number")
st["seek_hint"] = {"position": position, "ts": time.time()}
elif action == "skip":
current = st["track_id"]
candidates = sorted(
(
t for t in _library["tracks"].values()
if t.get("status") == "ready" and t["id"] != current
),
key=lambda t: t.get("added_at", 0),
reverse=True,
)
if candidates:
st["track_id"] = candidates[0]["id"]
st["desired"] = "playing"
st["seek_hint"] = None
else:
st["desired"] = "stopped"
else:
raise HTTPException(400, f"unknown action: {action}")
snapshot = _room_snapshot(room)
return {"ok": True, **snapshot}
@app.post("/media/api/choices")
async def choices(request: Request) -> dict:
"""Publish Hope's song suggestions so the UI can render clickable picks."""
body = await request.json()
room = str(body.get("room") or "default")
raw = body.get("choices") or []
cleaned = []
for c in raw[:5]:
if not isinstance(c, dict) or not c.get("video_id"):
continue
cleaned.append({
"video_id": str(c["video_id"]),
"title": str(c.get("title") or c["video_id"]),
"channel": str(c.get("channel") or ""),
"duration": c.get("duration"),
})
with _lock:
_room_state(room)["choices"] = cleaned
return {"ok": True, "choices": cleaned}
@app.get("/media/api/state")
def state(room: str = "default") -> dict:
with _lock:
return _room_snapshot(room)
@app.post("/media/api/report")
async def report(request: Request) -> dict:
"""The browser reports real playback state (source of truth for the player)."""
body = await request.json()
room = str(body.get("room") or "default")
try:
position = float(body.get("position") or 0.0)
except (TypeError, ValueError):
position = 0.0
with _lock:
st = _room_state(room)
st["reported"] = {
"track_id": body.get("track_id"),
"state": str(body.get("state") or "stopped"),
"position": position,
"duration": body.get("duration"),
"ts": time.time(),
}
return {"ok": True}
@app.post("/media/api/streams")
async def add_stream(request: Request) -> dict:
body = await request.json()
track = _resolve_play_target({"url": body.get("url"), "name": body.get("name")})
return {"ok": True, "track": _track_dict(track)}
# ── API: stream proxy (external http(s) audio over same-origin HTTPS) ───────
_HOP_HEADERS = ("content-type", "content-range", "accept-ranges", "content-length")
@app.get("/media/api/stream/{track_id}")
async def stream_proxy(track_id: str, request: Request):
with _lock:
track = _get_track(track_id)
job = next(
(j for j in _jobs.values()
if j["track_id"] == track_id and j["stage"] == "downloading"),
None,
)
if not track:
raise HTTPException(404, "unknown track")
if track["kind"] == "youtube":
# Progressive download: an active download is streamed in real time
# (chunked, no Content-Length) so playback starts while the file
# grows; a finished file is served directly with native Range support.
path = _youtube_media_file(track_id)
if path is None and job is not None:
path = await _wait_for_media_file(track_id)
if path is None:
raise HTTPException(
404, f"audio file not available (track is {track.get('status')})"
)
if job is not None:
return StreamingResponse(
_growing_file(track_id, job),
media_type=_media_type_for(path.name),
headers={"Cache-Control": "no-store"},
)
return FileResponse(path, media_type=_media_type_for(path.name))
range_header = request.headers.get("range")
headers = {"Range": range_header} if range_header else {}
client = httpx.AsyncClient(timeout=httpx.Timeout(30, read=600), follow_redirects=True)
try:
upstream = await client.send(client.build_request("GET", track["url"], headers=headers), stream=True)
except httpx.HTTPError as exc:
await client.aclose()
raise HTTPException(502, f"upstream stream failed: {exc}")
if upstream.status_code >= 400:
await upstream.aclose()
await client.aclose()
raise HTTPException(502, f"upstream returned {upstream.status_code}")
out_headers = {
name: value
for name, value in upstream.headers.items()
if name.lower() in _HOP_HEADERS
}
media_type = upstream.headers.get("content-type", "audio/mpeg")
async def relay():
try:
async for chunk in upstream.aiter_bytes(64 * 1024):
yield chunk
finally:
await upstream.aclose()
await client.aclose()
return StreamingResponse(
relay(),
status_code=upstream.status_code,
media_type=media_type,
headers=out_headers,
)
@app.exception_handler(HTTPException)
async def http_error(_request: Request, exc: HTTPException):
return JSONResponse({"error": exc.detail}, status_code=exc.status_code)
if __name__ == "__main__":
uvicorn.run(app, host=MEDIA_HOST, port=MEDIA_PORT, log_level="info")
+1 -300
View File
@@ -371,34 +371,6 @@ button {
transition: opacity 90ms linear, width 90ms linear, height 90ms linear; transition: opacity 90ms linear, width 90ms linear, height 90ms linear;
} }
/* Album art fills the square while a track is loaded outside a conversation. */
.led-art {
position: absolute;
inset: 12px;
width: calc(100% - 24px);
height: calc(100% - 24px);
object-fit: cover;
border-radius: 18px;
z-index: 1;
opacity: 0;
transform: scale(0.96);
transition: opacity 240ms ease, transform 240ms ease;
pointer-events: none;
}
.led-wall.show-art .led-art {
opacity: 1;
transform: scale(1);
}
.led-wall.show-art .led-dot {
opacity: 0 !important;
transition: none;
}
.led-wall.show-art {
filter: none;
}
/* Palette shifts by state — a cool blue while it listens, warm toward the /* Palette shifts by state — a cool blue while it listens, warm toward the
center as the agent speaks, amber while thinking, dim when idle. */ center as the agent speaks, amber while thinking, dim when idle. */
@@ -1201,276 +1173,6 @@ body[data-mode="text"] .mic-meter {
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); } .task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); } .task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
/* ── Media player ───────────────────────────────────────────────────────── */
/* Reuses .tasks-panel positioning (fixed right, slides in via .open); the
content scrolls under a sticky header. */
.media-panel {
overflow-y: auto;
padding-bottom: 1.2rem;
}
.media-panel .tasks-header {
position: sticky;
top: 0;
z-index: 1;
background: #0d1929;
}
.media-now {
display: flex;
align-items: center;
gap: 0.75rem;
margin: 1rem 1.1rem 0;
padding: 0.75rem;
border-radius: 12px;
background: var(--surface-raised);
}
.media-art,
.media-art-placeholder {
width: 56px;
height: 56px;
flex: 0 0 56px;
border-radius: 8px;
object-fit: cover;
background: rgba(23, 41, 65, 0.7);
}
.media-art-placeholder {
display: flex;
align-items: center;
justify-content: center;
color: var(--ink-muted);
}
.media-now-info {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
gap: 0.15rem;
}
.media-now-info strong {
color: var(--ink);
font-size: 0.88rem;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.media-now-info span {
color: var(--ink-muted);
font-size: 0.72rem;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.media-progress {
position: relative;
margin-top: 0.35rem;
display: flex;
align-items: center;
gap: 0.5rem;
}
/* Track bar behind the fill (fill width is driven inline by progress). */
.media-progress::before {
content: "";
position: absolute;
left: 0;
top: 50%;
transform: translateY(-50%);
width: 90px;
height: 4px;
border-radius: 2px;
background: rgba(139, 167, 255, 0.18);
}
.media-progress-fill {
height: 4px;
flex: 0 0 auto;
width: 0;
border-radius: 2px;
background: var(--iris);
transition: width 400ms ease;
}
.media-progress-label {
color: var(--ink-muted);
font-size: 0.66rem;
white-space: nowrap;
}
.media-controls {
display: flex;
align-items: center;
justify-content: center;
gap: 0.9rem;
margin: 0.9rem 1.1rem 0;
}
.media-controls .btn-icon {
color: var(--ink-soft);
}
.media-play {
width: 52px;
height: 52px;
border-radius: 50%;
display: inline-flex;
align-items: center;
justify-content: center;
color: #0b1523 !important;
background: var(--iris) !important;
border: none;
}
.media-play .play-icon,
.media-play .pause-icon {
display: none;
}
.media-play:not(.playing) .play-icon {
display: inline-block;
}
.media-play.playing .pause-icon {
display: inline-block;
}
.media-play:hover:not(:disabled) {
filter: brightness(1.1);
}
.media-seek {
display: flex;
align-items: center;
gap: 0.55rem;
margin: 0.75rem 1.1rem 0;
}
.media-time {
color: var(--ink-muted);
font-size: 0.68rem;
font-variant-numeric: tabular-nums;
min-width: 2.6rem;
text-align: center;
}
.media-seek input[type="range"],
.media-volume input[type="range"] {
flex: 1;
accent-color: var(--iris);
cursor: pointer;
}
.media-volume {
display: flex;
align-items: center;
gap: 0.55rem;
margin: 0.55rem 1.1rem 0;
color: var(--ink-muted);
}
.media-search {
display: flex;
gap: 0.45rem;
margin: 0 1.1rem;
}
.media-search input {
flex: 1;
min-width: 0;
padding: 0.5rem 0.7rem;
border: 1px solid var(--line-strong);
border-radius: 9px;
color: var(--ink);
background: rgba(5, 11, 20, 0.45);
font-size: 0.8rem;
}
.media-search input::placeholder {
color: var(--ink-muted);
}
.media-stream-form input:first-child {
flex: 0 0 38%;
}
.media-list {
display: flex;
flex-direction: column;
gap: 0.4rem;
margin: 0.65rem 1.1rem 1rem;
max-height: 240px;
overflow-y: auto;
}
.media-row {
display: flex;
align-items: center;
gap: 0.6rem;
padding: 0.55rem 0.65rem;
border-radius: 9px;
background: var(--surface-raised);
}
.media-row-info {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
gap: 0.1rem;
}
.media-row-info strong {
color: var(--ink);
font-size: 0.78rem;
font-weight: 600;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.media-row-info span {
color: var(--ink-muted);
font-size: 0.68rem;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.media-row-play {
flex: 0 0 auto;
padding: 0.3rem 0.7rem;
font-size: 0.7rem;
}
.media-row-delete {
flex: 0 0 auto;
color: var(--ink-muted);
}
.media-row-delete:hover {
color: #ffc2ba;
}
.media-empty {
padding: 0.8rem;
border-radius: 9px;
color: var(--ink-muted);
background: var(--surface-raised);
font-size: 0.74rem;
text-align: center;
}
.media-choice {
border: 1px solid rgba(139, 167, 255, 0.35);
background: rgba(60, 95, 201, 0.12);
}
/* ── Responsive ─────────────────────────────────────────────────────────── */ /* ── Responsive ─────────────────────────────────────────────────────────── */
@media (max-height: 760px) and (min-width: 881px) { @media (max-height: 760px) and (min-width: 881px) {
.workspace { .workspace {
@@ -1614,8 +1316,7 @@ body[data-mode="text"] .mic-meter {
padding: 1.25rem 1rem; padding: 1.25rem 1rem;
} }
.tasks-panel, .tasks-panel {
.media-panel {
width: 100vw; width: 100vw;
} }
+2 -2
View File
@@ -97,8 +97,8 @@ class Handler(BaseHTTPRequestHandler):
resp = json.dumps( resp = json.dumps(
{ {
"token": token, "token": token,
# The room name scopes media-server player state; the UI sends # Returned for diagnostics and reconnects. Music deliberately
# it with every /media/api/ call. # uses its own stable scope in the separate Hope Music service.
"room": room_name, "room": room_name,
"user": { "user": {
"id": user_id, "id": user_id,