refactor: split music streaming into Hope Music
Rebuild & Deploy / rebuild (push) Successful in 2m7s
Rebuild & Deploy / rebuild (push) Successful in 2m7s
Move the media server, downloader, library volume, and browser player to the standalone hope-media-music product. Keep Hope's music tools as a thin HTTP client of the LAN service on port 8094. Use a stable voice-room player scope instead of ephemeral LiveKit room names. The agent now owns the conversation handshake: fresh reported playback mutes its input, joining pauses music, and the last participant leaving resumes only music it paused. Peer failures remain non-fatal. Also move the voice deploy workflow onto the active Unraid Gitea runner and remove yt-dlp/ffmpeg/media health and volume dependencies from the image.
This commit is contained in:
1 parent
d4ea4fc984
commit
436b3fc8b1
23 files changed
+356
-2518
No files matched your search
+10
-12
@@ -29,18 +29,16 @@ FIRECRAWL_BASE=http://192.168.86.2:3002
|
|||||||
# Per-user memory profiles and Bearer tokens live in config/users.json.
|
# Per-user memory profiles and Bearer tokens live in config/users.json.
|
||||||
VOICE_DEFAULT_USER=shane
|
VOICE_DEFAULT_USER=shane
|
||||||
|
|
||||||
# ── Media player (music library + YouTube Music downloads) ─────────────────
|
# ── Hope Music (separate LAN service) ───────────────────────────────────────
|
||||||
# The permanent library lives in ./media (bind-mounted to /data/media):
|
# The voice agent keeps the music MCP tools, but the library, downloads,
|
||||||
# media/files/ transcoded mp3 tracks
|
# streaming, and browser player live in the hope-media-music product.
|
||||||
# media/thumbs/ cover art
|
MUSIC_BASE_URL=http://192.168.86.2:8094
|
||||||
# media/jobs/ download job status
|
# Stable player scope shared with the standalone Hope Music page. Do not use
|
||||||
# MEDIA_SERVER_PORT=8093 (8092 is taken by YOURLS/Apache on xNAS)
|
# the ephemeral LiveKit room name minted for each conversation.
|
||||||
#
|
MUSIC_ROOM=voice-room
|
||||||
# Optional: export cookies.txt from a browser logged into YouTube and drop it
|
# Player reports are refreshed every few seconds. Older "playing" reports are
|
||||||
# in ./media/cookies.txt to avoid "confirm you're not a bot" download blocks.
|
# treated as stale so a closed player cannot leave Hope's mic disabled.
|
||||||
# Anonymous access works from most residential IPs; cookies are the reliable
|
MUSIC_REPORT_TTL=30
|
||||||
# path.
|
|
||||||
# YTDLP_COOKIES=/data/media/cookies.txt
|
|
||||||
|
|
||||||
# Extra MCP servers (optional). JSON list of {url, transport} objects.
|
# Extra MCP servers (optional). JSON list of {url, transport} objects.
|
||||||
# transport: "sse" | "streamable_http" | omit for auto-detect
|
# transport: "sse" | "streamable_http" | omit for auto-detect
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ on:
|
|||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
rebuild:
|
rebuild:
|
||||||
runs-on: runner-voice
|
runs-on: unraid
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
@@ -26,7 +26,7 @@ jobs:
|
|||||||
--name voice \
|
--name voice \
|
||||||
--network host \
|
--network host \
|
||||||
--restart unless-stopped \
|
--restart unless-stopped \
|
||||||
--health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null' \
|
--health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null' \
|
||||||
--health-interval=15s \
|
--health-interval=15s \
|
||||||
--health-timeout=5s \
|
--health-timeout=5s \
|
||||||
--health-retries=3 \
|
--health-retries=3 \
|
||||||
@@ -34,7 +34,6 @@ jobs:
|
|||||||
-v /mnt/user/dev/hope-voice-api/certs:/etc/voice/certs \
|
-v /mnt/user/dev/hope-voice-api/certs:/etc/voice/certs \
|
||||||
-v /mnt/user/dev/hope-voice-api/skills:/skills \
|
-v /mnt/user/dev/hope-voice-api/skills:/skills \
|
||||||
-v /mnt/user/dev/hope-voice-api/livekit.yaml:/etc/livekit.yaml:ro \
|
-v /mnt/user/dev/hope-voice-api/livekit.yaml:/etc/livekit.yaml:ro \
|
||||||
-v /mnt/user/dev/hope-voice-api/media:/data/media \
|
|
||||||
-e LIVEKIT_URL=http://localhost:7880 \
|
-e LIVEKIT_URL=http://localhost:7880 \
|
||||||
-e LIVEKIT_API_KEY=devkey \
|
-e LIVEKIT_API_KEY=devkey \
|
||||||
-e LIVEKIT_API_SECRET=hopevoice-devsecret-2026-lan-only \
|
-e LIVEKIT_API_SECRET=hopevoice-devsecret-2026-lan-only \
|
||||||
@@ -49,9 +48,8 @@ jobs:
|
|||||||
-e WEB_PORT=9178 \
|
-e WEB_PORT=9178 \
|
||||||
-e VOICE_USERS_FILE=/etc/hope/users.json \
|
-e VOICE_USERS_FILE=/etc/hope/users.json \
|
||||||
-e VOICE_DEFAULT_USER=shane \
|
-e VOICE_DEFAULT_USER=shane \
|
||||||
-e MEDIA_DIR=/data/media \
|
-e MUSIC_BASE_URL=http://192.168.86.2:8094 \
|
||||||
-e MEDIA_SERVER_PORT=8093 \
|
-e MUSIC_ROOM=voice-room \
|
||||||
-e YTDLP_COOKIES=/data/media/cookies.txt \
|
|
||||||
hope-voice-api-voice:latest
|
hope-voice-api-voice:latest
|
||||||
|
|
||||||
- name: Verify deployment
|
- name: Verify deployment
|
||||||
|
|||||||
@@ -8,7 +8,3 @@ certs/
|
|||||||
.venv-tests/
|
.venv-tests/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
.pytest_cache/
|
.pytest_cache/
|
||||||
|
|
||||||
# Permanent music library (media server runtime data)
|
|
||||||
/media/*
|
|
||||||
!/media/.gitkeep
|
|
||||||
@@ -9,16 +9,13 @@ One Docker container runs four processes via supervisord:
|
|||||||
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
||||||
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||||
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
||||||
4. **Media server** — FastAPI music library/player coordinator (port 8093 internal, proxied at /media/api/)
|
4. **Token server** — authenticated LiveKit bootstrap endpoint (port 8091 internal, proxied at `/token`)
|
||||||
|
|
||||||
```
|
```
|
||||||
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
||||||
▲ │
|
▲ │
|
||||||
└────────── audio ◄────────────────────────────┘
|
└────────── audio ◄────────────────────────────┘
|
||||||
Azure STT → LLM (OpenWebUI) → Azure TTS
|
Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||||
|
|
||||||
Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media
|
|
||||||
└────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
@@ -41,10 +38,10 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp`
|
|||||||
|
|
||||||
| Port | Protocol | Service | Access |
|
| Port | Protocol | Service | Access |
|
||||||
|-------|----------|----------------------|--------------|
|
|-------|----------|----------------------|--------------|
|
||||||
|
| 8091 | TCP | Token server | internal only (proxied via nginx at /token) |
|
||||||
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
||||||
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
||||||
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
||||||
| 8093 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) |
|
|
||||||
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
@@ -58,36 +55,32 @@ Key variables:
|
|||||||
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
||||||
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
||||||
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
||||||
- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads
|
- `MUSIC_BASE_URL` — standalone Hope Music LAN API (default `http://192.168.86.2:8094`)
|
||||||
|
- `MUSIC_ROOM` — stable player scope shared with Hope Music (default `voice-room`)
|
||||||
|
|
||||||
## Media Player
|
## Hope Music Integration
|
||||||
|
|
||||||
Hope can play music on the web UI's media player: YouTube Music search →
|
Music is a separate product and deployment: `snowspeeder/hope-media-music`. It
|
||||||
yt-dlp raw best-audio download (webm/m4a, no transcode) → progressive
|
owns the library, YouTube Music search/download pipeline, stream proxy, and
|
||||||
streaming from the media server while the file grows → finished file served
|
the standalone browser player at `https://192.168.86.2:9179`. This repo keeps
|
||||||
with native HTTP Range for seeking. Songs start playing within seconds of the
|
only the agent-side control client.
|
||||||
download starting and remain in the library for instant offline replay.
|
|
||||||
Live audio stream URLs (internet radio) are added to the library and proxied
|
|
||||||
over HTTPS by the media server (mixed-content safe). Search results put ready
|
|
||||||
library matches first so Hope prefers a downloaded copy over a fresh lookup.
|
|
||||||
|
|
||||||
- The permanent library lives in `./media` (bind-mounted to `/data/media`):
|
- Hope's tools live in `agent/media_mcp.py` (MCP, id `media`):
|
||||||
`files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
|
|
||||||
- `web/media_server.py` owns the library, search, download jobs (one at a
|
|
||||||
time), and per-room player desired state. The browser is the actual player:
|
|
||||||
it polls `GET /media/api/state?room=…`, reconciles, and reports real
|
|
||||||
playback back via `POST /media/api/report`.
|
|
||||||
- Hope's tools live in `agent/media_mcp.py` (MCP, id "media"):
|
|
||||||
`search_music`, `play_song`, `offer_song_choices`, `pause_music`,
|
`search_music`, `play_song`, `offer_song_choices`, `pause_music`,
|
||||||
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
|
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
|
||||||
`list_library`, `add_stream`. The MCP process is spawned per session with
|
`list_library`, `add_stream`. They call `MUSIC_BASE_URL` over HTTP.
|
||||||
`MEDIA_ROOM` set to the LiveKit room name, scoping player state per room.
|
- Music uses the stable `MUSIC_ROOM` scope (`voice-room` by default), not the
|
||||||
- **Music and conversation are mutually exclusive.** When a track plays, the
|
ephemeral LiveKit room minted for each conversation. The standalone player
|
||||||
UI reports `{type: "media_state", state: "playing"}` on data-channel topic
|
defaults to that same scope, so voice commands control the open music page
|
||||||
"media-control" and the agent disables its mic (`_apply_listening` in
|
and playback survives across voice sessions.
|
||||||
agent.py). Engaging Hope (Start conversation / text message) pauses the
|
- **Music and conversation remain mutually exclusive.** The agent polls Hope
|
||||||
music; ending the conversation resumes what was playing. Voice commands
|
Music's fresh browser-reported state. Actual playback disables the agent's
|
||||||
(pause/skip/…) work whenever the conversation is active.
|
mic; joining a conversation pauses music, and the last participant leaving
|
||||||
|
resumes only music that the agent paused.
|
||||||
|
- A stale `playing` report (default: thirty seconds) is treated as stopped so
|
||||||
|
closing the music page cannot leave Hope's mic disabled.
|
||||||
|
- Hope Music is a peer service: an outage logs a warning but never takes down
|
||||||
|
the voice conversation. Its health and E2E tests belong in its own repo.
|
||||||
|
|
||||||
## Voice Selection
|
## Voice Selection
|
||||||
|
|
||||||
@@ -153,8 +146,7 @@ identical to a failure). Isolated single-tool probes will pass and mislead.
|
|||||||
via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit
|
via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit
|
||||||
and the one CI (rebuild.yml) deploys on every push to master. The local dev
|
and the one CI (rebuild.yml) deploys on every push to master. The local dev
|
||||||
compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178`
|
compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178`
|
||||||
to target it instead. Tests that can only pass on the deployment host
|
to target it instead. Tests that probe the local container through
|
||||||
(repo `./media` bind mount) or that probe the local container via
|
|
||||||
`docker exec voice` skip automatically on remote targets.
|
`docker exec voice` skip automatically on remote targets.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -164,8 +156,8 @@ docker compose ps
|
|||||||
# Run the suite against the Unraid deployment (default)
|
# Run the suite against the Unraid deployment (default)
|
||||||
python -m pytest tests/ -v -m "not slow"
|
python -m pytest tests/ -v -m "not slow"
|
||||||
|
|
||||||
# Real end-to-end YouTube Music download (network-heavy, gated)
|
# Verify the external Hope Music boundary + client configuration
|
||||||
HOPE_RUN_YTDLP_DOWNLOAD=1 python -m pytest tests/test_media.py -m slow
|
python -m pytest tests/test_media.py -v
|
||||||
|
|
||||||
# Check all logs (single service: "voice")
|
# Check all logs (single service: "voice")
|
||||||
docker compose logs -f voice
|
docker compose logs -f voice
|
||||||
@@ -184,7 +176,7 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
## File Layout
|
## File Layout
|
||||||
|
|
||||||
```
|
```
|
||||||
~/dev/voice/
|
~/dev/hope-voice-api/
|
||||||
├── AGENTS.md ← you are here
|
├── AGENTS.md ← you are here
|
||||||
├── .env.example ← config template (copy to .env)
|
├── .env.example ← config template (copy to .env)
|
||||||
├── .gitignore
|
├── .gitignore
|
||||||
@@ -192,24 +184,18 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
├── Dockerfile ← multi-stage build
|
├── Dockerfile ← multi-stage build
|
||||||
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
||||||
├── livekit.yaml ← LiveKit server config
|
├── livekit.yaml ← LiveKit server config
|
||||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy + /media/ (static + API)
|
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy
|
||||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server, media-server)
|
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server)
|
||||||
├── certs/ ← persisted self-signed cert (volume mount)
|
├── certs/ ← persisted self-signed cert (volume mount)
|
||||||
├── media/ ← permanent music library (bind-mounted to /data/media)
|
|
||||||
│ ├── files/ ← raw audio tracks (webm/m4a; media-server streamed, Range when complete)
|
|
||||||
│ ├── thumbs/ ← cover art
|
|
||||||
│ ├── jobs/ ← download job status snapshots
|
|
||||||
│ └── library.json ← library metadata (permanent)
|
|
||||||
├── agent/
|
├── agent/
|
||||||
│ ├── agent.py ← LiveKit Agents voice pipeline
|
│ ├── agent.py ← LiveKit Agents voice pipeline + Hope Music handshake
|
||||||
│ ├── media_mcp.py ← Hope's music tools (YouTube Music via media server)
|
│ ├── media_mcp.py ← thin HTTP client for the separate Hope Music service
|
||||||
│ ├── pyproject.toml ← Python deps (uv)
|
│ ├── pyproject.toml ← Python deps (uv)
|
||||||
│ └── uv.lock ← locked dependency versions
|
│ └── uv.lock ← locked dependency versions
|
||||||
└── web/
|
└── web/
|
||||||
├── index.html ← single-page voice UI (+ music panel)
|
├── index.html ← voice-only single-page UI
|
||||||
├── app.js ← LiveKit client logic (+ player reconciliation)
|
├── app.js ← LiveKit client logic (no music player)
|
||||||
├── media_server.py ← FastAPI library/search/download/player state
|
├── token_server.py ← LiveKit token endpoint (mints ephemeral room names)
|
||||||
├── token_server.py ← LiveKit token endpoint (mints room name)
|
|
||||||
├── manifest.json ← PWA manifest
|
├── manifest.json ← PWA manifest
|
||||||
├── favicon.svg ← site icon
|
├── favicon.svg ← site icon
|
||||||
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
||||||
@@ -218,9 +204,9 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- **Single container.** All services (LiveKit, agent, web, token endpoint) run in one Docker container via supervisord. No multi-service compose.
|
- **Single voice container.** LiveKit, the agent, nginx, and token endpoint run in one Docker container via supervisord. Hope Music is deliberately a separate product/container.
|
||||||
- **No published UDP ports in compose.** LiveKit binds its media ports directly on the host network (`network_mode: host`). This avoids the docker-proxy process explosion that hit hope-webui.
|
- **No published UDP ports in compose.** LiveKit binds its media ports directly on the host network (`network_mode: host`). This avoids the docker-proxy process explosion that hit hope-webui.
|
||||||
- **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server now pins the room name to "voice-room" server-side and no longer accepts arbitrary room names.
|
- **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server ignores browser-supplied room names and mints `voice-{user}-{uuid8}` server-side.
|
||||||
- **Interruption mode must be "vad".** `interruption={"mode": "adaptive"}` requires the LiveKit Cloud barge-in service (agent-gateway.livekit.cloud) and spams 401 retries on self-hosted setups.
|
- **Interruption mode must be "vad".** `interruption={"mode": "adaptive"}` requires the LiveKit Cloud barge-in service (agent-gateway.livekit.cloud) and spams 401 retries on self-hosted setups.
|
||||||
- **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`.
|
- **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`.
|
||||||
- **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access.
|
- **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access.
|
||||||
@@ -231,31 +217,27 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
||||||
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
||||||
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
||||||
- **Music downloads are raw audio, streamed as they grow.** yt-dlp fetches
|
- **Hope Music stays external.** `hope-media-music` owns yt-dlp, ffmpeg, the
|
||||||
bestaudio directly (`-f bestaudio/best`, no `-x`/transcode) into
|
library volume, streaming endpoints, and the player UI. Do not add those
|
||||||
`/data/media/files/{video_id}.{ext}`; the stream endpoint
|
dependencies or files back to this image. This repo owns only the MCP/HTTP
|
||||||
(`GET /media/api/stream/{id}`) serves the growing `.part` file chunked
|
client and the conversation/music handshake.
|
||||||
(no Content-Length) while the download runs, then serves the finished file
|
- **Music scope is stable.** LiveKit room names are intentionally ephemeral;
|
||||||
with native Range for seeking. `audio_url` is that endpoint for the whole
|
`MUSIC_ROOM` defaults to `voice-room`, which is also the standalone player's
|
||||||
lifecycle — a stable URL means the browser never reloads mid-playback. The
|
default. Never derive the music scope from `ctx.room.name`.
|
||||||
mp3 transcode was dropped: it delayed playback by the whole download+encode,
|
- **Playback truth comes from fresh reports.** The standalone player reports
|
||||||
and modern browsers play webm/opus and m4a everywhere. Never point the
|
real state every few seconds. The agent polls it and accepts `playing` only
|
||||||
player at YouTube directly.
|
while the report timestamp is fresh (`MUSIC_REPORT_TTL`, default thirty
|
||||||
- **Search prefers the library.** `/media/api/search` merges ready library
|
seconds), preventing a closed player from silencing the mic forever.
|
||||||
matches (title token/substring match) in front of the yt-dlp results, and
|
- **Peer failure is non-fatal.** Hope Music being unavailable must not stop the
|
||||||
the search_music tool docstring + system prompt tell Hope to always pick
|
voice agent or fail the voice container healthcheck. Music calls log and
|
||||||
`in_library=true`. Deterministic preference lives in the server merge, not
|
degrade gracefully.
|
||||||
in prompt hopes.
|
- **Never hard-require optional container env in supervisord.** `%(ENV_X)s`
|
||||||
- **Progress lines come from yt-dlp stderr.** `[download] N%` goes to stderr;
|
expansion aborts the whole config when a variable is missing. Settings with
|
||||||
only `--print-json` metadata goes to stdout. The download worker drains
|
safe in-code defaults (including Hope Music) should inherit the container
|
||||||
stderr in a thread (deadlock-safe) and reads stdout for the JSON — parsing
|
environment instead of being listed there.
|
||||||
progress from stdout silently never matched.
|
- **Healthchecks must use `curl -f`.** Without it, any HTTP response — including
|
||||||
- **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`.
|
a 503 from a squatter — counts as healthy. Voice health covers only LiveKit
|
||||||
- **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`.
|
and its HTTPS UI; Hope Music has its own independent healthcheck.
|
||||||
- **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.
|
|
||||||
- **Never hard-require container env in supervisord.** `%(ENV_X)s` expansion aborts the whole supervisord config when the var is missing, taking every program down (this silently killed the run-151 deploy). Programs with safe in-code defaults (media server) must inherit the container env instead of listing it.
|
|
||||||
- **Healthchecks must use `curl -f`.** Without it, any HTTP response — including a 503 from a squatter — counts as healthy. Run #15 deployed "healthy" against a broken media port this way.
|
|
||||||
- **Port 8092 is taken on xNAS.** An Apache/YOURLS stack occupies 8092 on the Unraid host; the media server uses 8093. Before picking a new internal port, probe the Unraid deployment host, not just the dev box.
|
|
||||||
|
|
||||||
## Git
|
## Git
|
||||||
|
|
||||||
|
|||||||
+4
-8
@@ -24,10 +24,10 @@ ENV PYTHONUNBUFFERED=1
|
|||||||
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
|
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
|
||||||
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
||||||
|
|
||||||
# Install LiveKit server binary + supervisord + nginx + ffmpeg (yt-dlp audio
|
# Install LiveKit server runtime dependencies, supervisord, and nginx.
|
||||||
# extraction/transcoding for the media player)
|
# Music download/transcode dependencies live in hope-media-music now.
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
curl ca-certificates supervisor nginx libasound2 iproute2 ffmpeg \
|
curl ca-certificates supervisor nginx libasound2 iproute2 \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Download LiveKit server (latest stable)
|
# Download LiveKit server (latest stable)
|
||||||
@@ -52,7 +52,7 @@ COPY agent/task_worker.py /opt/voice-agent/task_worker.py
|
|||||||
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
|
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
|
||||||
COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py
|
COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py
|
||||||
|
|
||||||
# Copy web frontend + token endpoint + media server
|
# Copy the voice frontend + token endpoint. Hope Music is a separate service.
|
||||||
COPY web/index.html /var/www/voice/
|
COPY web/index.html /var/www/voice/
|
||||||
COPY web/app.js /var/www/voice/
|
COPY web/app.js /var/www/voice/
|
||||||
COPY web/style.css /var/www/voice/
|
COPY web/style.css /var/www/voice/
|
||||||
@@ -60,10 +60,6 @@ COPY web/livekit-client.umd.js /var/www/voice/
|
|||||||
COPY web/manifest.json /var/www/voice/
|
COPY web/manifest.json /var/www/voice/
|
||||||
COPY web/favicon.svg /var/www/voice/
|
COPY web/favicon.svg /var/www/voice/
|
||||||
COPY web/token_server.py /opt/voice/token_server.py
|
COPY web/token_server.py /opt/voice/token_server.py
|
||||||
COPY web/media_server.py /opt/voice/media_server.py
|
|
||||||
|
|
||||||
# Permanent media library mount point (bind-mounted from ./media)
|
|
||||||
RUN mkdir -p /data/media/files /data/media/thumbs /data/media/jobs
|
|
||||||
|
|
||||||
# Config files
|
# Config files
|
||||||
COPY livekit.yaml /etc/livekit.yaml
|
COPY livekit.yaml /etc/livekit.yaml
|
||||||
|
|||||||
+144
-31
@@ -11,6 +11,7 @@ import logging
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import textwrap
|
import textwrap
|
||||||
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from dotenv import load_dotenv
|
from dotenv import load_dotenv
|
||||||
@@ -58,6 +59,18 @@ WEB_MCP_ENABLED = os.environ.get("WEB_MCP_ENABLED", "true").lower() in ("1", "tr
|
|||||||
FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
|
FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
|
||||||
SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
|
SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
|
||||||
VOICE_USERS_FILE = os.environ.get("VOICE_USERS_FILE", "/etc/hope/users.json")
|
VOICE_USERS_FILE = os.environ.get("VOICE_USERS_FILE", "/etc/hope/users.json")
|
||||||
|
# Hope Music is a separate LAN service. The agent is its control surface; the
|
||||||
|
# standalone Hope Music page is the browser audio player.
|
||||||
|
MUSIC_BASE_URL = os.environ.get(
|
||||||
|
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||||
|
).rstrip("/")
|
||||||
|
# This is deliberately stable rather than the short-lived LiveKit room name.
|
||||||
|
# The Hope Music page defaults to the same scope, so playback survives across
|
||||||
|
# voice sessions and both products address one virtual player.
|
||||||
|
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||||
|
# The player reports every few seconds while audio is active. A stale playing
|
||||||
|
# report must never leave Hope's microphone disabled after the page closes.
|
||||||
|
MUSIC_REPORT_TTL = float(os.environ.get("MUSIC_REPORT_TTL", "30"))
|
||||||
|
|
||||||
|
|
||||||
def load_user_profile(user_id: str) -> dict:
|
def load_user_profile(user_id: str) -> dict:
|
||||||
@@ -208,15 +221,15 @@ SYSTEM_PROMPT = textwrap.dedent("""\
|
|||||||
stalled); task_status can also be called to check up on a task. Use
|
stalled); task_status can also be called to check up on a task. Use
|
||||||
list_tasks to see every task you have dispatched.
|
list_tasks to see every task you have dispatched.
|
||||||
|
|
||||||
# Music (media player in the web UI)
|
# Music (standalone Hope Music player)
|
||||||
You can play music on the user's web player. For any music request, call
|
You can control the user's separate Hope Music web player. For any music
|
||||||
search_music first, then:
|
request, call search_music first, then:
|
||||||
- One clearly-right result → play_song with its video_id. Announce it in
|
- One clearly-right result → play_song with its video_id. Announce it in
|
||||||
one short sentence, then stop talking — playback pauses the
|
one short sentence, then stop talking — playback pauses the
|
||||||
conversation until the user comes back to you.
|
conversation until the user comes back to you.
|
||||||
- Ambiguous results → offer_song_choices with the top 2-3 options, name
|
- Ambiguous results → offer_song_choices with the top 2-3 options, name
|
||||||
them briefly, and let the user pick by voice or in the player panel.
|
them briefly, and let the user pick by voice or in Hope Music. Never
|
||||||
Never start playback on a guess.
|
start playback on a guess.
|
||||||
Prefer what is already saved: search_music puts downloaded library
|
Prefer what is already saved: search_music puts downloaded library
|
||||||
matches first (in_library=true) — always pick one of those over a fresh
|
matches first (in_library=true) — always pick one of those over a fresh
|
||||||
YouTube result, and list_library shows the whole saved collection which
|
YouTube result, and list_library shows the whole saved collection which
|
||||||
@@ -693,9 +706,9 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
|||||||
)
|
)
|
||||||
logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
|
logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
|
||||||
|
|
||||||
# Media tools talk to the local media server; the room name scopes player
|
# Media tools call the separate hope-media-music service over HTTP. A
|
||||||
# state so concurrent sessions don't fight over one virtual player.
|
# stable scope lets the standalone player and every voice session address
|
||||||
media_server_url = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093")
|
# the same virtual player.
|
||||||
media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py")
|
media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py")
|
||||||
toolsets.append(
|
toolsets.append(
|
||||||
mcp.MCPToolset(
|
mcp.MCPToolset(
|
||||||
@@ -705,14 +718,14 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
|||||||
args=[media_mcp_script],
|
args=[media_mcp_script],
|
||||||
env={
|
env={
|
||||||
**os.environ,
|
**os.environ,
|
||||||
"MEDIA_SERVER_URL": media_server_url,
|
"MUSIC_BASE_URL": MUSIC_BASE_URL,
|
||||||
"MEDIA_ROOM": os.path.basename(task_dir.rstrip("/")) or "default",
|
"MUSIC_ROOM": MUSIC_ROOM,
|
||||||
},
|
},
|
||||||
client_session_timeout_seconds=60,
|
client_session_timeout_seconds=60,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
logger.info("Media MCP toolset enabled (%s)", media_server_url)
|
logger.info("Media MCP enabled (%s, room %s)", MUSIC_BASE_URL, MUSIC_ROOM)
|
||||||
|
|
||||||
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
|
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
|
||||||
if extra:
|
if extra:
|
||||||
@@ -828,13 +841,12 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
except Exception as e: # noqa: BLE001
|
except Exception as e: # noqa: BLE001
|
||||||
logger.warning("failed to publish model list: %s", e)
|
logger.warning("failed to publish model list: %s", e)
|
||||||
|
|
||||||
# Listen for conversation controls (voice selection and voice/text mode).
|
# Listen for conversation controls (voice selection, voice/text mode, and
|
||||||
# Defined here so it captures this job's session via closure.
|
# model switching). Music state comes from the separate service, not from
|
||||||
# Music and conversation are mutually exclusive by design: while a track
|
# the voice UI's data channel.
|
||||||
# plays the agent's ears are off; re-engaging the conversation pauses the
|
|
||||||
# music, and ending the conversation lets the user resume it.
|
|
||||||
session_mode = {"value": "voice"} # UI voice/text switch
|
session_mode = {"value": "voice"} # UI voice/text switch
|
||||||
media_playback = {"value": "stopped"} # UI-reported player state
|
media_playback = {"value": "stopped"} # fresh service-reported state
|
||||||
|
music_paused_by_agent = {"value": False}
|
||||||
|
|
||||||
def _apply_listening() -> None:
|
def _apply_listening() -> None:
|
||||||
listening = (
|
listening = (
|
||||||
@@ -843,9 +855,108 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
)
|
)
|
||||||
session.input.set_audio_enabled(listening)
|
session.input.set_audio_enabled(listening)
|
||||||
|
|
||||||
|
async def _music_request(
|
||||||
|
method: str, path: str, *, warn: bool = True, **kwargs
|
||||||
|
) -> dict | None:
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
url = f"{MUSIC_BASE_URL}/media/api{path}"
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=10) as http:
|
||||||
|
response = await http.request(method, url, **kwargs)
|
||||||
|
response.raise_for_status()
|
||||||
|
return response.json()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
# Hope Music is a peer service: it being unavailable must never
|
||||||
|
# take the conversation down with it.
|
||||||
|
if warn:
|
||||||
|
logger.warning("Music service %s %s failed: %s", method, path, exc)
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def _music_snapshot(*, warn: bool = True) -> dict | None:
|
||||||
|
return await _music_request(
|
||||||
|
"GET", "/state", warn=warn, params={"room": MUSIC_ROOM}
|
||||||
|
)
|
||||||
|
|
||||||
|
def _music_is_playing(snapshot: dict | None) -> bool:
|
||||||
|
"""Return true only for fresh, browser-reported playback."""
|
||||||
|
reported = (snapshot or {}).get("reported") or {}
|
||||||
|
if reported.get("state") != "playing":
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
age = time.time() - float(reported.get("ts") or 0)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return False
|
||||||
|
return 0 <= age < MUSIC_REPORT_TTL
|
||||||
|
|
||||||
|
async def _pause_music_for_conversation() -> None:
|
||||||
|
"""Pause actual playback when the user engages Hope."""
|
||||||
|
if music_paused_by_agent["value"]:
|
||||||
|
return
|
||||||
|
if not _music_is_playing(await _music_snapshot()):
|
||||||
|
return
|
||||||
|
result = await _music_request(
|
||||||
|
"POST", "/control",
|
||||||
|
json={"room": MUSIC_ROOM, "action": "pause"},
|
||||||
|
)
|
||||||
|
if result is not None:
|
||||||
|
music_paused_by_agent["value"] = True
|
||||||
|
logger.info("Paused Hope Music for the conversation (%s)", MUSIC_ROOM)
|
||||||
|
|
||||||
|
async def _resume_music_after_conversation() -> None:
|
||||||
|
if not music_paused_by_agent["value"]:
|
||||||
|
return
|
||||||
|
result = await _music_request(
|
||||||
|
"POST", "/control",
|
||||||
|
json={"room": MUSIC_ROOM, "action": "resume"},
|
||||||
|
)
|
||||||
|
if result is not None:
|
||||||
|
music_paused_by_agent["value"] = False
|
||||||
|
logger.info("Resumed Hope Music after the conversation (%s)", MUSIC_ROOM)
|
||||||
|
|
||||||
|
async def _poll_music_state() -> None:
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
await asyncio.sleep(2)
|
||||||
|
# Jobs deliberately survive page reloads. Do not keep making
|
||||||
|
# network requests after the room has no human participant.
|
||||||
|
if not ctx.room.remote_participants:
|
||||||
|
continue
|
||||||
|
snapshot = await _music_snapshot(warn=False)
|
||||||
|
reported = (snapshot or {}).get("reported") or {}
|
||||||
|
state = reported.get("state") or "stopped"
|
||||||
|
if state == "playing" and not _music_is_playing(snapshot):
|
||||||
|
state = "stopped"
|
||||||
|
if state not in ("playing", "paused", "stopped"):
|
||||||
|
state = "stopped"
|
||||||
|
if state != media_playback["value"]:
|
||||||
|
logger.info("Hope Music playback -> %s", state)
|
||||||
|
media_playback["value"] = state
|
||||||
|
_apply_listening()
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
raise
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
logger.exception("Hope Music state poller failed")
|
||||||
|
|
||||||
|
def _on_participant_joined(*_args) -> None:
|
||||||
|
"""A returning participant starts a new conversation handshake."""
|
||||||
|
asyncio.create_task(_pause_music_for_conversation())
|
||||||
|
|
||||||
|
def _on_participant_left(*_args) -> None:
|
||||||
|
"""When the last human leaves, hand playback back to Hope Music."""
|
||||||
|
|
||||||
|
async def _deferred() -> None:
|
||||||
|
# The event can fire before the participant is removed from the
|
||||||
|
# room object; re-check the roster after it settles.
|
||||||
|
await asyncio.sleep(0.5)
|
||||||
|
if not ctx.room.remote_participants:
|
||||||
|
await _resume_music_after_conversation()
|
||||||
|
|
||||||
|
asyncio.create_task(_deferred())
|
||||||
|
|
||||||
def _on_room_data(packet) -> None:
|
def _on_room_data(packet) -> None:
|
||||||
topic = getattr(packet, "topic", None)
|
topic = getattr(packet, "topic", None)
|
||||||
if topic and topic not in ("voice-control", "conversation-control", "media-control"):
|
if topic and topic not in ("voice-control", "conversation-control"):
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
msg = json.loads(packet.data.decode("utf-8"))
|
msg = json.loads(packet.data.decode("utf-8"))
|
||||||
@@ -865,17 +976,6 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
session.output.set_audio_enabled(voice_enabled)
|
session.output.set_audio_enabled(voice_enabled)
|
||||||
_apply_listening()
|
_apply_listening()
|
||||||
return
|
return
|
||||||
|
|
||||||
if msg.get("type") == "media_state":
|
|
||||||
state = msg.get("state")
|
|
||||||
if state not in ("playing", "paused", "stopped"):
|
|
||||||
return
|
|
||||||
logger.info("Media playback -> %s (agent listening: %s)",
|
|
||||||
state, session_mode["value"] == "voice" and state != "playing")
|
|
||||||
media_playback["value"] = state
|
|
||||||
_apply_listening()
|
|
||||||
return
|
|
||||||
|
|
||||||
if msg.get("type") == "set_voice":
|
if msg.get("type") == "set_voice":
|
||||||
voice = msg.get("voice", "")
|
voice = msg.get("voice", "")
|
||||||
if not voice:
|
if not voice:
|
||||||
@@ -913,6 +1013,8 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
logger.warning("Failed to update model: %s", e)
|
logger.warning("Failed to update model: %s", e)
|
||||||
|
|
||||||
ctx.room.on("data_received", _on_room_data)
|
ctx.room.on("data_received", _on_room_data)
|
||||||
|
ctx.room.on("participant_connected", _on_participant_joined)
|
||||||
|
ctx.room.on("participant_disconnected", _on_participant_left)
|
||||||
|
|
||||||
# Publish user/agent transcripts to the room so the web UI can render them.
|
# Publish user/agent transcripts to the room so the web UI can render them.
|
||||||
async def publish_transcript(role: str, text: str) -> None:
|
async def publish_transcript(role: str, text: str) -> None:
|
||||||
@@ -933,9 +1035,9 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
role = getattr(msg, "role", None)
|
role = getattr(msg, "role", None)
|
||||||
text = getattr(msg, "text_content", None)
|
text = getattr(msg, "text_content", None)
|
||||||
if role == "user":
|
if role == "user":
|
||||||
asyncio.create_task(publish_transcript("user", text))
|
asyncio.create_task(publish_transcript("user", text or ""))
|
||||||
elif role == "assistant":
|
elif role == "assistant":
|
||||||
asyncio.create_task(publish_transcript("agent", text))
|
asyncio.create_task(publish_transcript("agent", text or ""))
|
||||||
|
|
||||||
@session.on("user_input_transcribed")
|
@session.on("user_input_transcribed")
|
||||||
def _on_user_transcribed(ev) -> None:
|
def _on_user_transcribed(ev) -> None:
|
||||||
@@ -1017,6 +1119,17 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
await ctx.connect()
|
await ctx.connect()
|
||||||
|
await _pause_music_for_conversation()
|
||||||
|
music_poll_task = asyncio.create_task(
|
||||||
|
_poll_music_state(), name=f"music-poller-{ctx.room.name}"
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _stop_music_poller(*_args) -> None:
|
||||||
|
music_poll_task.cancel()
|
||||||
|
await asyncio.gather(music_poll_task, return_exceptions=True)
|
||||||
|
await _resume_music_after_conversation()
|
||||||
|
|
||||||
|
ctx.add_shutdown_callback(_stop_music_poller)
|
||||||
# The participant can appear in the browser slightly before AgentSession is
|
# The participant can appear in the browser slightly before AgentSession is
|
||||||
# ready to accept lk.chat streams. Explicitly signal readiness so the first
|
# ready to accept lk.chat streams. Explicitly signal readiness so the first
|
||||||
# typed message is never raced against session startup.
|
# typed message is never raced against session startup.
|
||||||
|
|||||||
+24
-23
@@ -1,18 +1,17 @@
|
|||||||
"""Media MCP server — lets Hope find and control music by voice.
|
"""Media MCP client — lets Hope control the standalone Hope Music service.
|
||||||
|
|
||||||
Tools talk to the local media server (web/media_server.py) over HTTP, which
|
Tools call the separate ``hope-media-music`` LAN service over HTTP. That
|
||||||
owns the library, YouTube Music search/download jobs and per-room player
|
service owns the library, YouTube Music search/download jobs, stream proxy,
|
||||||
state. The browser is the actual player: it reconciles against the media
|
and per-room player state. Its standalone browser page is the actual audio
|
||||||
server and reports playback back, so voice controls and the web panel stay
|
player: it reconciles against the service and reports real playback.
|
||||||
in sync.
|
|
||||||
|
|
||||||
Per-session environment (set by the agent when spawning this MCP):
|
Environment (set by the voice agent when spawning this MCP):
|
||||||
- MEDIA_SERVER_URL e.g. http://127.0.0.1:8093
|
- MUSIC_BASE_URL e.g. http://192.168.86.2:8094
|
||||||
- MEDIA_ROOM LiveKit room name this session controls
|
- MUSIC_ROOM stable player scope shared with the Hope Music page
|
||||||
|
|
||||||
The "stop listening while music plays" rule lives in the agent itself: the
|
The voice agent polls the same service state to keep music and conversation
|
||||||
UI reports media_state over the data channel and the agent disables its mic
|
mutually exclusive; there is no music server or music player in the voice
|
||||||
while a track is playing.
|
container anymore.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -22,20 +21,22 @@ import os
|
|||||||
import httpx
|
import httpx
|
||||||
from mcp.server.fastmcp import FastMCP
|
from mcp.server.fastmcp import FastMCP
|
||||||
|
|
||||||
MEDIA_SERVER_URL = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093").rstrip("/")
|
MUSIC_BASE_URL = os.environ.get(
|
||||||
MEDIA_ROOM = os.environ.get("MEDIA_ROOM", "default")
|
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||||
|
).rstrip("/")
|
||||||
|
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||||
|
|
||||||
mcp = FastMCP("media")
|
mcp = FastMCP("media")
|
||||||
|
|
||||||
|
|
||||||
def _media_get(path: str, params: dict | None = None) -> dict:
|
def _media_get(path: str, params: dict | None = None) -> dict:
|
||||||
resp = httpx.get(f"{MEDIA_SERVER_URL}{path}", params=params, timeout=10)
|
resp = httpx.get(f"{MUSIC_BASE_URL}{path}", params=params, timeout=10)
|
||||||
resp.raise_for_status()
|
resp.raise_for_status()
|
||||||
return resp.json()
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
def _media_post(path: str, body: dict) -> dict:
|
def _media_post(path: str, body: dict) -> dict:
|
||||||
resp = httpx.post(f"{MEDIA_SERVER_URL}{path}", json=body, timeout=30)
|
resp = httpx.post(f"{MUSIC_BASE_URL}{path}", json=body, timeout=30)
|
||||||
if resp.status_code >= 400:
|
if resp.status_code >= 400:
|
||||||
try:
|
try:
|
||||||
detail = resp.json().get("error", resp.text)
|
detail = resp.json().get("error", resp.text)
|
||||||
@@ -46,7 +47,7 @@ def _media_post(path: str, body: dict) -> dict:
|
|||||||
|
|
||||||
|
|
||||||
def _control(action: str, **extra) -> str:
|
def _control(action: str, **extra) -> str:
|
||||||
data = _media_post("/media/api/control", {"room": MEDIA_ROOM, "action": action, **extra})
|
data = _media_post("/media/api/control", {"room": MUSIC_ROOM, "action": action, **extra})
|
||||||
if "error" in data:
|
if "error" in data:
|
||||||
return f"Could not {action} the music: {data['error']}"
|
return f"Could not {action} the music: {data['error']}"
|
||||||
track = data.get("track") or {}
|
track = data.get("track") or {}
|
||||||
@@ -87,14 +88,14 @@ def search_music(query: str, limit: int = 5) -> str:
|
|||||||
|
|
||||||
@mcp.tool()
|
@mcp.tool()
|
||||||
def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
||||||
"""Play a song in the user's web player. Pass the video_id from
|
"""Play a song in the user's standalone Hope Music player. Pass the video_id from
|
||||||
search_music — for an in_library result this plays the saved copy with no
|
search_music — for an in_library result this plays the saved copy with no
|
||||||
download; a new song downloads and starts playing automatically while it
|
download; a new song downloads and starts playing automatically while it
|
||||||
arrives. After calling this, tell the user what is playing in one short
|
arrives. After calling this, tell the user what is playing in one short
|
||||||
sentence and stop talking — the player takes over and the conversation
|
sentence and stop talking — the player takes over and the conversation
|
||||||
pauses until they return."""
|
pauses until they return."""
|
||||||
data = _media_post("/media/api/play", {
|
data = _media_post("/media/api/play", {
|
||||||
"room": MEDIA_ROOM, "video_id": video_id, "title": title, "channel": channel,
|
"room": MUSIC_ROOM, "video_id": video_id, "title": title, "channel": channel,
|
||||||
})
|
})
|
||||||
if "error" in data:
|
if "error" in data:
|
||||||
return f"Could not start playback: {data['error']}"
|
return f"Could not start playback: {data['error']}"
|
||||||
@@ -114,12 +115,12 @@ def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
|||||||
def offer_song_choices(choices: list[dict]) -> str:
|
def offer_song_choices(choices: list[dict]) -> str:
|
||||||
"""Offer the user 2-4 song options when a search is ambiguous. Pass a list
|
"""Offer the user 2-4 song options when a search is ambiguous. Pass a list
|
||||||
like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}].
|
like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}].
|
||||||
The choices appear as clickable cards in the music panel; the user can
|
The choices appear as clickable cards in Hope Music; the user can
|
||||||
also just say which one they want. Briefly name the options aloud, then
|
also just say which one they want. Briefly name the options aloud, then
|
||||||
wait for their pick."""
|
wait for their pick."""
|
||||||
if not choices:
|
if not choices:
|
||||||
return "No choices provided."
|
return "No choices provided."
|
||||||
data = _media_post("/media/api/choices", {"room": MEDIA_ROOM, "choices": choices})
|
data = _media_post("/media/api/choices", {"room": MUSIC_ROOM, "choices": choices})
|
||||||
if "error" in data:
|
if "error" in data:
|
||||||
return f"Could not present choices: {data['error']}"
|
return f"Could not present choices: {data['error']}"
|
||||||
names = [
|
names = [
|
||||||
@@ -162,7 +163,7 @@ def seek_music(delta_seconds: int) -> str:
|
|||||||
jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips
|
jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips
|
||||||
20 seconds forward)."""
|
20 seconds forward)."""
|
||||||
try:
|
try:
|
||||||
reported = (_media_get("/media/api/state", {"room": MEDIA_ROOM}).get("reported") or {})
|
reported = (_media_get("/media/api/state", {"room": MUSIC_ROOM}).get("reported") or {})
|
||||||
position = float(reported.get("position") or 0.0)
|
position = float(reported.get("position") or 0.0)
|
||||||
except Exception: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
position = 0.0
|
position = 0.0
|
||||||
@@ -176,7 +177,7 @@ def now_playing() -> str:
|
|||||||
its playback position. Use when the user asks what's playing or what this
|
its playback position. Use when the user asks what's playing or what this
|
||||||
song is."""
|
song is."""
|
||||||
try:
|
try:
|
||||||
st = _media_get("/media/api/state", {"room": MEDIA_ROOM})
|
st = _media_get("/media/api/state", {"room": MUSIC_ROOM})
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
return f"Media server unreachable: {exc}"
|
return f"Media server unreachable: {exc}"
|
||||||
track = st.get("track") or {}
|
track = st.get("track") or {}
|
||||||
|
|||||||
@@ -16,7 +16,6 @@ dependencies = [
|
|||||||
"livekit-plugins-openai~=1.7",
|
"livekit-plugins-openai~=1.7",
|
||||||
"python-dotenv",
|
"python-dotenv",
|
||||||
"uvicorn>=0.52.4",
|
"uvicorn>=0.52.4",
|
||||||
"yt-dlp>=2026.8.19",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
|
|||||||
Generated
+1
-12
@@ -664,7 +664,7 @@ name = "exceptiongroup"
|
|||||||
version = "1.3.1"
|
version = "1.3.1"
|
||||||
source = { registry = "https://pypi.org/simple" }
|
source = { registry = "https://pypi.org/simple" }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
{ name = "typing-extensions" },
|
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
|
||||||
]
|
]
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
|
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
|
||||||
wheels = [
|
wheels = [
|
||||||
@@ -2767,7 +2767,6 @@ dependencies = [
|
|||||||
{ name = "livekit-plugins-openai" },
|
{ name = "livekit-plugins-openai" },
|
||||||
{ name = "python-dotenv" },
|
{ name = "python-dotenv" },
|
||||||
{ name = "uvicorn" },
|
{ name = "uvicorn" },
|
||||||
{ name = "yt-dlp" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[package.metadata]
|
[package.metadata]
|
||||||
@@ -2779,7 +2778,6 @@ requires-dist = [
|
|||||||
{ name = "livekit-plugins-openai", specifier = "~=1.7" },
|
{ name = "livekit-plugins-openai", specifier = "~=1.7" },
|
||||||
{ name = "python-dotenv" },
|
{ name = "python-dotenv" },
|
||||||
{ name = "uvicorn", specifier = ">=0.52.4" },
|
{ name = "uvicorn", specifier = ">=0.52.4" },
|
||||||
{ name = "yt-dlp", specifier = ">=2026.8.19" },
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -3062,12 +3060,3 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
|
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
|
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
|
||||||
name = "yt-dlp"
|
|
||||||
version = "2026.8.19"
|
|
||||||
source = { registry = "https://pypi.org/simple" }
|
|
||||||
sdist = { url = "https://files.pythonhosted.org/packages/1e/e0/832fa4ca334b766a06933a196066edc3dba37cdb6f14cd98d59bcc69a4b4/yt_dlp-2026.8.19.tar.gz", hash = "sha256:9e213e48cea35c66b378e4447903f118f6392a5fa380a2b6d7070ec86f4e0af1", size = 3052025, upload-time = "2026-08-19T23:48:59.291Z" }
|
|
||||||
wheels = [
|
|
||||||
{ url = "https://files.pythonhosted.org/packages/69/b2/8cd1613f56eed7ceb64fbd4df3f1c01246bfb098e6f398228bafda22b80b/yt_dlp-2026.8.19-py3-none-any.whl", hash = "sha256:1d57897e94c6665a0a6f9bc54b34e584284e32c034ffab3a7df25d8f7b24eedf", size = 3185533, upload-time = "2026-08-19T23:48:56.925Z" },
|
|
||||||
]
|
|
||||||
@@ -10,4 +10,3 @@ Requires-Dist: livekit-plugins-azure~=1.7
|
|||||||
Requires-Dist: livekit-plugins-openai~=1.7
|
Requires-Dist: livekit-plugins-openai~=1.7
|
||||||
Requires-Dist: python-dotenv
|
Requires-Dist: python-dotenv
|
||||||
Requires-Dist: uvicorn>=0.52.4
|
Requires-Dist: uvicorn>=0.52.4
|
||||||
Requires-Dist: yt-dlp>=2026.8.19
|
|
||||||
@@ -5,4 +5,3 @@ livekit-plugins-azure~=1.7
|
|||||||
livekit-plugins-openai~=1.7
|
livekit-plugins-openai~=1.7
|
||||||
python-dotenv
|
python-dotenv
|
||||||
uvicorn>=0.52.4
|
uvicorn>=0.52.4
|
||||||
yt-dlp>=2026.8.19
|
|
||||||
+3
-5
@@ -18,16 +18,14 @@ services:
|
|||||||
WEB_PORT: "${WEB_PORT:-9178}"
|
WEB_PORT: "${WEB_PORT:-9178}"
|
||||||
VOICE_USERS_FILE: "/etc/hope/users.json"
|
VOICE_USERS_FILE: "/etc/hope/users.json"
|
||||||
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
|
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
|
||||||
MEDIA_DIR: "/data/media"
|
MUSIC_BASE_URL: "${MUSIC_BASE_URL:-http://192.168.86.2:8094}"
|
||||||
MEDIA_SERVER_PORT: "${MEDIA_SERVER_PORT:-8093}"
|
MUSIC_ROOM: "${MUSIC_ROOM:-voice-room}"
|
||||||
YTDLP_COOKIES: "${YTDLP_COOKIES:-/data/media/cookies.txt}"
|
|
||||||
volumes:
|
volumes:
|
||||||
- ./livekit.yaml:/etc/livekit.yaml:ro
|
- ./livekit.yaml:/etc/livekit.yaml:ro
|
||||||
- ./certs:/etc/voice/certs
|
- ./certs:/etc/voice/certs
|
||||||
- ./skills:/skills
|
- ./skills:/skills
|
||||||
- ./media:/data/media
|
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:$${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null"]
|
test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null"]
|
||||||
interval: 15s
|
interval: 15s
|
||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 3
|
retries: 3
|
||||||
|
|||||||
Whitespace-only changes.
-21
@@ -22,27 +22,6 @@ server {
|
|||||||
proxy_read_timeout 3600s;
|
proxy_read_timeout 3600s;
|
||||||
proxy_send_timeout 3600s;
|
proxy_send_timeout 3600s;
|
||||||
}
|
}
|
||||||
|
|
||||||
# Media server API (library, search, player state) and the proxy for
|
|
||||||
# external live-stream URLs (http:// sources would be blocked as mixed
|
|
||||||
# content in the browser, so they are re-served over HTTPS from here).
|
|
||||||
location /media/api/ {
|
|
||||||
proxy_pass http://127.0.0.1:8093/media/api/;
|
|
||||||
proxy_buffering off;
|
|
||||||
proxy_read_timeout 3600s;
|
|
||||||
proxy_send_timeout 3600s;
|
|
||||||
}
|
|
||||||
|
|
||||||
# Downloaded tracks are plain static files — nginx serves them with
|
|
||||||
# native HTTP Range support, which gives the web player seeking for free.
|
|
||||||
location /media/files/ {
|
|
||||||
alias /data/media/files/;
|
|
||||||
}
|
|
||||||
|
|
||||||
location /media/thumbs/ {
|
|
||||||
alias /data/media/thumbs/;
|
|
||||||
}
|
|
||||||
|
|
||||||
location / {
|
location / {
|
||||||
try_files $uri $uri/ =404;
|
try_files $uri $uri/ =404;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -56,18 +56,3 @@ environment=
|
|||||||
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
|
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
|
||||||
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
|
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
|
||||||
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
|
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
|
||||||
|
|
||||||
[program:media-server]
|
|
||||||
; Runs as root because the /data/media bind mount is a host (Unraid) share
|
|
||||||
; whose uid does not match the container's voiceuser.
|
|
||||||
; MEDIA_DIR / MEDIA_SERVER_PORT / YTDLP_COOKIES are inherited from the
|
|
||||||
; container environment (media_server.py has safe defaults for all three) —
|
|
||||||
; do NOT list them via %(ENV_x)s here: a missing var aborts supervisord's
|
|
||||||
; config expansion and takes every program down with it.
|
|
||||||
command=/opt/voice-agent/.venv/bin/python /opt/voice/media_server.py
|
|
||||||
user=root
|
|
||||||
autostart=true
|
|
||||||
autorestart=true
|
|
||||||
stdout_logfile=/dev/stdout
|
|
||||||
stdout_logfile_maxbytes=0
|
|
||||||
redirect_stderr=true
|
|
||||||
+2
-2
@@ -15,7 +15,7 @@ tool servers, and agent process.
|
|||||||
## Run
|
## Run
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /home/shane/dev/voice
|
cd ~/dev/hope-voice-api
|
||||||
python -m pytest tests/ -v # everything
|
python -m pytest tests/ -v # everything
|
||||||
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
||||||
```
|
```
|
||||||
@@ -27,7 +27,7 @@ python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
|||||||
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
||||||
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
||||||
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
||||||
| `test_media.py` | media server API + player UI + media MCP | library/player state flow, Range proxy, static mp3 Range, live YT Music search (`-m slow`), gated full download (`HOPE_RUN_YTDLP_DOWNLOAD=1`) |
|
| `test_media.py` | external Hope Music boundary + media MCP client | LAN service health, stable room scope, split regression guards, local-container MCP probes |
|
||||||
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
||||||
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
||||||
|
|
||||||
|
|||||||
+1
-2
@@ -27,8 +27,7 @@ WEB_BASE_URL = os.environ.get("VOICE_WEB_URL", "https://192.168.86.2:9178").rstr
|
|||||||
TOKEN_URL = os.environ.get(
|
TOKEN_URL = os.environ.get(
|
||||||
"VOICE_TOKEN_URL", f"{WEB_BASE_URL}/token"
|
"VOICE_TOKEN_URL", f"{WEB_BASE_URL}/token"
|
||||||
)
|
)
|
||||||
# Filesystem-coupled tests (repo ./media bind mount) only make sense when
|
# docker-exec integration probes only make sense when the test target is local.
|
||||||
# the test host IS the deployment host.
|
|
||||||
IS_LOCAL_DEPLOYMENT = urlparse(WEB_BASE_URL).hostname in ("localhost", "127.0.0.1")
|
IS_LOCAL_DEPLOYMENT = urlparse(WEB_BASE_URL).hostname in ("localhost", "127.0.0.1")
|
||||||
LLM_BASE_URL = (
|
LLM_BASE_URL = (
|
||||||
os.environ.get("LLM_BASE_URL")
|
os.environ.get("LLM_BASE_URL")
|
||||||
|
|||||||
+103
-534
@@ -1,484 +1,112 @@
|
|||||||
"""Tests for the media player subsystem.
|
"""Voice-side integration tests for the standalone Hope Music service.
|
||||||
|
|
||||||
Covers the media server API (via nginx HTTPS and direct), the stream proxy
|
The library, downloads, stream proxy, web player, and their exhaustive tests
|
||||||
with HTTP Range passthrough, static file serving of the ./media volume, the
|
live in ``hope-media-music`` now. This module guards the remaining product
|
||||||
web player markup, and Hope's media MCP toolset. Download tests are gated
|
boundary: the voice repo contains only the MCP/HTTP client, uses one stable
|
||||||
behind HOPE_RUN_YTDLP_DOWNLOAD=1 (network-heavy, occasionally flaky).
|
player scope, and can reach the LAN service.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
import os
|
||||||
import subprocess
|
import socket
|
||||||
import threading
|
import threading
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||||
|
from pathlib import Path
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from conftest import (
|
from conftest import IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
|
||||||
TOKEN_URL, docker_exec, docker_exec_python, IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
|
|
||||||
|
ROOT = Path(__file__).parents[1]
|
||||||
|
MUSIC_BASE_URL = os.environ.get(
|
||||||
|
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||||
|
).rstrip("/")
|
||||||
|
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||||
|
MEDIA_API = "/media/api"
|
||||||
|
|
||||||
|
|
||||||
|
def test_external_music_service_health_and_library():
|
||||||
|
"""The peer service used by the deployed agent is reachable over the LAN."""
|
||||||
|
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as client:
|
||||||
|
health = client.get(f"{MEDIA_API}/healthz")
|
||||||
|
assert health.status_code == 200
|
||||||
|
assert health.json()["ok"] is True
|
||||||
|
|
||||||
|
library = client.get(f"{MEDIA_API}/library")
|
||||||
|
assert library.status_code == 200
|
||||||
|
assert isinstance(library.json()["tracks"], list)
|
||||||
|
|
||||||
|
|
||||||
|
def test_voice_repo_contains_only_the_music_client():
|
||||||
|
"""Regression guard for the product split: backend/player cannot creep back."""
|
||||||
|
agent = (ROOT / "agent" / "agent.py").read_text()
|
||||||
|
client = (ROOT / "agent" / "media_mcp.py").read_text()
|
||||||
|
web_js = (ROOT / "web" / "app.js").read_text()
|
||||||
|
web_html = (ROOT / "web" / "index.html").read_text()
|
||||||
|
|
||||||
|
assert not (ROOT / "web" / "media_server.py").exists()
|
||||||
|
assert not (ROOT / "media").exists()
|
||||||
|
assert "MUSIC_BASE_URL" in agent and "MUSIC_ROOM" in agent
|
||||||
|
assert "MUSIC_BASE_URL" in client and "MUSIC_ROOM" in client
|
||||||
|
assert "MEDIA_SERVER_URL" not in agent + client
|
||||||
|
assert "media-control" not in agent + web_js
|
||||||
|
assert "media_state" not in agent + web_js
|
||||||
|
assert "/media/api" not in web_js
|
||||||
|
assert 'id="mediaPanel"' not in web_html
|
||||||
|
assert 'id="mediaAudio"' not in web_html
|
||||||
|
|
||||||
|
|
||||||
|
def test_voice_uses_a_stable_music_scope():
|
||||||
|
"""The ephemeral LiveKit room must not be the standalone player's scope."""
|
||||||
|
assert MUSIC_ROOM == "voice-room" or os.environ.get("MUSIC_ROOM")
|
||||||
|
source = (ROOT / "agent" / "agent.py").read_text()
|
||||||
|
assert 'MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")' in source
|
||||||
|
assert '"MUSIC_ROOM": MUSIC_ROOM' in source
|
||||||
|
|
||||||
|
|
||||||
|
# These exercise the actual MCP subprocess inside a local voice container. The
|
||||||
|
# default suite targets Unraid remotely, where docker-exec intentionally skips.
|
||||||
|
needs_local_container = pytest.mark.skipif(
|
||||||
|
not IS_LOCAL_DEPLOYMENT,
|
||||||
|
reason="docker-exec MCP probes require a local voice container",
|
||||||
)
|
)
|
||||||
|
|
||||||
MEDIA_API = "/media/api"
|
MEDIA_MCP_ENV = {
|
||||||
TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}"
|
"MUSIC_BASE_URL": MUSIC_BASE_URL,
|
||||||
|
"MUSIC_ROOM": "mcp-test-room",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
# ── local upstream for stream-proxy tests ───────────────────────────────────
|
|
||||||
|
|
||||||
def _serve_fake_audio(payload: bytes):
|
def _serve_fake_audio(payload: bytes):
|
||||||
"""Tiny Range-capable HTTP server on an ephemeral port.
|
"""Tiny Range-capable upstream reachable by a host-networked container."""
|
||||||
|
|
||||||
Binds 0.0.0.0 and advertises this host's LAN IP so the deployment host's
|
|
||||||
media server (which may be a different machine — Unraid) can reach it
|
|
||||||
over the LAN when fetching external stream URLs.
|
|
||||||
"""
|
|
||||||
import socket
|
|
||||||
|
|
||||||
class Handler(BaseHTTPRequestHandler):
|
class Handler(BaseHTTPRequestHandler):
|
||||||
def do_GET(self):
|
def do_GET(self):
|
||||||
rng = self.headers.get("Range")
|
self.send_response(200)
|
||||||
if rng and rng.startswith("bytes="):
|
|
||||||
start_s, _, end_s = rng[6:].partition("-")
|
|
||||||
start = int(start_s or 0)
|
|
||||||
end = int(end_s) if end_s else len(payload) - 1
|
|
||||||
end = min(end, len(payload) - 1)
|
|
||||||
body = payload[start : end + 1]
|
|
||||||
self.send_response(206)
|
|
||||||
self.send_header(
|
|
||||||
"Content-Range", f"bytes {start}-{end}/{len(payload)}"
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
body = payload
|
|
||||||
self.send_response(200)
|
|
||||||
self.send_header("Content-Type", "audio/mpeg")
|
self.send_header("Content-Type", "audio/mpeg")
|
||||||
self.send_header("Content-Length", str(len(body)))
|
self.send_header("Content-Length", str(len(payload)))
|
||||||
self.send_header("Accept-Ranges", "bytes")
|
|
||||||
self.end_headers()
|
self.end_headers()
|
||||||
self.wfile.write(body)
|
self.wfile.write(payload)
|
||||||
|
|
||||||
def log_message(self, *args):
|
def log_message(self, format, *args): # noqa: A002
|
||||||
pass
|
pass
|
||||||
|
|
||||||
server = HTTPServer(("0.0.0.0", 0), Handler)
|
server = HTTPServer(("0.0.0.0", 0), Handler)
|
||||||
port = server.server_address[1]
|
port = server.server_address[1]
|
||||||
# The local IP toward the deployment host (works across the LAN).
|
|
||||||
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||||
try:
|
try:
|
||||||
probe.connect((urlparse(WEB_BASE_URL).hostname or "192.168.86.2", 9178))
|
target = urlparse(WEB_BASE_URL).hostname or "192.168.86.2"
|
||||||
lan_ip = probe.getsockname()[0]
|
probe.connect((target, urlparse(WEB_BASE_URL).port or 9178))
|
||||||
|
host = probe.getsockname()[0]
|
||||||
except OSError:
|
except OSError:
|
||||||
lan_ip = "127.0.0.1"
|
host = "127.0.0.1"
|
||||||
finally:
|
finally:
|
||||||
probe.close()
|
probe.close()
|
||||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||||
return server, f"http://{lan_ip}:{port}/audio.mp3"
|
return server, f"http://{host}:{port}/audio.mp3"
|
||||||
|
|
||||||
|
|
||||||
# ── health + basic API shape ────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def test_media_server_health_via_nginx(client):
|
|
||||||
resp = client.get(f"{MEDIA_API}/healthz")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert resp.json()["ok"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_library_endpoint(client):
|
|
||||||
resp = client.get(f"{MEDIA_API}/library")
|
|
||||||
assert resp.status_code == 200
|
|
||||||
data = resp.json()
|
|
||||||
assert isinstance(data["tracks"], list)
|
|
||||||
assert data["version"] >= 1
|
|
||||||
for track in data["tracks"]:
|
|
||||||
assert {"id", "kind", "title", "status"} <= set(track)
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_search_requires_query(client):
|
|
||||||
resp = client.post(f"{MEDIA_API}/search", json={"query": ""})
|
|
||||||
assert resp.status_code == 400
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_play_requires_target(client):
|
|
||||||
resp = client.post(f"{MEDIA_API}/play", json={"room": TEST_ROOM})
|
|
||||||
assert resp.status_code == 400
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_unknown_stream_404(client):
|
|
||||||
resp = client.get(f"{MEDIA_API}/stream/does-not-exist")
|
|
||||||
assert resp.status_code == 404
|
|
||||||
|
|
||||||
|
|
||||||
# ── player state flow (uses a live stream track as the fixture) ─────────────
|
|
||||||
|
|
||||||
def test_media_player_state_flow(client):
|
|
||||||
"""add stream -> play -> state -> report -> pause -> seek hint -> stop."""
|
|
||||||
upstream, url = _serve_fake_audio(b"\x00" * 64)
|
|
||||||
try:
|
|
||||||
added = client.post(
|
|
||||||
f"{MEDIA_API}/streams", json={"name": "Test FM", "url": url}
|
|
||||||
)
|
|
||||||
assert added.status_code == 200
|
|
||||||
track = added.json()["track"]
|
|
||||||
track_id = track["id"]
|
|
||||||
assert track["kind"] == "stream"
|
|
||||||
assert track["status"] == "ready"
|
|
||||||
assert track["audio_url"].startswith(f"{MEDIA_API}/stream/")
|
|
||||||
|
|
||||||
played = client.post(
|
|
||||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
|
||||||
).json()
|
|
||||||
assert played["desired"] == "playing"
|
|
||||||
assert played["track"]["id"] == track_id
|
|
||||||
|
|
||||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
|
||||||
assert state["track"]["id"] == track_id
|
|
||||||
|
|
||||||
client.post(
|
|
||||||
f"{MEDIA_API}/report",
|
|
||||||
json={"room": TEST_ROOM, "track_id": track_id,
|
|
||||||
"state": "playing", "position": 12.5},
|
|
||||||
)
|
|
||||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
|
||||||
assert abs(state["reported"]["position"] - 12.5) < 0.01
|
|
||||||
|
|
||||||
client.post(f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "pause"})
|
|
||||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["desired"] == "paused"
|
|
||||||
|
|
||||||
client.post(
|
|
||||||
f"{MEDIA_API}/control",
|
|
||||||
json={"room": TEST_ROOM, "action": "seek", "position": 42},
|
|
||||||
)
|
|
||||||
hint = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["seek_hint"]
|
|
||||||
assert hint["position"] == 42
|
|
||||||
|
|
||||||
# start_paused variant: scope transfer without starting playback
|
|
||||||
transfer = client.post(
|
|
||||||
f"{MEDIA_API}/play",
|
|
||||||
json={"room": TEST_ROOM + "-b", "track_id": track_id, "start_paused": True},
|
|
||||||
).json()
|
|
||||||
assert transfer["desired"] == "paused"
|
|
||||||
|
|
||||||
# playing clears any open suggestion choices
|
|
||||||
client.post(
|
|
||||||
f"{MEDIA_API}/choices",
|
|
||||||
json={"room": TEST_ROOM, "choices": [{"video_id": "x", "title": "X"}]},
|
|
||||||
)
|
|
||||||
client.post(
|
|
||||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
|
||||||
)
|
|
||||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["choices"] == []
|
|
||||||
|
|
||||||
stopped = client.post(
|
|
||||||
f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "stop"}
|
|
||||||
).json()
|
|
||||||
assert stopped["desired"] == "stopped"
|
|
||||||
finally:
|
|
||||||
upstream.shutdown()
|
|
||||||
# Remove the test stream from the permanent library.
|
|
||||||
try:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
|
||||||
except (httpx.HTTPError, UnboundLocalError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_stream_proxy_and_range(client):
|
|
||||||
"""External audio URLs must be reachable same-origin (HTTPS) and support
|
|
||||||
Range passthrough so the browser can seek progressive streams.
|
|
||||||
|
|
||||||
Local deployments test against a controlled fake upstream; remote
|
|
||||||
deployments (Unraid) cannot reach the test host's ephemeral ports, so a
|
|
||||||
public reference mp3 is used and Range semantics are asserted without
|
|
||||||
byte equality."""
|
|
||||||
if IS_LOCAL_DEPLOYMENT:
|
|
||||||
payload = b"0123456789" * 100
|
|
||||||
upstream, url = _serve_fake_audio(payload)
|
|
||||||
track_id = None
|
|
||||||
try:
|
|
||||||
track_id = client.post(
|
|
||||||
f"{MEDIA_API}/streams", json={"name": "Range FM", "url": url}
|
|
||||||
).json()["track"]["id"]
|
|
||||||
|
|
||||||
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
|
||||||
assert full.status_code == 200
|
|
||||||
assert full.content == payload
|
|
||||||
|
|
||||||
ranged = client.get(
|
|
||||||
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=100-199"}
|
|
||||||
)
|
|
||||||
assert ranged.status_code == 206
|
|
||||||
assert ranged.content == payload[100:200]
|
|
||||||
assert ranged.headers["content-range"] == f"bytes 100-199/{len(payload)}"
|
|
||||||
finally:
|
|
||||||
upstream.shutdown()
|
|
||||||
if track_id:
|
|
||||||
try:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
|
||||||
except httpx.HTTPError:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
|
|
||||||
# Remote deployment: public reference audio (SoundHelix).
|
|
||||||
url = "https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3"
|
|
||||||
track_id = None
|
|
||||||
try:
|
|
||||||
track_id = client.post(
|
|
||||||
f"{MEDIA_API}/streams", json={"name": "Range FM public", "url": url}
|
|
||||||
).json()["track"]["id"]
|
|
||||||
|
|
||||||
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
|
||||||
assert full.status_code == 200
|
|
||||||
assert full.headers.get("content-type", "").startswith("audio/")
|
|
||||||
assert len(full.content) > 100_000
|
|
||||||
|
|
||||||
ranged = client.get(
|
|
||||||
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=0-1023"}
|
|
||||||
)
|
|
||||||
assert ranged.status_code == 206
|
|
||||||
assert len(ranged.content) == 1024
|
|
||||||
assert (ranged.headers.get("content-range") or "").startswith("bytes 0-1023/")
|
|
||||||
finally:
|
|
||||||
if track_id:
|
|
||||||
try:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
|
||||||
except httpx.HTTPError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_files_static_range_support(client):
|
|
||||||
"""Downloaded tracks are nginx static files with native Range support.
|
|
||||||
|
|
||||||
Writes a probe file into the repo's ./media bind mount, so it only runs
|
|
||||||
when the test host IS the deployment host (local dev deployment)."""
|
|
||||||
if not IS_LOCAL_DEPLOYMENT:
|
|
||||||
pytest.skip("writes to the repo ./media bind mount — local deployments only")
|
|
||||||
import pathlib
|
|
||||||
|
|
||||||
marker = uuid.uuid4().hex[:8]
|
|
||||||
host_media = pathlib.Path(__file__).parents[1] / "media" / "files"
|
|
||||||
host_media.mkdir(parents=True, exist_ok=True)
|
|
||||||
fname = f"test-{marker}.mp3"
|
|
||||||
payload = b"IMHEADDATA" + b"\x01" * 2048
|
|
||||||
(host_media / fname).write_bytes(payload)
|
|
||||||
try:
|
|
||||||
full = client.get(f"/media/files/{fname}")
|
|
||||||
assert full.status_code == 200
|
|
||||||
assert full.content == payload
|
|
||||||
|
|
||||||
ranged = client.get(f"/media/files/{fname}", headers={"Range": "bytes=0-9"})
|
|
||||||
assert ranged.status_code == 206
|
|
||||||
assert ranged.content == b"IMHEADDATA"
|
|
||||||
finally:
|
|
||||||
(host_media / fname).unlink(missing_ok=True)
|
|
||||||
|
|
||||||
|
|
||||||
def test_media_library_persists_in_volume(client):
|
|
||||||
"""The library JSON is written to the ./media bind mount (permanent).
|
|
||||||
|
|
||||||
Reads the repo's ./media directory — local deployments only."""
|
|
||||||
if not IS_LOCAL_DEPLOYMENT:
|
|
||||||
pytest.skip("reads the repo ./media bind mount — local deployments only")
|
|
||||||
import pathlib
|
|
||||||
|
|
||||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
|
||||||
track_id = None
|
|
||||||
try:
|
|
||||||
track_id = client.post(
|
|
||||||
f"{MEDIA_API}/streams", json={"name": "Persist FM", "url": url}
|
|
||||||
).json()["track"]["id"]
|
|
||||||
library_file = pathlib.Path(__file__).parents[1] / "media" / "library.json"
|
|
||||||
data = json.loads(library_file.read_text())
|
|
||||||
assert track_id in data["tracks"]
|
|
||||||
finally:
|
|
||||||
upstream.shutdown()
|
|
||||||
if track_id:
|
|
||||||
try:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
|
||||||
except httpx.HTTPError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ── web player UI ───────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def test_ui_media_panel_serves(client):
|
|
||||||
resp = client.get("/")
|
|
||||||
body = resp.text
|
|
||||||
assert 'id="mediaBtn"' in body
|
|
||||||
assert 'id="mediaPanel"' in body
|
|
||||||
assert 'id="playPauseBtn"' in body
|
|
||||||
assert 'id="seekBar"' in body
|
|
||||||
assert 'id="mediaAudio"' in body
|
|
||||||
|
|
||||||
|
|
||||||
def test_ui_media_logic_serves(client):
|
|
||||||
resp = client.get("/app.js")
|
|
||||||
body = resp.text
|
|
||||||
assert "reconcileMedia" in body
|
|
||||||
assert "media-control" in body # data-channel topic for the agent
|
|
||||||
assert "media_state" in body # playback state reports
|
|
||||||
assert "/media/api/state" in body # reconciliation polling
|
|
||||||
|
|
||||||
|
|
||||||
def test_ui_media_styles_serve(client):
|
|
||||||
resp = client.get("/style.css")
|
|
||||||
assert ".media-panel" in resp.text
|
|
||||||
assert ".media-row" in resp.text
|
|
||||||
|
|
||||||
|
|
||||||
def test_token_response_includes_room():
|
|
||||||
resp = httpx.post(TOKEN_URL, json={}, timeout=10, verify=False)
|
|
||||||
assert resp.status_code == 200
|
|
||||||
assert resp.json()["room"].startswith("voice-")
|
|
||||||
|
|
||||||
|
|
||||||
# ── Hope's media MCP toolset ────────────────────────────────────────────────
|
|
||||||
# These spawn the MCP server inside the LOCAL dev container (docker exec),
|
|
||||||
# whose media server is the local one — they can only be consistent when
|
|
||||||
# the test target IS the local deployment.
|
|
||||||
|
|
||||||
MEDIA_MCP_ENV = {
|
|
||||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8093",
|
|
||||||
"MEDIA_ROOM": "mcp-test-room",
|
|
||||||
}
|
|
||||||
|
|
||||||
needs_local_container = pytest.mark.skipif(
|
|
||||||
not IS_LOCAL_DEPLOYMENT,
|
|
||||||
reason="docker-exec MCP probes run against the local dev container",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── media server internals (in-process, inside the container) ───────────────
|
|
||||||
|
|
||||||
@needs_local_container
|
|
||||||
def test_media_search_prefers_library_matches(client):
|
|
||||||
"""Search returns ready library matches ahead of YouTube results, so
|
|
||||||
Hope prefers an already-downloaded copy. Runs the endpoint in-process
|
|
||||||
with _run_search stubbed — fully deterministic, no network."""
|
|
||||||
code = '''
|
|
||||||
import sys
|
|
||||||
sys.path.insert(0, "/opt/voice")
|
|
||||||
import media_server as ms
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
ms._library["tracks"]["libpref01"] = {
|
|
||||||
"id": "libpref01", "kind": "youtube", "video_id": "libpref01",
|
|
||||||
"title": "Preference Probe Song", "channel": "Probe Artist",
|
|
||||||
"duration": 120, "status": "ready", "ext": "webm",
|
|
||||||
"error": None, "added_at": 1.0,
|
|
||||||
}
|
|
||||||
ms._run_search = lambda q, limit: [
|
|
||||||
{"video_id": "remote01", "title": "Preference Probe Song (cover)",
|
|
||||||
"channel": "", "duration": 99, "view_count": 1,
|
|
||||||
"in_library": False, "track_id": None, "library_status": None},
|
|
||||||
]
|
|
||||||
|
|
||||||
c = TestClient(ms.app)
|
|
||||||
r = c.post("/media/api/search", json={"query": "preference probe", "limit": 5})
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
results = r.json()["results"]
|
|
||||||
assert results, "no results"
|
|
||||||
assert results[0]["video_id"] == "libpref01", results
|
|
||||||
assert results[0]["in_library"] is True
|
|
||||||
assert any(x["video_id"] == "remote01" for x in results), results
|
|
||||||
print("OK")
|
|
||||||
'''
|
|
||||||
proc = docker_exec_python(code, timeout=60, check=False)
|
|
||||||
assert proc.returncode == 0, proc.stderr[-800:]
|
|
||||||
assert "OK" in proc.stdout
|
|
||||||
|
|
||||||
|
|
||||||
@needs_local_container
|
|
||||||
def test_media_stream_endpoint_serves_ready_track(client):
|
|
||||||
"""GET /media/api/stream/{id} for a finished youtube track serves the
|
|
||||||
file bytes with native Range support (progressive-playback endpoint)."""
|
|
||||||
code = '''
|
|
||||||
import sys
|
|
||||||
sys.path.insert(0, "/opt/voice")
|
|
||||||
import media_server as ms
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
probe = ms.FILES_DIR / "probetrk.mp3"
|
|
||||||
probe.write_bytes(b"PROBEDATA" * 100)
|
|
||||||
ms._library["tracks"]["probetrk"] = {
|
|
||||||
"id": "probetrk", "kind": "youtube", "video_id": "probetrk",
|
|
||||||
"title": "Probe", "channel": "", "duration": 1,
|
|
||||||
"status": "ready", "ext": "mp3", "error": None, "added_at": 1.0,
|
|
||||||
}
|
|
||||||
try:
|
|
||||||
c = TestClient(ms.app)
|
|
||||||
full = c.get("/media/api/stream/probetrk")
|
|
||||||
assert full.status_code == 200, full.status_code
|
|
||||||
assert full.content == b"PROBEDATA" * 100
|
|
||||||
part = c.get("/media/api/stream/probetrk", headers={"Range": "bytes=0-8"})
|
|
||||||
assert part.status_code == 206, part.status_code
|
|
||||||
assert part.content == b"PROBEDATA"
|
|
||||||
assert (part.headers.get("content-range") or "").startswith("bytes 0-8/")
|
|
||||||
# track dict exposes a stable stream URL while downloading and ready
|
|
||||||
d = ms._track_dict(ms._library["tracks"]["probetrk"])
|
|
||||||
assert d["audio_url"] == "/media/api/stream/probetrk", d
|
|
||||||
print("OK")
|
|
||||||
finally:
|
|
||||||
probe.unlink(missing_ok=True)
|
|
||||||
ms._library["tracks"].pop("probetrk", None)
|
|
||||||
'''
|
|
||||||
proc = docker_exec_python(code, timeout=60, check=False)
|
|
||||||
assert proc.returncode == 0, proc.stderr[-800:]
|
|
||||||
assert "OK" in proc.stdout
|
|
||||||
|
|
||||||
|
|
||||||
@needs_local_container
|
|
||||||
def test_media_play_resolves_cached_ready_track(client):
|
|
||||||
"""Playing an ALREADY-cached song must resolve to the on-disk ready track
|
|
||||||
(no redownload) and hand the browser a playable stream URL, so a cached
|
|
||||||
pick starts instantly rather than re-fetching from YouTube.
|
|
||||||
|
|
||||||
Regression: a cached track could be left as paused with currentTime 0 if
|
|
||||||
the room the agent played into diverged from the room the browser polled,
|
|
||||||
or if autoplay was blocked. This pins the server-side resolve + track
|
|
||||||
dict so a ready file is always returned with a stable audio_url."""
|
|
||||||
code = '''
|
|
||||||
import sys
|
|
||||||
sys.path.insert(0, "/opt/voice")
|
|
||||||
import media_server as ms
|
|
||||||
from fastapi.testclient import TestClient
|
|
||||||
|
|
||||||
probe = ms.FILES_DIR / "cachetrk.webm"
|
|
||||||
probe.write_bytes(b"CACHEDBYTES" * 40)
|
|
||||||
ms._library["tracks"]["cachetrk"] = {
|
|
||||||
"id": "cachetrk", "kind": "youtube", "video_id": "cachetrk",
|
|
||||||
"title": "Cached Song", "channel": "Artist", "duration": 200,
|
|
||||||
"status": "ready", "ext": "webm", "error": None, "added_at": 1.0,
|
|
||||||
}
|
|
||||||
ms._start_download = lambda *a, **k: (_ for _ in ()).throw(
|
|
||||||
AssertionError("must not re-download an already cached track")
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
c = TestClient(ms.app)
|
|
||||||
# play by video_id (as Hope's play_song does) must reuse the ready track
|
|
||||||
r = c.post("/media/api/play", json={
|
|
||||||
"room": "cached-test-room", "video_id": "cachetrk",
|
|
||||||
"title": "Cached Song", "channel": "Artist",
|
|
||||||
}, timeout=10)
|
|
||||||
assert r.status_code == 200, r.text
|
|
||||||
body = r.json()
|
|
||||||
assert body["desired"] == "playing", body
|
|
||||||
track = body["track"]
|
|
||||||
assert track["status"] == "ready", track
|
|
||||||
assert track["audio_url"] == "/media/api/stream/cachetrk", track
|
|
||||||
# the stream endpoint serves the cached bytes (browser plays it directly)
|
|
||||||
data = c.get("/media/api/stream/cachetrk")
|
|
||||||
assert data.status_code == 200
|
|
||||||
assert data.content == b"CACHEDBYTES" * 40
|
|
||||||
print("OK")
|
|
||||||
finally:
|
|
||||||
probe.unlink(missing_ok=True)
|
|
||||||
ms._library["tracks"].pop("cachetrk", None)
|
|
||||||
'''
|
|
||||||
proc = docker_exec_python(code, timeout=60, check=False)
|
|
||||||
assert proc.returncode == 0, proc.stderr[-800:]
|
|
||||||
assert "OK" in proc.stdout
|
|
||||||
|
|
||||||
|
|
||||||
@needs_local_container
|
@needs_local_container
|
||||||
def test_mcp_media_list_library(mcp_client_factory):
|
def test_mcp_media_list_library(mcp_client_factory):
|
||||||
@@ -490,8 +118,9 @@ def test_mcp_media_list_library(mcp_client_factory):
|
|||||||
|
|
||||||
|
|
||||||
@needs_local_container
|
@needs_local_container
|
||||||
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory):
|
||||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||||
|
track_id = None
|
||||||
try:
|
try:
|
||||||
text, is_error = mcp_client_factory(
|
text, is_error = mcp_client_factory(
|
||||||
"media_mcp.py", "add_stream",
|
"media_mcp.py", "add_stream",
|
||||||
@@ -500,99 +129,39 @@ def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
|||||||
assert not is_error, text
|
assert not is_error, text
|
||||||
assert "Added" in text
|
assert "Added" in text
|
||||||
|
|
||||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||||
probe = next(t for t in library if t["title"] == "MCP Test Radio")
|
library = music.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||||
try:
|
probe = next(t for t in library if t["title"] == "MCP Test Radio")
|
||||||
text, is_error = mcp_client_factory(
|
track_id = probe["id"]
|
||||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
music.post(
|
||||||
)
|
f"{MEDIA_API}/play",
|
||||||
assert not is_error
|
json={"room": "mcp-test-room", "track_id": track_id},
|
||||||
# Nothing loaded in mcp-test-room: the tool must say so plainly.
|
).raise_for_status()
|
||||||
assert "Nothing is loaded" in text
|
|
||||||
|
|
||||||
client.post(
|
text, is_error = mcp_client_factory(
|
||||||
f"{MEDIA_API}/play", json={"room": "mcp-test-room", "track_id": probe["id"]}
|
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||||
)
|
)
|
||||||
text, is_error = mcp_client_factory(
|
assert not is_error
|
||||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
assert "MCP Test Radio" in text
|
||||||
)
|
|
||||||
assert not is_error
|
|
||||||
assert "MCP Test Radio" in text
|
|
||||||
finally:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{probe['id']}")
|
|
||||||
finally:
|
finally:
|
||||||
upstream.shutdown()
|
upstream.shutdown()
|
||||||
|
if track_id:
|
||||||
|
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||||
|
music.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||||
|
|
||||||
|
|
||||||
@needs_local_container
|
@needs_local_container
|
||||||
def test_mcp_media_resume_without_load(mcp_client_factory, client):
|
def test_mcp_media_resume_without_load(mcp_client_factory):
|
||||||
"""Errors from the media server reach the LLM as tool text (not MCP
|
room = "mcp-empty-room"
|
||||||
errors) so Hope can explain them naturally."""
|
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||||
client.post(
|
music.post(
|
||||||
f"{MEDIA_API}/control", json={"room": "mcp-empty-room", "action": "stop"}
|
f"{MEDIA_API}/control",
|
||||||
)
|
json={"room": room, "action": "stop"},
|
||||||
|
).raise_for_status()
|
||||||
|
|
||||||
text, is_error = mcp_client_factory(
|
text, is_error = mcp_client_factory(
|
||||||
"media_mcp.py", "resume_music", {}, env={
|
"media_mcp.py", "resume_music", {},
|
||||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8093",
|
env={"MUSIC_BASE_URL": MUSIC_BASE_URL, "MUSIC_ROOM": room},
|
||||||
"MEDIA_ROOM": "mcp-empty-room",
|
|
||||||
}
|
|
||||||
)
|
)
|
||||||
assert not is_error
|
assert not is_error
|
||||||
assert "nothing loaded" in text.lower()
|
assert "nothing loaded" in text.lower()
|
||||||
|
|
||||||
|
|
||||||
# ── live YouTube Music (network-dependent) ──────────────────────────────────
|
|
||||||
|
|
||||||
@pytest.mark.slow
|
|
||||||
def test_media_search_live(client):
|
|
||||||
resp = client.post(
|
|
||||||
f"{MEDIA_API}/search",
|
|
||||||
json={"query": "blind guardian timeless miracle", "limit": 3},
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200
|
|
||||||
results = resp.json()["results"]
|
|
||||||
assert len(results) >= 1
|
|
||||||
for r in results:
|
|
||||||
assert r["video_id"]
|
|
||||||
assert r["title"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.slow
|
|
||||||
def test_media_download_and_stream_end_to_end(client):
|
|
||||||
"""Full pipeline: search -> play -> yt-dlp download -> playable stream.
|
|
||||||
The audio_url stays on /media/api/stream/ for the whole lifecycle; once
|
|
||||||
ready it serves the finished file with Range support. Gated: run with
|
|
||||||
HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
|
|
||||||
if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
|
|
||||||
pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")
|
|
||||||
|
|
||||||
search = client.post(
|
|
||||||
f"{MEDIA_API}/search", json={"query": "kingdom hearts dearly beloved", "limit": 3}
|
|
||||||
)
|
|
||||||
results = search.json()["results"]
|
|
||||||
assert results, "search returned nothing"
|
|
||||||
video = results[0]
|
|
||||||
|
|
||||||
played = client.post(
|
|
||||||
f"{MEDIA_API}/play",
|
|
||||||
json={"room": TEST_ROOM, "video_id": video["video_id"],
|
|
||||||
"title": video["title"], "channel": video["channel"]},
|
|
||||||
).json()
|
|
||||||
track_id = played["track"]["id"]
|
|
||||||
|
|
||||||
try:
|
|
||||||
deadline = time.time() + 240
|
|
||||||
track = played["track"]
|
|
||||||
while time.time() < deadline:
|
|
||||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
|
||||||
track = next(t for t in library if t["id"] == track_id)
|
|
||||||
if track["status"] in ("ready", "failed"):
|
|
||||||
break
|
|
||||||
time.sleep(3)
|
|
||||||
assert track["status"] == "ready", f"download failed: {track.get('error')}"
|
|
||||||
|
|
||||||
stream = client.get(track["audio_url"], headers={"Range": "bytes=0-1023"})
|
|
||||||
assert stream.status_code in (200, 206)
|
|
||||||
assert len(stream.content) >= 1
|
|
||||||
finally:
|
|
||||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
|
||||||
-568
@@ -269,18 +269,6 @@ const LED_GROW = 1.4; // how much the dots grow toward the center
|
|||||||
let ledDots = [];
|
let ledDots = [];
|
||||||
let ledWallTime = 0;
|
let ledWallTime = 0;
|
||||||
|
|
||||||
// Decides whether the LED wall square should show the track's album art.
|
|
||||||
// Music and conversation are mutually exclusive: while a conversation is
|
|
||||||
// active the music is paused, so the wall pulses (agent audio) and any loaded
|
|
||||||
// track's cover art returns once the talk ends.
|
|
||||||
function updateAvatarMedia() {
|
|
||||||
const inConversation = !!(room && room.localParticipant);
|
|
||||||
const showArt = !inConversation && !!mediaCurrentTrackId && !!mediaCurrentArtUrl;
|
|
||||||
ledWallEl.classList.toggle("show-art", showArt);
|
|
||||||
if (showArt && ledArtEl.getAttribute("src") !== mediaCurrentArtUrl) {
|
|
||||||
ledArtEl.src = mediaCurrentArtUrl;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function buildLedWall() {
|
function buildLedWall() {
|
||||||
const inner = Math.floor(LED_COLS / 2);
|
const inner = Math.floor(LED_COLS / 2);
|
||||||
@@ -781,7 +769,6 @@ document.addEventListener("keydown", (event) => {
|
|||||||
if (event.key !== "Escape") return;
|
if (event.key !== "Escape") return;
|
||||||
if (settingsSheet.classList.contains("open")) closeSheet();
|
if (settingsSheet.classList.contains("open")) closeSheet();
|
||||||
if (tasksPanel.classList.contains("open")) setTasksOpen(false);
|
if (tasksPanel.classList.contains("open")) setTasksOpen(false);
|
||||||
if (mediaPanel.classList.contains("open")) setMediaOpen(false);
|
|
||||||
});
|
});
|
||||||
// ── Wake lock ───────────────────────────────────────────────────────────────
|
// ── Wake lock ───────────────────────────────────────────────────────────────
|
||||||
async function acquireWakeLock() {
|
async function acquireWakeLock() {
|
||||||
@@ -905,8 +892,6 @@ function createRoom(token) {
|
|||||||
stopBtn.disabled = true;
|
stopBtn.disabled = true;
|
||||||
agentParticipant = null;
|
agentParticipant = null;
|
||||||
agentReady = false;
|
agentReady = false;
|
||||||
// Leaving the conversation hands the room back to the music player.
|
|
||||||
resumePlaybackAfterTalk();
|
|
||||||
});
|
});
|
||||||
|
|
||||||
return newRoom.connect(LIVEKIT_URL, token, {
|
return newRoom.connect(LIVEKIT_URL, token, {
|
||||||
@@ -965,7 +950,6 @@ async function connectConversation() {
|
|||||||
|
|
||||||
// Get a signed access token from the token endpoint.
|
// Get a signed access token from the token endpoint.
|
||||||
const tokenData = await fetchToken(ROOM_NAME);
|
const tokenData = await fetchToken(ROOM_NAME);
|
||||||
await enterMediaScope(tokenData.room);
|
|
||||||
room = await createRoom(tokenData.token);
|
room = await createRoom(tokenData.token);
|
||||||
|
|
||||||
// The agent is dispatched when we join. If it misses the dispatch
|
// The agent is dispatched when we join. If it misses the dispatch
|
||||||
@@ -979,7 +963,6 @@ async function connectConversation() {
|
|||||||
agentReady = false;
|
agentReady = false;
|
||||||
await stale.disconnect();
|
await stale.disconnect();
|
||||||
const freshTokenData = await fetchToken(ROOM_NAME);
|
const freshTokenData = await fetchToken(ROOM_NAME);
|
||||||
await enterMediaScope(freshTokenData.room);
|
|
||||||
room = await createRoom(freshTokenData.token);
|
room = await createRoom(freshTokenData.token);
|
||||||
if (!(await waitForAgent(15000))) {
|
if (!(await waitForAgent(15000))) {
|
||||||
throw new Error("Assistant did not join — reload the page and try again");
|
throw new Error("Assistant did not join — reload the page and try again");
|
||||||
@@ -1011,7 +994,6 @@ async function connectConversation() {
|
|||||||
muteBtn.disabled = conversationMode !== "voice";
|
muteBtn.disabled = conversationMode !== "voice";
|
||||||
updateMuteIcon();
|
updateMuteIcon();
|
||||||
setState("listening");
|
setState("listening");
|
||||||
updateAvatarMedia();
|
|
||||||
return room;
|
return room;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error("Connection failed:", err);
|
console.error("Connection failed:", err);
|
||||||
@@ -1093,9 +1075,6 @@ stopBtn.addEventListener("click", () => {
|
|||||||
muteBtn.disabled = true;
|
muteBtn.disabled = true;
|
||||||
micMuted = false;
|
micMuted = false;
|
||||||
updateMuteIcon();
|
updateMuteIcon();
|
||||||
// Ending the conversation resumes music that was paused for the talk.
|
|
||||||
resumePlaybackAfterTalk();
|
|
||||||
updateAvatarMedia();
|
|
||||||
});
|
});
|
||||||
|
|
||||||
// ── Mic mute toggle ─────────────────────────────────────────────────────────
|
// ── Mic mute toggle ─────────────────────────────────────────────────────────
|
||||||
@@ -1162,551 +1141,4 @@ function sendModelMessage(model, targetRoom) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Media player ────────────────────────────────────────────────────────────
|
|
||||||
// The browser <audio> element is the player. The media server owns library +
|
|
||||||
// per-room desired state; this UI reconciles against GET /media/api/state
|
|
||||||
// (polling) and reports real playback back. Hope drives the same state via
|
|
||||||
// her media tools, so voice commands and panel clicks stay in sync.
|
|
||||||
//
|
|
||||||
// Conversation rule (mirrored in agent.py): while a track plays the agent's
|
|
||||||
// mic is off. Connecting to Hope pauses music; ending the conversation
|
|
||||||
// resumes what was playing.
|
|
||||||
|
|
||||||
const mediaBtn = document.getElementById("mediaBtn");
|
|
||||||
const mediaPanel = document.getElementById("mediaPanel");
|
|
||||||
const closeMediaBtn = document.getElementById("closeMediaBtn");
|
|
||||||
const mediaBadge = document.getElementById("mediaBadge");
|
|
||||||
const mediaArt = document.getElementById("mediaArt");
|
|
||||||
const mediaArtPlaceholder = document.getElementById("mediaArtPlaceholder");
|
|
||||||
const mediaTitleEl = document.getElementById("mediaTitle");
|
|
||||||
const mediaChannelEl = document.getElementById("mediaChannel");
|
|
||||||
const mediaProgressWrap = document.getElementById("mediaProgressWrap");
|
|
||||||
const mediaProgressFill = document.getElementById("mediaProgressFill");
|
|
||||||
const mediaProgressLabel = document.getElementById("mediaProgressLabel");
|
|
||||||
const rewBtn = document.getElementById("rewBtn");
|
|
||||||
const fwdBtn = document.getElementById("fwdBtn");
|
|
||||||
const playPauseBtn = document.getElementById("playPauseBtn");
|
|
||||||
const playIcon = document.getElementById("playIcon");
|
|
||||||
const pauseIcon = document.getElementById("pauseIcon");
|
|
||||||
const stopMediaBtn = document.getElementById("stopMediaBtn");
|
|
||||||
const seekBar = document.getElementById("seekBar");
|
|
||||||
const mediaCurEl = document.getElementById("mediaCur");
|
|
||||||
const mediaDurEl = document.getElementById("mediaDur");
|
|
||||||
const volumeBar = document.getElementById("volumeBar");
|
|
||||||
const mediaSearchForm = document.getElementById("mediaSearchForm");
|
|
||||||
const mediaSearchInput = document.getElementById("mediaSearchInput");
|
|
||||||
const mediaResultsEl = document.getElementById("mediaResults");
|
|
||||||
const mediaChoicesWrap = document.getElementById("mediaChoicesWrap");
|
|
||||||
const mediaChoicesEl = document.getElementById("mediaChoices");
|
|
||||||
const mediaLibraryEl = document.getElementById("mediaLibrary");
|
|
||||||
const streamForm = document.getElementById("streamForm");
|
|
||||||
const streamNameInput = document.getElementById("streamName");
|
|
||||||
const streamUrlInput = document.getElementById("streamUrl");
|
|
||||||
const mediaAudio = document.getElementById("mediaAudio");
|
|
||||||
const ledArtEl = document.getElementById("ledArt");
|
|
||||||
|
|
||||||
let mediaRoom = null; // LiveKit room name (player state scope)
|
|
||||||
let mediaCurrentTrackId = null; // track id loaded into the audio element
|
|
||||||
let mediaCurrentArtUrl = null; // current track's cover art
|
|
||||||
let mediaAppliedSeekTs = 0; // last applied seek_hint timestamp
|
|
||||||
let mediaWasPlayingBeforeTalk = false;
|
|
||||||
let mediaPollTimer = null;
|
|
||||||
let mediaLastReport = 0;
|
|
||||||
|
|
||||||
function mediaScope() {
|
|
||||||
// The LiveKit room is single-source of truth for the player scope. If a
|
|
||||||
// room is connected, poll the room the agent actually joined (room.name),
|
|
||||||
// so a "play X" request from the agent can never land in a stale scope.
|
|
||||||
if (room && room.name) return room.name;
|
|
||||||
return mediaRoom || "default";
|
|
||||||
}
|
|
||||||
|
|
||||||
async function mediaPost(path, body) {
|
|
||||||
const resp = await fetch(`/media/api${path}`, {
|
|
||||||
method: "POST",
|
|
||||||
headers: { "Content-Type": "application/json" },
|
|
||||||
body: JSON.stringify({ room: mediaScope(), ...body }),
|
|
||||||
});
|
|
||||||
if (!resp.ok) {
|
|
||||||
const detail = await resp.json().catch(() => ({}));
|
|
||||||
throw new Error(detail.error || `media API ${path} -> ${resp.status}`);
|
|
||||||
}
|
|
||||||
return resp.json();
|
|
||||||
}
|
|
||||||
|
|
||||||
// Called just before joining the LiveKit room: carry whatever the audio
|
|
||||||
// element holds into the room-scoped player state, paused (talking pauses
|
|
||||||
// music). resumePlaybackAfterTalk() reverses this when the talk ends.
|
|
||||||
async function enterMediaScope(newRoom) {
|
|
||||||
if (!newRoom || newRoom === mediaRoom) return;
|
|
||||||
const wasPlaying = !!mediaCurrentTrackId && !mediaAudio.paused;
|
|
||||||
mediaWasPlayingBeforeTalk = wasPlaying;
|
|
||||||
const carriedTrack = mediaCurrentTrackId;
|
|
||||||
mediaRoom = newRoom;
|
|
||||||
if (carriedTrack) {
|
|
||||||
try {
|
|
||||||
await mediaPost("/play", { room: mediaRoom, track_id: carriedTrack, start_paused: true });
|
|
||||||
} catch (e) { console.warn("media scope transfer failed:", e); }
|
|
||||||
mediaAudio.pause();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function resumePlaybackAfterTalk() {
|
|
||||||
if (!mediaWasPlayingBeforeTalk) return;
|
|
||||||
mediaWasPlayingBeforeTalk = false;
|
|
||||||
try { await mediaPost("/control", { action: "resume" }); } catch (e) { /* poll retries */ }
|
|
||||||
}
|
|
||||||
|
|
||||||
function publishMediaState(state) {
|
|
||||||
if (!room || !room.localParticipant) return;
|
|
||||||
const payload = new TextEncoder().encode(
|
|
||||||
JSON.stringify({ type: "media_state", state, track_id: mediaCurrentTrackId })
|
|
||||||
);
|
|
||||||
room.localParticipant.publishData(payload, { topic: "media-control", reliable: true });
|
|
||||||
}
|
|
||||||
|
|
||||||
function reportMediaPlayback(state) {
|
|
||||||
const body = {
|
|
||||||
state,
|
|
||||||
track_id: mediaCurrentTrackId,
|
|
||||||
position: isFinite(mediaAudio.currentTime) ? mediaAudio.currentTime : 0,
|
|
||||||
};
|
|
||||||
if (isFinite(mediaAudio.duration)) body.duration = mediaAudio.duration;
|
|
||||||
mediaPost("/report", body).catch(() => {});
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Polling + reconciliation ────────────────────────────────────────────────
|
|
||||||
function startMediaPolling() {
|
|
||||||
if (mediaPollTimer) return;
|
|
||||||
mediaPollTimer = setInterval(() => {
|
|
||||||
pollMediaState();
|
|
||||||
}, 1500);
|
|
||||||
}
|
|
||||||
|
|
||||||
async function pollMediaState() {
|
|
||||||
let s;
|
|
||||||
try {
|
|
||||||
const resp = await fetch(`/media/api/state?room=${encodeURIComponent(mediaScope())}`);
|
|
||||||
if (!resp.ok) return;
|
|
||||||
s = await resp.json();
|
|
||||||
} catch (e) {
|
|
||||||
return; // media server briefly unavailable; try again next tick
|
|
||||||
}
|
|
||||||
reconcileMedia(s);
|
|
||||||
}
|
|
||||||
|
|
||||||
function reconcileMedia(s) {
|
|
||||||
renderMediaChoices(s.choices || []);
|
|
||||||
updateMediaBadge(s);
|
|
||||||
|
|
||||||
const track = s.track;
|
|
||||||
const pendingJob = (s.pending || []).find(j => !track || j.track_id === track.id);
|
|
||||||
updateMediaProgress(track, pendingJob);
|
|
||||||
|
|
||||||
if (track && track.id !== mediaCurrentTrackId) {
|
|
||||||
loadMediaTrack(track);
|
|
||||||
} else if (track && track.audio_url && mediaAudio.getAttribute("src") !== track.audio_url) {
|
|
||||||
// The track was loaded before it had a stream (still downloading) and
|
|
||||||
// now has one — wire it up so playback can start without a reload.
|
|
||||||
mediaAudio.src = track.audio_url;
|
|
||||||
mediaAudio.load();
|
|
||||||
updateMediaNowCard(track);
|
|
||||||
}
|
|
||||||
if (!track && mediaCurrentTrackId && s.desired === "stopped") clearMediaAudio();
|
|
||||||
|
|
||||||
applyDesiredMediaState(s, track);
|
|
||||||
|
|
||||||
if (s.seek_hint && s.seek_hint.ts !== mediaAppliedSeekTs && isFinite(mediaAudio.duration)) {
|
|
||||||
mediaAppliedSeekTs = s.seek_hint.ts;
|
|
||||||
const target = Math.min(s.seek_hint.position, mediaAudio.duration);
|
|
||||||
if (Number.isFinite(target)) {
|
|
||||||
mediaAudio.currentTime = Math.max(0, target);
|
|
||||||
updateSeekUI();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function applyDesiredMediaState(s, track) {
|
|
||||||
if (s.desired === "playing") {
|
|
||||||
// audio_url presence means a playable stream exists — including a
|
|
||||||
// still-growing download, which plays progressively.
|
|
||||||
if (track && track.audio_url && mediaAudio.src && mediaAudio.paused) {
|
|
||||||
mediaAudio.play().catch((e) => console.warn("playback blocked:", e));
|
|
||||||
}
|
|
||||||
} else if (!mediaAudio.paused) {
|
|
||||||
mediaAudio.pause();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function loadMediaTrack(track) {
|
|
||||||
mediaCurrentTrackId = track.id;
|
|
||||||
updateMediaNowCard(track);
|
|
||||||
if (track.audio_url) {
|
|
||||||
if (mediaAudio.getAttribute("src") !== track.audio_url) {
|
|
||||||
mediaAudio.src = track.audio_url;
|
|
||||||
mediaAudio.load();
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Downloading or failed: no stream yet. reconcileMedia wires the
|
|
||||||
// stream URL in via polling as soon as one is available.
|
|
||||||
mediaAudio.removeAttribute("src");
|
|
||||||
mediaAudio.load();
|
|
||||||
updateMediaControls();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function clearMediaAudio() {
|
|
||||||
mediaCurrentTrackId = null;
|
|
||||||
mediaCurrentArtUrl = null;
|
|
||||||
mediaAudio.pause();
|
|
||||||
mediaAudio.removeAttribute("src");
|
|
||||||
mediaAudio.load();
|
|
||||||
mediaTitleEl.textContent = "Nothing loaded";
|
|
||||||
mediaChannelEl.textContent = "";
|
|
||||||
mediaArt.hidden = true;
|
|
||||||
mediaArtPlaceholder.hidden = false;
|
|
||||||
updateMediaControls();
|
|
||||||
updateAvatarMedia();
|
|
||||||
}
|
|
||||||
|
|
||||||
function updateMediaNowCard(track) {
|
|
||||||
mediaTitleEl.textContent = track.title || track.id;
|
|
||||||
mediaCurrentArtUrl = track.art_url || null;
|
|
||||||
if (track.status === "failed") {
|
|
||||||
mediaChannelEl.textContent = "Download failed — pick another result or try again.";
|
|
||||||
} else if (track.status === "downloading") {
|
|
||||||
// Progressive streaming: the track plays while the download runs.
|
|
||||||
mediaChannelEl.textContent = track.channel
|
|
||||||
? `${track.channel} · downloading` : "Downloading…";
|
|
||||||
} else {
|
|
||||||
mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
|
|
||||||
}
|
|
||||||
if (track.art_url) {
|
|
||||||
mediaArt.src = track.art_url;
|
|
||||||
mediaArt.hidden = false;
|
|
||||||
mediaArtPlaceholder.hidden = true;
|
|
||||||
} else {
|
|
||||||
mediaArt.hidden = true;
|
|
||||||
mediaArtPlaceholder.hidden = false;
|
|
||||||
}
|
|
||||||
updateAvatarMedia();
|
|
||||||
}
|
|
||||||
|
|
||||||
function updateMediaProgress(track, job) {
|
|
||||||
if (!job) {
|
|
||||||
mediaProgressWrap.hidden = true;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
mediaProgressWrap.hidden = false;
|
|
||||||
const pct = Math.round(job.progress || 0);
|
|
||||||
mediaProgressFill.style.width = `${pct}%`;
|
|
||||||
const stage = job.stage === "queued" ? "Queued…" : "Downloading…";
|
|
||||||
mediaProgressLabel.textContent = `${stage} ${pct}%`;
|
|
||||||
if (track) updateMediaNowCard(track);
|
|
||||||
}
|
|
||||||
|
|
||||||
function updateMediaBadge(s) {
|
|
||||||
const pending = (s.pending || []).length;
|
|
||||||
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
|
||||||
if (pending > 0) {
|
|
||||||
mediaBadge.hidden = false;
|
|
||||||
mediaBadge.textContent = String(pending);
|
|
||||||
} else if (playing) {
|
|
||||||
mediaBadge.hidden = false;
|
|
||||||
mediaBadge.innerHTML = "♫";
|
|
||||||
} else {
|
|
||||||
mediaBadge.hidden = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Playback controls ───────────────────────────────────────────────────────
|
|
||||||
function updateMediaControls() {
|
|
||||||
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
|
||||||
playPauseBtn.classList.toggle("playing", playing);
|
|
||||||
playPauseBtn.setAttribute("aria-label", playing ? "Pause" : "Play");
|
|
||||||
updateMediaBadge({ pending: [] });
|
|
||||||
}
|
|
||||||
|
|
||||||
function fmtTime(sec) {
|
|
||||||
if (!isFinite(sec) || sec < 0) return "0:00";
|
|
||||||
const s = Math.floor(sec % 60);
|
|
||||||
const m = Math.floor((sec / 60) % 60);
|
|
||||||
const h = Math.floor(sec / 3600);
|
|
||||||
return h ? `${h}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}` : `${m}:${String(s).padStart(2, "0")}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
function updateSeekUI() {
|
|
||||||
if (!isFinite(mediaAudio.duration) || mediaAudio.duration <= 0) {
|
|
||||||
seekBar.value = 0;
|
|
||||||
mediaDurEl.textContent = "0:00";
|
|
||||||
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
seekBar.max = Math.floor(mediaAudio.duration);
|
|
||||||
seekBar.value = Math.floor(mediaAudio.currentTime);
|
|
||||||
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
|
||||||
mediaDurEl.textContent = fmtTime(mediaAudio.duration);
|
|
||||||
}
|
|
||||||
|
|
||||||
playPauseBtn.addEventListener("click", () => {
|
|
||||||
if (!mediaCurrentTrackId || !mediaAudio.src) return;
|
|
||||||
if (mediaAudio.paused) {
|
|
||||||
mediaAudio.play().catch((e) => console.warn("play failed:", e));
|
|
||||||
mediaPost("/control", { action: "resume" }).catch(() => {});
|
|
||||||
} else {
|
|
||||||
mediaAudio.pause();
|
|
||||||
mediaPost("/control", { action: "pause" }).catch(() => {});
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
rewBtn.addEventListener("click", () => {
|
|
||||||
if (!mediaAudio.src) return;
|
|
||||||
mediaAudio.currentTime = Math.max(0, mediaAudio.currentTime - 10);
|
|
||||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
|
||||||
});
|
|
||||||
|
|
||||||
fwdBtn.addEventListener("click", () => {
|
|
||||||
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
|
||||||
mediaAudio.currentTime = Math.min(mediaAudio.duration - 0.5, mediaAudio.currentTime + 10);
|
|
||||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
|
||||||
});
|
|
||||||
|
|
||||||
stopMediaBtn.addEventListener("click", () => {
|
|
||||||
clearMediaAudio();
|
|
||||||
mediaPost("/control", { action: "stop" }).catch(() => {});
|
|
||||||
publishMediaState("stopped");
|
|
||||||
});
|
|
||||||
|
|
||||||
seekBar.addEventListener("change", () => {
|
|
||||||
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
|
||||||
mediaAudio.currentTime = Number(seekBar.value);
|
|
||||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
|
||||||
});
|
|
||||||
|
|
||||||
volumeBar.addEventListener("input", () => {
|
|
||||||
mediaAudio.volume = Number(volumeBar.value) / 100;
|
|
||||||
try { localStorage.setItem("media_volume", volumeBar.value); } catch (e) { /* ok */ }
|
|
||||||
});
|
|
||||||
mediaAudio.volume = Number(localStorage.getItem("media_volume") ?? 90) / 100;
|
|
||||||
volumeBar.value = Math.round(mediaAudio.volume * 100);
|
|
||||||
|
|
||||||
// Browsers block audio until a user gestured on the page. Hope often starts a
|
|
||||||
// cached song from a poll (voice command, no click on the player), so the
|
|
||||||
// <audio> element sits outside the LiveKit room's audio-unlock flow and its
|
|
||||||
// play() is rejected. Capture the first gesture and unlock the element.
|
|
||||||
let mediaAudioUnlocked = false;
|
|
||||||
function unlockMediaAudio() {
|
|
||||||
if (mediaAudioUnlocked) return;
|
|
||||||
mediaAudioUnlocked = true;
|
|
||||||
removeEventListener("pointerdown", unlockMediaAudio, true);
|
|
||||||
removeEventListener("keydown", unlockMediaAudio, true);
|
|
||||||
removeEventListener("touchstart", unlockMediaAudio, true);
|
|
||||||
// Play the current src (if any) inside the gesture to whitelist the
|
|
||||||
// element for autoplay, so poll-driven play() later isn't blocked.
|
|
||||||
if (mediaAudio.getAttribute("src") && mediaAudio.paused) {
|
|
||||||
const p = mediaAudio.play();
|
|
||||||
if (p) p.catch(() => {});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
addEventListener("pointerdown", unlockMediaAudio, true);
|
|
||||||
addEventListener("keydown", unlockMediaAudio, true);
|
|
||||||
addEventListener("touchstart", unlockMediaAudio, true);
|
|
||||||
|
|
||||||
mediaAudio.addEventListener("play", () => {
|
|
||||||
updateMediaControls();
|
|
||||||
reportMediaPlayback("playing");
|
|
||||||
publishMediaState("playing");
|
|
||||||
});
|
|
||||||
mediaAudio.addEventListener("pause", () => {
|
|
||||||
updateMediaControls();
|
|
||||||
if (mediaCurrentTrackId) {
|
|
||||||
reportMediaPlayback("paused");
|
|
||||||
publishMediaState("paused");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
mediaAudio.addEventListener("ended", () => {
|
|
||||||
updateMediaControls();
|
|
||||||
reportMediaPlayback("stopped");
|
|
||||||
publishMediaState("stopped");
|
|
||||||
mediaPost("/control", { action: "stop" }).catch(() => {});
|
|
||||||
});
|
|
||||||
mediaAudio.addEventListener("timeupdate", () => {
|
|
||||||
updateSeekUI();
|
|
||||||
const now = Date.now();
|
|
||||||
if (!mediaAudio.paused && now - mediaLastReport > 5000) {
|
|
||||||
mediaLastReport = now;
|
|
||||||
reportMediaPlayback("playing");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
mediaAudio.addEventListener("durationchange", updateSeekUI);
|
|
||||||
mediaAudio.addEventListener("error", () => {
|
|
||||||
// An empty src (track still downloading) also fires this — only surface
|
|
||||||
// real playback failures.
|
|
||||||
if (mediaCurrentTrackId && mediaAudio.getAttribute("src")) {
|
|
||||||
mediaChannelEl.textContent = "Playback error — the track may still be preparing.";
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// ── Search / choices / library ──────────────────────────────────────────────
|
|
||||||
function fmtListDuration(sec) {
|
|
||||||
const s = Number(sec);
|
|
||||||
return isFinite(s) && s > 0 ? fmtTime(s) : "";
|
|
||||||
}
|
|
||||||
|
|
||||||
function mediaRowActions() {
|
|
||||||
const play = document.createElement("button");
|
|
||||||
play.className = "btn btn-secondary media-row-play";
|
|
||||||
play.type = "button";
|
|
||||||
play.textContent = "Play";
|
|
||||||
return play;
|
|
||||||
}
|
|
||||||
|
|
||||||
function renderMediaResults(results) {
|
|
||||||
mediaResultsEl.innerHTML = "";
|
|
||||||
if (!results || results.length === 0) {
|
|
||||||
mediaResultsEl.innerHTML = '<div class="media-empty">No results.</div>';
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
for (const r of results) {
|
|
||||||
const row = document.createElement("div");
|
|
||||||
row.className = "media-row";
|
|
||||||
const info = document.createElement("div");
|
|
||||||
info.className = "media-row-info";
|
|
||||||
const dur = fmtListDuration(r.duration);
|
|
||||||
info.innerHTML = `<strong>${escapeHtml(r.title)}</strong>` +
|
|
||||||
`<span>${escapeHtml(r.channel || "")}${dur ? " · " + dur : ""}` +
|
|
||||||
`${r.in_library ? " · in library" : ""}</span>`;
|
|
||||||
const play = mediaRowActions();
|
|
||||||
play.addEventListener("click", () => playVideoId(r));
|
|
||||||
row.appendChild(info);
|
|
||||||
row.appendChild(play);
|
|
||||||
mediaResultsEl.appendChild(row);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function playVideoId(r) {
|
|
||||||
try {
|
|
||||||
await mediaPost("/play", {
|
|
||||||
video_id: r.video_id,
|
|
||||||
title: r.title,
|
|
||||||
channel: r.channel || "",
|
|
||||||
});
|
|
||||||
} catch (e) {
|
|
||||||
mediaChannelEl.textContent = `Could not start: ${e.message}`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
mediaSearchForm.addEventListener("submit", async (event) => {
|
|
||||||
event.preventDefault();
|
|
||||||
const query = mediaSearchInput.value.trim();
|
|
||||||
if (!query) return;
|
|
||||||
mediaResultsEl.innerHTML = '<div class="media-empty">Searching YouTube Music…</div>';
|
|
||||||
try {
|
|
||||||
const data = await mediaPost("/search", { query, limit: 6 });
|
|
||||||
renderMediaResults(data.results);
|
|
||||||
} catch (e) {
|
|
||||||
mediaResultsEl.innerHTML = `<div class="media-empty">Search failed: ${escapeHtml(e.message)}</div>`;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
function renderMediaChoices(choices) {
|
|
||||||
mediaChoicesWrap.hidden = choices.length === 0;
|
|
||||||
mediaChoicesEl.innerHTML = "";
|
|
||||||
for (const c of choices) {
|
|
||||||
const row = document.createElement("div");
|
|
||||||
row.className = "media-row media-choice";
|
|
||||||
const info = document.createElement("div");
|
|
||||||
info.className = "media-row-info";
|
|
||||||
const dur = fmtListDuration(c.duration);
|
|
||||||
info.innerHTML = `<strong>${escapeHtml(c.title)}</strong>` +
|
|
||||||
`<span>${escapeHtml(c.channel || "")}${dur ? " · " + dur : ""}</span>`;
|
|
||||||
const play = mediaRowActions();
|
|
||||||
play.addEventListener("click", () => playVideoId(c));
|
|
||||||
row.appendChild(info);
|
|
||||||
row.appendChild(play);
|
|
||||||
mediaChoicesEl.appendChild(row);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async function refreshLibrary() {
|
|
||||||
let data;
|
|
||||||
try {
|
|
||||||
data = await fetch("/media/api/library").then((r) => r.json());
|
|
||||||
} catch (e) {
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
mediaLibraryEl.innerHTML = "";
|
|
||||||
const tracks = data.tracks || [];
|
|
||||||
if (tracks.length === 0) {
|
|
||||||
mediaLibraryEl.innerHTML = '<div class="media-empty">Downloaded songs and streams live here.</div>';
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
for (const t of tracks) {
|
|
||||||
const row = document.createElement("div");
|
|
||||||
row.className = "media-row";
|
|
||||||
const info = document.createElement("div");
|
|
||||||
info.className = "media-row-info";
|
|
||||||
const dur = fmtListDuration(t.duration);
|
|
||||||
const statusTag = t.status === "ready" ? "" : ` · ${t.status}`;
|
|
||||||
info.innerHTML = `<strong>${escapeHtml(t.title)}</strong>` +
|
|
||||||
`<span>${escapeHtml(t.channel || (t.kind === "stream" ? "Live stream" : ""))}` +
|
|
||||||
`${dur ? " · " + dur : ""}${statusTag}</span>`;
|
|
||||||
const play = mediaRowActions();
|
|
||||||
play.disabled = t.status !== "ready";
|
|
||||||
play.addEventListener("click", () => {
|
|
||||||
mediaPost("/play", { track_id: t.id }).catch((e) => {
|
|
||||||
mediaChannelEl.textContent = `Could not play: ${e.message}`;
|
|
||||||
});
|
|
||||||
});
|
|
||||||
const del = document.createElement("button");
|
|
||||||
del.className = "btn btn-icon media-row-delete";
|
|
||||||
del.type = "button";
|
|
||||||
del.setAttribute("aria-label", `Remove ${t.title}`);
|
|
||||||
del.title = "Remove from library";
|
|
||||||
del.innerHTML = '<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>';
|
|
||||||
del.addEventListener("click", async () => {
|
|
||||||
try {
|
|
||||||
await fetch(`/media/api/tracks/${encodeURIComponent(t.id)}`, { method: "DELETE" });
|
|
||||||
if (t.id === mediaCurrentTrackId) clearMediaAudio();
|
|
||||||
refreshLibrary();
|
|
||||||
} catch (e) { /* leave the row; next refresh decides */ }
|
|
||||||
});
|
|
||||||
row.appendChild(info);
|
|
||||||
row.appendChild(play);
|
|
||||||
row.appendChild(del);
|
|
||||||
mediaLibraryEl.appendChild(row);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
streamForm.addEventListener("submit", async (event) => {
|
|
||||||
event.preventDefault();
|
|
||||||
const name = streamNameInput.value.trim() || streamUrlInput.value.trim();
|
|
||||||
const url = streamUrlInput.value.trim();
|
|
||||||
if (!url) return;
|
|
||||||
try {
|
|
||||||
await mediaPost("/streams", { name, url });
|
|
||||||
streamNameInput.value = "";
|
|
||||||
streamUrlInput.value = "";
|
|
||||||
refreshLibrary();
|
|
||||||
} catch (e) {
|
|
||||||
mediaChannelEl.textContent = `Could not add stream: ${e.message}`;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// ── Panel open/close ────────────────────────────────────────────────────────
|
|
||||||
function setMediaOpen(open) {
|
|
||||||
mediaPanel.classList.toggle("open", open);
|
|
||||||
mediaBtn.setAttribute("aria-expanded", String(open));
|
|
||||||
if (open) {
|
|
||||||
refreshLibrary();
|
|
||||||
pollMediaState();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
mediaBtn.setAttribute("aria-expanded", "false");
|
|
||||||
mediaBtn.addEventListener("click", () => setMediaOpen(!mediaPanel.classList.contains("open")));
|
|
||||||
closeMediaBtn.addEventListener("click", () => setMediaOpen(false));
|
|
||||||
|
|
||||||
startMediaPolling();
|
|
||||||
updateMediaControls();
|
|
||||||
buildLedWall();
|
buildLedWall();
|
||||||
@@ -42,11 +42,6 @@
|
|||||||
<span>Tasks</span>
|
<span>Tasks</span>
|
||||||
<span class="tasks-badge" id="tasksBadge" hidden>0</span>
|
<span class="tasks-badge" id="tasksBadge" hidden>0</span>
|
||||||
</button>
|
</button>
|
||||||
<button id="mediaBtn" class="btn btn-secondary tasks-trigger" aria-label="Music player" title="Music player">
|
|
||||||
<svg viewBox="0 0 24 24" width="17" height="17" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
|
||||||
<span>Music</span>
|
|
||||||
<span class="tasks-badge" id="mediaBadge" hidden>♫</span>
|
|
||||||
</button>
|
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
@@ -59,7 +54,6 @@
|
|||||||
|
|
||||||
<div class="orb-stage">
|
<div class="orb-stage">
|
||||||
<div class="led-wall" id="ledWall" data-state="idle" aria-hidden="true">
|
<div class="led-wall" id="ledWall" data-state="idle" aria-hidden="true">
|
||||||
<img id="ledArt" class="led-art" alt="">
|
|
||||||
</div>
|
</div>
|
||||||
<div class="orb-labels" aria-live="polite">
|
<div class="orb-labels" aria-live="polite">
|
||||||
<span class="state-label" id="stateLabel">Ready when you are</span>
|
<span class="state-label" id="stateLabel">Ready when you are</span>
|
||||||
@@ -151,85 +145,6 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="tasks-panel media-panel" id="mediaPanel" aria-label="Music player">
|
|
||||||
<div class="tasks-header">
|
|
||||||
<div>
|
|
||||||
<h2>Music</h2>
|
|
||||||
<p>YouTube Music downloads and live streams, streamed to you.</p>
|
|
||||||
</div>
|
|
||||||
<button id="closeMediaBtn" class="btn btn-icon" aria-label="Close music player">
|
|
||||||
<svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="media-now">
|
|
||||||
<img id="mediaArt" class="media-art" alt="" hidden>
|
|
||||||
<div class="media-art-placeholder" id="mediaArtPlaceholder" aria-hidden="true">
|
|
||||||
<svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
|
||||||
</div>
|
|
||||||
<div class="media-now-info">
|
|
||||||
<strong id="mediaTitle">Nothing loaded</strong>
|
|
||||||
<span id="mediaChannel"></span>
|
|
||||||
<div class="media-progress" id="mediaProgressWrap" hidden>
|
|
||||||
<div class="media-progress-fill" id="mediaProgressFill"></div>
|
|
||||||
<span class="media-progress-label" id="mediaProgressLabel">Fetching…</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="media-controls" role="group" aria-label="Playback controls">
|
|
||||||
<button id="rewBtn" class="btn btn-icon" aria-label="Back 10 seconds" title="Back 10s">
|
|
||||||
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 17a5 5 0 1 0-5-5"></path><path d="M11 17H6v-5"></path><path d="m19 5-4 4h-3"></path><path d="m15 9 4-4"></path></svg>
|
|
||||||
</button>
|
|
||||||
<button id="playPauseBtn" class="btn btn-icon media-play" aria-label="Play or pause" title="Play / pause">
|
|
||||||
<svg id="playIcon" class="play-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M8 5v14l11-7Z"></path></svg>
|
|
||||||
<svg id="pauseIcon" class="pause-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M7 5h4v14H7zM13 5h4v14h-4z"></path></svg>
|
|
||||||
</button>
|
|
||||||
<button id="fwdBtn" class="btn btn-icon" aria-label="Forward 10 seconds" title="Forward 10s">
|
|
||||||
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M13 17a5 5 0 1 1 5-5"></path><path d="M13 17h5v-5"></path><path d="m5 5 4 4h3"></path><path d="m9 9 4-4"></path></svg>
|
|
||||||
</button>
|
|
||||||
<button id="stopMediaBtn" class="btn btn-icon" aria-label="Stop" title="Stop">
|
|
||||||
<svg viewBox="0 0 24 24" width="18" height="18" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"></rect></svg>
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="media-seek">
|
|
||||||
<span class="media-time" id="mediaCur">0:00</span>
|
|
||||||
<input type="range" id="seekBar" min="0" max="1000" value="0" step="1" aria-label="Seek position">
|
|
||||||
<span class="media-time" id="mediaDur">0:00</span>
|
|
||||||
</div>
|
|
||||||
<div class="media-volume">
|
|
||||||
<svg viewBox="0 0 24 24" width="15" height="15" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 5 6 9H2v6h4l5 4V5Z"></path><path d="M15.5 8.5a5 5 0 0 1 0 7"></path></svg>
|
|
||||||
<input type="range" id="volumeBar" min="0" max="100" value="90" aria-label="Volume">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3 class="sheet-section-title">Search YouTube Music</h3>
|
|
||||||
<form class="media-search" id="mediaSearchForm">
|
|
||||||
<input id="mediaSearchInput" type="search" placeholder="Song, artist, lyrics…" autocomplete="off">
|
|
||||||
<button class="btn btn-secondary" type="submit">Find</button>
|
|
||||||
</form>
|
|
||||||
<div class="media-list" id="mediaResults"></div>
|
|
||||||
|
|
||||||
<div id="mediaChoicesWrap" hidden>
|
|
||||||
<h3 class="sheet-section-title">Hope’s suggestions</h3>
|
|
||||||
<div class="media-list" id="mediaChoices"></div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3 class="sheet-section-title">Library</h3>
|
|
||||||
<div class="media-list" id="mediaLibrary">
|
|
||||||
<div class="media-empty">Downloaded songs and streams live here.</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<h3 class="sheet-section-title">Add live stream</h3>
|
|
||||||
<form class="media-search media-stream-form" id="streamForm">
|
|
||||||
<input id="streamName" type="text" placeholder="Station name" autocomplete="off">
|
|
||||||
<input id="streamUrl" type="url" placeholder="https://… audio stream URL" required>
|
|
||||||
<button class="btn btn-secondary" type="submit">Add</button>
|
|
||||||
</form>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<audio id="mediaAudio" preload="metadata"></audio>
|
|
||||||
|
|
||||||
<script src="livekit-client.umd.js"></script>
|
<script src="livekit-client.umd.js"></script>
|
||||||
<script src="app.js"></script>
|
<script src="app.js"></script>
|
||||||
</body>
|
</body>
|
||||||
|
|||||||
@@ -1,810 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Media server for the Hope voice assistant.
|
|
||||||
|
|
||||||
A small FastAPI service that owns the permanent music library and player
|
|
||||||
coordination:
|
|
||||||
|
|
||||||
- Library: downloaded YouTube Music tracks (raw best-audio files, saved by
|
|
||||||
yt-dlp with no transcode) and user-added live audio stream URLs, persisted
|
|
||||||
in library.json.
|
|
||||||
- Search: yt-dlp ``ytsearch`` (YouTube Music pages share video ids), used by
|
|
||||||
Hope's media MCP toolset and by the web UI search box. Ready library tracks
|
|
||||||
that match the query are returned first so Hope prefers a downloaded copy.
|
|
||||||
- Download jobs: one-at-a-time worker thread; progress is parsed from
|
|
||||||
yt-dlp stderr and surfaced through GET /media/api/state. Playback starts
|
|
||||||
while the file is still growing (progressive streaming), and the finished
|
|
||||||
file stays in the library for instant replay.
|
|
||||||
- Player state: per-room desired state (loaded track, playing/paused/stopped,
|
|
||||||
seek hints). The browser is the actual player and reconciles against this;
|
|
||||||
it reports real playback state back so Hope's voice tools can answer
|
|
||||||
"what's playing".
|
|
||||||
|
|
||||||
Security posture matches the token server: bind 127.0.0.1 only, reachability
|
|
||||||
exclusively through the nginx HTTPS proxy (/media/api/). Growing downloads
|
|
||||||
are streamed by this service (chunked, no Content-Length); finished files are
|
|
||||||
served here too with native Range support for seeking. Live stream URLs are
|
|
||||||
proxied (mixed-content: an http:// radio URL would be blocked by the browser,
|
|
||||||
so it is fetched server-side and re-served over HTTPS).
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import httpx
|
|
||||||
import uvicorn
|
|
||||||
from fastapi import FastAPI, HTTPException, Request
|
|
||||||
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
|
|
||||||
|
|
||||||
MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/data/media"))
|
|
||||||
FILES_DIR = MEDIA_DIR / "files"
|
|
||||||
THUMBS_DIR = MEDIA_DIR / "thumbs"
|
|
||||||
JOBS_DIR = MEDIA_DIR / "jobs"
|
|
||||||
LIBRARY_FILE = MEDIA_DIR / "library.json"
|
|
||||||
COOKIES_FILE = os.environ.get("YTDLP_COOKIES", "")
|
|
||||||
MEDIA_HOST = os.environ.get("MEDIA_SERVER_HOST", "127.0.0.1")
|
|
||||||
# 8093, NOT 8092: 8092 is occupied by an Apache/YOURLS stack on the Unraid
|
|
||||||
# deployment host (xNAS) — a bind conflict there looks like a healthy
|
|
||||||
# deployment because the healthcheck curl (without -f) accepts any response.
|
|
||||||
MEDIA_PORT = int(os.environ.get("MEDIA_SERVER_PORT", "8093"))
|
|
||||||
|
|
||||||
DOWNLOAD_TIMEOUT = int(os.environ.get("MEDIA_DOWNLOAD_TIMEOUT", "600"))
|
|
||||||
SEARCH_TIMEOUT = int(os.environ.get("MEDIA_SEARCH_TIMEOUT", "45"))
|
|
||||||
|
|
||||||
app = FastAPI(title="hope-media", docs_url=None, redoc_url=None, openapi_url=None)
|
|
||||||
|
|
||||||
_lock = threading.Lock() # guards library + rooms below
|
|
||||||
_library: dict[str, Any] = {"tracks": {}, "version": 1}
|
|
||||||
_rooms: dict[str, dict[str, Any]] = {}
|
|
||||||
_jobs: dict[str, dict[str, Any]] = {}
|
|
||||||
_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="media-dl")
|
|
||||||
|
|
||||||
for _d in (FILES_DIR, THUMBS_DIR, JOBS_DIR):
|
|
||||||
_d.mkdir(parents=True, exist_ok=True)
|
|
||||||
# The bind mount may be inspected/cleaned from the host by a non-root
|
|
||||||
# user (Unraid share convention) — keep the dirs world-traversable.
|
|
||||||
try:
|
|
||||||
os.chmod(_d, 0o777)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ── Library store ───────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _load_library() -> None:
|
|
||||||
if LIBRARY_FILE.exists():
|
|
||||||
try:
|
|
||||||
data = json.loads(LIBRARY_FILE.read_text(encoding="utf-8"))
|
|
||||||
tracks = data.get("tracks", {})
|
|
||||||
if isinstance(tracks, dict):
|
|
||||||
_library["tracks"] = tracks
|
|
||||||
_library["version"] = int(data.get("version", 1))
|
|
||||||
except (json.JSONDecodeError, OSError):
|
|
||||||
# Corrupt library: keep the broken file aside, start fresh.
|
|
||||||
try:
|
|
||||||
LIBRARY_FILE.replace(LIBRARY_FILE.with_suffix(".json.broken"))
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _save_library() -> None:
|
|
||||||
tmp = LIBRARY_FILE.with_suffix(".json.tmp")
|
|
||||||
tmp.write_text(json.dumps(_library), encoding="utf-8")
|
|
||||||
tmp.replace(LIBRARY_FILE)
|
|
||||||
|
|
||||||
|
|
||||||
def _bump_version() -> int:
|
|
||||||
_library["version"] = int(_library.get("version", 1)) + 1
|
|
||||||
return _library["version"]
|
|
||||||
|
|
||||||
|
|
||||||
def _track_dict(track: dict) -> dict:
|
|
||||||
out = dict(track)
|
|
||||||
if track["kind"] == "youtube":
|
|
||||||
# The stream endpoint serves the growing file during download and the
|
|
||||||
# finished file afterwards; the URL is stable across the transition
|
|
||||||
# so the browser never has to reload mid-playback.
|
|
||||||
if track.get("status") in ("downloading", "ready"):
|
|
||||||
out["audio_url"] = f"/media/api/stream/{track['id']}"
|
|
||||||
out["art_url"] = (
|
|
||||||
f"/media/thumbs/{track['id']}.jpg"
|
|
||||||
if (THUMBS_DIR / f"{track['id']}.jpg").exists()
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
out["audio_url"] = f"/media/api/stream/{track['id']}"
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _get_track(track_id: str) -> dict | None:
|
|
||||||
return _library["tracks"].get(track_id)
|
|
||||||
|
|
||||||
|
|
||||||
def _upsert_track(track: dict) -> None:
|
|
||||||
_library["tracks"][track["id"]] = track
|
|
||||||
_bump_version()
|
|
||||||
_save_library()
|
|
||||||
|
|
||||||
|
|
||||||
_load_library()
|
|
||||||
|
|
||||||
|
|
||||||
# ── Player state (per room) ─────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _room_state(room: str) -> dict:
|
|
||||||
return _rooms.setdefault(
|
|
||||||
room,
|
|
||||||
{"track_id": None, "desired": "stopped", "reported": None,
|
|
||||||
"seek_hint": None, "choices": []},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _room_snapshot(room: str) -> dict:
|
|
||||||
st = _room_state(room)
|
|
||||||
track = _get_track(st["track_id"]) if st["track_id"] else None
|
|
||||||
pending = [
|
|
||||||
{
|
|
||||||
"job_id": j["id"],
|
|
||||||
"track_id": j["track_id"],
|
|
||||||
"stage": j["stage"],
|
|
||||||
"progress": j["progress"],
|
|
||||||
"error": j.get("error"),
|
|
||||||
}
|
|
||||||
for j in _jobs.values()
|
|
||||||
if j["stage"] in ("queued", "downloading", "transcoding")
|
|
||||||
]
|
|
||||||
return {
|
|
||||||
"room": room,
|
|
||||||
"track": _track_dict(track) if track else None,
|
|
||||||
"desired": st["desired"],
|
|
||||||
"reported": st["reported"],
|
|
||||||
"seek_hint": st["seek_hint"],
|
|
||||||
"choices": st["choices"],
|
|
||||||
"pending": pending,
|
|
||||||
"version": _library["version"],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ── yt-dlp helpers ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _ytdlp_base_args() -> list[str]:
|
|
||||||
args = [sys.executable, "-m", "yt_dlp", "--no-playlist", "--no-warnings"]
|
|
||||||
if COOKIES_FILE and Path(COOKIES_FILE).exists():
|
|
||||||
args += ["--cookies", COOKIES_FILE]
|
|
||||||
return args
|
|
||||||
|
|
||||||
|
|
||||||
_PROGRESS_RE = re.compile(r"\[download\]\s+([\d.]+)%")
|
|
||||||
|
|
||||||
# Audio containers yt-dlp can produce for a raw best-audio download.
|
|
||||||
_AUDIO_EXTS = {
|
|
||||||
".webm", ".opus", ".ogg", ".m4a", ".mp4", ".m4b", ".aac",
|
|
||||||
".mp3", ".mka", ".flac", ".wav",
|
|
||||||
}
|
|
||||||
_AUDIO_MIME = {
|
|
||||||
".webm": "audio/webm",
|
|
||||||
".opus": "audio/opus",
|
|
||||||
".ogg": "audio/ogg",
|
|
||||||
".m4a": "audio/mp4",
|
|
||||||
".mp4": "audio/mp4",
|
|
||||||
".m4b": "audio/mp4",
|
|
||||||
".aac": "audio/aac",
|
|
||||||
".mp3": "audio/mpeg",
|
|
||||||
".mka": "audio/x-matroska",
|
|
||||||
".flac": "audio/flac",
|
|
||||||
".wav": "audio/wav",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _media_type_for(name: str) -> str:
|
|
||||||
if name.endswith(".part"):
|
|
||||||
name = name[:-5]
|
|
||||||
return _AUDIO_MIME.get(Path(name).suffix.lower(), "audio/mpeg")
|
|
||||||
|
|
||||||
|
|
||||||
def _youtube_media_file(track_id: str) -> Path | None:
|
|
||||||
"""The on-disk audio file for a youtube track, if any.
|
|
||||||
|
|
||||||
During an active download yt-dlp writes ``{id}.{ext}.part`` and renames it
|
|
||||||
to ``{id}.{ext}`` on completion — return whichever currently exists (the
|
|
||||||
partial file wins, thumbnails and other sidecars are ignored).
|
|
||||||
"""
|
|
||||||
complete: Path | None = None
|
|
||||||
for p in FILES_DIR.glob(f"{track_id}.*"):
|
|
||||||
suffix = p.suffix.lower()
|
|
||||||
if suffix == ".part":
|
|
||||||
if p.stat().st_size > 0:
|
|
||||||
return p
|
|
||||||
elif suffix in _AUDIO_EXTS:
|
|
||||||
if complete is None or p.name < complete.name:
|
|
||||||
complete = p
|
|
||||||
return complete
|
|
||||||
|
|
||||||
|
|
||||||
async def _wait_for_media_file(track_id: str, timeout: float = 15.0) -> Path | None:
|
|
||||||
"""Wait for yt-dlp to create the output file after a play request."""
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
path = _youtube_media_file(track_id)
|
|
||||||
if path is not None:
|
|
||||||
return path
|
|
||||||
await asyncio.sleep(0.5)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
async def _growing_file(track_id: str, job: dict, chunk_size: int = 64 * 1024):
|
|
||||||
"""Yield bytes from a file that yt-dlp is still writing.
|
|
||||||
|
|
||||||
The response stays open with no Content-Length until the download
|
|
||||||
finishes, so the browser plays the track as it grows. The open descriptor
|
|
||||||
keeps reading the old inode across yt-dlp's .part -> final rename, and the
|
|
||||||
path is re-resolved if the file disappears between resolution and open.
|
|
||||||
"""
|
|
||||||
fd = -1
|
|
||||||
pos = 0
|
|
||||||
stall = 0
|
|
||||||
try:
|
|
||||||
while True:
|
|
||||||
if fd == -1:
|
|
||||||
path = _youtube_media_file(track_id)
|
|
||||||
if path is not None:
|
|
||||||
try:
|
|
||||||
fd = os.open(path, os.O_RDONLY)
|
|
||||||
except FileNotFoundError:
|
|
||||||
fd = -1 # renamed between glob and open — retry
|
|
||||||
if fd != -1:
|
|
||||||
data = await asyncio.to_thread(os.pread, fd, chunk_size, pos)
|
|
||||||
if data:
|
|
||||||
pos += len(data)
|
|
||||||
stall = 0
|
|
||||||
yield data
|
|
||||||
continue
|
|
||||||
with _lock:
|
|
||||||
active = job.get("stage") == "downloading"
|
|
||||||
if not active:
|
|
||||||
break
|
|
||||||
stall += 1
|
|
||||||
if stall > 60: # ~30s with no new bytes — stop waiting
|
|
||||||
break
|
|
||||||
await asyncio.sleep(0.5)
|
|
||||||
finally:
|
|
||||||
if fd != -1:
|
|
||||||
os.close(fd)
|
|
||||||
|
|
||||||
|
|
||||||
def _run_search(query: str, limit: int) -> list[dict]:
|
|
||||||
"""Flat ytsearch lookup — cheap, no download, works anonymously."""
|
|
||||||
cmd = _ytdlp_base_args() + [
|
|
||||||
"--flat-playlist", "--dump-json",
|
|
||||||
f"ytsearch{limit}:{query}",
|
|
||||||
]
|
|
||||||
try:
|
|
||||||
proc = subprocess.run(
|
|
||||||
cmd, capture_output=True, text=True, timeout=SEARCH_TIMEOUT, check=False
|
|
||||||
)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
raise HTTPException(504, "YouTube Music search timed out")
|
|
||||||
if proc.returncode != 0 and not proc.stdout.strip():
|
|
||||||
detail = (proc.stderr or "").strip().splitlines()[-1:] or ["unknown error"]
|
|
||||||
raise HTTPException(502, f"Search failed: {detail[0][:300]}")
|
|
||||||
results = []
|
|
||||||
for line in proc.stdout.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if not line.startswith("{"):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
entry = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
video_id = entry.get("id")
|
|
||||||
if not video_id:
|
|
||||||
continue
|
|
||||||
existing = _get_track(video_id)
|
|
||||||
results.append({
|
|
||||||
"video_id": video_id,
|
|
||||||
"title": entry.get("title") or video_id,
|
|
||||||
"channel": entry.get("uploader") or entry.get("channel") or "",
|
|
||||||
"duration": entry.get("duration"),
|
|
||||||
"view_count": entry.get("view_count"),
|
|
||||||
"in_library": bool(existing),
|
|
||||||
"track_id": existing["id"] if existing else None,
|
|
||||||
"library_status": existing["status"] if existing else None,
|
|
||||||
})
|
|
||||||
return results
|
|
||||||
|
|
||||||
|
|
||||||
def _job_dict(job: dict) -> dict:
|
|
||||||
return {k: job.get(k) for k in
|
|
||||||
("id", "track_id", "stage", "progress", "error", "started_at")}
|
|
||||||
|
|
||||||
|
|
||||||
def _persist_job(job: dict) -> None:
|
|
||||||
try:
|
|
||||||
(JOBS_DIR / f"{job['id']}.json").write_text(
|
|
||||||
json.dumps(_job_dict(job)), encoding="utf-8"
|
|
||||||
)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _fail_job(job: dict, track: dict, message: str) -> None:
|
|
||||||
job["stage"] = "error"
|
|
||||||
job["error"] = message[:500]
|
|
||||||
_persist_job(job)
|
|
||||||
with _lock:
|
|
||||||
track["status"] = "failed"
|
|
||||||
track["error"] = message[:500]
|
|
||||||
_bump_version()
|
|
||||||
_save_library()
|
|
||||||
|
|
||||||
|
|
||||||
def _download_worker(job: dict, track: dict, video_id: str) -> None:
|
|
||||||
"""Runs in the single-worker executor: raw best-audio download.
|
|
||||||
|
|
||||||
No mp3 transcode — the audio container is streamed straight into the
|
|
||||||
library so playback can begin while the file is still growing (the stream
|
|
||||||
endpoint serves the partial file), and the finished file stays playable
|
|
||||||
offline. Thumbnail handling is unchanged.
|
|
||||||
"""
|
|
||||||
cmd = _ytdlp_base_args() + [
|
|
||||||
"-f", "bestaudio/best",
|
|
||||||
"--newline", "--progress",
|
|
||||||
"--print-json",
|
|
||||||
"--write-thumbnail", "--convert-thumbnails", "jpg",
|
|
||||||
"--retries", "3", "--fragment-retries", "3",
|
|
||||||
"-P", str(FILES_DIR),
|
|
||||||
"-o", f"{track['id']}.%(ext)s",
|
|
||||||
f"https://www.youtube.com/watch?v={video_id}",
|
|
||||||
]
|
|
||||||
job["stage"] = "downloading"
|
|
||||||
info: dict = {}
|
|
||||||
# Progress lines go to stderr, the metadata JSON to stdout. Drain stderr
|
|
||||||
# in a thread so the stdout read loop can never deadlock the pipe; the
|
|
||||||
# tail is kept for the failure detail.
|
|
||||||
stderr_tail: list[str] = []
|
|
||||||
proc = None
|
|
||||||
|
|
||||||
def _drain_stderr() -> None:
|
|
||||||
assert proc is not None and proc.stderr is not None
|
|
||||||
for line in proc.stderr:
|
|
||||||
stderr_tail.append(line)
|
|
||||||
if len(stderr_tail) > 200:
|
|
||||||
stderr_tail.pop(0)
|
|
||||||
m = _PROGRESS_RE.search(line)
|
|
||||||
if m:
|
|
||||||
job["progress"] = min(99.0, float(m.group(1)))
|
|
||||||
_persist_job(job)
|
|
||||||
|
|
||||||
try:
|
|
||||||
proc = subprocess.Popen(
|
|
||||||
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
|
||||||
text=True, errors="replace",
|
|
||||||
)
|
|
||||||
threading.Thread(target=_drain_stderr, daemon=True).start()
|
|
||||||
deadline = time.monotonic() + DOWNLOAD_TIMEOUT
|
|
||||||
assert proc.stdout is not None
|
|
||||||
for line in proc.stdout:
|
|
||||||
if time.monotonic() > deadline:
|
|
||||||
proc.kill()
|
|
||||||
proc.wait(timeout=30)
|
|
||||||
raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
|
|
||||||
line = line.strip()
|
|
||||||
if line.startswith("{"):
|
|
||||||
try:
|
|
||||||
info = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
pass
|
|
||||||
proc.wait(timeout=30)
|
|
||||||
if proc.returncode != 0:
|
|
||||||
detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
|
|
||||||
raise RuntimeError(detail.strip())
|
|
||||||
media = _youtube_media_file(track["id"])
|
|
||||||
if media is None:
|
|
||||||
raise RuntimeError("yt-dlp finished but no audio file was produced")
|
|
||||||
except Exception as exc: # noqa: BLE001 — any failure marks the track failed
|
|
||||||
_fail_job(job, track, str(exc))
|
|
||||||
return
|
|
||||||
|
|
||||||
thumb_src = FILES_DIR / f"{track['id']}.jpg"
|
|
||||||
try:
|
|
||||||
if thumb_src.exists():
|
|
||||||
thumb_src.replace(THUMBS_DIR / f"{track['id']}.jpg")
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
job["stage"] = "ready"
|
|
||||||
job["progress"] = 100.0
|
|
||||||
_persist_job(job)
|
|
||||||
with _lock:
|
|
||||||
track["status"] = "ready"
|
|
||||||
track["error"] = None
|
|
||||||
track["ext"] = media.suffix.lstrip(".").lower()
|
|
||||||
if info:
|
|
||||||
if info.get("title"):
|
|
||||||
track["title"] = info["title"]
|
|
||||||
if info.get("uploader") or info.get("channel"):
|
|
||||||
track["channel"] = info.get("uploader") or info.get("channel")
|
|
||||||
if info.get("duration"):
|
|
||||||
track["duration"] = round(float(info["duration"]))
|
|
||||||
_bump_version()
|
|
||||||
_save_library()
|
|
||||||
|
|
||||||
|
|
||||||
def _start_download(track: dict, video_id: str) -> dict:
|
|
||||||
job = {
|
|
||||||
"id": uuid.uuid4().hex[:12],
|
|
||||||
"track_id": track["id"],
|
|
||||||
"stage": "queued",
|
|
||||||
"progress": 0.0,
|
|
||||||
"error": None,
|
|
||||||
"started_at": time.time(),
|
|
||||||
}
|
|
||||||
_jobs[job["id"]] = job
|
|
||||||
_persist_job(job)
|
|
||||||
_executor.submit(_download_worker, job, track, video_id)
|
|
||||||
return job
|
|
||||||
|
|
||||||
|
|
||||||
# ── API: health, library ────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
@app.get("/media/api/healthz")
|
|
||||||
def healthz() -> dict:
|
|
||||||
return {"ok": True, "version": _library["version"]}
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/media/api/library")
|
|
||||||
def get_library() -> dict:
|
|
||||||
with _lock:
|
|
||||||
tracks = sorted(
|
|
||||||
(_track_dict(t) for t in _library["tracks"].values()),
|
|
||||||
key=lambda t: t.get("added_at", 0),
|
|
||||||
reverse=True,
|
|
||||||
)
|
|
||||||
return {"tracks": tracks, "version": _library["version"]}
|
|
||||||
|
|
||||||
|
|
||||||
def _delete_track_files(track: dict) -> None:
|
|
||||||
if track["kind"] != "youtube":
|
|
||||||
return
|
|
||||||
paths = [
|
|
||||||
FILES_DIR / f"{track['id']}.{ext}" for ext in
|
|
||||||
(s.lstrip(".") for s in _AUDIO_EXTS)
|
|
||||||
] + [FILES_DIR / f"{track['id']}.jpg", THUMBS_DIR / f"{track['id']}.jpg"]
|
|
||||||
for path in paths:
|
|
||||||
try:
|
|
||||||
path.unlink(missing_ok=True)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/media/api/tracks/{track_id}")
|
|
||||||
def delete_track(track_id: str) -> dict:
|
|
||||||
with _lock:
|
|
||||||
track = _get_track(track_id)
|
|
||||||
if not track:
|
|
||||||
raise HTTPException(404, "unknown track")
|
|
||||||
del _library["tracks"][track_id]
|
|
||||||
_bump_version()
|
|
||||||
_save_library()
|
|
||||||
for room, st in _rooms.items():
|
|
||||||
if st["track_id"] == track_id:
|
|
||||||
st["track_id"] = None
|
|
||||||
st["desired"] = "stopped"
|
|
||||||
_delete_track_files(track)
|
|
||||||
return {"ok": True}
|
|
||||||
|
|
||||||
|
|
||||||
# ── API: search + play + control ────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _library_search(query: str, limit: int) -> list[dict]:
|
|
||||||
"""Ready library tracks whose title matches the query.
|
|
||||||
|
|
||||||
These are merged in front of the YouTube results so Hope prefers an
|
|
||||||
already-downloaded copy over fetching a new one.
|
|
||||||
"""
|
|
||||||
q = query.strip().lower()
|
|
||||||
tokens = {t for t in re.split(r"\W+", q) if len(t) > 2}
|
|
||||||
hits: list[dict] = []
|
|
||||||
if not q:
|
|
||||||
return hits
|
|
||||||
for track in _library["tracks"].values():
|
|
||||||
if track.get("kind") != "youtube" or track.get("status") != "ready":
|
|
||||||
continue
|
|
||||||
title = (track.get("title") or "").strip().lower()
|
|
||||||
if not title:
|
|
||||||
continue
|
|
||||||
title_tokens = {t for t in re.split(r"\W+", title) if t}
|
|
||||||
if not (q in title or title in q or (tokens and tokens <= title_tokens)):
|
|
||||||
continue
|
|
||||||
hits.append({
|
|
||||||
"video_id": track["id"],
|
|
||||||
"track_id": track["id"],
|
|
||||||
"title": track.get("title") or track["id"],
|
|
||||||
"channel": track.get("channel") or "",
|
|
||||||
"duration": track.get("duration"),
|
|
||||||
"view_count": None,
|
|
||||||
"in_library": True,
|
|
||||||
"library_status": "ready",
|
|
||||||
})
|
|
||||||
return hits[:limit]
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/search")
|
|
||||||
async def search(request: Request) -> dict:
|
|
||||||
body = await request.json()
|
|
||||||
query = str(body.get("query", "")).strip()
|
|
||||||
if not query:
|
|
||||||
raise HTTPException(400, "query is required")
|
|
||||||
try:
|
|
||||||
limit = max(1, min(int(body.get("limit", 6)), 15))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
raise HTTPException(400, "limit must be an integer")
|
|
||||||
local = _library_search(query, limit)
|
|
||||||
remote = _run_search(query, limit)
|
|
||||||
seen = {r["video_id"] for r in local}
|
|
||||||
merged = local + [r for r in remote if r["video_id"] not in seen]
|
|
||||||
return {"query": query, "results": merged[:limit]}
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_play_target(body: dict) -> dict:
|
|
||||||
"""Find or create the track a play request refers to.
|
|
||||||
|
|
||||||
Accepts video_id (YouTube), track_id (existing library entry) or a raw
|
|
||||||
url+name (live stream). Returns the library track dict.
|
|
||||||
"""
|
|
||||||
video_id = str(body.get("video_id") or "").strip()
|
|
||||||
track_id = str(body.get("track_id") or "").strip()
|
|
||||||
url = str(body.get("url") or "").strip()
|
|
||||||
|
|
||||||
if video_id:
|
|
||||||
existing = _get_track(video_id)
|
|
||||||
if existing:
|
|
||||||
return existing
|
|
||||||
track = {
|
|
||||||
"id": video_id,
|
|
||||||
"kind": "youtube",
|
|
||||||
"video_id": video_id,
|
|
||||||
"title": str(body.get("title") or video_id),
|
|
||||||
"channel": str(body.get("channel") or ""),
|
|
||||||
"duration": body.get("duration"),
|
|
||||||
"status": "downloading",
|
|
||||||
"error": None,
|
|
||||||
"added_at": time.time(),
|
|
||||||
}
|
|
||||||
_upsert_track(track)
|
|
||||||
_start_download(track, video_id)
|
|
||||||
return track
|
|
||||||
|
|
||||||
if track_id:
|
|
||||||
track = _get_track(track_id)
|
|
||||||
if not track:
|
|
||||||
raise HTTPException(404, "unknown track")
|
|
||||||
return track
|
|
||||||
|
|
||||||
if url:
|
|
||||||
if not re.match(r"^https?://", url):
|
|
||||||
raise HTTPException(400, "url must be http(s)")
|
|
||||||
track_id = "s" + uuid.uuid4().hex[:10]
|
|
||||||
track = {
|
|
||||||
"id": track_id,
|
|
||||||
"kind": "stream",
|
|
||||||
"video_id": None,
|
|
||||||
"title": str(body.get("name") or url),
|
|
||||||
"channel": str(body.get("channel") or ""),
|
|
||||||
"duration": None,
|
|
||||||
"status": "ready",
|
|
||||||
"error": None,
|
|
||||||
"url": url,
|
|
||||||
"added_at": time.time(),
|
|
||||||
}
|
|
||||||
_upsert_track(track)
|
|
||||||
return track
|
|
||||||
|
|
||||||
raise HTTPException(400, "one of video_id, track_id or url is required")
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/play")
|
|
||||||
async def play(request: Request) -> dict:
|
|
||||||
body = await request.json()
|
|
||||||
room = str(body.get("room") or "default")
|
|
||||||
track = _resolve_play_target(body)
|
|
||||||
with _lock:
|
|
||||||
st = _room_state(room)
|
|
||||||
st["track_id"] = track["id"]
|
|
||||||
# start_paused carries a track into a new scope without starting it —
|
|
||||||
# the UI uses this when a conversation begins (talking pauses music).
|
|
||||||
st["desired"] = "paused" if body.get("start_paused") else "playing"
|
|
||||||
st["choices"] = [] # a pick resolves any open suggestion list
|
|
||||||
st["seek_hint"] = None
|
|
||||||
snapshot = _room_snapshot(room)
|
|
||||||
return {"ok": True, **snapshot}
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/control")
|
|
||||||
async def control(request: Request) -> dict:
|
|
||||||
body = await request.json()
|
|
||||||
room = str(body.get("room") or "default")
|
|
||||||
action = str(body.get("action") or "")
|
|
||||||
with _lock:
|
|
||||||
st = _room_state(room)
|
|
||||||
if action == "pause":
|
|
||||||
st["desired"] = "paused"
|
|
||||||
elif action == "resume":
|
|
||||||
if not st["track_id"]:
|
|
||||||
raise HTTPException(409, "nothing loaded")
|
|
||||||
st["desired"] = "playing"
|
|
||||||
elif action == "stop":
|
|
||||||
st["desired"] = "stopped"
|
|
||||||
elif action == "seek":
|
|
||||||
try:
|
|
||||||
position = max(0.0, float(body.get("position", 0)))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
raise HTTPException(400, "position must be a number")
|
|
||||||
st["seek_hint"] = {"position": position, "ts": time.time()}
|
|
||||||
elif action == "skip":
|
|
||||||
current = st["track_id"]
|
|
||||||
candidates = sorted(
|
|
||||||
(
|
|
||||||
t for t in _library["tracks"].values()
|
|
||||||
if t.get("status") == "ready" and t["id"] != current
|
|
||||||
),
|
|
||||||
key=lambda t: t.get("added_at", 0),
|
|
||||||
reverse=True,
|
|
||||||
)
|
|
||||||
if candidates:
|
|
||||||
st["track_id"] = candidates[0]["id"]
|
|
||||||
st["desired"] = "playing"
|
|
||||||
st["seek_hint"] = None
|
|
||||||
else:
|
|
||||||
st["desired"] = "stopped"
|
|
||||||
else:
|
|
||||||
raise HTTPException(400, f"unknown action: {action}")
|
|
||||||
snapshot = _room_snapshot(room)
|
|
||||||
return {"ok": True, **snapshot}
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/choices")
|
|
||||||
async def choices(request: Request) -> dict:
|
|
||||||
"""Publish Hope's song suggestions so the UI can render clickable picks."""
|
|
||||||
body = await request.json()
|
|
||||||
room = str(body.get("room") or "default")
|
|
||||||
raw = body.get("choices") or []
|
|
||||||
cleaned = []
|
|
||||||
for c in raw[:5]:
|
|
||||||
if not isinstance(c, dict) or not c.get("video_id"):
|
|
||||||
continue
|
|
||||||
cleaned.append({
|
|
||||||
"video_id": str(c["video_id"]),
|
|
||||||
"title": str(c.get("title") or c["video_id"]),
|
|
||||||
"channel": str(c.get("channel") or ""),
|
|
||||||
"duration": c.get("duration"),
|
|
||||||
})
|
|
||||||
with _lock:
|
|
||||||
_room_state(room)["choices"] = cleaned
|
|
||||||
return {"ok": True, "choices": cleaned}
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/media/api/state")
|
|
||||||
def state(room: str = "default") -> dict:
|
|
||||||
with _lock:
|
|
||||||
return _room_snapshot(room)
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/report")
|
|
||||||
async def report(request: Request) -> dict:
|
|
||||||
"""The browser reports real playback state (source of truth for the player)."""
|
|
||||||
body = await request.json()
|
|
||||||
room = str(body.get("room") or "default")
|
|
||||||
try:
|
|
||||||
position = float(body.get("position") or 0.0)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
position = 0.0
|
|
||||||
with _lock:
|
|
||||||
st = _room_state(room)
|
|
||||||
st["reported"] = {
|
|
||||||
"track_id": body.get("track_id"),
|
|
||||||
"state": str(body.get("state") or "stopped"),
|
|
||||||
"position": position,
|
|
||||||
"duration": body.get("duration"),
|
|
||||||
"ts": time.time(),
|
|
||||||
}
|
|
||||||
return {"ok": True}
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/media/api/streams")
|
|
||||||
async def add_stream(request: Request) -> dict:
|
|
||||||
body = await request.json()
|
|
||||||
track = _resolve_play_target({"url": body.get("url"), "name": body.get("name")})
|
|
||||||
return {"ok": True, "track": _track_dict(track)}
|
|
||||||
|
|
||||||
|
|
||||||
# ── API: stream proxy (external http(s) audio over same-origin HTTPS) ───────
|
|
||||||
|
|
||||||
_HOP_HEADERS = ("content-type", "content-range", "accept-ranges", "content-length")
|
|
||||||
|
|
||||||
|
|
||||||
@app.get("/media/api/stream/{track_id}")
|
|
||||||
async def stream_proxy(track_id: str, request: Request):
|
|
||||||
with _lock:
|
|
||||||
track = _get_track(track_id)
|
|
||||||
job = next(
|
|
||||||
(j for j in _jobs.values()
|
|
||||||
if j["track_id"] == track_id and j["stage"] == "downloading"),
|
|
||||||
None,
|
|
||||||
)
|
|
||||||
if not track:
|
|
||||||
raise HTTPException(404, "unknown track")
|
|
||||||
|
|
||||||
if track["kind"] == "youtube":
|
|
||||||
# Progressive download: an active download is streamed in real time
|
|
||||||
# (chunked, no Content-Length) so playback starts while the file
|
|
||||||
# grows; a finished file is served directly with native Range support.
|
|
||||||
path = _youtube_media_file(track_id)
|
|
||||||
if path is None and job is not None:
|
|
||||||
path = await _wait_for_media_file(track_id)
|
|
||||||
if path is None:
|
|
||||||
raise HTTPException(
|
|
||||||
404, f"audio file not available (track is {track.get('status')})"
|
|
||||||
)
|
|
||||||
if job is not None:
|
|
||||||
return StreamingResponse(
|
|
||||||
_growing_file(track_id, job),
|
|
||||||
media_type=_media_type_for(path.name),
|
|
||||||
headers={"Cache-Control": "no-store"},
|
|
||||||
)
|
|
||||||
return FileResponse(path, media_type=_media_type_for(path.name))
|
|
||||||
|
|
||||||
range_header = request.headers.get("range")
|
|
||||||
headers = {"Range": range_header} if range_header else {}
|
|
||||||
client = httpx.AsyncClient(timeout=httpx.Timeout(30, read=600), follow_redirects=True)
|
|
||||||
try:
|
|
||||||
upstream = await client.send(client.build_request("GET", track["url"], headers=headers), stream=True)
|
|
||||||
except httpx.HTTPError as exc:
|
|
||||||
await client.aclose()
|
|
||||||
raise HTTPException(502, f"upstream stream failed: {exc}")
|
|
||||||
|
|
||||||
if upstream.status_code >= 400:
|
|
||||||
await upstream.aclose()
|
|
||||||
await client.aclose()
|
|
||||||
raise HTTPException(502, f"upstream returned {upstream.status_code}")
|
|
||||||
|
|
||||||
out_headers = {
|
|
||||||
name: value
|
|
||||||
for name, value in upstream.headers.items()
|
|
||||||
if name.lower() in _HOP_HEADERS
|
|
||||||
}
|
|
||||||
media_type = upstream.headers.get("content-type", "audio/mpeg")
|
|
||||||
|
|
||||||
async def relay():
|
|
||||||
try:
|
|
||||||
async for chunk in upstream.aiter_bytes(64 * 1024):
|
|
||||||
yield chunk
|
|
||||||
finally:
|
|
||||||
await upstream.aclose()
|
|
||||||
await client.aclose()
|
|
||||||
|
|
||||||
return StreamingResponse(
|
|
||||||
relay(),
|
|
||||||
status_code=upstream.status_code,
|
|
||||||
media_type=media_type,
|
|
||||||
headers=out_headers,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@app.exception_handler(HTTPException)
|
|
||||||
async def http_error(_request: Request, exc: HTTPException):
|
|
||||||
return JSONResponse({"error": exc.detail}, status_code=exc.status_code)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
uvicorn.run(app, host=MEDIA_HOST, port=MEDIA_PORT, log_level="info")
|
|
||||||
+1
-300
@@ -371,34 +371,6 @@ button {
|
|||||||
transition: opacity 90ms linear, width 90ms linear, height 90ms linear;
|
transition: opacity 90ms linear, width 90ms linear, height 90ms linear;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Album art fills the square while a track is loaded outside a conversation. */
|
|
||||||
.led-art {
|
|
||||||
position: absolute;
|
|
||||||
inset: 12px;
|
|
||||||
width: calc(100% - 24px);
|
|
||||||
height: calc(100% - 24px);
|
|
||||||
object-fit: cover;
|
|
||||||
border-radius: 18px;
|
|
||||||
z-index: 1;
|
|
||||||
opacity: 0;
|
|
||||||
transform: scale(0.96);
|
|
||||||
transition: opacity 240ms ease, transform 240ms ease;
|
|
||||||
pointer-events: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.led-wall.show-art .led-art {
|
|
||||||
opacity: 1;
|
|
||||||
transform: scale(1);
|
|
||||||
}
|
|
||||||
|
|
||||||
.led-wall.show-art .led-dot {
|
|
||||||
opacity: 0 !important;
|
|
||||||
transition: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.led-wall.show-art {
|
|
||||||
filter: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Palette shifts by state — a cool blue while it listens, warm toward the
|
/* Palette shifts by state — a cool blue while it listens, warm toward the
|
||||||
center as the agent speaks, amber while thinking, dim when idle. */
|
center as the agent speaks, amber while thinking, dim when idle. */
|
||||||
@@ -1201,276 +1173,6 @@ body[data-mode="text"] .mic-meter {
|
|||||||
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
|
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
|
||||||
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
|
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
|
||||||
|
|
||||||
/* ── Media player ───────────────────────────────────────────────────────── */
|
|
||||||
/* Reuses .tasks-panel positioning (fixed right, slides in via .open); the
|
|
||||||
content scrolls under a sticky header. */
|
|
||||||
.media-panel {
|
|
||||||
overflow-y: auto;
|
|
||||||
padding-bottom: 1.2rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-panel .tasks-header {
|
|
||||||
position: sticky;
|
|
||||||
top: 0;
|
|
||||||
z-index: 1;
|
|
||||||
background: #0d1929;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-now {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.75rem;
|
|
||||||
margin: 1rem 1.1rem 0;
|
|
||||||
padding: 0.75rem;
|
|
||||||
border-radius: 12px;
|
|
||||||
background: var(--surface-raised);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-art,
|
|
||||||
.media-art-placeholder {
|
|
||||||
width: 56px;
|
|
||||||
height: 56px;
|
|
||||||
flex: 0 0 56px;
|
|
||||||
border-radius: 8px;
|
|
||||||
object-fit: cover;
|
|
||||||
background: rgba(23, 41, 65, 0.7);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-art-placeholder {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
color: var(--ink-muted);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-now-info {
|
|
||||||
flex: 1;
|
|
||||||
min-width: 0;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 0.15rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-now-info strong {
|
|
||||||
color: var(--ink);
|
|
||||||
font-size: 0.88rem;
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-now-info span {
|
|
||||||
color: var(--ink-muted);
|
|
||||||
font-size: 0.72rem;
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-progress {
|
|
||||||
position: relative;
|
|
||||||
margin-top: 0.35rem;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.5rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* Track bar behind the fill (fill width is driven inline by progress). */
|
|
||||||
.media-progress::before {
|
|
||||||
content: "";
|
|
||||||
position: absolute;
|
|
||||||
left: 0;
|
|
||||||
top: 50%;
|
|
||||||
transform: translateY(-50%);
|
|
||||||
width: 90px;
|
|
||||||
height: 4px;
|
|
||||||
border-radius: 2px;
|
|
||||||
background: rgba(139, 167, 255, 0.18);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-progress-fill {
|
|
||||||
height: 4px;
|
|
||||||
flex: 0 0 auto;
|
|
||||||
width: 0;
|
|
||||||
border-radius: 2px;
|
|
||||||
background: var(--iris);
|
|
||||||
transition: width 400ms ease;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-progress-label {
|
|
||||||
color: var(--ink-muted);
|
|
||||||
font-size: 0.66rem;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-controls {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
gap: 0.9rem;
|
|
||||||
margin: 0.9rem 1.1rem 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-controls .btn-icon {
|
|
||||||
color: var(--ink-soft);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-play {
|
|
||||||
width: 52px;
|
|
||||||
height: 52px;
|
|
||||||
border-radius: 50%;
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
color: #0b1523 !important;
|
|
||||||
background: var(--iris) !important;
|
|
||||||
border: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-play .play-icon,
|
|
||||||
.media-play .pause-icon {
|
|
||||||
display: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-play:not(.playing) .play-icon {
|
|
||||||
display: inline-block;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-play.playing .pause-icon {
|
|
||||||
display: inline-block;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-play:hover:not(:disabled) {
|
|
||||||
filter: brightness(1.1);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-seek {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.55rem;
|
|
||||||
margin: 0.75rem 1.1rem 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-time {
|
|
||||||
color: var(--ink-muted);
|
|
||||||
font-size: 0.68rem;
|
|
||||||
font-variant-numeric: tabular-nums;
|
|
||||||
min-width: 2.6rem;
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-seek input[type="range"],
|
|
||||||
.media-volume input[type="range"] {
|
|
||||||
flex: 1;
|
|
||||||
accent-color: var(--iris);
|
|
||||||
cursor: pointer;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-volume {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.55rem;
|
|
||||||
margin: 0.55rem 1.1rem 0;
|
|
||||||
color: var(--ink-muted);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-search {
|
|
||||||
display: flex;
|
|
||||||
gap: 0.45rem;
|
|
||||||
margin: 0 1.1rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-search input {
|
|
||||||
flex: 1;
|
|
||||||
min-width: 0;
|
|
||||||
padding: 0.5rem 0.7rem;
|
|
||||||
border: 1px solid var(--line-strong);
|
|
||||||
border-radius: 9px;
|
|
||||||
color: var(--ink);
|
|
||||||
background: rgba(5, 11, 20, 0.45);
|
|
||||||
font-size: 0.8rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-search input::placeholder {
|
|
||||||
color: var(--ink-muted);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-stream-form input:first-child {
|
|
||||||
flex: 0 0 38%;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-list {
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 0.4rem;
|
|
||||||
margin: 0.65rem 1.1rem 1rem;
|
|
||||||
max-height: 240px;
|
|
||||||
overflow-y: auto;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row {
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 0.6rem;
|
|
||||||
padding: 0.55rem 0.65rem;
|
|
||||||
border-radius: 9px;
|
|
||||||
background: var(--surface-raised);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-info {
|
|
||||||
flex: 1;
|
|
||||||
min-width: 0;
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 0.1rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-info strong {
|
|
||||||
color: var(--ink);
|
|
||||||
font-size: 0.78rem;
|
|
||||||
font-weight: 600;
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-info span {
|
|
||||||
color: var(--ink-muted);
|
|
||||||
font-size: 0.68rem;
|
|
||||||
overflow: hidden;
|
|
||||||
text-overflow: ellipsis;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-play {
|
|
||||||
flex: 0 0 auto;
|
|
||||||
padding: 0.3rem 0.7rem;
|
|
||||||
font-size: 0.7rem;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-delete {
|
|
||||||
flex: 0 0 auto;
|
|
||||||
color: var(--ink-muted);
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-row-delete:hover {
|
|
||||||
color: #ffc2ba;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-empty {
|
|
||||||
padding: 0.8rem;
|
|
||||||
border-radius: 9px;
|
|
||||||
color: var(--ink-muted);
|
|
||||||
background: var(--surface-raised);
|
|
||||||
font-size: 0.74rem;
|
|
||||||
text-align: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
.media-choice {
|
|
||||||
border: 1px solid rgba(139, 167, 255, 0.35);
|
|
||||||
background: rgba(60, 95, 201, 0.12);
|
|
||||||
}
|
|
||||||
|
|
||||||
/* ── Responsive ─────────────────────────────────────────────────────────── */
|
/* ── Responsive ─────────────────────────────────────────────────────────── */
|
||||||
@media (max-height: 760px) and (min-width: 881px) {
|
@media (max-height: 760px) and (min-width: 881px) {
|
||||||
.workspace {
|
.workspace {
|
||||||
@@ -1614,8 +1316,7 @@ body[data-mode="text"] .mic-meter {
|
|||||||
padding: 1.25rem 1rem;
|
padding: 1.25rem 1rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
.tasks-panel,
|
.tasks-panel {
|
||||||
.media-panel {
|
|
||||||
width: 100vw;
|
width: 100vw;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -97,8 +97,8 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
resp = json.dumps(
|
resp = json.dumps(
|
||||||
{
|
{
|
||||||
"token": token,
|
"token": token,
|
||||||
# The room name scopes media-server player state; the UI sends
|
# Returned for diagnostics and reconnects. Music deliberately
|
||||||
# it with every /media/api/ call.
|
# uses its own stable scope in the separate Hope Music service.
|
||||||
"room": room_name,
|
"room": room_name,
|
||||||
"user": {
|
"user": {
|
||||||
"id": user_id,
|
"id": user_id,
|
||||||
|
|||||||
Reference in new issue
Block a user