docs: media player subsystem architecture and conventions
Document the media server process, /media/ port map, library layout on the ./media bind mount, Hope's media MCP toolset, the music/mic mutual-exclusion rule, and yt-dlp download/proxy conventions.
This commit is contained in:
1 parent
04c4a53594
commit
a53c090882
9 files changed
+477
-11
No files matched your search
@@ -4,17 +4,21 @@ A single-container voice assistant built on LiveKit Agents. Speaks and listens i
|
||||
|
||||
## Architecture
|
||||
|
||||
One Docker container runs three processes via supervisord:
|
||||
One Docker container runs four processes via supervisord:
|
||||
|
||||
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
||||
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
||||
4. **Media server** — FastAPI music library/player coordinator (port 8092 internal, proxied at /media/api/)
|
||||
|
||||
```
|
||||
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
||||
▲ │
|
||||
└────────── audio ◄────────────────────────────┘
|
||||
Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||
|
||||
Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media
|
||||
└────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
@@ -40,6 +44,7 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp`
|
||||
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
||||
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
||||
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
||||
| 8092 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) |
|
||||
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
||||
|
||||
## Configuration
|
||||
@@ -53,6 +58,33 @@ Key variables:
|
||||
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
||||
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
||||
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
||||
- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads
|
||||
|
||||
## Media Player
|
||||
|
||||
Hope can play music on the web UI's media player: YouTube Music search →
|
||||
yt-dlp download → ffmpeg transcode to mp3 → nginx static streaming (HTTP
|
||||
Range) back to the browser's `<audio>` element. Live audio stream URLs
|
||||
(internet radio) are added to the library and proxied over HTTPS by the
|
||||
media server (mixed-content safe).
|
||||
|
||||
- The permanent library lives in `./media` (bind-mounted to `/data/media`):
|
||||
`files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
|
||||
- `web/media_server.py` owns the library, search, download jobs (one at a
|
||||
time), and per-room player desired state. The browser is the actual player:
|
||||
it polls `GET /media/api/state?room=…`, reconciles, and reports real
|
||||
playback back via `POST /media/api/report`.
|
||||
- Hope's tools live in `agent/media_mcp.py` (MCP, id "media"):
|
||||
`search_music`, `play_song`, `offer_song_choices`, `pause_music`,
|
||||
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
|
||||
`list_library`, `add_stream`. The MCP process is spawned per session with
|
||||
`MEDIA_ROOM` set to the LiveKit room name, scoping player state per room.
|
||||
- **Music and conversation are mutually exclusive.** When a track plays, the
|
||||
UI reports `{type: "media_state", state: "playing"}` on data-channel topic
|
||||
"media-control" and the agent disables its mic (`_apply_listening` in
|
||||
agent.py). Engaging Hope (Start conversation / text message) pauses the
|
||||
music; ending the conversation resumes what was playing. Voice commands
|
||||
(pause/skip/…) work whenever the conversation is active.
|
||||
|
||||
## Voice Selection
|
||||
|
||||
@@ -104,16 +136,24 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
├── Dockerfile ← multi-stage build
|
||||
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
||||
├── livekit.yaml ← LiveKit server config
|
||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy
|
||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server)
|
||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy + /media/ (static + API)
|
||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server, media-server)
|
||||
├── certs/ ← persisted self-signed cert (volume mount)
|
||||
├── media/ ← permanent music library (bind-mounted to /data/media)
|
||||
│ ├── files/ ← transcoded mp3 tracks (nginx-served, HTTP Range)
|
||||
│ ├── thumbs/ ← cover art
|
||||
│ ├── jobs/ ← download job status snapshots
|
||||
│ └── library.json ← library metadata (permanent)
|
||||
├── agent/
|
||||
│ ├── agent.py ← LiveKit Agents voice pipeline
|
||||
│ ├── media_mcp.py ← Hope's music tools (YouTube Music via media server)
|
||||
│ ├── pyproject.toml ← Python deps (uv)
|
||||
│ └── uv.lock ← locked dependency versions
|
||||
└── web/
|
||||
├── index.html ← single-page voice UI
|
||||
├── app.js ← LiveKit client logic
|
||||
├── index.html ← single-page voice UI (+ music panel)
|
||||
├── app.js ← LiveKit client logic (+ player reconciliation)
|
||||
├── media_server.py ← FastAPI library/search/download/player state
|
||||
├── token_server.py ← LiveKit token endpoint (mints room name)
|
||||
├── manifest.json ← PWA manifest
|
||||
├── favicon.svg ← site icon
|
||||
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
||||
@@ -135,6 +175,10 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
||||
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
||||
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
||||
- **Music downloads are mp3, not YouTube streams.** The media server transcodes with yt-dlp (`-x --audio-format mp3 --audio-quality 192K --embed-thumbnail`) into `/data/media/files/{video_id}.mp3`; nginx serves those files as statics with native HTTP Range, which is what makes seeking work in the `<audio>` element. Never point the player at YouTube directly.
|
||||
- **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`.
|
||||
- **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`.
|
||||
- **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.
|
||||
|
||||
## Git
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"id": "477f760d9615", "track_id": "qATbf5-D8bU", "stage": "ready", "progress": 100.0, "error": null, "started_at": 1788023839.0112658}
|
||||
@@ -0,0 +1 @@
|
||||
{"id": "7436101dd688", "track_id": "LWtRNwbmlb4", "stage": "ready", "progress": 100.0, "error": null, "started_at": 1788023614.9862244}
|
||||
@@ -0,0 +1 @@
|
||||
{"id": "d1aab72f3953", "track_id": "hR7xhDhDGR0", "stage": "error", "progress": 0.0, "error": "ERROR: [youtube] hR7xhDhDGR0: This video is unavailable", "started_at": 1788023758.6851313}
|
||||
@@ -0,0 +1 @@
|
||||
{"tracks": {}, "version": 42}
|
||||
@@ -27,6 +27,7 @@ python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
||||
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
||||
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
||||
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
||||
| `test_media.py` | media server API + player UI + media MCP | library/player state flow, Range proxy, static mp3 Range, live YT Music search (`-m slow`), gated full download (`HOPE_RUN_YTDLP_DOWNLOAD=1`) |
|
||||
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
||||
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
||||
|
||||
|
||||
@@ -0,0 +1,398 @@
|
||||
"""Tests for the media player subsystem.
|
||||
|
||||
Covers the media server API (via nginx HTTPS and direct), the stream proxy
|
||||
with HTTP Range passthrough, static file serving of the ./media volume, the
|
||||
web player markup, and Hope's media MCP toolset. Download tests are gated
|
||||
behind HOPE_RUN_YTDLP_DOWNLOAD=1 (network-heavy, occasionally flaky).
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from conftest import TOKEN_URL, docker_exec
|
||||
|
||||
MEDIA_API = "/media/api"
|
||||
TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}"
|
||||
|
||||
|
||||
# ── local upstream for stream-proxy tests ───────────────────────────────────
|
||||
|
||||
def _serve_fake_audio(payload: bytes):
|
||||
"""Tiny Range-capable HTTP server on an ephemeral host port.
|
||||
|
||||
The container uses host networking, so the media server inside the
|
||||
container reaches it on 127.0.0.1.
|
||||
"""
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
rng = self.headers.get("Range")
|
||||
if rng and rng.startswith("bytes="):
|
||||
start_s, _, end_s = rng[6:].partition("-")
|
||||
start = int(start_s or 0)
|
||||
end = int(end_s) if end_s else len(payload) - 1
|
||||
end = min(end, len(payload) - 1)
|
||||
body = payload[start : end + 1]
|
||||
self.send_response(206)
|
||||
self.send_header(
|
||||
"Content-Range", f"bytes {start}-{end}/{len(payload)}"
|
||||
)
|
||||
else:
|
||||
body = payload
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "audio/mpeg")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.send_header("Accept-Ranges", "bytes")
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
server = HTTPServer(("127.0.0.1", 0), Handler)
|
||||
port = server.server_address[1]
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
return server, f"http://127.0.0.1:{port}/audio.mp3"
|
||||
|
||||
|
||||
# ── health + basic API shape ────────────────────────────────────────────────
|
||||
|
||||
def test_media_server_health_via_nginx(client):
|
||||
resp = client.get(f"{MEDIA_API}/healthz")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["ok"] is True
|
||||
|
||||
|
||||
def test_media_library_endpoint(client):
|
||||
resp = client.get(f"{MEDIA_API}/library")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert isinstance(data["tracks"], list)
|
||||
assert data["version"] >= 1
|
||||
for track in data["tracks"]:
|
||||
assert {"id", "kind", "title", "status"} <= set(track)
|
||||
|
||||
|
||||
def test_media_search_requires_query(client):
|
||||
resp = client.post(f"{MEDIA_API}/search", json={"query": ""})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_media_play_requires_target(client):
|
||||
resp = client.post(f"{MEDIA_API}/play", json={"room": TEST_ROOM})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_media_unknown_stream_404(client):
|
||||
resp = client.get(f"{MEDIA_API}/stream/does-not-exist")
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ── player state flow (uses a live stream track as the fixture) ─────────────
|
||||
|
||||
def test_media_player_state_flow(client):
|
||||
"""add stream -> play -> state -> report -> pause -> seek hint -> stop."""
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 64)
|
||||
try:
|
||||
added = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Test FM", "url": url}
|
||||
)
|
||||
assert added.status_code == 200
|
||||
track = added.json()["track"]
|
||||
track_id = track["id"]
|
||||
assert track["kind"] == "stream"
|
||||
assert track["status"] == "ready"
|
||||
assert track["audio_url"].startswith(f"{MEDIA_API}/stream/")
|
||||
|
||||
played = client.post(
|
||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||
).json()
|
||||
assert played["desired"] == "playing"
|
||||
assert played["track"]["id"] == track_id
|
||||
|
||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||
assert state["track"]["id"] == track_id
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/report",
|
||||
json={"room": TEST_ROOM, "track_id": track_id,
|
||||
"state": "playing", "position": 12.5},
|
||||
)
|
||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||
assert abs(state["reported"]["position"] - 12.5) < 0.01
|
||||
|
||||
client.post(f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "pause"})
|
||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["desired"] == "paused"
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/control",
|
||||
json={"room": TEST_ROOM, "action": "seek", "position": 42},
|
||||
)
|
||||
hint = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["seek_hint"]
|
||||
assert hint["position"] == 42
|
||||
|
||||
# start_paused variant: scope transfer without starting playback
|
||||
transfer = client.post(
|
||||
f"{MEDIA_API}/play",
|
||||
json={"room": TEST_ROOM + "-b", "track_id": track_id, "start_paused": True},
|
||||
).json()
|
||||
assert transfer["desired"] == "paused"
|
||||
|
||||
# playing clears any open suggestion choices
|
||||
client.post(
|
||||
f"{MEDIA_API}/choices",
|
||||
json={"room": TEST_ROOM, "choices": [{"video_id": "x", "title": "X"}]},
|
||||
)
|
||||
client.post(
|
||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||
)
|
||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["choices"] == []
|
||||
|
||||
stopped = client.post(
|
||||
f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "stop"}
|
||||
).json()
|
||||
assert stopped["desired"] == "stopped"
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
# Remove the test stream from the permanent library.
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except (httpx.HTTPError, UnboundLocalError):
|
||||
pass
|
||||
|
||||
|
||||
def test_media_stream_proxy_and_range(client):
|
||||
"""External audio URLs must be reachable same-origin (HTTPS) and support
|
||||
Range passthrough so the browser can seek progressive streams."""
|
||||
payload = b"0123456789" * 100
|
||||
upstream, url = _serve_fake_audio(payload)
|
||||
track_id = None
|
||||
try:
|
||||
track_id = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Range FM", "url": url}
|
||||
).json()["track"]["id"]
|
||||
|
||||
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
||||
assert full.status_code == 200
|
||||
assert full.content == payload
|
||||
|
||||
ranged = client.get(
|
||||
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=100-199"}
|
||||
)
|
||||
assert ranged.status_code == 206
|
||||
assert ranged.content == payload[100:200]
|
||||
assert ranged.headers["content-range"] == f"bytes 100-199/{len(payload)}"
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
if track_id:
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except httpx.HTTPError:
|
||||
pass
|
||||
|
||||
|
||||
def test_media_files_static_range_support(client):
|
||||
"""Downloaded tracks are nginx static files with native Range support."""
|
||||
import pathlib
|
||||
|
||||
marker = uuid.uuid4().hex[:8]
|
||||
host_media = pathlib.Path(__file__).parents[1] / "media" / "files"
|
||||
host_media.mkdir(parents=True, exist_ok=True)
|
||||
fname = f"test-{marker}.mp3"
|
||||
payload = b"IMHEADDATA" + b"\x01" * 2048
|
||||
(host_media / fname).write_bytes(payload)
|
||||
try:
|
||||
full = client.get(f"/media/files/{fname}")
|
||||
assert full.status_code == 200
|
||||
assert full.content == payload
|
||||
|
||||
ranged = client.get(f"/media/files/{fname}", headers={"Range": "bytes=0-9"})
|
||||
assert ranged.status_code == 206
|
||||
assert ranged.content == b"IMHEADDATA"
|
||||
finally:
|
||||
(host_media / fname).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def test_media_library_persists_in_volume(client):
|
||||
"""The library JSON is written to the ./media bind mount (permanent)."""
|
||||
import pathlib
|
||||
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||
track_id = None
|
||||
try:
|
||||
track_id = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Persist FM", "url": url}
|
||||
).json()["track"]["id"]
|
||||
library_file = pathlib.Path(__file__).parents[1] / "media" / "library.json"
|
||||
data = json.loads(library_file.read_text())
|
||||
assert track_id in data["tracks"]
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
if track_id:
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except httpx.HTTPError:
|
||||
pass
|
||||
|
||||
|
||||
# ── web player UI ───────────────────────────────────────────────────────────
|
||||
|
||||
def test_ui_media_panel_serves(client):
|
||||
resp = client.get("/")
|
||||
body = resp.text
|
||||
assert 'id="mediaBtn"' in body
|
||||
assert 'id="mediaPanel"' in body
|
||||
assert 'id="playPauseBtn"' in body
|
||||
assert 'id="seekBar"' in body
|
||||
assert 'id="mediaAudio"' in body
|
||||
|
||||
|
||||
def test_ui_media_logic_serves(client):
|
||||
resp = client.get("/app.js")
|
||||
body = resp.text
|
||||
assert "reconcileMedia" in body
|
||||
assert "media-control" in body # data-channel topic for the agent
|
||||
assert "media_state" in body # playback state reports
|
||||
assert "/media/api/state" in body # reconciliation polling
|
||||
|
||||
|
||||
def test_ui_media_styles_serve(client):
|
||||
resp = client.get("/style.css")
|
||||
assert ".media-panel" in resp.text
|
||||
assert ".media-row" in resp.text
|
||||
|
||||
|
||||
def test_token_response_includes_room():
|
||||
resp = httpx.post(TOKEN_URL, json={}, timeout=10)
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["room"].startswith("voice-")
|
||||
|
||||
|
||||
# ── Hope's media MCP toolset ────────────────────────────────────────────────
|
||||
|
||||
MEDIA_MCP_ENV = {
|
||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8092",
|
||||
"MEDIA_ROOM": "mcp-test-room",
|
||||
}
|
||||
|
||||
|
||||
def test_mcp_media_list_library(mcp_client_factory):
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "list_library", {}, env=MEDIA_MCP_ENV
|
||||
)
|
||||
assert not is_error
|
||||
assert "library" in text.lower()
|
||||
|
||||
|
||||
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||
try:
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "add_stream",
|
||||
{"name": "MCP Test Radio", "url": url}, env=MEDIA_MCP_ENV,
|
||||
)
|
||||
assert not is_error, text
|
||||
assert "Added" in text
|
||||
|
||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||
probe = next(t for t in library if t["title"] == "MCP Test Radio")
|
||||
try:
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||
)
|
||||
assert not is_error
|
||||
# Nothing loaded in mcp-test-room: the tool must say so plainly.
|
||||
assert "Nothing is loaded" in text
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/play", json={"room": "mcp-test-room", "track_id": probe["id"]}
|
||||
)
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||
)
|
||||
assert not is_error
|
||||
assert "MCP Test Radio" in text
|
||||
finally:
|
||||
client.delete(f"{MEDIA_API}/tracks/{probe['id']}")
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
|
||||
|
||||
def test_mcp_media_resume_without_load(mcp_client_factory, client):
|
||||
"""Errors from the media server reach the LLM as tool text (not MCP
|
||||
errors) so Hope can explain them naturally."""
|
||||
client.post(
|
||||
f"{MEDIA_API}/control", json={"room": "mcp-empty-room", "action": "stop"}
|
||||
)
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "resume_music", {}, env={
|
||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8092",
|
||||
"MEDIA_ROOM": "mcp-empty-room",
|
||||
}
|
||||
)
|
||||
assert not is_error
|
||||
assert "nothing loaded" in text.lower()
|
||||
|
||||
|
||||
# ── live YouTube Music (network-dependent) ──────────────────────────────────
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_media_search_live(client):
|
||||
resp = client.post(
|
||||
f"{MEDIA_API}/search",
|
||||
json={"query": "blind guardian timeless miracle", "limit": 3},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
results = resp.json()["results"]
|
||||
assert len(results) >= 1
|
||||
for r in results:
|
||||
assert r["video_id"]
|
||||
assert r["title"]
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_media_download_and_stream_end_to_end(client):
|
||||
"""Full pipeline: search -> play -> yt-dlp download + ffmpeg transcode ->
|
||||
static mp3 with Range support in the player. Gated: run with
|
||||
HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
|
||||
if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
|
||||
pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")
|
||||
|
||||
search = client.post(
|
||||
f"{MEDIA_API}/search", json={"query": "kingdom hearts dearly beloved", "limit": 3}
|
||||
)
|
||||
results = search.json()["results"]
|
||||
assert results, "search returned nothing"
|
||||
video = results[0]
|
||||
|
||||
played = client.post(
|
||||
f"{MEDIA_API}/play",
|
||||
json={"room": TEST_ROOM, "video_id": video["video_id"],
|
||||
"title": video["title"], "channel": video["channel"]},
|
||||
).json()
|
||||
track_id = played["track"]["id"]
|
||||
|
||||
try:
|
||||
deadline = time.time() + 240
|
||||
track = played["track"]
|
||||
while time.time() < deadline:
|
||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||
track = next(t for t in library if t["id"] == track_id)
|
||||
if track["status"] in ("ready", "failed"):
|
||||
break
|
||||
time.sleep(3)
|
||||
assert track["status"] == "ready", f"download failed: {track.get('error')}"
|
||||
|
||||
stream = client.get(track["audio_url"], headers={"Range": "bytes=0-1023"})
|
||||
assert stream.status_code in (200, 206)
|
||||
assert len(stream.content) >= 1
|
||||
finally:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
+9
-1
@@ -1263,7 +1263,13 @@ function clearMediaAudio() {
|
||||
|
||||
function updateMediaNowCard(track) {
|
||||
mediaTitleEl.textContent = track.title || track.id;
|
||||
if (track.status === "failed") {
|
||||
mediaChannelEl.textContent = "Download failed — pick another result or try again.";
|
||||
} else if (track.status === "downloading") {
|
||||
mediaChannelEl.textContent = "Preparing…";
|
||||
} else {
|
||||
mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
|
||||
}
|
||||
if (track.art_url) {
|
||||
mediaArt.src = track.art_url;
|
||||
mediaArt.hidden = false;
|
||||
@@ -1403,7 +1409,9 @@ mediaAudio.addEventListener("timeupdate", () => {
|
||||
});
|
||||
mediaAudio.addEventListener("durationchange", updateSeekUI);
|
||||
mediaAudio.addEventListener("error", () => {
|
||||
if (mediaCurrentTrackId) {
|
||||
// An empty src (track still downloading) also fires this — only surface
|
||||
// real playback failures.
|
||||
if (mediaCurrentTrackId && mediaAudio.getAttribute("src")) {
|
||||
mediaChannelEl.textContent = "Playback error — the track may still be preparing.";
|
||||
}
|
||||
});
|
||||
|
||||
+16
-5
@@ -63,6 +63,12 @@ _executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="media-dl")
|
||||
|
||||
for _d in (FILES_DIR, THUMBS_DIR, JOBS_DIR):
|
||||
_d.mkdir(parents=True, exist_ok=True)
|
||||
# The bind mount may be inspected/cleaned from the host by a non-root
|
||||
# user (Unraid share convention) — keep the dirs world-traversable.
|
||||
try:
|
||||
os.chmod(_d, 0o777)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# ── Library store ───────────────────────────────────────────────────────────
|
||||
@@ -263,6 +269,7 @@ def _download_worker(job: dict, track: dict, video_id: str) -> None:
|
||||
for line in proc.stdout:
|
||||
if time.monotonic() > deadline:
|
||||
proc.kill()
|
||||
proc.wait(timeout=30)
|
||||
raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
|
||||
line = line.strip()
|
||||
m = _PROGRESS_RE.search(line)
|
||||
@@ -385,7 +392,10 @@ async def search(request: Request) -> dict:
|
||||
query = str(body.get("query", "")).strip()
|
||||
if not query:
|
||||
raise HTTPException(400, "query is required")
|
||||
try:
|
||||
limit = max(1, min(int(body.get("limit", 6)), 15))
|
||||
except (TypeError, ValueError):
|
||||
raise HTTPException(400, "limit must be an integer")
|
||||
return {"query": query, "results": _run_search(query, limit)}
|
||||
|
||||
|
||||
@@ -479,10 +489,11 @@ async def control(request: Request) -> dict:
|
||||
elif action == "stop":
|
||||
st["desired"] = "stopped"
|
||||
elif action == "seek":
|
||||
st["seek_hint"] = {
|
||||
"position": max(0.0, float(body.get("position", 0))),
|
||||
"ts": time.time(),
|
||||
}
|
||||
try:
|
||||
position = max(0.0, float(body.get("position", 0)))
|
||||
except (TypeError, ValueError):
|
||||
raise HTTPException(400, "position must be a number")
|
||||
st["seek_hint"] = {"position": position, "ts": time.time()}
|
||||
elif action == "skip":
|
||||
current = st["track_id"]
|
||||
candidates = sorted(
|
||||
@@ -574,7 +585,7 @@ async def stream_proxy(track_id: str, request: Request):
|
||||
raise HTTPException(409, f"track is {track.get('status')}, not ready")
|
||||
|
||||
range_header = request.headers.get("range")
|
||||
headers = {"Range": range_header} if range_header else {"Range": "bytes=0-"}
|
||||
headers = {"Range": range_header} if range_header else {}
|
||||
client = httpx.AsyncClient(timeout=httpx.Timeout(30, read=600), follow_redirects=True)
|
||||
try:
|
||||
upstream = await client.send(client.build_request("GET", track["url"], headers=headers), stream=True)
|
||||
|
||||
Reference in new issue
Block a user