From a53c090882ea6358160efd79d053d4fc26d9936b Mon Sep 17 00:00:00 2001 From: Shane Maynard Date: Sat, 29 Aug 2026 13:22:01 -0400 Subject: [PATCH] docs: media player subsystem architecture and conventions Document the media server process, /media/ port map, library layout on the ./media bind mount, Hope's media MCP toolset, the music/mic mutual-exclusion rule, and yt-dlp download/proxy conventions. --- AGENTS.md | 54 ++++- media/jobs/477f760d9615.json | 1 + media/jobs/7436101dd688.json | 1 + media/jobs/d1aab72f3953.json | 1 + media/library.json | 1 + tests/README.md | 1 + tests/test_media.py | 398 +++++++++++++++++++++++++++++++++++ web/app.js | 12 +- web/media_server.py | 23 +- 9 files changed, 479 insertions(+), 13 deletions(-) create mode 100644 media/jobs/477f760d9615.json create mode 100644 media/jobs/7436101dd688.json create mode 100644 media/jobs/d1aab72f3953.json create mode 100644 media/library.json create mode 100644 tests/test_media.py diff --git a/AGENTS.md b/AGENTS.md index 927038a..f048734 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,17 +4,21 @@ A single-container voice assistant built on LiveKit Agents. Speaks and listens i ## Architecture -One Docker container runs three processes via supervisord: +One Docker container runs four processes via supervisord: 1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media) 2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS 3. **Web frontend** — static HTML served by nginx (port 9178 by default) +4. **Media server** — FastAPI music library/player coordinator (port 8092 internal, proxied at /media/api/) ``` Browser ──WebRTC──► LiveKit Server ──audio──► Agent ▲ │ └────────── audio ◄────────────────────────────┘ Azure STT → LLM (OpenWebUI) → Azure TTS + +Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media + └────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘ ``` ## Quick Start @@ -40,6 +44,7 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp` | 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) | | 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN | | 7882 | UDP | LiveKit RTC media (muxed) | LAN | +| 8092 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) | | 9178 | TCP | Web frontend (HTTPS) | LAN | ## Configuration @@ -53,6 +58,33 @@ Key variables: - `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI) - `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI) - `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`) +- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads + +## Media Player + +Hope can play music on the web UI's media player: YouTube Music search → +yt-dlp download → ffmpeg transcode to mp3 → nginx static streaming (HTTP +Range) back to the browser's `