refactor: split music streaming into Hope Music
Rebuild & Deploy / rebuild (push) Successful in 2m7s
Rebuild & Deploy / rebuild (push) Successful in 2m7s
Move the media server, downloader, library volume, and browser player to the standalone hope-media-music product. Keep Hope's music tools as a thin HTTP client of the LAN service on port 8094. Use a stable voice-room player scope instead of ephemeral LiveKit room names. The agent now owns the conversation handshake: fresh reported playback mutes its input, joining pauses music, and the last participant leaving resumes only music it paused. Peer failures remain non-fatal. Also move the voice deploy workflow onto the active Unraid Gitea runner and remove yt-dlp/ffmpeg/media health and volume dependencies from the image.
This commit is contained in:
snowspeeder authored and 
Shane Maynard committed
1 parent
d4ea4fc984
commit
436b3fc8b1
23 files changed
+349
-2511
No files matched your search
+10
-12
@@ -29,18 +29,16 @@ FIRECRAWL_BASE=http://192.168.86.2:3002
|
||||
# Per-user memory profiles and Bearer tokens live in config/users.json.
|
||||
VOICE_DEFAULT_USER=shane
|
||||
|
||||
# ── Media player (music library + YouTube Music downloads) ─────────────────
|
||||
# The permanent library lives in ./media (bind-mounted to /data/media):
|
||||
# media/files/ transcoded mp3 tracks
|
||||
# media/thumbs/ cover art
|
||||
# media/jobs/ download job status
|
||||
# MEDIA_SERVER_PORT=8093 (8092 is taken by YOURLS/Apache on xNAS)
|
||||
#
|
||||
# Optional: export cookies.txt from a browser logged into YouTube and drop it
|
||||
# in ./media/cookies.txt to avoid "confirm you're not a bot" download blocks.
|
||||
# Anonymous access works from most residential IPs; cookies are the reliable
|
||||
# path.
|
||||
# YTDLP_COOKIES=/data/media/cookies.txt
|
||||
# ── Hope Music (separate LAN service) ───────────────────────────────────────
|
||||
# The voice agent keeps the music MCP tools, but the library, downloads,
|
||||
# streaming, and browser player live in the hope-media-music product.
|
||||
MUSIC_BASE_URL=http://192.168.86.2:8094
|
||||
# Stable player scope shared with the standalone Hope Music page. Do not use
|
||||
# the ephemeral LiveKit room name minted for each conversation.
|
||||
MUSIC_ROOM=voice-room
|
||||
# Player reports are refreshed every few seconds. Older "playing" reports are
|
||||
# treated as stale so a closed player cannot leave Hope's mic disabled.
|
||||
MUSIC_REPORT_TTL=30
|
||||
|
||||
# Extra MCP servers (optional). JSON list of {url, transport} objects.
|
||||
# transport: "sse" | "streamable_http" | omit for auto-detect
|
||||
|
||||
@@ -8,7 +8,7 @@ on:
|
||||
|
||||
jobs:
|
||||
rebuild:
|
||||
runs-on: runner-voice
|
||||
runs-on: unraid
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
@@ -26,7 +26,7 @@ jobs:
|
||||
--name voice \
|
||||
--network host \
|
||||
--restart unless-stopped \
|
||||
--health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null' \
|
||||
--health-cmd='curl -sfk https://localhost:${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null' \
|
||||
--health-interval=15s \
|
||||
--health-timeout=5s \
|
||||
--health-retries=3 \
|
||||
@@ -34,7 +34,6 @@ jobs:
|
||||
-v /mnt/user/dev/hope-voice-api/certs:/etc/voice/certs \
|
||||
-v /mnt/user/dev/hope-voice-api/skills:/skills \
|
||||
-v /mnt/user/dev/hope-voice-api/livekit.yaml:/etc/livekit.yaml:ro \
|
||||
-v /mnt/user/dev/hope-voice-api/media:/data/media \
|
||||
-e LIVEKIT_URL=http://localhost:7880 \
|
||||
-e LIVEKIT_API_KEY=devkey \
|
||||
-e LIVEKIT_API_SECRET=hopevoice-devsecret-2026-lan-only \
|
||||
@@ -49,9 +48,8 @@ jobs:
|
||||
-e WEB_PORT=9178 \
|
||||
-e VOICE_USERS_FILE=/etc/hope/users.json \
|
||||
-e VOICE_DEFAULT_USER=shane \
|
||||
-e MEDIA_DIR=/data/media \
|
||||
-e MEDIA_SERVER_PORT=8093 \
|
||||
-e YTDLP_COOKIES=/data/media/cookies.txt \
|
||||
-e MUSIC_BASE_URL=http://192.168.86.2:8094 \
|
||||
-e MUSIC_ROOM=voice-room \
|
||||
hope-voice-api-voice:latest
|
||||
|
||||
- name: Verify deployment
|
||||
|
||||
@@ -8,7 +8,3 @@ certs/
|
||||
.venv-tests/
|
||||
__pycache__/
|
||||
.pytest_cache/
|
||||
|
||||
# Permanent music library (media server runtime data)
|
||||
/media/*
|
||||
!/media/.gitkeep
|
||||
@@ -9,16 +9,13 @@ One Docker container runs four processes via supervisord:
|
||||
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
||||
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
||||
4. **Media server** — FastAPI music library/player coordinator (port 8093 internal, proxied at /media/api/)
|
||||
4. **Token server** — authenticated LiveKit bootstrap endpoint (port 8091 internal, proxied at `/token`)
|
||||
|
||||
```
|
||||
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
||||
▲ │
|
||||
└────────── audio ◄────────────────────────────┘
|
||||
Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||
|
||||
Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media
|
||||
└────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
@@ -41,10 +38,10 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp`
|
||||
|
||||
| Port | Protocol | Service | Access |
|
||||
|-------|----------|----------------------|--------------|
|
||||
| 8091 | TCP | Token server | internal only (proxied via nginx at /token) |
|
||||
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
||||
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
||||
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
||||
| 8093 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) |
|
||||
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
||||
|
||||
## Configuration
|
||||
@@ -58,36 +55,32 @@ Key variables:
|
||||
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
||||
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
||||
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
||||
- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads
|
||||
- `MUSIC_BASE_URL` — standalone Hope Music LAN API (default `http://192.168.86.2:8094`)
|
||||
- `MUSIC_ROOM` — stable player scope shared with Hope Music (default `voice-room`)
|
||||
|
||||
## Media Player
|
||||
## Hope Music Integration
|
||||
|
||||
Hope can play music on the web UI's media player: YouTube Music search →
|
||||
yt-dlp raw best-audio download (webm/m4a, no transcode) → progressive
|
||||
streaming from the media server while the file grows → finished file served
|
||||
with native HTTP Range for seeking. Songs start playing within seconds of the
|
||||
download starting and remain in the library for instant offline replay.
|
||||
Live audio stream URLs (internet radio) are added to the library and proxied
|
||||
over HTTPS by the media server (mixed-content safe). Search results put ready
|
||||
library matches first so Hope prefers a downloaded copy over a fresh lookup.
|
||||
Music is a separate product and deployment: `snowspeeder/hope-media-music`. It
|
||||
owns the library, YouTube Music search/download pipeline, stream proxy, and
|
||||
the standalone browser player at `https://192.168.86.2:9179`. This repo keeps
|
||||
only the agent-side control client.
|
||||
|
||||
- The permanent library lives in `./media` (bind-mounted to `/data/media`):
|
||||
`files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
|
||||
- `web/media_server.py` owns the library, search, download jobs (one at a
|
||||
time), and per-room player desired state. The browser is the actual player:
|
||||
it polls `GET /media/api/state?room=…`, reconciles, and reports real
|
||||
playback back via `POST /media/api/report`.
|
||||
- Hope's tools live in `agent/media_mcp.py` (MCP, id "media"):
|
||||
- Hope's tools live in `agent/media_mcp.py` (MCP, id `media`):
|
||||
`search_music`, `play_song`, `offer_song_choices`, `pause_music`,
|
||||
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
|
||||
`list_library`, `add_stream`. The MCP process is spawned per session with
|
||||
`MEDIA_ROOM` set to the LiveKit room name, scoping player state per room.
|
||||
- **Music and conversation are mutually exclusive.** When a track plays, the
|
||||
UI reports `{type: "media_state", state: "playing"}` on data-channel topic
|
||||
"media-control" and the agent disables its mic (`_apply_listening` in
|
||||
agent.py). Engaging Hope (Start conversation / text message) pauses the
|
||||
music; ending the conversation resumes what was playing. Voice commands
|
||||
(pause/skip/…) work whenever the conversation is active.
|
||||
`list_library`, `add_stream`. They call `MUSIC_BASE_URL` over HTTP.
|
||||
- Music uses the stable `MUSIC_ROOM` scope (`voice-room` by default), not the
|
||||
ephemeral LiveKit room minted for each conversation. The standalone player
|
||||
defaults to that same scope, so voice commands control the open music page
|
||||
and playback survives across voice sessions.
|
||||
- **Music and conversation remain mutually exclusive.** The agent polls Hope
|
||||
Music's fresh browser-reported state. Actual playback disables the agent's
|
||||
mic; joining a conversation pauses music, and the last participant leaving
|
||||
resumes only music that the agent paused.
|
||||
- A stale `playing` report (default: thirty seconds) is treated as stopped so
|
||||
closing the music page cannot leave Hope's mic disabled.
|
||||
- Hope Music is a peer service: an outage logs a warning but never takes down
|
||||
the voice conversation. Its health and E2E tests belong in its own repo.
|
||||
|
||||
## Voice Selection
|
||||
|
||||
@@ -153,8 +146,7 @@ identical to a failure). Isolated single-tool probes will pass and mislead.
|
||||
via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit
|
||||
and the one CI (rebuild.yml) deploys on every push to master. The local dev
|
||||
compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178`
|
||||
to target it instead. Tests that can only pass on the deployment host
|
||||
(repo `./media` bind mount) or that probe the local container via
|
||||
to target it instead. Tests that probe the local container through
|
||||
`docker exec voice` skip automatically on remote targets.
|
||||
|
||||
```bash
|
||||
@@ -164,8 +156,8 @@ docker compose ps
|
||||
# Run the suite against the Unraid deployment (default)
|
||||
python -m pytest tests/ -v -m "not slow"
|
||||
|
||||
# Real end-to-end YouTube Music download (network-heavy, gated)
|
||||
HOPE_RUN_YTDLP_DOWNLOAD=1 python -m pytest tests/test_media.py -m slow
|
||||
# Verify the external Hope Music boundary + client configuration
|
||||
python -m pytest tests/test_media.py -v
|
||||
|
||||
# Check all logs (single service: "voice")
|
||||
docker compose logs -f voice
|
||||
@@ -184,7 +176,7 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
## File Layout
|
||||
|
||||
```
|
||||
~/dev/voice/
|
||||
~/dev/hope-voice-api/
|
||||
├── AGENTS.md ← you are here
|
||||
├── .env.example ← config template (copy to .env)
|
||||
├── .gitignore
|
||||
@@ -192,24 +184,18 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
├── Dockerfile ← multi-stage build
|
||||
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
||||
├── livekit.yaml ← LiveKit server config
|
||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy + /media/ (static + API)
|
||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server, media-server)
|
||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy
|
||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server)
|
||||
├── certs/ ← persisted self-signed cert (volume mount)
|
||||
├── media/ ← permanent music library (bind-mounted to /data/media)
|
||||
│ ├── files/ ← raw audio tracks (webm/m4a; media-server streamed, Range when complete)
|
||||
│ ├── thumbs/ ← cover art
|
||||
│ ├── jobs/ ← download job status snapshots
|
||||
│ └── library.json ← library metadata (permanent)
|
||||
├── agent/
|
||||
│ ├── agent.py ← LiveKit Agents voice pipeline
|
||||
│ ├── media_mcp.py ← Hope's music tools (YouTube Music via media server)
|
||||
│ ├── agent.py ← LiveKit Agents voice pipeline + Hope Music handshake
|
||||
│ ├── media_mcp.py ← thin HTTP client for the separate Hope Music service
|
||||
│ ├── pyproject.toml ← Python deps (uv)
|
||||
│ └── uv.lock ← locked dependency versions
|
||||
└── web/
|
||||
├── index.html ← single-page voice UI (+ music panel)
|
||||
├── app.js ← LiveKit client logic (+ player reconciliation)
|
||||
├── media_server.py ← FastAPI library/search/download/player state
|
||||
├── token_server.py ← LiveKit token endpoint (mints room name)
|
||||
├── index.html ← voice-only single-page UI
|
||||
├── app.js ← LiveKit client logic (no music player)
|
||||
├── token_server.py ← LiveKit token endpoint (mints ephemeral room names)
|
||||
├── manifest.json ← PWA manifest
|
||||
├── favicon.svg ← site icon
|
||||
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
||||
@@ -218,9 +204,9 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Single container.** All services (LiveKit, agent, web, token endpoint) run in one Docker container via supervisord. No multi-service compose.
|
||||
- **Single voice container.** LiveKit, the agent, nginx, and token endpoint run in one Docker container via supervisord. Hope Music is deliberately a separate product/container.
|
||||
- **No published UDP ports in compose.** LiveKit binds its media ports directly on the host network (`network_mode: host`). This avoids the docker-proxy process explosion that hit hope-webui.
|
||||
- **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server now pins the room name to "voice-room" server-side and no longer accepts arbitrary room names.
|
||||
- **Agent dispatch via roomConfig token claim.** LiveKit only dispatches agents to rooms that request them; a room auto-created by a participant join gets none. The token endpoint embeds `roomConfig.agents` in every JWT so the agent is dispatched when the browser joins. Do not pre-create rooms instead — if the agent worker isn't registered yet (first ~15s after container start), the dispatch silently fails and never retries; joining later re-fires it. The token server ignores browser-supplied room names and mints `voice-{user}-{uuid8}` server-side.
|
||||
- **Interruption mode must be "vad".** `interruption={"mode": "adaptive"}` requires the LiveKit Cloud barge-in service (agent-gateway.livekit.cloud) and spams 401 retries on self-hosted setups.
|
||||
- **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`.
|
||||
- **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access.
|
||||
@@ -231,31 +217,27 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
||||
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
||||
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
||||
- **Music downloads are raw audio, streamed as they grow.** yt-dlp fetches
|
||||
bestaudio directly (`-f bestaudio/best`, no `-x`/transcode) into
|
||||
`/data/media/files/{video_id}.{ext}`; the stream endpoint
|
||||
(`GET /media/api/stream/{id}`) serves the growing `.part` file chunked
|
||||
(no Content-Length) while the download runs, then serves the finished file
|
||||
with native Range for seeking. `audio_url` is that endpoint for the whole
|
||||
lifecycle — a stable URL means the browser never reloads mid-playback. The
|
||||
mp3 transcode was dropped: it delayed playback by the whole download+encode,
|
||||
and modern browsers play webm/opus and m4a everywhere. Never point the
|
||||
player at YouTube directly.
|
||||
- **Search prefers the library.** `/media/api/search` merges ready library
|
||||
matches (title token/substring match) in front of the yt-dlp results, and
|
||||
the search_music tool docstring + system prompt tell Hope to always pick
|
||||
`in_library=true`. Deterministic preference lives in the server merge, not
|
||||
in prompt hopes.
|
||||
- **Progress lines come from yt-dlp stderr.** `[download] N%` goes to stderr;
|
||||
only `--print-json` metadata goes to stdout. The download worker drains
|
||||
stderr in a thread (deadlock-safe) and reads stdout for the JSON — parsing
|
||||
progress from stdout silently never matched.
|
||||
- **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`.
|
||||
- **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`.
|
||||
- **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.
|
||||
- **Never hard-require container env in supervisord.** `%(ENV_X)s` expansion aborts the whole supervisord config when the var is missing, taking every program down (this silently killed the run-151 deploy). Programs with safe in-code defaults (media server) must inherit the container env instead of listing it.
|
||||
- **Healthchecks must use `curl -f`.** Without it, any HTTP response — including a 503 from a squatter — counts as healthy. Run #15 deployed "healthy" against a broken media port this way.
|
||||
- **Port 8092 is taken on xNAS.** An Apache/YOURLS stack occupies 8092 on the Unraid host; the media server uses 8093. Before picking a new internal port, probe the Unraid deployment host, not just the dev box.
|
||||
- **Hope Music stays external.** `hope-media-music` owns yt-dlp, ffmpeg, the
|
||||
library volume, streaming endpoints, and the player UI. Do not add those
|
||||
dependencies or files back to this image. This repo owns only the MCP/HTTP
|
||||
client and the conversation/music handshake.
|
||||
- **Music scope is stable.** LiveKit room names are intentionally ephemeral;
|
||||
`MUSIC_ROOM` defaults to `voice-room`, which is also the standalone player's
|
||||
default. Never derive the music scope from `ctx.room.name`.
|
||||
- **Playback truth comes from fresh reports.** The standalone player reports
|
||||
real state every few seconds. The agent polls it and accepts `playing` only
|
||||
while the report timestamp is fresh (`MUSIC_REPORT_TTL`, default thirty
|
||||
seconds), preventing a closed player from silencing the mic forever.
|
||||
- **Peer failure is non-fatal.** Hope Music being unavailable must not stop the
|
||||
voice agent or fail the voice container healthcheck. Music calls log and
|
||||
degrade gracefully.
|
||||
- **Never hard-require optional container env in supervisord.** `%(ENV_X)s`
|
||||
expansion aborts the whole config when a variable is missing. Settings with
|
||||
safe in-code defaults (including Hope Music) should inherit the container
|
||||
environment instead of being listed there.
|
||||
- **Healthchecks must use `curl -f`.** Without it, any HTTP response — including
|
||||
a 503 from a squatter — counts as healthy. Voice health covers only LiveKit
|
||||
and its HTTPS UI; Hope Music has its own independent healthcheck.
|
||||
|
||||
## Git
|
||||
|
||||
|
||||
+4
-8
@@ -24,10 +24,10 @@ ENV PYTHONUNBUFFERED=1
|
||||
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
|
||||
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
||||
|
||||
# Install LiveKit server binary + supervisord + nginx + ffmpeg (yt-dlp audio
|
||||
# extraction/transcoding for the media player)
|
||||
# Install LiveKit server runtime dependencies, supervisord, and nginx.
|
||||
# Music download/transcode dependencies live in hope-media-music now.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
curl ca-certificates supervisor nginx libasound2 iproute2 ffmpeg \
|
||||
curl ca-certificates supervisor nginx libasound2 iproute2 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Download LiveKit server (latest stable)
|
||||
@@ -52,7 +52,7 @@ COPY agent/task_worker.py /opt/voice-agent/task_worker.py
|
||||
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
|
||||
COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py
|
||||
|
||||
# Copy web frontend + token endpoint + media server
|
||||
# Copy the voice frontend + token endpoint. Hope Music is a separate service.
|
||||
COPY web/index.html /var/www/voice/
|
||||
COPY web/app.js /var/www/voice/
|
||||
COPY web/style.css /var/www/voice/
|
||||
@@ -60,10 +60,6 @@ COPY web/livekit-client.umd.js /var/www/voice/
|
||||
COPY web/manifest.json /var/www/voice/
|
||||
COPY web/favicon.svg /var/www/voice/
|
||||
COPY web/token_server.py /opt/voice/token_server.py
|
||||
COPY web/media_server.py /opt/voice/media_server.py
|
||||
|
||||
# Permanent media library mount point (bind-mounted from ./media)
|
||||
RUN mkdir -p /data/media/files /data/media/thumbs /data/media/jobs
|
||||
|
||||
# Config files
|
||||
COPY livekit.yaml /etc/livekit.yaml
|
||||
|
||||
+144
-31
@@ -11,6 +11,7 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import textwrap
|
||||
import time
|
||||
import uuid
|
||||
|
||||
from dotenv import load_dotenv
|
||||
@@ -58,6 +59,18 @@ WEB_MCP_ENABLED = os.environ.get("WEB_MCP_ENABLED", "true").lower() in ("1", "tr
|
||||
FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
|
||||
SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
|
||||
VOICE_USERS_FILE = os.environ.get("VOICE_USERS_FILE", "/etc/hope/users.json")
|
||||
# Hope Music is a separate LAN service. The agent is its control surface; the
|
||||
# standalone Hope Music page is the browser audio player.
|
||||
MUSIC_BASE_URL = os.environ.get(
|
||||
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||
).rstrip("/")
|
||||
# This is deliberately stable rather than the short-lived LiveKit room name.
|
||||
# The Hope Music page defaults to the same scope, so playback survives across
|
||||
# voice sessions and both products address one virtual player.
|
||||
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||
# The player reports every few seconds while audio is active. A stale playing
|
||||
# report must never leave Hope's microphone disabled after the page closes.
|
||||
MUSIC_REPORT_TTL = float(os.environ.get("MUSIC_REPORT_TTL", "30"))
|
||||
|
||||
|
||||
def load_user_profile(user_id: str) -> dict:
|
||||
@@ -208,15 +221,15 @@ SYSTEM_PROMPT = textwrap.dedent("""\
|
||||
stalled); task_status can also be called to check up on a task. Use
|
||||
list_tasks to see every task you have dispatched.
|
||||
|
||||
# Music (media player in the web UI)
|
||||
You can play music on the user's web player. For any music request, call
|
||||
search_music first, then:
|
||||
# Music (standalone Hope Music player)
|
||||
You can control the user's separate Hope Music web player. For any music
|
||||
request, call search_music first, then:
|
||||
- One clearly-right result → play_song with its video_id. Announce it in
|
||||
one short sentence, then stop talking — playback pauses the
|
||||
conversation until the user comes back to you.
|
||||
- Ambiguous results → offer_song_choices with the top 2-3 options, name
|
||||
them briefly, and let the user pick by voice or in the player panel.
|
||||
Never start playback on a guess.
|
||||
them briefly, and let the user pick by voice or in Hope Music. Never
|
||||
start playback on a guess.
|
||||
Prefer what is already saved: search_music puts downloaded library
|
||||
matches first (in_library=true) — always pick one of those over a fresh
|
||||
YouTube result, and list_library shows the whole saved collection which
|
||||
@@ -693,9 +706,9 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
||||
)
|
||||
logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
|
||||
|
||||
# Media tools talk to the local media server; the room name scopes player
|
||||
# state so concurrent sessions don't fight over one virtual player.
|
||||
media_server_url = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093")
|
||||
# Media tools call the separate hope-media-music service over HTTP. A
|
||||
# stable scope lets the standalone player and every voice session address
|
||||
# the same virtual player.
|
||||
media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py")
|
||||
toolsets.append(
|
||||
mcp.MCPToolset(
|
||||
@@ -705,14 +718,14 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
||||
args=[media_mcp_script],
|
||||
env={
|
||||
**os.environ,
|
||||
"MEDIA_SERVER_URL": media_server_url,
|
||||
"MEDIA_ROOM": os.path.basename(task_dir.rstrip("/")) or "default",
|
||||
"MUSIC_BASE_URL": MUSIC_BASE_URL,
|
||||
"MUSIC_ROOM": MUSIC_ROOM,
|
||||
},
|
||||
client_session_timeout_seconds=60,
|
||||
),
|
||||
)
|
||||
)
|
||||
logger.info("Media MCP toolset enabled (%s)", media_server_url)
|
||||
logger.info("Media MCP enabled (%s, room %s)", MUSIC_BASE_URL, MUSIC_ROOM)
|
||||
|
||||
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
|
||||
if extra:
|
||||
@@ -828,13 +841,12 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("failed to publish model list: %s", e)
|
||||
|
||||
# Listen for conversation controls (voice selection and voice/text mode).
|
||||
# Defined here so it captures this job's session via closure.
|
||||
# Music and conversation are mutually exclusive by design: while a track
|
||||
# plays the agent's ears are off; re-engaging the conversation pauses the
|
||||
# music, and ending the conversation lets the user resume it.
|
||||
# Listen for conversation controls (voice selection, voice/text mode, and
|
||||
# model switching). Music state comes from the separate service, not from
|
||||
# the voice UI's data channel.
|
||||
session_mode = {"value": "voice"} # UI voice/text switch
|
||||
media_playback = {"value": "stopped"} # UI-reported player state
|
||||
media_playback = {"value": "stopped"} # fresh service-reported state
|
||||
music_paused_by_agent = {"value": False}
|
||||
|
||||
def _apply_listening() -> None:
|
||||
listening = (
|
||||
@@ -843,9 +855,108 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
)
|
||||
session.input.set_audio_enabled(listening)
|
||||
|
||||
async def _music_request(
|
||||
method: str, path: str, *, warn: bool = True, **kwargs
|
||||
) -> dict | None:
|
||||
import httpx
|
||||
|
||||
url = f"{MUSIC_BASE_URL}/media/api{path}"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10) as http:
|
||||
response = await http.request(method, url, **kwargs)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# Hope Music is a peer service: it being unavailable must never
|
||||
# take the conversation down with it.
|
||||
if warn:
|
||||
logger.warning("Music service %s %s failed: %s", method, path, exc)
|
||||
return None
|
||||
|
||||
async def _music_snapshot(*, warn: bool = True) -> dict | None:
|
||||
return await _music_request(
|
||||
"GET", "/state", warn=warn, params={"room": MUSIC_ROOM}
|
||||
)
|
||||
|
||||
def _music_is_playing(snapshot: dict | None) -> bool:
|
||||
"""Return true only for fresh, browser-reported playback."""
|
||||
reported = (snapshot or {}).get("reported") or {}
|
||||
if reported.get("state") != "playing":
|
||||
return False
|
||||
try:
|
||||
age = time.time() - float(reported.get("ts") or 0)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
return 0 <= age < MUSIC_REPORT_TTL
|
||||
|
||||
async def _pause_music_for_conversation() -> None:
|
||||
"""Pause actual playback when the user engages Hope."""
|
||||
if music_paused_by_agent["value"]:
|
||||
return
|
||||
if not _music_is_playing(await _music_snapshot()):
|
||||
return
|
||||
result = await _music_request(
|
||||
"POST", "/control",
|
||||
json={"room": MUSIC_ROOM, "action": "pause"},
|
||||
)
|
||||
if result is not None:
|
||||
music_paused_by_agent["value"] = True
|
||||
logger.info("Paused Hope Music for the conversation (%s)", MUSIC_ROOM)
|
||||
|
||||
async def _resume_music_after_conversation() -> None:
|
||||
if not music_paused_by_agent["value"]:
|
||||
return
|
||||
result = await _music_request(
|
||||
"POST", "/control",
|
||||
json={"room": MUSIC_ROOM, "action": "resume"},
|
||||
)
|
||||
if result is not None:
|
||||
music_paused_by_agent["value"] = False
|
||||
logger.info("Resumed Hope Music after the conversation (%s)", MUSIC_ROOM)
|
||||
|
||||
async def _poll_music_state() -> None:
|
||||
while True:
|
||||
try:
|
||||
await asyncio.sleep(2)
|
||||
# Jobs deliberately survive page reloads. Do not keep making
|
||||
# network requests after the room has no human participant.
|
||||
if not ctx.room.remote_participants:
|
||||
continue
|
||||
snapshot = await _music_snapshot(warn=False)
|
||||
reported = (snapshot or {}).get("reported") or {}
|
||||
state = reported.get("state") or "stopped"
|
||||
if state == "playing" and not _music_is_playing(snapshot):
|
||||
state = "stopped"
|
||||
if state not in ("playing", "paused", "stopped"):
|
||||
state = "stopped"
|
||||
if state != media_playback["value"]:
|
||||
logger.info("Hope Music playback -> %s", state)
|
||||
media_playback["value"] = state
|
||||
_apply_listening()
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception("Hope Music state poller failed")
|
||||
|
||||
def _on_participant_joined(*_args) -> None:
|
||||
"""A returning participant starts a new conversation handshake."""
|
||||
asyncio.create_task(_pause_music_for_conversation())
|
||||
|
||||
def _on_participant_left(*_args) -> None:
|
||||
"""When the last human leaves, hand playback back to Hope Music."""
|
||||
|
||||
async def _deferred() -> None:
|
||||
# The event can fire before the participant is removed from the
|
||||
# room object; re-check the roster after it settles.
|
||||
await asyncio.sleep(0.5)
|
||||
if not ctx.room.remote_participants:
|
||||
await _resume_music_after_conversation()
|
||||
|
||||
asyncio.create_task(_deferred())
|
||||
|
||||
def _on_room_data(packet) -> None:
|
||||
topic = getattr(packet, "topic", None)
|
||||
if topic and topic not in ("voice-control", "conversation-control", "media-control"):
|
||||
if topic and topic not in ("voice-control", "conversation-control"):
|
||||
return
|
||||
try:
|
||||
msg = json.loads(packet.data.decode("utf-8"))
|
||||
@@ -865,17 +976,6 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
session.output.set_audio_enabled(voice_enabled)
|
||||
_apply_listening()
|
||||
return
|
||||
|
||||
if msg.get("type") == "media_state":
|
||||
state = msg.get("state")
|
||||
if state not in ("playing", "paused", "stopped"):
|
||||
return
|
||||
logger.info("Media playback -> %s (agent listening: %s)",
|
||||
state, session_mode["value"] == "voice" and state != "playing")
|
||||
media_playback["value"] = state
|
||||
_apply_listening()
|
||||
return
|
||||
|
||||
if msg.get("type") == "set_voice":
|
||||
voice = msg.get("voice", "")
|
||||
if not voice:
|
||||
@@ -913,6 +1013,8 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
logger.warning("Failed to update model: %s", e)
|
||||
|
||||
ctx.room.on("data_received", _on_room_data)
|
||||
ctx.room.on("participant_connected", _on_participant_joined)
|
||||
ctx.room.on("participant_disconnected", _on_participant_left)
|
||||
|
||||
# Publish user/agent transcripts to the room so the web UI can render them.
|
||||
async def publish_transcript(role: str, text: str) -> None:
|
||||
@@ -933,9 +1035,9 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
role = getattr(msg, "role", None)
|
||||
text = getattr(msg, "text_content", None)
|
||||
if role == "user":
|
||||
asyncio.create_task(publish_transcript("user", text))
|
||||
asyncio.create_task(publish_transcript("user", text or ""))
|
||||
elif role == "assistant":
|
||||
asyncio.create_task(publish_transcript("agent", text))
|
||||
asyncio.create_task(publish_transcript("agent", text or ""))
|
||||
|
||||
@session.on("user_input_transcribed")
|
||||
def _on_user_transcribed(ev) -> None:
|
||||
@@ -1017,6 +1119,17 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
)
|
||||
|
||||
await ctx.connect()
|
||||
await _pause_music_for_conversation()
|
||||
music_poll_task = asyncio.create_task(
|
||||
_poll_music_state(), name=f"music-poller-{ctx.room.name}"
|
||||
)
|
||||
|
||||
async def _stop_music_poller(*_args) -> None:
|
||||
music_poll_task.cancel()
|
||||
await asyncio.gather(music_poll_task, return_exceptions=True)
|
||||
await _resume_music_after_conversation()
|
||||
|
||||
ctx.add_shutdown_callback(_stop_music_poller)
|
||||
# The participant can appear in the browser slightly before AgentSession is
|
||||
# ready to accept lk.chat streams. Explicitly signal readiness so the first
|
||||
# typed message is never raced against session startup.
|
||||
|
||||
+24
-23
@@ -1,18 +1,17 @@
|
||||
"""Media MCP server — lets Hope find and control music by voice.
|
||||
"""Media MCP client — lets Hope control the standalone Hope Music service.
|
||||
|
||||
Tools talk to the local media server (web/media_server.py) over HTTP, which
|
||||
owns the library, YouTube Music search/download jobs and per-room player
|
||||
state. The browser is the actual player: it reconciles against the media
|
||||
server and reports playback back, so voice controls and the web panel stay
|
||||
in sync.
|
||||
Tools call the separate ``hope-media-music`` LAN service over HTTP. That
|
||||
service owns the library, YouTube Music search/download jobs, stream proxy,
|
||||
and per-room player state. Its standalone browser page is the actual audio
|
||||
player: it reconciles against the service and reports real playback.
|
||||
|
||||
Per-session environment (set by the agent when spawning this MCP):
|
||||
- MEDIA_SERVER_URL e.g. http://127.0.0.1:8093
|
||||
- MEDIA_ROOM LiveKit room name this session controls
|
||||
Environment (set by the voice agent when spawning this MCP):
|
||||
- MUSIC_BASE_URL e.g. http://192.168.86.2:8094
|
||||
- MUSIC_ROOM stable player scope shared with the Hope Music page
|
||||
|
||||
The "stop listening while music plays" rule lives in the agent itself: the
|
||||
UI reports media_state over the data channel and the agent disables its mic
|
||||
while a track is playing.
|
||||
The voice agent polls the same service state to keep music and conversation
|
||||
mutually exclusive; there is no music server or music player in the voice
|
||||
container anymore.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -22,20 +21,22 @@ import os
|
||||
import httpx
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
|
||||
MEDIA_SERVER_URL = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8093").rstrip("/")
|
||||
MEDIA_ROOM = os.environ.get("MEDIA_ROOM", "default")
|
||||
MUSIC_BASE_URL = os.environ.get(
|
||||
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||
).rstrip("/")
|
||||
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||
|
||||
mcp = FastMCP("media")
|
||||
|
||||
|
||||
def _media_get(path: str, params: dict | None = None) -> dict:
|
||||
resp = httpx.get(f"{MEDIA_SERVER_URL}{path}", params=params, timeout=10)
|
||||
resp = httpx.get(f"{MUSIC_BASE_URL}{path}", params=params, timeout=10)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
def _media_post(path: str, body: dict) -> dict:
|
||||
resp = httpx.post(f"{MEDIA_SERVER_URL}{path}", json=body, timeout=30)
|
||||
resp = httpx.post(f"{MUSIC_BASE_URL}{path}", json=body, timeout=30)
|
||||
if resp.status_code >= 400:
|
||||
try:
|
||||
detail = resp.json().get("error", resp.text)
|
||||
@@ -46,7 +47,7 @@ def _media_post(path: str, body: dict) -> dict:
|
||||
|
||||
|
||||
def _control(action: str, **extra) -> str:
|
||||
data = _media_post("/media/api/control", {"room": MEDIA_ROOM, "action": action, **extra})
|
||||
data = _media_post("/media/api/control", {"room": MUSIC_ROOM, "action": action, **extra})
|
||||
if "error" in data:
|
||||
return f"Could not {action} the music: {data['error']}"
|
||||
track = data.get("track") or {}
|
||||
@@ -87,14 +88,14 @@ def search_music(query: str, limit: int = 5) -> str:
|
||||
|
||||
@mcp.tool()
|
||||
def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
||||
"""Play a song in the user's web player. Pass the video_id from
|
||||
"""Play a song in the user's standalone Hope Music player. Pass the video_id from
|
||||
search_music — for an in_library result this plays the saved copy with no
|
||||
download; a new song downloads and starts playing automatically while it
|
||||
arrives. After calling this, tell the user what is playing in one short
|
||||
sentence and stop talking — the player takes over and the conversation
|
||||
pauses until they return."""
|
||||
data = _media_post("/media/api/play", {
|
||||
"room": MEDIA_ROOM, "video_id": video_id, "title": title, "channel": channel,
|
||||
"room": MUSIC_ROOM, "video_id": video_id, "title": title, "channel": channel,
|
||||
})
|
||||
if "error" in data:
|
||||
return f"Could not start playback: {data['error']}"
|
||||
@@ -114,12 +115,12 @@ def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
||||
def offer_song_choices(choices: list[dict]) -> str:
|
||||
"""Offer the user 2-4 song options when a search is ambiguous. Pass a list
|
||||
like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}].
|
||||
The choices appear as clickable cards in the music panel; the user can
|
||||
The choices appear as clickable cards in Hope Music; the user can
|
||||
also just say which one they want. Briefly name the options aloud, then
|
||||
wait for their pick."""
|
||||
if not choices:
|
||||
return "No choices provided."
|
||||
data = _media_post("/media/api/choices", {"room": MEDIA_ROOM, "choices": choices})
|
||||
data = _media_post("/media/api/choices", {"room": MUSIC_ROOM, "choices": choices})
|
||||
if "error" in data:
|
||||
return f"Could not present choices: {data['error']}"
|
||||
names = [
|
||||
@@ -162,7 +163,7 @@ def seek_music(delta_seconds: int) -> str:
|
||||
jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips
|
||||
20 seconds forward)."""
|
||||
try:
|
||||
reported = (_media_get("/media/api/state", {"room": MEDIA_ROOM}).get("reported") or {})
|
||||
reported = (_media_get("/media/api/state", {"room": MUSIC_ROOM}).get("reported") or {})
|
||||
position = float(reported.get("position") or 0.0)
|
||||
except Exception: # noqa: BLE001
|
||||
position = 0.0
|
||||
@@ -176,7 +177,7 @@ def now_playing() -> str:
|
||||
its playback position. Use when the user asks what's playing or what this
|
||||
song is."""
|
||||
try:
|
||||
st = _media_get("/media/api/state", {"room": MEDIA_ROOM})
|
||||
st = _media_get("/media/api/state", {"room": MUSIC_ROOM})
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return f"Media server unreachable: {exc}"
|
||||
track = st.get("track") or {}
|
||||
|
||||
@@ -16,7 +16,6 @@ dependencies = [
|
||||
"livekit-plugins-openai~=1.7",
|
||||
"python-dotenv",
|
||||
"uvicorn>=0.52.4",
|
||||
"yt-dlp>=2026.8.19",
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
|
||||
Generated
+1
-12
@@ -664,7 +664,7 @@ name = "exceptiongroup"
|
||||
version = "1.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "typing-extensions" },
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
|
||||
wheels = [
|
||||
@@ -2767,7 +2767,6 @@ dependencies = [
|
||||
{ name = "livekit-plugins-openai" },
|
||||
{ name = "python-dotenv" },
|
||||
{ name = "uvicorn" },
|
||||
{ name = "yt-dlp" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
@@ -2779,7 +2778,6 @@ requires-dist = [
|
||||
{ name = "livekit-plugins-openai", specifier = "~=1.7" },
|
||||
{ name = "python-dotenv" },
|
||||
{ name = "uvicorn", specifier = ">=0.52.4" },
|
||||
{ name = "yt-dlp", specifier = ">=2026.8.19" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3062,12 +3060,3 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "yt-dlp"
|
||||
version = "2026.8.19"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1e/e0/832fa4ca334b766a06933a196066edc3dba37cdb6f14cd98d59bcc69a4b4/yt_dlp-2026.8.19.tar.gz", hash = "sha256:9e213e48cea35c66b378e4447903f118f6392a5fa380a2b6d7070ec86f4e0af1", size = 3052025, upload-time = "2026-08-19T23:48:59.291Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/69/b2/8cd1613f56eed7ceb64fbd4df3f1c01246bfb098e6f398228bafda22b80b/yt_dlp-2026.8.19-py3-none-any.whl", hash = "sha256:1d57897e94c6665a0a6f9bc54b34e584284e32c034ffab3a7df25d8f7b24eedf", size = 3185533, upload-time = "2026-08-19T23:48:56.925Z" },
|
||||
]
|
||||
@@ -10,4 +10,3 @@ Requires-Dist: livekit-plugins-azure~=1.7
|
||||
Requires-Dist: livekit-plugins-openai~=1.7
|
||||
Requires-Dist: python-dotenv
|
||||
Requires-Dist: uvicorn>=0.52.4
|
||||
Requires-Dist: yt-dlp>=2026.8.19
|
||||
@@ -5,4 +5,3 @@ livekit-plugins-azure~=1.7
|
||||
livekit-plugins-openai~=1.7
|
||||
python-dotenv
|
||||
uvicorn>=0.52.4
|
||||
yt-dlp>=2026.8.19
|
||||
+3
-5
@@ -18,16 +18,14 @@ services:
|
||||
WEB_PORT: "${WEB_PORT:-9178}"
|
||||
VOICE_USERS_FILE: "/etc/hope/users.json"
|
||||
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
|
||||
MEDIA_DIR: "/data/media"
|
||||
MEDIA_SERVER_PORT: "${MEDIA_SERVER_PORT:-8093}"
|
||||
YTDLP_COOKIES: "${YTDLP_COOKIES:-/data/media/cookies.txt}"
|
||||
MUSIC_BASE_URL: "${MUSIC_BASE_URL:-http://192.168.86.2:8094}"
|
||||
MUSIC_ROOM: "${MUSIC_ROOM:-voice-room}"
|
||||
volumes:
|
||||
- ./livekit.yaml:/etc/livekit.yaml:ro
|
||||
- ./certs:/etc/voice/certs
|
||||
- ./skills:/skills
|
||||
- ./media:/data/media
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null && curl -sf http://localhost:$${MEDIA_SERVER_PORT:-8093}/media/api/healthz -o /dev/null"]
|
||||
test: ["CMD-SHELL", "curl -sfk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -sf http://localhost:7880/ -o /dev/null"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
|
||||
Whitespace-only changes.
-21
@@ -22,27 +22,6 @@ server {
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
}
|
||||
|
||||
# Media server API (library, search, player state) and the proxy for
|
||||
# external live-stream URLs (http:// sources would be blocked as mixed
|
||||
# content in the browser, so they are re-served over HTTPS from here).
|
||||
location /media/api/ {
|
||||
proxy_pass http://127.0.0.1:8093/media/api/;
|
||||
proxy_buffering off;
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
}
|
||||
|
||||
# Downloaded tracks are plain static files — nginx serves them with
|
||||
# native HTTP Range support, which gives the web player seeking for free.
|
||||
location /media/files/ {
|
||||
alias /data/media/files/;
|
||||
}
|
||||
|
||||
location /media/thumbs/ {
|
||||
alias /data/media/thumbs/;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
|
||||
@@ -56,18 +56,3 @@ environment=
|
||||
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
|
||||
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
|
||||
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
|
||||
|
||||
[program:media-server]
|
||||
; Runs as root because the /data/media bind mount is a host (Unraid) share
|
||||
; whose uid does not match the container's voiceuser.
|
||||
; MEDIA_DIR / MEDIA_SERVER_PORT / YTDLP_COOKIES are inherited from the
|
||||
; container environment (media_server.py has safe defaults for all three) —
|
||||
; do NOT list them via %(ENV_x)s here: a missing var aborts supervisord's
|
||||
; config expansion and takes every program down with it.
|
||||
command=/opt/voice-agent/.venv/bin/python /opt/voice/media_server.py
|
||||
user=root
|
||||
autostart=true
|
||||
autorestart=true
|
||||
stdout_logfile=/dev/stdout
|
||||
stdout_logfile_maxbytes=0
|
||||
redirect_stderr=true
|
||||
+2
-2
@@ -15,7 +15,7 @@ tool servers, and agent process.
|
||||
## Run
|
||||
|
||||
```bash
|
||||
cd /home/shane/dev/voice
|
||||
cd ~/dev/hope-voice-api
|
||||
python -m pytest tests/ -v # everything
|
||||
python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
||||
```
|
||||
@@ -27,7 +27,7 @@ python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
||||
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
||||
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
||||
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
||||
| `test_media.py` | media server API + player UI + media MCP | library/player state flow, Range proxy, static mp3 Range, live YT Music search (`-m slow`), gated full download (`HOPE_RUN_YTDLP_DOWNLOAD=1`) |
|
||||
| `test_media.py` | external Hope Music boundary + media MCP client | LAN service health, stable room scope, split regression guards, local-container MCP probes |
|
||||
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
||||
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
||||
|
||||
|
||||
+1
-2
@@ -27,8 +27,7 @@ WEB_BASE_URL = os.environ.get("VOICE_WEB_URL", "https://192.168.86.2:9178").rstr
|
||||
TOKEN_URL = os.environ.get(
|
||||
"VOICE_TOKEN_URL", f"{WEB_BASE_URL}/token"
|
||||
)
|
||||
# Filesystem-coupled tests (repo ./media bind mount) only make sense when
|
||||
# the test host IS the deployment host.
|
||||
# docker-exec integration probes only make sense when the test target is local.
|
||||
IS_LOCAL_DEPLOYMENT = urlparse(WEB_BASE_URL).hostname in ("localhost", "127.0.0.1")
|
||||
LLM_BASE_URL = (
|
||||
os.environ.get("LLM_BASE_URL")
|
||||
|
||||
+96
-527
@@ -1,484 +1,112 @@
|
||||
"""Tests for the media player subsystem.
|
||||
"""Voice-side integration tests for the standalone Hope Music service.
|
||||
|
||||
Covers the media server API (via nginx HTTPS and direct), the stream proxy
|
||||
with HTTP Range passthrough, static file serving of the ./media volume, the
|
||||
web player markup, and Hope's media MCP toolset. Download tests are gated
|
||||
behind HOPE_RUN_YTDLP_DOWNLOAD=1 (network-heavy, occasionally flaky).
|
||||
The library, downloads, stream proxy, web player, and their exhaustive tests
|
||||
live in ``hope-media-music`` now. This module guards the remaining product
|
||||
boundary: the voice repo contains only the MCP/HTTP client, uses one stable
|
||||
player scope, and can reach the LAN service.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from conftest import (
|
||||
TOKEN_URL, docker_exec, docker_exec_python, IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
|
||||
from conftest import IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
|
||||
|
||||
ROOT = Path(__file__).parents[1]
|
||||
MUSIC_BASE_URL = os.environ.get(
|
||||
"MUSIC_BASE_URL", "http://192.168.86.2:8094"
|
||||
).rstrip("/")
|
||||
MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")
|
||||
MEDIA_API = "/media/api"
|
||||
|
||||
|
||||
def test_external_music_service_health_and_library():
|
||||
"""The peer service used by the deployed agent is reachable over the LAN."""
|
||||
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as client:
|
||||
health = client.get(f"{MEDIA_API}/healthz")
|
||||
assert health.status_code == 200
|
||||
assert health.json()["ok"] is True
|
||||
|
||||
library = client.get(f"{MEDIA_API}/library")
|
||||
assert library.status_code == 200
|
||||
assert isinstance(library.json()["tracks"], list)
|
||||
|
||||
|
||||
def test_voice_repo_contains_only_the_music_client():
|
||||
"""Regression guard for the product split: backend/player cannot creep back."""
|
||||
agent = (ROOT / "agent" / "agent.py").read_text()
|
||||
client = (ROOT / "agent" / "media_mcp.py").read_text()
|
||||
web_js = (ROOT / "web" / "app.js").read_text()
|
||||
web_html = (ROOT / "web" / "index.html").read_text()
|
||||
|
||||
assert not (ROOT / "web" / "media_server.py").exists()
|
||||
assert not (ROOT / "media").exists()
|
||||
assert "MUSIC_BASE_URL" in agent and "MUSIC_ROOM" in agent
|
||||
assert "MUSIC_BASE_URL" in client and "MUSIC_ROOM" in client
|
||||
assert "MEDIA_SERVER_URL" not in agent + client
|
||||
assert "media-control" not in agent + web_js
|
||||
assert "media_state" not in agent + web_js
|
||||
assert "/media/api" not in web_js
|
||||
assert 'id="mediaPanel"' not in web_html
|
||||
assert 'id="mediaAudio"' not in web_html
|
||||
|
||||
|
||||
def test_voice_uses_a_stable_music_scope():
|
||||
"""The ephemeral LiveKit room must not be the standalone player's scope."""
|
||||
assert MUSIC_ROOM == "voice-room" or os.environ.get("MUSIC_ROOM")
|
||||
source = (ROOT / "agent" / "agent.py").read_text()
|
||||
assert 'MUSIC_ROOM = os.environ.get("MUSIC_ROOM", "voice-room")' in source
|
||||
assert '"MUSIC_ROOM": MUSIC_ROOM' in source
|
||||
|
||||
|
||||
# These exercise the actual MCP subprocess inside a local voice container. The
|
||||
# default suite targets Unraid remotely, where docker-exec intentionally skips.
|
||||
needs_local_container = pytest.mark.skipif(
|
||||
not IS_LOCAL_DEPLOYMENT,
|
||||
reason="docker-exec MCP probes require a local voice container",
|
||||
)
|
||||
|
||||
MEDIA_API = "/media/api"
|
||||
TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}"
|
||||
MEDIA_MCP_ENV = {
|
||||
"MUSIC_BASE_URL": MUSIC_BASE_URL,
|
||||
"MUSIC_ROOM": "mcp-test-room",
|
||||
}
|
||||
|
||||
|
||||
# ── local upstream for stream-proxy tests ───────────────────────────────────
|
||||
|
||||
def _serve_fake_audio(payload: bytes):
|
||||
"""Tiny Range-capable HTTP server on an ephemeral port.
|
||||
|
||||
Binds 0.0.0.0 and advertises this host's LAN IP so the deployment host's
|
||||
media server (which may be a different machine — Unraid) can reach it
|
||||
over the LAN when fetching external stream URLs.
|
||||
"""
|
||||
import socket
|
||||
"""Tiny Range-capable upstream reachable by a host-networked container."""
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
rng = self.headers.get("Range")
|
||||
if rng and rng.startswith("bytes="):
|
||||
start_s, _, end_s = rng[6:].partition("-")
|
||||
start = int(start_s or 0)
|
||||
end = int(end_s) if end_s else len(payload) - 1
|
||||
end = min(end, len(payload) - 1)
|
||||
body = payload[start : end + 1]
|
||||
self.send_response(206)
|
||||
self.send_header(
|
||||
"Content-Range", f"bytes {start}-{end}/{len(payload)}"
|
||||
)
|
||||
else:
|
||||
body = payload
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "audio/mpeg")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.send_header("Accept-Ranges", "bytes")
|
||||
self.send_header("Content-Length", str(len(payload)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
self.wfile.write(payload)
|
||||
|
||||
def log_message(self, *args):
|
||||
def log_message(self, format, *args): # noqa: A002
|
||||
pass
|
||||
|
||||
server = HTTPServer(("0.0.0.0", 0), Handler)
|
||||
port = server.server_address[1]
|
||||
# The local IP toward the deployment host (works across the LAN).
|
||||
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
try:
|
||||
probe.connect((urlparse(WEB_BASE_URL).hostname or "192.168.86.2", 9178))
|
||||
lan_ip = probe.getsockname()[0]
|
||||
target = urlparse(WEB_BASE_URL).hostname or "192.168.86.2"
|
||||
probe.connect((target, urlparse(WEB_BASE_URL).port or 9178))
|
||||
host = probe.getsockname()[0]
|
||||
except OSError:
|
||||
lan_ip = "127.0.0.1"
|
||||
host = "127.0.0.1"
|
||||
finally:
|
||||
probe.close()
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
return server, f"http://{lan_ip}:{port}/audio.mp3"
|
||||
return server, f"http://{host}:{port}/audio.mp3"
|
||||
|
||||
|
||||
# ── health + basic API shape ────────────────────────────────────────────────
|
||||
|
||||
def test_media_server_health_via_nginx(client):
|
||||
resp = client.get(f"{MEDIA_API}/healthz")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["ok"] is True
|
||||
|
||||
|
||||
def test_media_library_endpoint(client):
|
||||
resp = client.get(f"{MEDIA_API}/library")
|
||||
assert resp.status_code == 200
|
||||
data = resp.json()
|
||||
assert isinstance(data["tracks"], list)
|
||||
assert data["version"] >= 1
|
||||
for track in data["tracks"]:
|
||||
assert {"id", "kind", "title", "status"} <= set(track)
|
||||
|
||||
|
||||
def test_media_search_requires_query(client):
|
||||
resp = client.post(f"{MEDIA_API}/search", json={"query": ""})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_media_play_requires_target(client):
|
||||
resp = client.post(f"{MEDIA_API}/play", json={"room": TEST_ROOM})
|
||||
assert resp.status_code == 400
|
||||
|
||||
|
||||
def test_media_unknown_stream_404(client):
|
||||
resp = client.get(f"{MEDIA_API}/stream/does-not-exist")
|
||||
assert resp.status_code == 404
|
||||
|
||||
|
||||
# ── player state flow (uses a live stream track as the fixture) ─────────────
|
||||
|
||||
def test_media_player_state_flow(client):
|
||||
"""add stream -> play -> state -> report -> pause -> seek hint -> stop."""
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 64)
|
||||
try:
|
||||
added = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Test FM", "url": url}
|
||||
)
|
||||
assert added.status_code == 200
|
||||
track = added.json()["track"]
|
||||
track_id = track["id"]
|
||||
assert track["kind"] == "stream"
|
||||
assert track["status"] == "ready"
|
||||
assert track["audio_url"].startswith(f"{MEDIA_API}/stream/")
|
||||
|
||||
played = client.post(
|
||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||
).json()
|
||||
assert played["desired"] == "playing"
|
||||
assert played["track"]["id"] == track_id
|
||||
|
||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||
assert state["track"]["id"] == track_id
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/report",
|
||||
json={"room": TEST_ROOM, "track_id": track_id,
|
||||
"state": "playing", "position": 12.5},
|
||||
)
|
||||
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||
assert abs(state["reported"]["position"] - 12.5) < 0.01
|
||||
|
||||
client.post(f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "pause"})
|
||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["desired"] == "paused"
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/control",
|
||||
json={"room": TEST_ROOM, "action": "seek", "position": 42},
|
||||
)
|
||||
hint = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["seek_hint"]
|
||||
assert hint["position"] == 42
|
||||
|
||||
# start_paused variant: scope transfer without starting playback
|
||||
transfer = client.post(
|
||||
f"{MEDIA_API}/play",
|
||||
json={"room": TEST_ROOM + "-b", "track_id": track_id, "start_paused": True},
|
||||
).json()
|
||||
assert transfer["desired"] == "paused"
|
||||
|
||||
# playing clears any open suggestion choices
|
||||
client.post(
|
||||
f"{MEDIA_API}/choices",
|
||||
json={"room": TEST_ROOM, "choices": [{"video_id": "x", "title": "X"}]},
|
||||
)
|
||||
client.post(
|
||||
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||
)
|
||||
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["choices"] == []
|
||||
|
||||
stopped = client.post(
|
||||
f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "stop"}
|
||||
).json()
|
||||
assert stopped["desired"] == "stopped"
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
# Remove the test stream from the permanent library.
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except (httpx.HTTPError, UnboundLocalError):
|
||||
pass
|
||||
|
||||
|
||||
def test_media_stream_proxy_and_range(client):
|
||||
"""External audio URLs must be reachable same-origin (HTTPS) and support
|
||||
Range passthrough so the browser can seek progressive streams.
|
||||
|
||||
Local deployments test against a controlled fake upstream; remote
|
||||
deployments (Unraid) cannot reach the test host's ephemeral ports, so a
|
||||
public reference mp3 is used and Range semantics are asserted without
|
||||
byte equality."""
|
||||
if IS_LOCAL_DEPLOYMENT:
|
||||
payload = b"0123456789" * 100
|
||||
upstream, url = _serve_fake_audio(payload)
|
||||
track_id = None
|
||||
try:
|
||||
track_id = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Range FM", "url": url}
|
||||
).json()["track"]["id"]
|
||||
|
||||
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
||||
assert full.status_code == 200
|
||||
assert full.content == payload
|
||||
|
||||
ranged = client.get(
|
||||
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=100-199"}
|
||||
)
|
||||
assert ranged.status_code == 206
|
||||
assert ranged.content == payload[100:200]
|
||||
assert ranged.headers["content-range"] == f"bytes 100-199/{len(payload)}"
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
if track_id:
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except httpx.HTTPError:
|
||||
pass
|
||||
return
|
||||
|
||||
# Remote deployment: public reference audio (SoundHelix).
|
||||
url = "https://www.soundhelix.com/examples/mp3/SoundHelix-Song-1.mp3"
|
||||
track_id = None
|
||||
try:
|
||||
track_id = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Range FM public", "url": url}
|
||||
).json()["track"]["id"]
|
||||
|
||||
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
||||
assert full.status_code == 200
|
||||
assert full.headers.get("content-type", "").startswith("audio/")
|
||||
assert len(full.content) > 100_000
|
||||
|
||||
ranged = client.get(
|
||||
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=0-1023"}
|
||||
)
|
||||
assert ranged.status_code == 206
|
||||
assert len(ranged.content) == 1024
|
||||
assert (ranged.headers.get("content-range") or "").startswith("bytes 0-1023/")
|
||||
finally:
|
||||
if track_id:
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except httpx.HTTPError:
|
||||
pass
|
||||
|
||||
|
||||
def test_media_files_static_range_support(client):
|
||||
"""Downloaded tracks are nginx static files with native Range support.
|
||||
|
||||
Writes a probe file into the repo's ./media bind mount, so it only runs
|
||||
when the test host IS the deployment host (local dev deployment)."""
|
||||
if not IS_LOCAL_DEPLOYMENT:
|
||||
pytest.skip("writes to the repo ./media bind mount — local deployments only")
|
||||
import pathlib
|
||||
|
||||
marker = uuid.uuid4().hex[:8]
|
||||
host_media = pathlib.Path(__file__).parents[1] / "media" / "files"
|
||||
host_media.mkdir(parents=True, exist_ok=True)
|
||||
fname = f"test-{marker}.mp3"
|
||||
payload = b"IMHEADDATA" + b"\x01" * 2048
|
||||
(host_media / fname).write_bytes(payload)
|
||||
try:
|
||||
full = client.get(f"/media/files/{fname}")
|
||||
assert full.status_code == 200
|
||||
assert full.content == payload
|
||||
|
||||
ranged = client.get(f"/media/files/{fname}", headers={"Range": "bytes=0-9"})
|
||||
assert ranged.status_code == 206
|
||||
assert ranged.content == b"IMHEADDATA"
|
||||
finally:
|
||||
(host_media / fname).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def test_media_library_persists_in_volume(client):
|
||||
"""The library JSON is written to the ./media bind mount (permanent).
|
||||
|
||||
Reads the repo's ./media directory — local deployments only."""
|
||||
if not IS_LOCAL_DEPLOYMENT:
|
||||
pytest.skip("reads the repo ./media bind mount — local deployments only")
|
||||
import pathlib
|
||||
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||
track_id = None
|
||||
try:
|
||||
track_id = client.post(
|
||||
f"{MEDIA_API}/streams", json={"name": "Persist FM", "url": url}
|
||||
).json()["track"]["id"]
|
||||
library_file = pathlib.Path(__file__).parents[1] / "media" / "library.json"
|
||||
data = json.loads(library_file.read_text())
|
||||
assert track_id in data["tracks"]
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
if track_id:
|
||||
try:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
except httpx.HTTPError:
|
||||
pass
|
||||
|
||||
|
||||
# ── web player UI ───────────────────────────────────────────────────────────
|
||||
|
||||
def test_ui_media_panel_serves(client):
|
||||
resp = client.get("/")
|
||||
body = resp.text
|
||||
assert 'id="mediaBtn"' in body
|
||||
assert 'id="mediaPanel"' in body
|
||||
assert 'id="playPauseBtn"' in body
|
||||
assert 'id="seekBar"' in body
|
||||
assert 'id="mediaAudio"' in body
|
||||
|
||||
|
||||
def test_ui_media_logic_serves(client):
|
||||
resp = client.get("/app.js")
|
||||
body = resp.text
|
||||
assert "reconcileMedia" in body
|
||||
assert "media-control" in body # data-channel topic for the agent
|
||||
assert "media_state" in body # playback state reports
|
||||
assert "/media/api/state" in body # reconciliation polling
|
||||
|
||||
|
||||
def test_ui_media_styles_serve(client):
|
||||
resp = client.get("/style.css")
|
||||
assert ".media-panel" in resp.text
|
||||
assert ".media-row" in resp.text
|
||||
|
||||
|
||||
def test_token_response_includes_room():
|
||||
resp = httpx.post(TOKEN_URL, json={}, timeout=10, verify=False)
|
||||
assert resp.status_code == 200
|
||||
assert resp.json()["room"].startswith("voice-")
|
||||
|
||||
|
||||
# ── Hope's media MCP toolset ────────────────────────────────────────────────
|
||||
# These spawn the MCP server inside the LOCAL dev container (docker exec),
|
||||
# whose media server is the local one — they can only be consistent when
|
||||
# the test target IS the local deployment.
|
||||
|
||||
MEDIA_MCP_ENV = {
|
||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8093",
|
||||
"MEDIA_ROOM": "mcp-test-room",
|
||||
}
|
||||
|
||||
needs_local_container = pytest.mark.skipif(
|
||||
not IS_LOCAL_DEPLOYMENT,
|
||||
reason="docker-exec MCP probes run against the local dev container",
|
||||
)
|
||||
|
||||
|
||||
# ── media server internals (in-process, inside the container) ───────────────
|
||||
|
||||
@needs_local_container
|
||||
def test_media_search_prefers_library_matches(client):
|
||||
"""Search returns ready library matches ahead of YouTube results, so
|
||||
Hope prefers an already-downloaded copy. Runs the endpoint in-process
|
||||
with _run_search stubbed — fully deterministic, no network."""
|
||||
code = '''
|
||||
import sys
|
||||
sys.path.insert(0, "/opt/voice")
|
||||
import media_server as ms
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
ms._library["tracks"]["libpref01"] = {
|
||||
"id": "libpref01", "kind": "youtube", "video_id": "libpref01",
|
||||
"title": "Preference Probe Song", "channel": "Probe Artist",
|
||||
"duration": 120, "status": "ready", "ext": "webm",
|
||||
"error": None, "added_at": 1.0,
|
||||
}
|
||||
ms._run_search = lambda q, limit: [
|
||||
{"video_id": "remote01", "title": "Preference Probe Song (cover)",
|
||||
"channel": "", "duration": 99, "view_count": 1,
|
||||
"in_library": False, "track_id": None, "library_status": None},
|
||||
]
|
||||
|
||||
c = TestClient(ms.app)
|
||||
r = c.post("/media/api/search", json={"query": "preference probe", "limit": 5})
|
||||
assert r.status_code == 200, r.text
|
||||
results = r.json()["results"]
|
||||
assert results, "no results"
|
||||
assert results[0]["video_id"] == "libpref01", results
|
||||
assert results[0]["in_library"] is True
|
||||
assert any(x["video_id"] == "remote01" for x in results), results
|
||||
print("OK")
|
||||
'''
|
||||
proc = docker_exec_python(code, timeout=60, check=False)
|
||||
assert proc.returncode == 0, proc.stderr[-800:]
|
||||
assert "OK" in proc.stdout
|
||||
|
||||
|
||||
@needs_local_container
|
||||
def test_media_stream_endpoint_serves_ready_track(client):
|
||||
"""GET /media/api/stream/{id} for a finished youtube track serves the
|
||||
file bytes with native Range support (progressive-playback endpoint)."""
|
||||
code = '''
|
||||
import sys
|
||||
sys.path.insert(0, "/opt/voice")
|
||||
import media_server as ms
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
probe = ms.FILES_DIR / "probetrk.mp3"
|
||||
probe.write_bytes(b"PROBEDATA" * 100)
|
||||
ms._library["tracks"]["probetrk"] = {
|
||||
"id": "probetrk", "kind": "youtube", "video_id": "probetrk",
|
||||
"title": "Probe", "channel": "", "duration": 1,
|
||||
"status": "ready", "ext": "mp3", "error": None, "added_at": 1.0,
|
||||
}
|
||||
try:
|
||||
c = TestClient(ms.app)
|
||||
full = c.get("/media/api/stream/probetrk")
|
||||
assert full.status_code == 200, full.status_code
|
||||
assert full.content == b"PROBEDATA" * 100
|
||||
part = c.get("/media/api/stream/probetrk", headers={"Range": "bytes=0-8"})
|
||||
assert part.status_code == 206, part.status_code
|
||||
assert part.content == b"PROBEDATA"
|
||||
assert (part.headers.get("content-range") or "").startswith("bytes 0-8/")
|
||||
# track dict exposes a stable stream URL while downloading and ready
|
||||
d = ms._track_dict(ms._library["tracks"]["probetrk"])
|
||||
assert d["audio_url"] == "/media/api/stream/probetrk", d
|
||||
print("OK")
|
||||
finally:
|
||||
probe.unlink(missing_ok=True)
|
||||
ms._library["tracks"].pop("probetrk", None)
|
||||
'''
|
||||
proc = docker_exec_python(code, timeout=60, check=False)
|
||||
assert proc.returncode == 0, proc.stderr[-800:]
|
||||
assert "OK" in proc.stdout
|
||||
|
||||
|
||||
@needs_local_container
|
||||
def test_media_play_resolves_cached_ready_track(client):
|
||||
"""Playing an ALREADY-cached song must resolve to the on-disk ready track
|
||||
(no redownload) and hand the browser a playable stream URL, so a cached
|
||||
pick starts instantly rather than re-fetching from YouTube.
|
||||
|
||||
Regression: a cached track could be left as paused with currentTime 0 if
|
||||
the room the agent played into diverged from the room the browser polled,
|
||||
or if autoplay was blocked. This pins the server-side resolve + track
|
||||
dict so a ready file is always returned with a stable audio_url."""
|
||||
code = '''
|
||||
import sys
|
||||
sys.path.insert(0, "/opt/voice")
|
||||
import media_server as ms
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
probe = ms.FILES_DIR / "cachetrk.webm"
|
||||
probe.write_bytes(b"CACHEDBYTES" * 40)
|
||||
ms._library["tracks"]["cachetrk"] = {
|
||||
"id": "cachetrk", "kind": "youtube", "video_id": "cachetrk",
|
||||
"title": "Cached Song", "channel": "Artist", "duration": 200,
|
||||
"status": "ready", "ext": "webm", "error": None, "added_at": 1.0,
|
||||
}
|
||||
ms._start_download = lambda *a, **k: (_ for _ in ()).throw(
|
||||
AssertionError("must not re-download an already cached track")
|
||||
)
|
||||
try:
|
||||
c = TestClient(ms.app)
|
||||
# play by video_id (as Hope's play_song does) must reuse the ready track
|
||||
r = c.post("/media/api/play", json={
|
||||
"room": "cached-test-room", "video_id": "cachetrk",
|
||||
"title": "Cached Song", "channel": "Artist",
|
||||
}, timeout=10)
|
||||
assert r.status_code == 200, r.text
|
||||
body = r.json()
|
||||
assert body["desired"] == "playing", body
|
||||
track = body["track"]
|
||||
assert track["status"] == "ready", track
|
||||
assert track["audio_url"] == "/media/api/stream/cachetrk", track
|
||||
# the stream endpoint serves the cached bytes (browser plays it directly)
|
||||
data = c.get("/media/api/stream/cachetrk")
|
||||
assert data.status_code == 200
|
||||
assert data.content == b"CACHEDBYTES" * 40
|
||||
print("OK")
|
||||
finally:
|
||||
probe.unlink(missing_ok=True)
|
||||
ms._library["tracks"].pop("cachetrk", None)
|
||||
'''
|
||||
proc = docker_exec_python(code, timeout=60, check=False)
|
||||
assert proc.returncode == 0, proc.stderr[-800:]
|
||||
assert "OK" in proc.stdout
|
||||
|
||||
|
||||
@needs_local_container
|
||||
def test_mcp_media_list_library(mcp_client_factory):
|
||||
@@ -490,8 +118,9 @@ def test_mcp_media_list_library(mcp_client_factory):
|
||||
|
||||
|
||||
@needs_local_container
|
||||
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
||||
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory):
|
||||
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||
track_id = None
|
||||
try:
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "add_stream",
|
||||
@@ -500,99 +129,39 @@ def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
||||
assert not is_error, text
|
||||
assert "Added" in text
|
||||
|
||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||
library = music.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||
probe = next(t for t in library if t["title"] == "MCP Test Radio")
|
||||
try:
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||
)
|
||||
assert not is_error
|
||||
# Nothing loaded in mcp-test-room: the tool must say so plainly.
|
||||
assert "Nothing is loaded" in text
|
||||
track_id = probe["id"]
|
||||
music.post(
|
||||
f"{MEDIA_API}/play",
|
||||
json={"room": "mcp-test-room", "track_id": track_id},
|
||||
).raise_for_status()
|
||||
|
||||
client.post(
|
||||
f"{MEDIA_API}/play", json={"room": "mcp-test-room", "track_id": probe["id"]}
|
||||
)
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||
)
|
||||
assert not is_error
|
||||
assert "MCP Test Radio" in text
|
||||
finally:
|
||||
client.delete(f"{MEDIA_API}/tracks/{probe['id']}")
|
||||
finally:
|
||||
upstream.shutdown()
|
||||
if track_id:
|
||||
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||
music.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
|
||||
|
||||
@needs_local_container
|
||||
def test_mcp_media_resume_without_load(mcp_client_factory, client):
|
||||
"""Errors from the media server reach the LLM as tool text (not MCP
|
||||
errors) so Hope can explain them naturally."""
|
||||
client.post(
|
||||
f"{MEDIA_API}/control", json={"room": "mcp-empty-room", "action": "stop"}
|
||||
)
|
||||
def test_mcp_media_resume_without_load(mcp_client_factory):
|
||||
room = "mcp-empty-room"
|
||||
with httpx.Client(base_url=MUSIC_BASE_URL, timeout=15) as music:
|
||||
music.post(
|
||||
f"{MEDIA_API}/control",
|
||||
json={"room": room, "action": "stop"},
|
||||
).raise_for_status()
|
||||
|
||||
text, is_error = mcp_client_factory(
|
||||
"media_mcp.py", "resume_music", {}, env={
|
||||
"MEDIA_SERVER_URL": "http://127.0.0.1:8093",
|
||||
"MEDIA_ROOM": "mcp-empty-room",
|
||||
}
|
||||
"media_mcp.py", "resume_music", {},
|
||||
env={"MUSIC_BASE_URL": MUSIC_BASE_URL, "MUSIC_ROOM": room},
|
||||
)
|
||||
assert not is_error
|
||||
assert "nothing loaded" in text.lower()
|
||||
|
||||
|
||||
# ── live YouTube Music (network-dependent) ──────────────────────────────────
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_media_search_live(client):
|
||||
resp = client.post(
|
||||
f"{MEDIA_API}/search",
|
||||
json={"query": "blind guardian timeless miracle", "limit": 3},
|
||||
)
|
||||
assert resp.status_code == 200
|
||||
results = resp.json()["results"]
|
||||
assert len(results) >= 1
|
||||
for r in results:
|
||||
assert r["video_id"]
|
||||
assert r["title"]
|
||||
|
||||
|
||||
@pytest.mark.slow
|
||||
def test_media_download_and_stream_end_to_end(client):
|
||||
"""Full pipeline: search -> play -> yt-dlp download -> playable stream.
|
||||
The audio_url stays on /media/api/stream/ for the whole lifecycle; once
|
||||
ready it serves the finished file with Range support. Gated: run with
|
||||
HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
|
||||
if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
|
||||
pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")
|
||||
|
||||
search = client.post(
|
||||
f"{MEDIA_API}/search", json={"query": "kingdom hearts dearly beloved", "limit": 3}
|
||||
)
|
||||
results = search.json()["results"]
|
||||
assert results, "search returned nothing"
|
||||
video = results[0]
|
||||
|
||||
played = client.post(
|
||||
f"{MEDIA_API}/play",
|
||||
json={"room": TEST_ROOM, "video_id": video["video_id"],
|
||||
"title": video["title"], "channel": video["channel"]},
|
||||
).json()
|
||||
track_id = played["track"]["id"]
|
||||
|
||||
try:
|
||||
deadline = time.time() + 240
|
||||
track = played["track"]
|
||||
while time.time() < deadline:
|
||||
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||
track = next(t for t in library if t["id"] == track_id)
|
||||
if track["status"] in ("ready", "failed"):
|
||||
break
|
||||
time.sleep(3)
|
||||
assert track["status"] == "ready", f"download failed: {track.get('error')}"
|
||||
|
||||
stream = client.get(track["audio_url"], headers={"Range": "bytes=0-1023"})
|
||||
assert stream.status_code in (200, 206)
|
||||
assert len(stream.content) >= 1
|
||||
finally:
|
||||
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||
-568
@@ -269,18 +269,6 @@ const LED_GROW = 1.4; // how much the dots grow toward the center
|
||||
let ledDots = [];
|
||||
let ledWallTime = 0;
|
||||
|
||||
// Decides whether the LED wall square should show the track's album art.
|
||||
// Music and conversation are mutually exclusive: while a conversation is
|
||||
// active the music is paused, so the wall pulses (agent audio) and any loaded
|
||||
// track's cover art returns once the talk ends.
|
||||
function updateAvatarMedia() {
|
||||
const inConversation = !!(room && room.localParticipant);
|
||||
const showArt = !inConversation && !!mediaCurrentTrackId && !!mediaCurrentArtUrl;
|
||||
ledWallEl.classList.toggle("show-art", showArt);
|
||||
if (showArt && ledArtEl.getAttribute("src") !== mediaCurrentArtUrl) {
|
||||
ledArtEl.src = mediaCurrentArtUrl;
|
||||
}
|
||||
}
|
||||
|
||||
function buildLedWall() {
|
||||
const inner = Math.floor(LED_COLS / 2);
|
||||
@@ -781,7 +769,6 @@ document.addEventListener("keydown", (event) => {
|
||||
if (event.key !== "Escape") return;
|
||||
if (settingsSheet.classList.contains("open")) closeSheet();
|
||||
if (tasksPanel.classList.contains("open")) setTasksOpen(false);
|
||||
if (mediaPanel.classList.contains("open")) setMediaOpen(false);
|
||||
});
|
||||
// ── Wake lock ───────────────────────────────────────────────────────────────
|
||||
async function acquireWakeLock() {
|
||||
@@ -905,8 +892,6 @@ function createRoom(token) {
|
||||
stopBtn.disabled = true;
|
||||
agentParticipant = null;
|
||||
agentReady = false;
|
||||
// Leaving the conversation hands the room back to the music player.
|
||||
resumePlaybackAfterTalk();
|
||||
});
|
||||
|
||||
return newRoom.connect(LIVEKIT_URL, token, {
|
||||
@@ -965,7 +950,6 @@ async function connectConversation() {
|
||||
|
||||
// Get a signed access token from the token endpoint.
|
||||
const tokenData = await fetchToken(ROOM_NAME);
|
||||
await enterMediaScope(tokenData.room);
|
||||
room = await createRoom(tokenData.token);
|
||||
|
||||
// The agent is dispatched when we join. If it misses the dispatch
|
||||
@@ -979,7 +963,6 @@ async function connectConversation() {
|
||||
agentReady = false;
|
||||
await stale.disconnect();
|
||||
const freshTokenData = await fetchToken(ROOM_NAME);
|
||||
await enterMediaScope(freshTokenData.room);
|
||||
room = await createRoom(freshTokenData.token);
|
||||
if (!(await waitForAgent(15000))) {
|
||||
throw new Error("Assistant did not join — reload the page and try again");
|
||||
@@ -1011,7 +994,6 @@ async function connectConversation() {
|
||||
muteBtn.disabled = conversationMode !== "voice";
|
||||
updateMuteIcon();
|
||||
setState("listening");
|
||||
updateAvatarMedia();
|
||||
return room;
|
||||
} catch (err) {
|
||||
console.error("Connection failed:", err);
|
||||
@@ -1093,9 +1075,6 @@ stopBtn.addEventListener("click", () => {
|
||||
muteBtn.disabled = true;
|
||||
micMuted = false;
|
||||
updateMuteIcon();
|
||||
// Ending the conversation resumes music that was paused for the talk.
|
||||
resumePlaybackAfterTalk();
|
||||
updateAvatarMedia();
|
||||
});
|
||||
|
||||
// ── Mic mute toggle ─────────────────────────────────────────────────────────
|
||||
@@ -1162,551 +1141,4 @@ function sendModelMessage(model, targetRoom) {
|
||||
});
|
||||
}
|
||||
|
||||
// ── Media player ────────────────────────────────────────────────────────────
|
||||
// The browser <audio> element is the player. The media server owns library +
|
||||
// per-room desired state; this UI reconciles against GET /media/api/state
|
||||
// (polling) and reports real playback back. Hope drives the same state via
|
||||
// her media tools, so voice commands and panel clicks stay in sync.
|
||||
//
|
||||
// Conversation rule (mirrored in agent.py): while a track plays the agent's
|
||||
// mic is off. Connecting to Hope pauses music; ending the conversation
|
||||
// resumes what was playing.
|
||||
|
||||
const mediaBtn = document.getElementById("mediaBtn");
|
||||
const mediaPanel = document.getElementById("mediaPanel");
|
||||
const closeMediaBtn = document.getElementById("closeMediaBtn");
|
||||
const mediaBadge = document.getElementById("mediaBadge");
|
||||
const mediaArt = document.getElementById("mediaArt");
|
||||
const mediaArtPlaceholder = document.getElementById("mediaArtPlaceholder");
|
||||
const mediaTitleEl = document.getElementById("mediaTitle");
|
||||
const mediaChannelEl = document.getElementById("mediaChannel");
|
||||
const mediaProgressWrap = document.getElementById("mediaProgressWrap");
|
||||
const mediaProgressFill = document.getElementById("mediaProgressFill");
|
||||
const mediaProgressLabel = document.getElementById("mediaProgressLabel");
|
||||
const rewBtn = document.getElementById("rewBtn");
|
||||
const fwdBtn = document.getElementById("fwdBtn");
|
||||
const playPauseBtn = document.getElementById("playPauseBtn");
|
||||
const playIcon = document.getElementById("playIcon");
|
||||
const pauseIcon = document.getElementById("pauseIcon");
|
||||
const stopMediaBtn = document.getElementById("stopMediaBtn");
|
||||
const seekBar = document.getElementById("seekBar");
|
||||
const mediaCurEl = document.getElementById("mediaCur");
|
||||
const mediaDurEl = document.getElementById("mediaDur");
|
||||
const volumeBar = document.getElementById("volumeBar");
|
||||
const mediaSearchForm = document.getElementById("mediaSearchForm");
|
||||
const mediaSearchInput = document.getElementById("mediaSearchInput");
|
||||
const mediaResultsEl = document.getElementById("mediaResults");
|
||||
const mediaChoicesWrap = document.getElementById("mediaChoicesWrap");
|
||||
const mediaChoicesEl = document.getElementById("mediaChoices");
|
||||
const mediaLibraryEl = document.getElementById("mediaLibrary");
|
||||
const streamForm = document.getElementById("streamForm");
|
||||
const streamNameInput = document.getElementById("streamName");
|
||||
const streamUrlInput = document.getElementById("streamUrl");
|
||||
const mediaAudio = document.getElementById("mediaAudio");
|
||||
const ledArtEl = document.getElementById("ledArt");
|
||||
|
||||
let mediaRoom = null; // LiveKit room name (player state scope)
|
||||
let mediaCurrentTrackId = null; // track id loaded into the audio element
|
||||
let mediaCurrentArtUrl = null; // current track's cover art
|
||||
let mediaAppliedSeekTs = 0; // last applied seek_hint timestamp
|
||||
let mediaWasPlayingBeforeTalk = false;
|
||||
let mediaPollTimer = null;
|
||||
let mediaLastReport = 0;
|
||||
|
||||
function mediaScope() {
|
||||
// The LiveKit room is single-source of truth for the player scope. If a
|
||||
// room is connected, poll the room the agent actually joined (room.name),
|
||||
// so a "play X" request from the agent can never land in a stale scope.
|
||||
if (room && room.name) return room.name;
|
||||
return mediaRoom || "default";
|
||||
}
|
||||
|
||||
async function mediaPost(path, body) {
|
||||
const resp = await fetch(`/media/api${path}`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ room: mediaScope(), ...body }),
|
||||
});
|
||||
if (!resp.ok) {
|
||||
const detail = await resp.json().catch(() => ({}));
|
||||
throw new Error(detail.error || `media API ${path} -> ${resp.status}`);
|
||||
}
|
||||
return resp.json();
|
||||
}
|
||||
|
||||
// Called just before joining the LiveKit room: carry whatever the audio
|
||||
// element holds into the room-scoped player state, paused (talking pauses
|
||||
// music). resumePlaybackAfterTalk() reverses this when the talk ends.
|
||||
async function enterMediaScope(newRoom) {
|
||||
if (!newRoom || newRoom === mediaRoom) return;
|
||||
const wasPlaying = !!mediaCurrentTrackId && !mediaAudio.paused;
|
||||
mediaWasPlayingBeforeTalk = wasPlaying;
|
||||
const carriedTrack = mediaCurrentTrackId;
|
||||
mediaRoom = newRoom;
|
||||
if (carriedTrack) {
|
||||
try {
|
||||
await mediaPost("/play", { room: mediaRoom, track_id: carriedTrack, start_paused: true });
|
||||
} catch (e) { console.warn("media scope transfer failed:", e); }
|
||||
mediaAudio.pause();
|
||||
}
|
||||
}
|
||||
|
||||
async function resumePlaybackAfterTalk() {
|
||||
if (!mediaWasPlayingBeforeTalk) return;
|
||||
mediaWasPlayingBeforeTalk = false;
|
||||
try { await mediaPost("/control", { action: "resume" }); } catch (e) { /* poll retries */ }
|
||||
}
|
||||
|
||||
function publishMediaState(state) {
|
||||
if (!room || !room.localParticipant) return;
|
||||
const payload = new TextEncoder().encode(
|
||||
JSON.stringify({ type: "media_state", state, track_id: mediaCurrentTrackId })
|
||||
);
|
||||
room.localParticipant.publishData(payload, { topic: "media-control", reliable: true });
|
||||
}
|
||||
|
||||
function reportMediaPlayback(state) {
|
||||
const body = {
|
||||
state,
|
||||
track_id: mediaCurrentTrackId,
|
||||
position: isFinite(mediaAudio.currentTime) ? mediaAudio.currentTime : 0,
|
||||
};
|
||||
if (isFinite(mediaAudio.duration)) body.duration = mediaAudio.duration;
|
||||
mediaPost("/report", body).catch(() => {});
|
||||
}
|
||||
|
||||
// ── Polling + reconciliation ────────────────────────────────────────────────
|
||||
function startMediaPolling() {
|
||||
if (mediaPollTimer) return;
|
||||
mediaPollTimer = setInterval(() => {
|
||||
pollMediaState();
|
||||
}, 1500);
|
||||
}
|
||||
|
||||
async function pollMediaState() {
|
||||
let s;
|
||||
try {
|
||||
const resp = await fetch(`/media/api/state?room=${encodeURIComponent(mediaScope())}`);
|
||||
if (!resp.ok) return;
|
||||
s = await resp.json();
|
||||
} catch (e) {
|
||||
return; // media server briefly unavailable; try again next tick
|
||||
}
|
||||
reconcileMedia(s);
|
||||
}
|
||||
|
||||
function reconcileMedia(s) {
|
||||
renderMediaChoices(s.choices || []);
|
||||
updateMediaBadge(s);
|
||||
|
||||
const track = s.track;
|
||||
const pendingJob = (s.pending || []).find(j => !track || j.track_id === track.id);
|
||||
updateMediaProgress(track, pendingJob);
|
||||
|
||||
if (track && track.id !== mediaCurrentTrackId) {
|
||||
loadMediaTrack(track);
|
||||
} else if (track && track.audio_url && mediaAudio.getAttribute("src") !== track.audio_url) {
|
||||
// The track was loaded before it had a stream (still downloading) and
|
||||
// now has one — wire it up so playback can start without a reload.
|
||||
mediaAudio.src = track.audio_url;
|
||||
mediaAudio.load();
|
||||
updateMediaNowCard(track);
|
||||
}
|
||||
if (!track && mediaCurrentTrackId && s.desired === "stopped") clearMediaAudio();
|
||||
|
||||
applyDesiredMediaState(s, track);
|
||||
|
||||
if (s.seek_hint && s.seek_hint.ts !== mediaAppliedSeekTs && isFinite(mediaAudio.duration)) {
|
||||
mediaAppliedSeekTs = s.seek_hint.ts;
|
||||
const target = Math.min(s.seek_hint.position, mediaAudio.duration);
|
||||
if (Number.isFinite(target)) {
|
||||
mediaAudio.currentTime = Math.max(0, target);
|
||||
updateSeekUI();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function applyDesiredMediaState(s, track) {
|
||||
if (s.desired === "playing") {
|
||||
// audio_url presence means a playable stream exists — including a
|
||||
// still-growing download, which plays progressively.
|
||||
if (track && track.audio_url && mediaAudio.src && mediaAudio.paused) {
|
||||
mediaAudio.play().catch((e) => console.warn("playback blocked:", e));
|
||||
}
|
||||
} else if (!mediaAudio.paused) {
|
||||
mediaAudio.pause();
|
||||
}
|
||||
}
|
||||
|
||||
function loadMediaTrack(track) {
|
||||
mediaCurrentTrackId = track.id;
|
||||
updateMediaNowCard(track);
|
||||
if (track.audio_url) {
|
||||
if (mediaAudio.getAttribute("src") !== track.audio_url) {
|
||||
mediaAudio.src = track.audio_url;
|
||||
mediaAudio.load();
|
||||
}
|
||||
} else {
|
||||
// Downloading or failed: no stream yet. reconcileMedia wires the
|
||||
// stream URL in via polling as soon as one is available.
|
||||
mediaAudio.removeAttribute("src");
|
||||
mediaAudio.load();
|
||||
updateMediaControls();
|
||||
}
|
||||
}
|
||||
|
||||
function clearMediaAudio() {
|
||||
mediaCurrentTrackId = null;
|
||||
mediaCurrentArtUrl = null;
|
||||
mediaAudio.pause();
|
||||
mediaAudio.removeAttribute("src");
|
||||
mediaAudio.load();
|
||||
mediaTitleEl.textContent = "Nothing loaded";
|
||||
mediaChannelEl.textContent = "";
|
||||
mediaArt.hidden = true;
|
||||
mediaArtPlaceholder.hidden = false;
|
||||
updateMediaControls();
|
||||
updateAvatarMedia();
|
||||
}
|
||||
|
||||
function updateMediaNowCard(track) {
|
||||
mediaTitleEl.textContent = track.title || track.id;
|
||||
mediaCurrentArtUrl = track.art_url || null;
|
||||
if (track.status === "failed") {
|
||||
mediaChannelEl.textContent = "Download failed — pick another result or try again.";
|
||||
} else if (track.status === "downloading") {
|
||||
// Progressive streaming: the track plays while the download runs.
|
||||
mediaChannelEl.textContent = track.channel
|
||||
? `${track.channel} · downloading` : "Downloading…";
|
||||
} else {
|
||||
mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
|
||||
}
|
||||
if (track.art_url) {
|
||||
mediaArt.src = track.art_url;
|
||||
mediaArt.hidden = false;
|
||||
mediaArtPlaceholder.hidden = true;
|
||||
} else {
|
||||
mediaArt.hidden = true;
|
||||
mediaArtPlaceholder.hidden = false;
|
||||
}
|
||||
updateAvatarMedia();
|
||||
}
|
||||
|
||||
function updateMediaProgress(track, job) {
|
||||
if (!job) {
|
||||
mediaProgressWrap.hidden = true;
|
||||
return;
|
||||
}
|
||||
mediaProgressWrap.hidden = false;
|
||||
const pct = Math.round(job.progress || 0);
|
||||
mediaProgressFill.style.width = `${pct}%`;
|
||||
const stage = job.stage === "queued" ? "Queued…" : "Downloading…";
|
||||
mediaProgressLabel.textContent = `${stage} ${pct}%`;
|
||||
if (track) updateMediaNowCard(track);
|
||||
}
|
||||
|
||||
function updateMediaBadge(s) {
|
||||
const pending = (s.pending || []).length;
|
||||
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
||||
if (pending > 0) {
|
||||
mediaBadge.hidden = false;
|
||||
mediaBadge.textContent = String(pending);
|
||||
} else if (playing) {
|
||||
mediaBadge.hidden = false;
|
||||
mediaBadge.innerHTML = "♫";
|
||||
} else {
|
||||
mediaBadge.hidden = true;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Playback controls ───────────────────────────────────────────────────────
|
||||
function updateMediaControls() {
|
||||
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
||||
playPauseBtn.classList.toggle("playing", playing);
|
||||
playPauseBtn.setAttribute("aria-label", playing ? "Pause" : "Play");
|
||||
updateMediaBadge({ pending: [] });
|
||||
}
|
||||
|
||||
function fmtTime(sec) {
|
||||
if (!isFinite(sec) || sec < 0) return "0:00";
|
||||
const s = Math.floor(sec % 60);
|
||||
const m = Math.floor((sec / 60) % 60);
|
||||
const h = Math.floor(sec / 3600);
|
||||
return h ? `${h}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}` : `${m}:${String(s).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
function updateSeekUI() {
|
||||
if (!isFinite(mediaAudio.duration) || mediaAudio.duration <= 0) {
|
||||
seekBar.value = 0;
|
||||
mediaDurEl.textContent = "0:00";
|
||||
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
||||
return;
|
||||
}
|
||||
seekBar.max = Math.floor(mediaAudio.duration);
|
||||
seekBar.value = Math.floor(mediaAudio.currentTime);
|
||||
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
||||
mediaDurEl.textContent = fmtTime(mediaAudio.duration);
|
||||
}
|
||||
|
||||
playPauseBtn.addEventListener("click", () => {
|
||||
if (!mediaCurrentTrackId || !mediaAudio.src) return;
|
||||
if (mediaAudio.paused) {
|
||||
mediaAudio.play().catch((e) => console.warn("play failed:", e));
|
||||
mediaPost("/control", { action: "resume" }).catch(() => {});
|
||||
} else {
|
||||
mediaAudio.pause();
|
||||
mediaPost("/control", { action: "pause" }).catch(() => {});
|
||||
}
|
||||
});
|
||||
|
||||
rewBtn.addEventListener("click", () => {
|
||||
if (!mediaAudio.src) return;
|
||||
mediaAudio.currentTime = Math.max(0, mediaAudio.currentTime - 10);
|
||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||
});
|
||||
|
||||
fwdBtn.addEventListener("click", () => {
|
||||
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
||||
mediaAudio.currentTime = Math.min(mediaAudio.duration - 0.5, mediaAudio.currentTime + 10);
|
||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||
});
|
||||
|
||||
stopMediaBtn.addEventListener("click", () => {
|
||||
clearMediaAudio();
|
||||
mediaPost("/control", { action: "stop" }).catch(() => {});
|
||||
publishMediaState("stopped");
|
||||
});
|
||||
|
||||
seekBar.addEventListener("change", () => {
|
||||
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
||||
mediaAudio.currentTime = Number(seekBar.value);
|
||||
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||
});
|
||||
|
||||
volumeBar.addEventListener("input", () => {
|
||||
mediaAudio.volume = Number(volumeBar.value) / 100;
|
||||
try { localStorage.setItem("media_volume", volumeBar.value); } catch (e) { /* ok */ }
|
||||
});
|
||||
mediaAudio.volume = Number(localStorage.getItem("media_volume") ?? 90) / 100;
|
||||
volumeBar.value = Math.round(mediaAudio.volume * 100);
|
||||
|
||||
// Browsers block audio until a user gestured on the page. Hope often starts a
|
||||
// cached song from a poll (voice command, no click on the player), so the
|
||||
// <audio> element sits outside the LiveKit room's audio-unlock flow and its
|
||||
// play() is rejected. Capture the first gesture and unlock the element.
|
||||
let mediaAudioUnlocked = false;
|
||||
function unlockMediaAudio() {
|
||||
if (mediaAudioUnlocked) return;
|
||||
mediaAudioUnlocked = true;
|
||||
removeEventListener("pointerdown", unlockMediaAudio, true);
|
||||
removeEventListener("keydown", unlockMediaAudio, true);
|
||||
removeEventListener("touchstart", unlockMediaAudio, true);
|
||||
// Play the current src (if any) inside the gesture to whitelist the
|
||||
// element for autoplay, so poll-driven play() later isn't blocked.
|
||||
if (mediaAudio.getAttribute("src") && mediaAudio.paused) {
|
||||
const p = mediaAudio.play();
|
||||
if (p) p.catch(() => {});
|
||||
}
|
||||
}
|
||||
addEventListener("pointerdown", unlockMediaAudio, true);
|
||||
addEventListener("keydown", unlockMediaAudio, true);
|
||||
addEventListener("touchstart", unlockMediaAudio, true);
|
||||
|
||||
mediaAudio.addEventListener("play", () => {
|
||||
updateMediaControls();
|
||||
reportMediaPlayback("playing");
|
||||
publishMediaState("playing");
|
||||
});
|
||||
mediaAudio.addEventListener("pause", () => {
|
||||
updateMediaControls();
|
||||
if (mediaCurrentTrackId) {
|
||||
reportMediaPlayback("paused");
|
||||
publishMediaState("paused");
|
||||
}
|
||||
});
|
||||
mediaAudio.addEventListener("ended", () => {
|
||||
updateMediaControls();
|
||||
reportMediaPlayback("stopped");
|
||||
publishMediaState("stopped");
|
||||
mediaPost("/control", { action: "stop" }).catch(() => {});
|
||||
});
|
||||
mediaAudio.addEventListener("timeupdate", () => {
|
||||
updateSeekUI();
|
||||
const now = Date.now();
|
||||
if (!mediaAudio.paused && now - mediaLastReport > 5000) {
|
||||
mediaLastReport = now;
|
||||
reportMediaPlayback("playing");
|
||||
}
|
||||
});
|
||||
mediaAudio.addEventListener("durationchange", updateSeekUI);
|
||||
mediaAudio.addEventListener("error", () => {
|
||||
// An empty src (track still downloading) also fires this — only surface
|
||||
// real playback failures.
|
||||
if (mediaCurrentTrackId && mediaAudio.getAttribute("src")) {
|
||||
mediaChannelEl.textContent = "Playback error — the track may still be preparing.";
|
||||
}
|
||||
});
|
||||
|
||||
// ── Search / choices / library ──────────────────────────────────────────────
|
||||
function fmtListDuration(sec) {
|
||||
const s = Number(sec);
|
||||
return isFinite(s) && s > 0 ? fmtTime(s) : "";
|
||||
}
|
||||
|
||||
function mediaRowActions() {
|
||||
const play = document.createElement("button");
|
||||
play.className = "btn btn-secondary media-row-play";
|
||||
play.type = "button";
|
||||
play.textContent = "Play";
|
||||
return play;
|
||||
}
|
||||
|
||||
function renderMediaResults(results) {
|
||||
mediaResultsEl.innerHTML = "";
|
||||
if (!results || results.length === 0) {
|
||||
mediaResultsEl.innerHTML = '<div class="media-empty">No results.</div>';
|
||||
return;
|
||||
}
|
||||
for (const r of results) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "media-row";
|
||||
const info = document.createElement("div");
|
||||
info.className = "media-row-info";
|
||||
const dur = fmtListDuration(r.duration);
|
||||
info.innerHTML = `<strong>${escapeHtml(r.title)}</strong>` +
|
||||
`<span>${escapeHtml(r.channel || "")}${dur ? " · " + dur : ""}` +
|
||||
`${r.in_library ? " · in library" : ""}</span>`;
|
||||
const play = mediaRowActions();
|
||||
play.addEventListener("click", () => playVideoId(r));
|
||||
row.appendChild(info);
|
||||
row.appendChild(play);
|
||||
mediaResultsEl.appendChild(row);
|
||||
}
|
||||
}
|
||||
|
||||
async function playVideoId(r) {
|
||||
try {
|
||||
await mediaPost("/play", {
|
||||
video_id: r.video_id,
|
||||
title: r.title,
|
||||
channel: r.channel || "",
|
||||
});
|
||||
} catch (e) {
|
||||
mediaChannelEl.textContent = `Could not start: ${e.message}`;
|
||||
}
|
||||
}
|
||||
|
||||
mediaSearchForm.addEventListener("submit", async (event) => {
|
||||
event.preventDefault();
|
||||
const query = mediaSearchInput.value.trim();
|
||||
if (!query) return;
|
||||
mediaResultsEl.innerHTML = '<div class="media-empty">Searching YouTube Music…</div>';
|
||||
try {
|
||||
const data = await mediaPost("/search", { query, limit: 6 });
|
||||
renderMediaResults(data.results);
|
||||
} catch (e) {
|
||||
mediaResultsEl.innerHTML = `<div class="media-empty">Search failed: ${escapeHtml(e.message)}</div>`;
|
||||
}
|
||||
});
|
||||
|
||||
function renderMediaChoices(choices) {
|
||||
mediaChoicesWrap.hidden = choices.length === 0;
|
||||
mediaChoicesEl.innerHTML = "";
|
||||
for (const c of choices) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "media-row media-choice";
|
||||
const info = document.createElement("div");
|
||||
info.className = "media-row-info";
|
||||
const dur = fmtListDuration(c.duration);
|
||||
info.innerHTML = `<strong>${escapeHtml(c.title)}</strong>` +
|
||||
`<span>${escapeHtml(c.channel || "")}${dur ? " · " + dur : ""}</span>`;
|
||||
const play = mediaRowActions();
|
||||
play.addEventListener("click", () => playVideoId(c));
|
||||
row.appendChild(info);
|
||||
row.appendChild(play);
|
||||
mediaChoicesEl.appendChild(row);
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshLibrary() {
|
||||
let data;
|
||||
try {
|
||||
data = await fetch("/media/api/library").then((r) => r.json());
|
||||
} catch (e) {
|
||||
return;
|
||||
}
|
||||
mediaLibraryEl.innerHTML = "";
|
||||
const tracks = data.tracks || [];
|
||||
if (tracks.length === 0) {
|
||||
mediaLibraryEl.innerHTML = '<div class="media-empty">Downloaded songs and streams live here.</div>';
|
||||
return;
|
||||
}
|
||||
for (const t of tracks) {
|
||||
const row = document.createElement("div");
|
||||
row.className = "media-row";
|
||||
const info = document.createElement("div");
|
||||
info.className = "media-row-info";
|
||||
const dur = fmtListDuration(t.duration);
|
||||
const statusTag = t.status === "ready" ? "" : ` · ${t.status}`;
|
||||
info.innerHTML = `<strong>${escapeHtml(t.title)}</strong>` +
|
||||
`<span>${escapeHtml(t.channel || (t.kind === "stream" ? "Live stream" : ""))}` +
|
||||
`${dur ? " · " + dur : ""}${statusTag}</span>`;
|
||||
const play = mediaRowActions();
|
||||
play.disabled = t.status !== "ready";
|
||||
play.addEventListener("click", () => {
|
||||
mediaPost("/play", { track_id: t.id }).catch((e) => {
|
||||
mediaChannelEl.textContent = `Could not play: ${e.message}`;
|
||||
});
|
||||
});
|
||||
const del = document.createElement("button");
|
||||
del.className = "btn btn-icon media-row-delete";
|
||||
del.type = "button";
|
||||
del.setAttribute("aria-label", `Remove ${t.title}`);
|
||||
del.title = "Remove from library";
|
||||
del.innerHTML = '<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>';
|
||||
del.addEventListener("click", async () => {
|
||||
try {
|
||||
await fetch(`/media/api/tracks/${encodeURIComponent(t.id)}`, { method: "DELETE" });
|
||||
if (t.id === mediaCurrentTrackId) clearMediaAudio();
|
||||
refreshLibrary();
|
||||
} catch (e) { /* leave the row; next refresh decides */ }
|
||||
});
|
||||
row.appendChild(info);
|
||||
row.appendChild(play);
|
||||
row.appendChild(del);
|
||||
mediaLibraryEl.appendChild(row);
|
||||
}
|
||||
}
|
||||
|
||||
streamForm.addEventListener("submit", async (event) => {
|
||||
event.preventDefault();
|
||||
const name = streamNameInput.value.trim() || streamUrlInput.value.trim();
|
||||
const url = streamUrlInput.value.trim();
|
||||
if (!url) return;
|
||||
try {
|
||||
await mediaPost("/streams", { name, url });
|
||||
streamNameInput.value = "";
|
||||
streamUrlInput.value = "";
|
||||
refreshLibrary();
|
||||
} catch (e) {
|
||||
mediaChannelEl.textContent = `Could not add stream: ${e.message}`;
|
||||
}
|
||||
});
|
||||
|
||||
// ── Panel open/close ────────────────────────────────────────────────────────
|
||||
function setMediaOpen(open) {
|
||||
mediaPanel.classList.toggle("open", open);
|
||||
mediaBtn.setAttribute("aria-expanded", String(open));
|
||||
if (open) {
|
||||
refreshLibrary();
|
||||
pollMediaState();
|
||||
}
|
||||
}
|
||||
|
||||
mediaBtn.setAttribute("aria-expanded", "false");
|
||||
mediaBtn.addEventListener("click", () => setMediaOpen(!mediaPanel.classList.contains("open")));
|
||||
closeMediaBtn.addEventListener("click", () => setMediaOpen(false));
|
||||
|
||||
startMediaPolling();
|
||||
updateMediaControls();
|
||||
buildLedWall();
|
||||
@@ -42,11 +42,6 @@
|
||||
<span>Tasks</span>
|
||||
<span class="tasks-badge" id="tasksBadge" hidden>0</span>
|
||||
</button>
|
||||
<button id="mediaBtn" class="btn btn-secondary tasks-trigger" aria-label="Music player" title="Music player">
|
||||
<svg viewBox="0 0 24 24" width="17" height="17" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
||||
<span>Music</span>
|
||||
<span class="tasks-badge" id="mediaBadge" hidden>♫</span>
|
||||
</button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
@@ -59,7 +54,6 @@
|
||||
|
||||
<div class="orb-stage">
|
||||
<div class="led-wall" id="ledWall" data-state="idle" aria-hidden="true">
|
||||
<img id="ledArt" class="led-art" alt="">
|
||||
</div>
|
||||
<div class="orb-labels" aria-live="polite">
|
||||
<span class="state-label" id="stateLabel">Ready when you are</span>
|
||||
@@ -151,85 +145,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="tasks-panel media-panel" id="mediaPanel" aria-label="Music player">
|
||||
<div class="tasks-header">
|
||||
<div>
|
||||
<h2>Music</h2>
|
||||
<p>YouTube Music downloads and live streams, streamed to you.</p>
|
||||
</div>
|
||||
<button id="closeMediaBtn" class="btn btn-icon" aria-label="Close music player">
|
||||
<svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="media-now">
|
||||
<img id="mediaArt" class="media-art" alt="" hidden>
|
||||
<div class="media-art-placeholder" id="mediaArtPlaceholder" aria-hidden="true">
|
||||
<svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
||||
</div>
|
||||
<div class="media-now-info">
|
||||
<strong id="mediaTitle">Nothing loaded</strong>
|
||||
<span id="mediaChannel"></span>
|
||||
<div class="media-progress" id="mediaProgressWrap" hidden>
|
||||
<div class="media-progress-fill" id="mediaProgressFill"></div>
|
||||
<span class="media-progress-label" id="mediaProgressLabel">Fetching…</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="media-controls" role="group" aria-label="Playback controls">
|
||||
<button id="rewBtn" class="btn btn-icon" aria-label="Back 10 seconds" title="Back 10s">
|
||||
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 17a5 5 0 1 0-5-5"></path><path d="M11 17H6v-5"></path><path d="m19 5-4 4h-3"></path><path d="m15 9 4-4"></path></svg>
|
||||
</button>
|
||||
<button id="playPauseBtn" class="btn btn-icon media-play" aria-label="Play or pause" title="Play / pause">
|
||||
<svg id="playIcon" class="play-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M8 5v14l11-7Z"></path></svg>
|
||||
<svg id="pauseIcon" class="pause-icon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M7 5h4v14H7zM13 5h4v14h-4z"></path></svg>
|
||||
</button>
|
||||
<button id="fwdBtn" class="btn btn-icon" aria-label="Forward 10 seconds" title="Forward 10s">
|
||||
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M13 17a5 5 0 1 1 5-5"></path><path d="M13 17h5v-5"></path><path d="m5 5 4 4h3"></path><path d="m9 9 4-4"></path></svg>
|
||||
</button>
|
||||
<button id="stopMediaBtn" class="btn btn-icon" aria-label="Stop" title="Stop">
|
||||
<svg viewBox="0 0 24 24" width="18" height="18" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"></rect></svg>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<div class="media-seek">
|
||||
<span class="media-time" id="mediaCur">0:00</span>
|
||||
<input type="range" id="seekBar" min="0" max="1000" value="0" step="1" aria-label="Seek position">
|
||||
<span class="media-time" id="mediaDur">0:00</span>
|
||||
</div>
|
||||
<div class="media-volume">
|
||||
<svg viewBox="0 0 24 24" width="15" height="15" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 5 6 9H2v6h4l5 4V5Z"></path><path d="M15.5 8.5a5 5 0 0 1 0 7"></path></svg>
|
||||
<input type="range" id="volumeBar" min="0" max="100" value="90" aria-label="Volume">
|
||||
</div>
|
||||
|
||||
<h3 class="sheet-section-title">Search YouTube Music</h3>
|
||||
<form class="media-search" id="mediaSearchForm">
|
||||
<input id="mediaSearchInput" type="search" placeholder="Song, artist, lyrics…" autocomplete="off">
|
||||
<button class="btn btn-secondary" type="submit">Find</button>
|
||||
</form>
|
||||
<div class="media-list" id="mediaResults"></div>
|
||||
|
||||
<div id="mediaChoicesWrap" hidden>
|
||||
<h3 class="sheet-section-title">Hope’s suggestions</h3>
|
||||
<div class="media-list" id="mediaChoices"></div>
|
||||
</div>
|
||||
|
||||
<h3 class="sheet-section-title">Library</h3>
|
||||
<div class="media-list" id="mediaLibrary">
|
||||
<div class="media-empty">Downloaded songs and streams live here.</div>
|
||||
</div>
|
||||
|
||||
<h3 class="sheet-section-title">Add live stream</h3>
|
||||
<form class="media-search media-stream-form" id="streamForm">
|
||||
<input id="streamName" type="text" placeholder="Station name" autocomplete="off">
|
||||
<input id="streamUrl" type="url" placeholder="https://… audio stream URL" required>
|
||||
<button class="btn btn-secondary" type="submit">Add</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<audio id="mediaAudio" preload="metadata"></audio>
|
||||
|
||||
<script src="livekit-client.umd.js"></script>
|
||||
<script src="app.js"></script>
|
||||
</body>
|
||||
|
||||
@@ -1,810 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Media server for the Hope voice assistant.
|
||||
|
||||
A small FastAPI service that owns the permanent music library and player
|
||||
coordination:
|
||||
|
||||
- Library: downloaded YouTube Music tracks (raw best-audio files, saved by
|
||||
yt-dlp with no transcode) and user-added live audio stream URLs, persisted
|
||||
in library.json.
|
||||
- Search: yt-dlp ``ytsearch`` (YouTube Music pages share video ids), used by
|
||||
Hope's media MCP toolset and by the web UI search box. Ready library tracks
|
||||
that match the query are returned first so Hope prefers a downloaded copy.
|
||||
- Download jobs: one-at-a-time worker thread; progress is parsed from
|
||||
yt-dlp stderr and surfaced through GET /media/api/state. Playback starts
|
||||
while the file is still growing (progressive streaming), and the finished
|
||||
file stays in the library for instant replay.
|
||||
- Player state: per-room desired state (loaded track, playing/paused/stopped,
|
||||
seek hints). The browser is the actual player and reconciles against this;
|
||||
it reports real playback state back so Hope's voice tools can answer
|
||||
"what's playing".
|
||||
|
||||
Security posture matches the token server: bind 127.0.0.1 only, reachability
|
||||
exclusively through the nginx HTTPS proxy (/media/api/). Growing downloads
|
||||
are streamed by this service (chunked, no Content-Length); finished files are
|
||||
served here too with native Range support for seeking. Live stream URLs are
|
||||
proxied (mixed-content: an http:// radio URL would be blocked by the browser,
|
||||
so it is fetched server-side and re-served over HTTPS).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
import uvicorn
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
|
||||
|
||||
MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/data/media"))
|
||||
FILES_DIR = MEDIA_DIR / "files"
|
||||
THUMBS_DIR = MEDIA_DIR / "thumbs"
|
||||
JOBS_DIR = MEDIA_DIR / "jobs"
|
||||
LIBRARY_FILE = MEDIA_DIR / "library.json"
|
||||
COOKIES_FILE = os.environ.get("YTDLP_COOKIES", "")
|
||||
MEDIA_HOST = os.environ.get("MEDIA_SERVER_HOST", "127.0.0.1")
|
||||
# 8093, NOT 8092: 8092 is occupied by an Apache/YOURLS stack on the Unraid
|
||||
# deployment host (xNAS) — a bind conflict there looks like a healthy
|
||||
# deployment because the healthcheck curl (without -f) accepts any response.
|
||||
MEDIA_PORT = int(os.environ.get("MEDIA_SERVER_PORT", "8093"))
|
||||
|
||||
DOWNLOAD_TIMEOUT = int(os.environ.get("MEDIA_DOWNLOAD_TIMEOUT", "600"))
|
||||
SEARCH_TIMEOUT = int(os.environ.get("MEDIA_SEARCH_TIMEOUT", "45"))
|
||||
|
||||
app = FastAPI(title="hope-media", docs_url=None, redoc_url=None, openapi_url=None)
|
||||
|
||||
_lock = threading.Lock() # guards library + rooms below
|
||||
_library: dict[str, Any] = {"tracks": {}, "version": 1}
|
||||
_rooms: dict[str, dict[str, Any]] = {}
|
||||
_jobs: dict[str, dict[str, Any]] = {}
|
||||
_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="media-dl")
|
||||
|
||||
for _d in (FILES_DIR, THUMBS_DIR, JOBS_DIR):
|
||||
_d.mkdir(parents=True, exist_ok=True)
|
||||
# The bind mount may be inspected/cleaned from the host by a non-root
|
||||
# user (Unraid share convention) — keep the dirs world-traversable.
|
||||
try:
|
||||
os.chmod(_d, 0o777)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# ── Library store ───────────────────────────────────────────────────────────
|
||||
|
||||
def _load_library() -> None:
|
||||
if LIBRARY_FILE.exists():
|
||||
try:
|
||||
data = json.loads(LIBRARY_FILE.read_text(encoding="utf-8"))
|
||||
tracks = data.get("tracks", {})
|
||||
if isinstance(tracks, dict):
|
||||
_library["tracks"] = tracks
|
||||
_library["version"] = int(data.get("version", 1))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
# Corrupt library: keep the broken file aside, start fresh.
|
||||
try:
|
||||
LIBRARY_FILE.replace(LIBRARY_FILE.with_suffix(".json.broken"))
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _save_library() -> None:
|
||||
tmp = LIBRARY_FILE.with_suffix(".json.tmp")
|
||||
tmp.write_text(json.dumps(_library), encoding="utf-8")
|
||||
tmp.replace(LIBRARY_FILE)
|
||||
|
||||
|
||||
def _bump_version() -> int:
|
||||
_library["version"] = int(_library.get("version", 1)) + 1
|
||||
return _library["version"]
|
||||
|
||||
|
||||
def _track_dict(track: dict) -> dict:
|
||||
out = dict(track)
|
||||
if track["kind"] == "youtube":
|
||||
# The stream endpoint serves the growing file during download and the
|
||||
# finished file afterwards; the URL is stable across the transition
|
||||
# so the browser never has to reload mid-playback.
|
||||
if track.get("status") in ("downloading", "ready"):
|
||||
out["audio_url"] = f"/media/api/stream/{track['id']}"
|
||||
out["art_url"] = (
|
||||
f"/media/thumbs/{track['id']}.jpg"
|
||||
if (THUMBS_DIR / f"{track['id']}.jpg").exists()
|
||||
else None
|
||||
)
|
||||
else:
|
||||
out["audio_url"] = f"/media/api/stream/{track['id']}"
|
||||
return out
|
||||
|
||||
|
||||
def _get_track(track_id: str) -> dict | None:
|
||||
return _library["tracks"].get(track_id)
|
||||
|
||||
|
||||
def _upsert_track(track: dict) -> None:
|
||||
_library["tracks"][track["id"]] = track
|
||||
_bump_version()
|
||||
_save_library()
|
||||
|
||||
|
||||
_load_library()
|
||||
|
||||
|
||||
# ── Player state (per room) ─────────────────────────────────────────────────
|
||||
|
||||
def _room_state(room: str) -> dict:
|
||||
return _rooms.setdefault(
|
||||
room,
|
||||
{"track_id": None, "desired": "stopped", "reported": None,
|
||||
"seek_hint": None, "choices": []},
|
||||
)
|
||||
|
||||
|
||||
def _room_snapshot(room: str) -> dict:
|
||||
st = _room_state(room)
|
||||
track = _get_track(st["track_id"]) if st["track_id"] else None
|
||||
pending = [
|
||||
{
|
||||
"job_id": j["id"],
|
||||
"track_id": j["track_id"],
|
||||
"stage": j["stage"],
|
||||
"progress": j["progress"],
|
||||
"error": j.get("error"),
|
||||
}
|
||||
for j in _jobs.values()
|
||||
if j["stage"] in ("queued", "downloading", "transcoding")
|
||||
]
|
||||
return {
|
||||
"room": room,
|
||||
"track": _track_dict(track) if track else None,
|
||||
"desired": st["desired"],
|
||||
"reported": st["reported"],
|
||||
"seek_hint": st["seek_hint"],
|
||||
"choices": st["choices"],
|
||||
"pending": pending,
|
||||
"version": _library["version"],
|
||||
}
|
||||
|
||||
|
||||
# ── yt-dlp helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
def _ytdlp_base_args() -> list[str]:
|
||||
args = [sys.executable, "-m", "yt_dlp", "--no-playlist", "--no-warnings"]
|
||||
if COOKIES_FILE and Path(COOKIES_FILE).exists():
|
||||
args += ["--cookies", COOKIES_FILE]
|
||||
return args
|
||||
|
||||
|
||||
_PROGRESS_RE = re.compile(r"\[download\]\s+([\d.]+)%")
|
||||
|
||||
# Audio containers yt-dlp can produce for a raw best-audio download.
|
||||
_AUDIO_EXTS = {
|
||||
".webm", ".opus", ".ogg", ".m4a", ".mp4", ".m4b", ".aac",
|
||||
".mp3", ".mka", ".flac", ".wav",
|
||||
}
|
||||
_AUDIO_MIME = {
|
||||
".webm": "audio/webm",
|
||||
".opus": "audio/opus",
|
||||
".ogg": "audio/ogg",
|
||||
".m4a": "audio/mp4",
|
||||
".mp4": "audio/mp4",
|
||||
".m4b": "audio/mp4",
|
||||
".aac": "audio/aac",
|
||||
".mp3": "audio/mpeg",
|
||||
".mka": "audio/x-matroska",
|
||||
".flac": "audio/flac",
|
||||
".wav": "audio/wav",
|
||||
}
|
||||
|
||||
|
||||
def _media_type_for(name: str) -> str:
|
||||
if name.endswith(".part"):
|
||||
name = name[:-5]
|
||||
return _AUDIO_MIME.get(Path(name).suffix.lower(), "audio/mpeg")
|
||||
|
||||
|
||||
def _youtube_media_file(track_id: str) -> Path | None:
|
||||
"""The on-disk audio file for a youtube track, if any.
|
||||
|
||||
During an active download yt-dlp writes ``{id}.{ext}.part`` and renames it
|
||||
to ``{id}.{ext}`` on completion — return whichever currently exists (the
|
||||
partial file wins, thumbnails and other sidecars are ignored).
|
||||
"""
|
||||
complete: Path | None = None
|
||||
for p in FILES_DIR.glob(f"{track_id}.*"):
|
||||
suffix = p.suffix.lower()
|
||||
if suffix == ".part":
|
||||
if p.stat().st_size > 0:
|
||||
return p
|
||||
elif suffix in _AUDIO_EXTS:
|
||||
if complete is None or p.name < complete.name:
|
||||
complete = p
|
||||
return complete
|
||||
|
||||
|
||||
async def _wait_for_media_file(track_id: str, timeout: float = 15.0) -> Path | None:
|
||||
"""Wait for yt-dlp to create the output file after a play request."""
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
path = _youtube_media_file(track_id)
|
||||
if path is not None:
|
||||
return path
|
||||
await asyncio.sleep(0.5)
|
||||
return None
|
||||
|
||||
|
||||
async def _growing_file(track_id: str, job: dict, chunk_size: int = 64 * 1024):
|
||||
"""Yield bytes from a file that yt-dlp is still writing.
|
||||
|
||||
The response stays open with no Content-Length until the download
|
||||
finishes, so the browser plays the track as it grows. The open descriptor
|
||||
keeps reading the old inode across yt-dlp's .part -> final rename, and the
|
||||
path is re-resolved if the file disappears between resolution and open.
|
||||
"""
|
||||
fd = -1
|
||||
pos = 0
|
||||
stall = 0
|
||||
try:
|
||||
while True:
|
||||
if fd == -1:
|
||||
path = _youtube_media_file(track_id)
|
||||
if path is not None:
|
||||
try:
|
||||
fd = os.open(path, os.O_RDONLY)
|
||||
except FileNotFoundError:
|
||||
fd = -1 # renamed between glob and open — retry
|
||||
if fd != -1:
|
||||
data = await asyncio.to_thread(os.pread, fd, chunk_size, pos)
|
||||
if data:
|
||||
pos += len(data)
|
||||
stall = 0
|
||||
yield data
|
||||
continue
|
||||
with _lock:
|
||||
active = job.get("stage") == "downloading"
|
||||
if not active:
|
||||
break
|
||||
stall += 1
|
||||
if stall > 60: # ~30s with no new bytes — stop waiting
|
||||
break
|
||||
await asyncio.sleep(0.5)
|
||||
finally:
|
||||
if fd != -1:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def _run_search(query: str, limit: int) -> list[dict]:
|
||||
"""Flat ytsearch lookup — cheap, no download, works anonymously."""
|
||||
cmd = _ytdlp_base_args() + [
|
||||
"--flat-playlist", "--dump-json",
|
||||
f"ytsearch{limit}:{query}",
|
||||
]
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
cmd, capture_output=True, text=True, timeout=SEARCH_TIMEOUT, check=False
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
raise HTTPException(504, "YouTube Music search timed out")
|
||||
if proc.returncode != 0 and not proc.stdout.strip():
|
||||
detail = (proc.stderr or "").strip().splitlines()[-1:] or ["unknown error"]
|
||||
raise HTTPException(502, f"Search failed: {detail[0][:300]}")
|
||||
results = []
|
||||
for line in proc.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line.startswith("{"):
|
||||
continue
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
video_id = entry.get("id")
|
||||
if not video_id:
|
||||
continue
|
||||
existing = _get_track(video_id)
|
||||
results.append({
|
||||
"video_id": video_id,
|
||||
"title": entry.get("title") or video_id,
|
||||
"channel": entry.get("uploader") or entry.get("channel") or "",
|
||||
"duration": entry.get("duration"),
|
||||
"view_count": entry.get("view_count"),
|
||||
"in_library": bool(existing),
|
||||
"track_id": existing["id"] if existing else None,
|
||||
"library_status": existing["status"] if existing else None,
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
def _job_dict(job: dict) -> dict:
|
||||
return {k: job.get(k) for k in
|
||||
("id", "track_id", "stage", "progress", "error", "started_at")}
|
||||
|
||||
|
||||
def _persist_job(job: dict) -> None:
|
||||
try:
|
||||
(JOBS_DIR / f"{job['id']}.json").write_text(
|
||||
json.dumps(_job_dict(job)), encoding="utf-8"
|
||||
)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _fail_job(job: dict, track: dict, message: str) -> None:
|
||||
job["stage"] = "error"
|
||||
job["error"] = message[:500]
|
||||
_persist_job(job)
|
||||
with _lock:
|
||||
track["status"] = "failed"
|
||||
track["error"] = message[:500]
|
||||
_bump_version()
|
||||
_save_library()
|
||||
|
||||
|
||||
def _download_worker(job: dict, track: dict, video_id: str) -> None:
|
||||
"""Runs in the single-worker executor: raw best-audio download.
|
||||
|
||||
No mp3 transcode — the audio container is streamed straight into the
|
||||
library so playback can begin while the file is still growing (the stream
|
||||
endpoint serves the partial file), and the finished file stays playable
|
||||
offline. Thumbnail handling is unchanged.
|
||||
"""
|
||||
cmd = _ytdlp_base_args() + [
|
||||
"-f", "bestaudio/best",
|
||||
"--newline", "--progress",
|
||||
"--print-json",
|
||||
"--write-thumbnail", "--convert-thumbnails", "jpg",
|
||||
"--retries", "3", "--fragment-retries", "3",
|
||||
"-P", str(FILES_DIR),
|
||||
"-o", f"{track['id']}.%(ext)s",
|
||||
f"https://www.youtube.com/watch?v={video_id}",
|
||||
]
|
||||
job["stage"] = "downloading"
|
||||
info: dict = {}
|
||||
# Progress lines go to stderr, the metadata JSON to stdout. Drain stderr
|
||||
# in a thread so the stdout read loop can never deadlock the pipe; the
|
||||
# tail is kept for the failure detail.
|
||||
stderr_tail: list[str] = []
|
||||
proc = None
|
||||
|
||||
def _drain_stderr() -> None:
|
||||
assert proc is not None and proc.stderr is not None
|
||||
for line in proc.stderr:
|
||||
stderr_tail.append(line)
|
||||
if len(stderr_tail) > 200:
|
||||
stderr_tail.pop(0)
|
||||
m = _PROGRESS_RE.search(line)
|
||||
if m:
|
||||
job["progress"] = min(99.0, float(m.group(1)))
|
||||
_persist_job(job)
|
||||
|
||||
try:
|
||||
proc = subprocess.Popen(
|
||||
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
||||
text=True, errors="replace",
|
||||
)
|
||||
threading.Thread(target=_drain_stderr, daemon=True).start()
|
||||
deadline = time.monotonic() + DOWNLOAD_TIMEOUT
|
||||
assert proc.stdout is not None
|
||||
for line in proc.stdout:
|
||||
if time.monotonic() > deadline:
|
||||
proc.kill()
|
||||
proc.wait(timeout=30)
|
||||
raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
|
||||
line = line.strip()
|
||||
if line.startswith("{"):
|
||||
try:
|
||||
info = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
proc.wait(timeout=30)
|
||||
if proc.returncode != 0:
|
||||
detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
|
||||
raise RuntimeError(detail.strip())
|
||||
media = _youtube_media_file(track["id"])
|
||||
if media is None:
|
||||
raise RuntimeError("yt-dlp finished but no audio file was produced")
|
||||
except Exception as exc: # noqa: BLE001 — any failure marks the track failed
|
||||
_fail_job(job, track, str(exc))
|
||||
return
|
||||
|
||||
thumb_src = FILES_DIR / f"{track['id']}.jpg"
|
||||
try:
|
||||
if thumb_src.exists():
|
||||
thumb_src.replace(THUMBS_DIR / f"{track['id']}.jpg")
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
job["stage"] = "ready"
|
||||
job["progress"] = 100.0
|
||||
_persist_job(job)
|
||||
with _lock:
|
||||
track["status"] = "ready"
|
||||
track["error"] = None
|
||||
track["ext"] = media.suffix.lstrip(".").lower()
|
||||
if info:
|
||||
if info.get("title"):
|
||||
track["title"] = info["title"]
|
||||
if info.get("uploader") or info.get("channel"):
|
||||
track["channel"] = info.get("uploader") or info.get("channel")
|
||||
if info.get("duration"):
|
||||
track["duration"] = round(float(info["duration"]))
|
||||
_bump_version()
|
||||
_save_library()
|
||||
|
||||
|
||||
def _start_download(track: dict, video_id: str) -> dict:
|
||||
job = {
|
||||
"id": uuid.uuid4().hex[:12],
|
||||
"track_id": track["id"],
|
||||
"stage": "queued",
|
||||
"progress": 0.0,
|
||||
"error": None,
|
||||
"started_at": time.time(),
|
||||
}
|
||||
_jobs[job["id"]] = job
|
||||
_persist_job(job)
|
||||
_executor.submit(_download_worker, job, track, video_id)
|
||||
return job
|
||||
|
||||
|
||||
# ── API: health, library ────────────────────────────────────────────────────
|
||||
|
||||
@app.get("/media/api/healthz")
|
||||
def healthz() -> dict:
|
||||
return {"ok": True, "version": _library["version"]}
|
||||
|
||||
|
||||
@app.get("/media/api/library")
|
||||
def get_library() -> dict:
|
||||
with _lock:
|
||||
tracks = sorted(
|
||||
(_track_dict(t) for t in _library["tracks"].values()),
|
||||
key=lambda t: t.get("added_at", 0),
|
||||
reverse=True,
|
||||
)
|
||||
return {"tracks": tracks, "version": _library["version"]}
|
||||
|
||||
|
||||
def _delete_track_files(track: dict) -> None:
|
||||
if track["kind"] != "youtube":
|
||||
return
|
||||
paths = [
|
||||
FILES_DIR / f"{track['id']}.{ext}" for ext in
|
||||
(s.lstrip(".") for s in _AUDIO_EXTS)
|
||||
] + [FILES_DIR / f"{track['id']}.jpg", THUMBS_DIR / f"{track['id']}.jpg"]
|
||||
for path in paths:
|
||||
try:
|
||||
path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
@app.delete("/media/api/tracks/{track_id}")
|
||||
def delete_track(track_id: str) -> dict:
|
||||
with _lock:
|
||||
track = _get_track(track_id)
|
||||
if not track:
|
||||
raise HTTPException(404, "unknown track")
|
||||
del _library["tracks"][track_id]
|
||||
_bump_version()
|
||||
_save_library()
|
||||
for room, st in _rooms.items():
|
||||
if st["track_id"] == track_id:
|
||||
st["track_id"] = None
|
||||
st["desired"] = "stopped"
|
||||
_delete_track_files(track)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
# ── API: search + play + control ────────────────────────────────────────────
|
||||
|
||||
def _library_search(query: str, limit: int) -> list[dict]:
|
||||
"""Ready library tracks whose title matches the query.
|
||||
|
||||
These are merged in front of the YouTube results so Hope prefers an
|
||||
already-downloaded copy over fetching a new one.
|
||||
"""
|
||||
q = query.strip().lower()
|
||||
tokens = {t for t in re.split(r"\W+", q) if len(t) > 2}
|
||||
hits: list[dict] = []
|
||||
if not q:
|
||||
return hits
|
||||
for track in _library["tracks"].values():
|
||||
if track.get("kind") != "youtube" or track.get("status") != "ready":
|
||||
continue
|
||||
title = (track.get("title") or "").strip().lower()
|
||||
if not title:
|
||||
continue
|
||||
title_tokens = {t for t in re.split(r"\W+", title) if t}
|
||||
if not (q in title or title in q or (tokens and tokens <= title_tokens)):
|
||||
continue
|
||||
hits.append({
|
||||
"video_id": track["id"],
|
||||
"track_id": track["id"],
|
||||
"title": track.get("title") or track["id"],
|
||||
"channel": track.get("channel") or "",
|
||||
"duration": track.get("duration"),
|
||||
"view_count": None,
|
||||
"in_library": True,
|
||||
"library_status": "ready",
|
||||
})
|
||||
return hits[:limit]
|
||||
|
||||
|
||||
@app.post("/media/api/search")
|
||||
async def search(request: Request) -> dict:
|
||||
body = await request.json()
|
||||
query = str(body.get("query", "")).strip()
|
||||
if not query:
|
||||
raise HTTPException(400, "query is required")
|
||||
try:
|
||||
limit = max(1, min(int(body.get("limit", 6)), 15))
|
||||
except (TypeError, ValueError):
|
||||
raise HTTPException(400, "limit must be an integer")
|
||||
local = _library_search(query, limit)
|
||||
remote = _run_search(query, limit)
|
||||
seen = {r["video_id"] for r in local}
|
||||
merged = local + [r for r in remote if r["video_id"] not in seen]
|
||||
return {"query": query, "results": merged[:limit]}
|
||||
|
||||
|
||||
def _resolve_play_target(body: dict) -> dict:
|
||||
"""Find or create the track a play request refers to.
|
||||
|
||||
Accepts video_id (YouTube), track_id (existing library entry) or a raw
|
||||
url+name (live stream). Returns the library track dict.
|
||||
"""
|
||||
video_id = str(body.get("video_id") or "").strip()
|
||||
track_id = str(body.get("track_id") or "").strip()
|
||||
url = str(body.get("url") or "").strip()
|
||||
|
||||
if video_id:
|
||||
existing = _get_track(video_id)
|
||||
if existing:
|
||||
return existing
|
||||
track = {
|
||||
"id": video_id,
|
||||
"kind": "youtube",
|
||||
"video_id": video_id,
|
||||
"title": str(body.get("title") or video_id),
|
||||
"channel": str(body.get("channel") or ""),
|
||||
"duration": body.get("duration"),
|
||||
"status": "downloading",
|
||||
"error": None,
|
||||
"added_at": time.time(),
|
||||
}
|
||||
_upsert_track(track)
|
||||
_start_download(track, video_id)
|
||||
return track
|
||||
|
||||
if track_id:
|
||||
track = _get_track(track_id)
|
||||
if not track:
|
||||
raise HTTPException(404, "unknown track")
|
||||
return track
|
||||
|
||||
if url:
|
||||
if not re.match(r"^https?://", url):
|
||||
raise HTTPException(400, "url must be http(s)")
|
||||
track_id = "s" + uuid.uuid4().hex[:10]
|
||||
track = {
|
||||
"id": track_id,
|
||||
"kind": "stream",
|
||||
"video_id": None,
|
||||
"title": str(body.get("name") or url),
|
||||
"channel": str(body.get("channel") or ""),
|
||||
"duration": None,
|
||||
"status": "ready",
|
||||
"error": None,
|
||||
"url": url,
|
||||
"added_at": time.time(),
|
||||
}
|
||||
_upsert_track(track)
|
||||
return track
|
||||
|
||||
raise HTTPException(400, "one of video_id, track_id or url is required")
|
||||
|
||||
|
||||
@app.post("/media/api/play")
|
||||
async def play(request: Request) -> dict:
|
||||
body = await request.json()
|
||||
room = str(body.get("room") or "default")
|
||||
track = _resolve_play_target(body)
|
||||
with _lock:
|
||||
st = _room_state(room)
|
||||
st["track_id"] = track["id"]
|
||||
# start_paused carries a track into a new scope without starting it —
|
||||
# the UI uses this when a conversation begins (talking pauses music).
|
||||
st["desired"] = "paused" if body.get("start_paused") else "playing"
|
||||
st["choices"] = [] # a pick resolves any open suggestion list
|
||||
st["seek_hint"] = None
|
||||
snapshot = _room_snapshot(room)
|
||||
return {"ok": True, **snapshot}
|
||||
|
||||
|
||||
@app.post("/media/api/control")
|
||||
async def control(request: Request) -> dict:
|
||||
body = await request.json()
|
||||
room = str(body.get("room") or "default")
|
||||
action = str(body.get("action") or "")
|
||||
with _lock:
|
||||
st = _room_state(room)
|
||||
if action == "pause":
|
||||
st["desired"] = "paused"
|
||||
elif action == "resume":
|
||||
if not st["track_id"]:
|
||||
raise HTTPException(409, "nothing loaded")
|
||||
st["desired"] = "playing"
|
||||
elif action == "stop":
|
||||
st["desired"] = "stopped"
|
||||
elif action == "seek":
|
||||
try:
|
||||
position = max(0.0, float(body.get("position", 0)))
|
||||
except (TypeError, ValueError):
|
||||
raise HTTPException(400, "position must be a number")
|
||||
st["seek_hint"] = {"position": position, "ts": time.time()}
|
||||
elif action == "skip":
|
||||
current = st["track_id"]
|
||||
candidates = sorted(
|
||||
(
|
||||
t for t in _library["tracks"].values()
|
||||
if t.get("status") == "ready" and t["id"] != current
|
||||
),
|
||||
key=lambda t: t.get("added_at", 0),
|
||||
reverse=True,
|
||||
)
|
||||
if candidates:
|
||||
st["track_id"] = candidates[0]["id"]
|
||||
st["desired"] = "playing"
|
||||
st["seek_hint"] = None
|
||||
else:
|
||||
st["desired"] = "stopped"
|
||||
else:
|
||||
raise HTTPException(400, f"unknown action: {action}")
|
||||
snapshot = _room_snapshot(room)
|
||||
return {"ok": True, **snapshot}
|
||||
|
||||
|
||||
@app.post("/media/api/choices")
|
||||
async def choices(request: Request) -> dict:
|
||||
"""Publish Hope's song suggestions so the UI can render clickable picks."""
|
||||
body = await request.json()
|
||||
room = str(body.get("room") or "default")
|
||||
raw = body.get("choices") or []
|
||||
cleaned = []
|
||||
for c in raw[:5]:
|
||||
if not isinstance(c, dict) or not c.get("video_id"):
|
||||
continue
|
||||
cleaned.append({
|
||||
"video_id": str(c["video_id"]),
|
||||
"title": str(c.get("title") or c["video_id"]),
|
||||
"channel": str(c.get("channel") or ""),
|
||||
"duration": c.get("duration"),
|
||||
})
|
||||
with _lock:
|
||||
_room_state(room)["choices"] = cleaned
|
||||
return {"ok": True, "choices": cleaned}
|
||||
|
||||
|
||||
@app.get("/media/api/state")
|
||||
def state(room: str = "default") -> dict:
|
||||
with _lock:
|
||||
return _room_snapshot(room)
|
||||
|
||||
|
||||
@app.post("/media/api/report")
|
||||
async def report(request: Request) -> dict:
|
||||
"""The browser reports real playback state (source of truth for the player)."""
|
||||
body = await request.json()
|
||||
room = str(body.get("room") or "default")
|
||||
try:
|
||||
position = float(body.get("position") or 0.0)
|
||||
except (TypeError, ValueError):
|
||||
position = 0.0
|
||||
with _lock:
|
||||
st = _room_state(room)
|
||||
st["reported"] = {
|
||||
"track_id": body.get("track_id"),
|
||||
"state": str(body.get("state") or "stopped"),
|
||||
"position": position,
|
||||
"duration": body.get("duration"),
|
||||
"ts": time.time(),
|
||||
}
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@app.post("/media/api/streams")
|
||||
async def add_stream(request: Request) -> dict:
|
||||
body = await request.json()
|
||||
track = _resolve_play_target({"url": body.get("url"), "name": body.get("name")})
|
||||
return {"ok": True, "track": _track_dict(track)}
|
||||
|
||||
|
||||
# ── API: stream proxy (external http(s) audio over same-origin HTTPS) ───────
|
||||
|
||||
_HOP_HEADERS = ("content-type", "content-range", "accept-ranges", "content-length")
|
||||
|
||||
|
||||
@app.get("/media/api/stream/{track_id}")
|
||||
async def stream_proxy(track_id: str, request: Request):
|
||||
with _lock:
|
||||
track = _get_track(track_id)
|
||||
job = next(
|
||||
(j for j in _jobs.values()
|
||||
if j["track_id"] == track_id and j["stage"] == "downloading"),
|
||||
None,
|
||||
)
|
||||
if not track:
|
||||
raise HTTPException(404, "unknown track")
|
||||
|
||||
if track["kind"] == "youtube":
|
||||
# Progressive download: an active download is streamed in real time
|
||||
# (chunked, no Content-Length) so playback starts while the file
|
||||
# grows; a finished file is served directly with native Range support.
|
||||
path = _youtube_media_file(track_id)
|
||||
if path is None and job is not None:
|
||||
path = await _wait_for_media_file(track_id)
|
||||
if path is None:
|
||||
raise HTTPException(
|
||||
404, f"audio file not available (track is {track.get('status')})"
|
||||
)
|
||||
if job is not None:
|
||||
return StreamingResponse(
|
||||
_growing_file(track_id, job),
|
||||
media_type=_media_type_for(path.name),
|
||||
headers={"Cache-Control": "no-store"},
|
||||
)
|
||||
return FileResponse(path, media_type=_media_type_for(path.name))
|
||||
|
||||
range_header = request.headers.get("range")
|
||||
headers = {"Range": range_header} if range_header else {}
|
||||
client = httpx.AsyncClient(timeout=httpx.Timeout(30, read=600), follow_redirects=True)
|
||||
try:
|
||||
upstream = await client.send(client.build_request("GET", track["url"], headers=headers), stream=True)
|
||||
except httpx.HTTPError as exc:
|
||||
await client.aclose()
|
||||
raise HTTPException(502, f"upstream stream failed: {exc}")
|
||||
|
||||
if upstream.status_code >= 400:
|
||||
await upstream.aclose()
|
||||
await client.aclose()
|
||||
raise HTTPException(502, f"upstream returned {upstream.status_code}")
|
||||
|
||||
out_headers = {
|
||||
name: value
|
||||
for name, value in upstream.headers.items()
|
||||
if name.lower() in _HOP_HEADERS
|
||||
}
|
||||
media_type = upstream.headers.get("content-type", "audio/mpeg")
|
||||
|
||||
async def relay():
|
||||
try:
|
||||
async for chunk in upstream.aiter_bytes(64 * 1024):
|
||||
yield chunk
|
||||
finally:
|
||||
await upstream.aclose()
|
||||
await client.aclose()
|
||||
|
||||
return StreamingResponse(
|
||||
relay(),
|
||||
status_code=upstream.status_code,
|
||||
media_type=media_type,
|
||||
headers=out_headers,
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(HTTPException)
|
||||
async def http_error(_request: Request, exc: HTTPException):
|
||||
return JSONResponse({"error": exc.detail}, status_code=exc.status_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
uvicorn.run(app, host=MEDIA_HOST, port=MEDIA_PORT, log_level="info")
|
||||
+1
-300
@@ -371,34 +371,6 @@ button {
|
||||
transition: opacity 90ms linear, width 90ms linear, height 90ms linear;
|
||||
}
|
||||
|
||||
/* Album art fills the square while a track is loaded outside a conversation. */
|
||||
.led-art {
|
||||
position: absolute;
|
||||
inset: 12px;
|
||||
width: calc(100% - 24px);
|
||||
height: calc(100% - 24px);
|
||||
object-fit: cover;
|
||||
border-radius: 18px;
|
||||
z-index: 1;
|
||||
opacity: 0;
|
||||
transform: scale(0.96);
|
||||
transition: opacity 240ms ease, transform 240ms ease;
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.led-wall.show-art .led-art {
|
||||
opacity: 1;
|
||||
transform: scale(1);
|
||||
}
|
||||
|
||||
.led-wall.show-art .led-dot {
|
||||
opacity: 0 !important;
|
||||
transition: none;
|
||||
}
|
||||
|
||||
.led-wall.show-art {
|
||||
filter: none;
|
||||
}
|
||||
|
||||
/* Palette shifts by state — a cool blue while it listens, warm toward the
|
||||
center as the agent speaks, amber while thinking, dim when idle. */
|
||||
@@ -1201,276 +1173,6 @@ body[data-mode="text"] .mic-meter {
|
||||
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
|
||||
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
|
||||
|
||||
/* ── Media player ───────────────────────────────────────────────────────── */
|
||||
/* Reuses .tasks-panel positioning (fixed right, slides in via .open); the
|
||||
content scrolls under a sticky header. */
|
||||
.media-panel {
|
||||
overflow-y: auto;
|
||||
padding-bottom: 1.2rem;
|
||||
}
|
||||
|
||||
.media-panel .tasks-header {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 1;
|
||||
background: #0d1929;
|
||||
}
|
||||
|
||||
.media-now {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.75rem;
|
||||
margin: 1rem 1.1rem 0;
|
||||
padding: 0.75rem;
|
||||
border-radius: 12px;
|
||||
background: var(--surface-raised);
|
||||
}
|
||||
|
||||
.media-art,
|
||||
.media-art-placeholder {
|
||||
width: 56px;
|
||||
height: 56px;
|
||||
flex: 0 0 56px;
|
||||
border-radius: 8px;
|
||||
object-fit: cover;
|
||||
background: rgba(23, 41, 65, 0.7);
|
||||
}
|
||||
|
||||
.media-art-placeholder {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.media-now-info {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.15rem;
|
||||
}
|
||||
|
||||
.media-now-info strong {
|
||||
color: var(--ink);
|
||||
font-size: 0.88rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.media-now-info span {
|
||||
color: var(--ink-muted);
|
||||
font-size: 0.72rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.media-progress {
|
||||
position: relative;
|
||||
margin-top: 0.35rem;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
/* Track bar behind the fill (fill width is driven inline by progress). */
|
||||
.media-progress::before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
left: 0;
|
||||
top: 50%;
|
||||
transform: translateY(-50%);
|
||||
width: 90px;
|
||||
height: 4px;
|
||||
border-radius: 2px;
|
||||
background: rgba(139, 167, 255, 0.18);
|
||||
}
|
||||
|
||||
.media-progress-fill {
|
||||
height: 4px;
|
||||
flex: 0 0 auto;
|
||||
width: 0;
|
||||
border-radius: 2px;
|
||||
background: var(--iris);
|
||||
transition: width 400ms ease;
|
||||
}
|
||||
|
||||
.media-progress-label {
|
||||
color: var(--ink-muted);
|
||||
font-size: 0.66rem;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.media-controls {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 0.9rem;
|
||||
margin: 0.9rem 1.1rem 0;
|
||||
}
|
||||
|
||||
.media-controls .btn-icon {
|
||||
color: var(--ink-soft);
|
||||
}
|
||||
|
||||
.media-play {
|
||||
width: 52px;
|
||||
height: 52px;
|
||||
border-radius: 50%;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: #0b1523 !important;
|
||||
background: var(--iris) !important;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.media-play .play-icon,
|
||||
.media-play .pause-icon {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.media-play:not(.playing) .play-icon {
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.media-play.playing .pause-icon {
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.media-play:hover:not(:disabled) {
|
||||
filter: brightness(1.1);
|
||||
}
|
||||
|
||||
.media-seek {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.55rem;
|
||||
margin: 0.75rem 1.1rem 0;
|
||||
}
|
||||
|
||||
.media-time {
|
||||
color: var(--ink-muted);
|
||||
font-size: 0.68rem;
|
||||
font-variant-numeric: tabular-nums;
|
||||
min-width: 2.6rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.media-seek input[type="range"],
|
||||
.media-volume input[type="range"] {
|
||||
flex: 1;
|
||||
accent-color: var(--iris);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.media-volume {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.55rem;
|
||||
margin: 0.55rem 1.1rem 0;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.media-search {
|
||||
display: flex;
|
||||
gap: 0.45rem;
|
||||
margin: 0 1.1rem;
|
||||
}
|
||||
|
||||
.media-search input {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
padding: 0.5rem 0.7rem;
|
||||
border: 1px solid var(--line-strong);
|
||||
border-radius: 9px;
|
||||
color: var(--ink);
|
||||
background: rgba(5, 11, 20, 0.45);
|
||||
font-size: 0.8rem;
|
||||
}
|
||||
|
||||
.media-search input::placeholder {
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.media-stream-form input:first-child {
|
||||
flex: 0 0 38%;
|
||||
}
|
||||
|
||||
.media-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.4rem;
|
||||
margin: 0.65rem 1.1rem 1rem;
|
||||
max-height: 240px;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.media-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
padding: 0.55rem 0.65rem;
|
||||
border-radius: 9px;
|
||||
background: var(--surface-raised);
|
||||
}
|
||||
|
||||
.media-row-info {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.1rem;
|
||||
}
|
||||
|
||||
.media-row-info strong {
|
||||
color: var(--ink);
|
||||
font-size: 0.78rem;
|
||||
font-weight: 600;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.media-row-info span {
|
||||
color: var(--ink-muted);
|
||||
font-size: 0.68rem;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.media-row-play {
|
||||
flex: 0 0 auto;
|
||||
padding: 0.3rem 0.7rem;
|
||||
font-size: 0.7rem;
|
||||
}
|
||||
|
||||
.media-row-delete {
|
||||
flex: 0 0 auto;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.media-row-delete:hover {
|
||||
color: #ffc2ba;
|
||||
}
|
||||
|
||||
.media-empty {
|
||||
padding: 0.8rem;
|
||||
border-radius: 9px;
|
||||
color: var(--ink-muted);
|
||||
background: var(--surface-raised);
|
||||
font-size: 0.74rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.media-choice {
|
||||
border: 1px solid rgba(139, 167, 255, 0.35);
|
||||
background: rgba(60, 95, 201, 0.12);
|
||||
}
|
||||
|
||||
/* ── Responsive ─────────────────────────────────────────────────────────── */
|
||||
@media (max-height: 760px) and (min-width: 881px) {
|
||||
.workspace {
|
||||
@@ -1614,8 +1316,7 @@ body[data-mode="text"] .mic-meter {
|
||||
padding: 1.25rem 1rem;
|
||||
}
|
||||
|
||||
.tasks-panel,
|
||||
.media-panel {
|
||||
.tasks-panel {
|
||||
width: 100vw;
|
||||
}
|
||||
|
||||
|
||||
+2
-2
@@ -97,8 +97,8 @@ class Handler(BaseHTTPRequestHandler):
|
||||
resp = json.dumps(
|
||||
{
|
||||
"token": token,
|
||||
# The room name scopes media-server player state; the UI sends
|
||||
# it with every /media/api/ call.
|
||||
# Returned for diagnostics and reconnects. Music deliberately
|
||||
# uses its own stable scope in the separate Hope Music service.
|
||||
"room": room_name,
|
||||
"user": {
|
||||
"id": user_id,
|
||||
|
||||
Reference in new issue
Block a user