Compare commits
4
Commits
a4ae96f01b
...
48b92585c6
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
48b92585c6 | ||
|
|
f534b8ccb5 | ||
|
|
a53c090882 | ||
|
|
04c4a53594 |
No files matched your search
@@ -29,6 +29,19 @@ FIRECRAWL_BASE=http://192.168.86.2:3002
|
|||||||
# Per-user memory profiles and Bearer tokens live in config/users.json.
|
# Per-user memory profiles and Bearer tokens live in config/users.json.
|
||||||
VOICE_DEFAULT_USER=shane
|
VOICE_DEFAULT_USER=shane
|
||||||
|
|
||||||
|
# ── Media player (music library + YouTube Music downloads) ─────────────────
|
||||||
|
# The permanent library lives in ./media (bind-mounted to /data/media):
|
||||||
|
# media/files/ transcoded mp3 tracks
|
||||||
|
# media/thumbs/ cover art
|
||||||
|
# media/jobs/ download job status
|
||||||
|
# MEDIA_SERVER_PORT=8092
|
||||||
|
#
|
||||||
|
# Optional: export cookies.txt from a browser logged into YouTube and drop it
|
||||||
|
# in ./media/cookies.txt to avoid "confirm you're not a bot" download blocks.
|
||||||
|
# Anonymous access works from most residential IPs; cookies are the reliable
|
||||||
|
# path.
|
||||||
|
# YTDLP_COOKIES=/data/media/cookies.txt
|
||||||
|
|
||||||
# Extra MCP servers (optional). JSON list of {url, transport} objects.
|
# Extra MCP servers (optional). JSON list of {url, transport} objects.
|
||||||
# transport: "sse" | "streamable_http" | omit for auto-detect
|
# transport: "sse" | "streamable_http" | omit for auto-detect
|
||||||
# EXTRA_MCP_SERVERS=[{"url":"http://192.168.86.2:3001/mcp","transport":"streamable_http"}]
|
# EXTRA_MCP_SERVERS=[{"url":"http://192.168.86.2:3001/mcp","transport":"streamable_http"}]
|
||||||
@@ -8,3 +8,7 @@ certs/
|
|||||||
.venv-tests/
|
.venv-tests/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
.pytest_cache/
|
.pytest_cache/
|
||||||
|
|
||||||
|
# Permanent music library (media server runtime data)
|
||||||
|
/media/*
|
||||||
|
!/media/.gitkeep
|
||||||
@@ -4,17 +4,21 @@ A single-container voice assistant built on LiveKit Agents. Speaks and listens i
|
|||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
One Docker container runs three processes via supervisord:
|
One Docker container runs four processes via supervisord:
|
||||||
|
|
||||||
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
|
||||||
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||||
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
3. **Web frontend** — static HTML served by nginx (port 9178 by default)
|
||||||
|
4. **Media server** — FastAPI music library/player coordinator (port 8092 internal, proxied at /media/api/)
|
||||||
|
|
||||||
```
|
```
|
||||||
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
Browser ──WebRTC──► LiveKit Server ──audio──► Agent
|
||||||
▲ │
|
▲ │
|
||||||
└────────── audio ◄────────────────────────────┘
|
└────────── audio ◄────────────────────────────┘
|
||||||
Azure STT → LLM (OpenWebUI) → Azure TTS
|
Azure STT → LLM (OpenWebUI) → Azure TTS
|
||||||
|
|
||||||
|
Browser ──HTTP──► nginx /media/api/ ──► Media server ──► yt-dlp/ffmpeg ──► ./media
|
||||||
|
└────── audio element ◄── /media/files/ (nginx static, Range) ──────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
@@ -40,6 +44,7 @@ LAN access requires firewall rules: `9178/tcp` (UI + signaling), `7882/udp`
|
|||||||
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
| 7880 | TCP | LiveKit HTTP/WS | internal only (proxied via nginx at /livekit/) |
|
||||||
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
| 7881 | TCP | LiveKit RTC media (TCP fallback) | LAN |
|
||||||
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
| 7882 | UDP | LiveKit RTC media (muxed) | LAN |
|
||||||
|
| 8092 | TCP | Media server (library, player state, stream proxy) | internal only (proxied via nginx at /media/api/) |
|
||||||
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
| 9178 | TCP | Web frontend (HTTPS) | LAN |
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
@@ -53,6 +58,33 @@ Key variables:
|
|||||||
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
|
||||||
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
- `LLM_MODEL` — default model (default `qwen2.5-vl-3b`; switchable live from the web UI)
|
||||||
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
|
||||||
|
- `YTDLP_COOKIES` — optional path to a cookies.txt inside the container (e.g. `/data/media/cookies.txt`) for reliable YouTube downloads
|
||||||
|
|
||||||
|
## Media Player
|
||||||
|
|
||||||
|
Hope can play music on the web UI's media player: YouTube Music search →
|
||||||
|
yt-dlp download → ffmpeg transcode to mp3 → nginx static streaming (HTTP
|
||||||
|
Range) back to the browser's `<audio>` element. Live audio stream URLs
|
||||||
|
(internet radio) are added to the library and proxied over HTTPS by the
|
||||||
|
media server (mixed-content safe).
|
||||||
|
|
||||||
|
- The permanent library lives in `./media` (bind-mounted to `/data/media`):
|
||||||
|
`files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
|
||||||
|
- `web/media_server.py` owns the library, search, download jobs (one at a
|
||||||
|
time), and per-room player desired state. The browser is the actual player:
|
||||||
|
it polls `GET /media/api/state?room=…`, reconciles, and reports real
|
||||||
|
playback back via `POST /media/api/report`.
|
||||||
|
- Hope's tools live in `agent/media_mcp.py` (MCP, id "media"):
|
||||||
|
`search_music`, `play_song`, `offer_song_choices`, `pause_music`,
|
||||||
|
`resume_music`, `stop_music`, `skip_song`, `seek_music`, `now_playing`,
|
||||||
|
`list_library`, `add_stream`. The MCP process is spawned per session with
|
||||||
|
`MEDIA_ROOM` set to the LiveKit room name, scoping player state per room.
|
||||||
|
- **Music and conversation are mutually exclusive.** When a track plays, the
|
||||||
|
UI reports `{type: "media_state", state: "playing"}` on data-channel topic
|
||||||
|
"media-control" and the agent disables its mic (`_apply_listening` in
|
||||||
|
agent.py). Engaging Hope (Start conversation / text message) pauses the
|
||||||
|
music; ending the conversation resumes what was playing. Voice commands
|
||||||
|
(pause/skip/…) work whenever the conversation is active.
|
||||||
|
|
||||||
## Voice Selection
|
## Voice Selection
|
||||||
|
|
||||||
@@ -104,16 +136,24 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
├── Dockerfile ← multi-stage build
|
├── Dockerfile ← multi-stage build
|
||||||
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
├── entrypoint.sh ← regenerates self-signed cert with LAN IP at start
|
||||||
├── livekit.yaml ← LiveKit server config
|
├── livekit.yaml ← LiveKit server config
|
||||||
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy
|
├── nginx.conf ← HTTPS UI + /livekit/ WS proxy + /media/ (static + API)
|
||||||
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server)
|
├── supervisord.conf ← process manager (livekit, agent, nginx, token-server, media-server)
|
||||||
├── certs/ ← persisted self-signed cert (volume mount)
|
├── certs/ ← persisted self-signed cert (volume mount)
|
||||||
|
├── media/ ← permanent music library (bind-mounted to /data/media)
|
||||||
|
│ ├── files/ ← transcoded mp3 tracks (nginx-served, HTTP Range)
|
||||||
|
│ ├── thumbs/ ← cover art
|
||||||
|
│ ├── jobs/ ← download job status snapshots
|
||||||
|
│ └── library.json ← library metadata (permanent)
|
||||||
├── agent/
|
├── agent/
|
||||||
│ ├── agent.py ← LiveKit Agents voice pipeline
|
│ ├── agent.py ← LiveKit Agents voice pipeline
|
||||||
|
│ ├── media_mcp.py ← Hope's music tools (YouTube Music via media server)
|
||||||
│ ├── pyproject.toml ← Python deps (uv)
|
│ ├── pyproject.toml ← Python deps (uv)
|
||||||
│ └── uv.lock ← locked dependency versions
|
│ └── uv.lock ← locked dependency versions
|
||||||
└── web/
|
└── web/
|
||||||
├── index.html ← single-page voice UI
|
├── index.html ← single-page voice UI (+ music panel)
|
||||||
├── app.js ← LiveKit client logic
|
├── app.js ← LiveKit client logic (+ player reconciliation)
|
||||||
|
├── media_server.py ← FastAPI library/search/download/player state
|
||||||
|
├── token_server.py ← LiveKit token endpoint (mints room name)
|
||||||
├── manifest.json ← PWA manifest
|
├── manifest.json ← PWA manifest
|
||||||
├── favicon.svg ← site icon
|
├── favicon.svg ← site icon
|
||||||
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
├── livekit-client.umd.js ← vendored LiveKit JS SDK (no CDN)
|
||||||
@@ -135,6 +175,10 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
|||||||
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
||||||
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
||||||
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
||||||
|
- **Music downloads are mp3, not YouTube streams.** The media server transcodes with yt-dlp (`-x --audio-format mp3 --audio-quality 192K --embed-thumbnail`) into `/data/media/files/{video_id}.mp3`; nginx serves those files as statics with native HTTP Range, which is what makes seeking work in the `<audio>` element. Never point the player at YouTube directly.
|
||||||
|
- **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`.
|
||||||
|
- **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`.
|
||||||
|
- **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.
|
||||||
|
|
||||||
## Git
|
## Git
|
||||||
|
|
||||||
|
|||||||
+9
-3
@@ -24,9 +24,10 @@ ENV PYTHONUNBUFFERED=1
|
|||||||
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
|
# Fix SSL cert path for the slim base image (ca-certificates installs to /etc/ssl/certs)
|
||||||
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
|
||||||
|
|
||||||
# Install LiveKit server binary + supervisord + nginx
|
# Install LiveKit server binary + supervisord + nginx + ffmpeg (yt-dlp audio
|
||||||
|
# extraction/transcoding for the media player)
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
curl ca-certificates supervisor nginx libasound2 iproute2 \
|
curl ca-certificates supervisor nginx libasound2 iproute2 ffmpeg \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Download LiveKit server (latest stable)
|
# Download LiveKit server (latest stable)
|
||||||
@@ -48,8 +49,9 @@ COPY agent/task_registry.py /opt/voice-agent/task_registry.py
|
|||||||
COPY agent/task_router.py /opt/voice-agent/task_router.py
|
COPY agent/task_router.py /opt/voice-agent/task_router.py
|
||||||
COPY agent/task_worker.py /opt/voice-agent/task_worker.py
|
COPY agent/task_worker.py /opt/voice-agent/task_worker.py
|
||||||
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
|
COPY agent/dispatch_mcp.py /opt/voice-agent/dispatch_mcp.py
|
||||||
|
COPY agent/media_mcp.py /opt/voice-agent/media_mcp.py
|
||||||
|
|
||||||
# Copy web frontend + token endpoint
|
# Copy web frontend + token endpoint + media server
|
||||||
COPY web/index.html /var/www/voice/
|
COPY web/index.html /var/www/voice/
|
||||||
COPY web/app.js /var/www/voice/
|
COPY web/app.js /var/www/voice/
|
||||||
COPY web/style.css /var/www/voice/
|
COPY web/style.css /var/www/voice/
|
||||||
@@ -57,6 +59,10 @@ COPY web/livekit-client.umd.js /var/www/voice/
|
|||||||
COPY web/manifest.json /var/www/voice/
|
COPY web/manifest.json /var/www/voice/
|
||||||
COPY web/favicon.svg /var/www/voice/
|
COPY web/favicon.svg /var/www/voice/
|
||||||
COPY web/token_server.py /opt/voice/token_server.py
|
COPY web/token_server.py /opt/voice/token_server.py
|
||||||
|
COPY web/media_server.py /opt/voice/media_server.py
|
||||||
|
|
||||||
|
# Permanent media library mount point (bind-mounted from ./media)
|
||||||
|
RUN mkdir -p /data/media/files /data/media/thumbs /data/media/jobs
|
||||||
|
|
||||||
# Config files
|
# Config files
|
||||||
COPY livekit.yaml /etc/livekit.yaml
|
COPY livekit.yaml /etc/livekit.yaml
|
||||||
|
|||||||
+64
-2
@@ -196,6 +196,23 @@ SYSTEM_PROMPT = textwrap.dedent("""\
|
|||||||
you receive it, share the findings naturally in your conversational style.
|
you receive it, share the findings naturally in your conversational style.
|
||||||
You can have multiple tasks running at once.
|
You can have multiple tasks running at once.
|
||||||
|
|
||||||
|
# Music (media player in the web UI)
|
||||||
|
You can play music on the user's web player. For any music request, call
|
||||||
|
search_music first, then:
|
||||||
|
- One clearly-right result → play_song with its video_id. Announce it in
|
||||||
|
one short sentence, then stop talking — playback pauses the
|
||||||
|
conversation until the user comes back to you.
|
||||||
|
- Ambiguous results → offer_song_choices with the top 2-3 options, name
|
||||||
|
them briefly, and let the user pick by voice or in the player panel.
|
||||||
|
Never start playback on a guess.
|
||||||
|
The library is permanent: list_library shows what is already downloaded,
|
||||||
|
and those tracks play instantly without searching. If the user names a
|
||||||
|
library track exactly, just play it. Control playback with pause_music,
|
||||||
|
resume_music, skip_song and seek_music; check what's on with now_playing.
|
||||||
|
add_stream registers a live radio or direct audio URL as a library track.
|
||||||
|
While music is playing you cannot hear the user — they will come back to
|
||||||
|
you when they want to talk again, and the music pauses automatically.
|
||||||
|
|
||||||
# Skills
|
# Skills
|
||||||
You have a skill library of learned procedures. Use skill_recall when you need
|
You have a skill library of learned procedures. Use skill_recall when you need
|
||||||
to perform a task you've done before — it will give you the steps. When you
|
to perform a task you've done before — it will give you the steps. When you
|
||||||
@@ -624,6 +641,27 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
|||||||
)
|
)
|
||||||
logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
|
logger.info("Dispatch MCP toolset enabled (%s)", task_dir)
|
||||||
|
|
||||||
|
# Media tools talk to the local media server; the room name scopes player
|
||||||
|
# state so concurrent sessions don't fight over one virtual player.
|
||||||
|
media_server_url = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8092")
|
||||||
|
media_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media_mcp.py")
|
||||||
|
toolsets.append(
|
||||||
|
mcp.MCPToolset(
|
||||||
|
id="media",
|
||||||
|
mcp_server=mcp.MCPServerStdio(
|
||||||
|
command=python_bin,
|
||||||
|
args=[media_mcp_script],
|
||||||
|
env={
|
||||||
|
**os.environ,
|
||||||
|
"MEDIA_SERVER_URL": media_server_url,
|
||||||
|
"MEDIA_ROOM": os.path.basename(task_dir.rstrip("/")) or "default",
|
||||||
|
},
|
||||||
|
client_session_timeout_seconds=60,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
logger.info("Media MCP toolset enabled (%s)", media_server_url)
|
||||||
|
|
||||||
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
|
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
|
||||||
if extra:
|
if extra:
|
||||||
try:
|
try:
|
||||||
@@ -740,9 +778,22 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
|
|
||||||
# Listen for conversation controls (voice selection and voice/text mode).
|
# Listen for conversation controls (voice selection and voice/text mode).
|
||||||
# Defined here so it captures this job's session via closure.
|
# Defined here so it captures this job's session via closure.
|
||||||
|
# Music and conversation are mutually exclusive by design: while a track
|
||||||
|
# plays the agent's ears are off; re-engaging the conversation pauses the
|
||||||
|
# music, and ending the conversation lets the user resume it.
|
||||||
|
session_mode = {"value": "voice"} # UI voice/text switch
|
||||||
|
media_playback = {"value": "stopped"} # UI-reported player state
|
||||||
|
|
||||||
|
def _apply_listening() -> None:
|
||||||
|
listening = (
|
||||||
|
session_mode["value"] == "voice"
|
||||||
|
and media_playback["value"] != "playing"
|
||||||
|
)
|
||||||
|
session.input.set_audio_enabled(listening)
|
||||||
|
|
||||||
def _on_room_data(packet) -> None:
|
def _on_room_data(packet) -> None:
|
||||||
topic = getattr(packet, "topic", None)
|
topic = getattr(packet, "topic", None)
|
||||||
if topic and topic not in ("voice-control", "conversation-control"):
|
if topic and topic not in ("voice-control", "conversation-control", "media-control"):
|
||||||
return
|
return
|
||||||
try:
|
try:
|
||||||
msg = json.loads(packet.data.decode("utf-8"))
|
msg = json.loads(packet.data.decode("utf-8"))
|
||||||
@@ -755,11 +806,22 @@ async def handle_job(ctx: JobContext) -> None:
|
|||||||
return
|
return
|
||||||
voice_enabled = mode == "voice"
|
voice_enabled = mode == "voice"
|
||||||
logger.info("Switching conversation mode to %s", mode)
|
logger.info("Switching conversation mode to %s", mode)
|
||||||
|
session_mode["value"] = mode
|
||||||
if not voice_enabled:
|
if not voice_enabled:
|
||||||
# interrupt() returns an already-scheduled Future, not a coroutine.
|
# interrupt() returns an already-scheduled Future, not a coroutine.
|
||||||
session.interrupt()
|
session.interrupt()
|
||||||
session.input.set_audio_enabled(voice_enabled)
|
|
||||||
session.output.set_audio_enabled(voice_enabled)
|
session.output.set_audio_enabled(voice_enabled)
|
||||||
|
_apply_listening()
|
||||||
|
return
|
||||||
|
|
||||||
|
if msg.get("type") == "media_state":
|
||||||
|
state = msg.get("state")
|
||||||
|
if state not in ("playing", "paused", "stopped"):
|
||||||
|
return
|
||||||
|
logger.info("Media playback -> %s (agent listening: %s)",
|
||||||
|
state, session_mode["value"] == "voice" and state != "playing")
|
||||||
|
media_playback["value"] = state
|
||||||
|
_apply_listening()
|
||||||
return
|
return
|
||||||
|
|
||||||
if msg.get("type") == "set_voice":
|
if msg.get("type") == "set_voice":
|
||||||
|
|||||||
@@ -0,0 +1,229 @@
|
|||||||
|
"""Media MCP server — lets Hope find and control music by voice.
|
||||||
|
|
||||||
|
Tools talk to the local media server (web/media_server.py) over HTTP, which
|
||||||
|
owns the library, YouTube Music search/download jobs and per-room player
|
||||||
|
state. The browser is the actual player: it reconciles against the media
|
||||||
|
server and reports playback back, so voice controls and the web panel stay
|
||||||
|
in sync.
|
||||||
|
|
||||||
|
Per-session environment (set by the agent when spawning this MCP):
|
||||||
|
- MEDIA_SERVER_URL e.g. http://127.0.0.1:8092
|
||||||
|
- MEDIA_ROOM LiveKit room name this session controls
|
||||||
|
|
||||||
|
The "stop listening while music plays" rule lives in the agent itself: the
|
||||||
|
UI reports media_state over the data channel and the agent disables its mic
|
||||||
|
while a track is playing.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from mcp.server.fastmcp import FastMCP
|
||||||
|
|
||||||
|
MEDIA_SERVER_URL = os.environ.get("MEDIA_SERVER_URL", "http://127.0.0.1:8092").rstrip("/")
|
||||||
|
MEDIA_ROOM = os.environ.get("MEDIA_ROOM", "default")
|
||||||
|
|
||||||
|
mcp = FastMCP("media")
|
||||||
|
|
||||||
|
|
||||||
|
def _media_get(path: str, params: dict | None = None) -> dict:
|
||||||
|
resp = httpx.get(f"{MEDIA_SERVER_URL}{path}", params=params, timeout=10)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _media_post(path: str, body: dict) -> dict:
|
||||||
|
resp = httpx.post(f"{MEDIA_SERVER_URL}{path}", json=body, timeout=30)
|
||||||
|
if resp.status_code >= 400:
|
||||||
|
try:
|
||||||
|
detail = resp.json().get("error", resp.text)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
detail = resp.text
|
||||||
|
return {"error": f"{resp.status_code}: {detail}"}
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _control(action: str, **extra) -> str:
|
||||||
|
data = _media_post("/media/api/control", {"room": MEDIA_ROOM, "action": action, **extra})
|
||||||
|
if "error" in data:
|
||||||
|
return f"Could not {action} the music: {data['error']}"
|
||||||
|
track = data.get("track") or {}
|
||||||
|
desired = data.get("desired", "")
|
||||||
|
name = track.get("title") or "nothing"
|
||||||
|
return f"OK — {action} applied. Loaded track: {name} (player state: {desired})."
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_duration(seconds) -> str:
|
||||||
|
try:
|
||||||
|
seconds = int(seconds or 0)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return ""
|
||||||
|
if seconds <= 0:
|
||||||
|
return ""
|
||||||
|
minutes, secs = divmod(seconds, 60)
|
||||||
|
hours, minutes = divmod(minutes, 60)
|
||||||
|
return f"{hours}:{minutes:02d}:{secs:02d}" if hours else f"{minutes}:{secs:02d}"
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def search_music(query: str, limit: int = 5) -> str:
|
||||||
|
"""Search YouTube Music for songs. Call this before playing anything the
|
||||||
|
user describes by name, artist or lyrics. Returns JSON results with
|
||||||
|
video_id, title, channel, duration and in_library flags. Pick the result
|
||||||
|
that clearly matches what the user asked for; if two or more could be
|
||||||
|
right, use offer_song_choices instead of guessing."""
|
||||||
|
data = _media_post("/media/api/search", {"query": query, "limit": max(1, min(limit, 10))})
|
||||||
|
if "error" in data:
|
||||||
|
return f"Search failed: {data['error']}"
|
||||||
|
results = data.get("results", [])
|
||||||
|
if not results:
|
||||||
|
return f"No YouTube Music results for '{query}'."
|
||||||
|
return json.dumps(results, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def play_song(video_id: str, title: str = "", channel: str = "") -> str:
|
||||||
|
"""Play a song in the user's web player. Pass the video_id from
|
||||||
|
search_music (or an in_library track_id). Downloads and transcodes first
|
||||||
|
if needed; playback starts automatically once ready. After calling this,
|
||||||
|
tell the user what is playing in one short sentence and stop talking —
|
||||||
|
the player takes over and the conversation pauses until they return."""
|
||||||
|
data = _media_post("/media/api/play", {
|
||||||
|
"room": MEDIA_ROOM, "video_id": video_id, "title": title, "channel": channel,
|
||||||
|
})
|
||||||
|
if "error" in data:
|
||||||
|
return f"Could not start playback: {data['error']}"
|
||||||
|
track = data.get("track") or {}
|
||||||
|
status = track.get("status")
|
||||||
|
name = track.get("title") or title or video_id
|
||||||
|
if status == "downloading":
|
||||||
|
return (
|
||||||
|
f"'{name}' is downloading and will start playing automatically when "
|
||||||
|
"ready (usually under a minute). Say it's on the way, then hand over "
|
||||||
|
"to the player."
|
||||||
|
)
|
||||||
|
return f"Playing '{name}' now. Hand over to the player with a short sign-off."
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def offer_song_choices(choices: list[dict]) -> str:
|
||||||
|
"""Offer the user 2-4 song options when a search is ambiguous. Pass a list
|
||||||
|
like [{"video_id": "...", "title": "...", "channel": "...", "duration": 213}].
|
||||||
|
The choices appear as clickable cards in the music panel; the user can
|
||||||
|
also just say which one they want. Briefly name the options aloud, then
|
||||||
|
wait for their pick."""
|
||||||
|
if not choices:
|
||||||
|
return "No choices provided."
|
||||||
|
data = _media_post("/media/api/choices", {"room": MEDIA_ROOM, "choices": choices})
|
||||||
|
if "error" in data:
|
||||||
|
return f"Could not present choices: {data['error']}"
|
||||||
|
names = [
|
||||||
|
f"{c.get('title')}" + (f" by {c['channel']}" if c.get("channel") else "")
|
||||||
|
for c in choices
|
||||||
|
]
|
||||||
|
return "Choices shown in the player: " + "; ".join(names) + ". Ask the user to pick one."
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def pause_music() -> str:
|
||||||
|
"""Pause the currently playing music. The user stays in the conversation
|
||||||
|
afterwards."""
|
||||||
|
return _control("pause")
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def resume_music() -> str:
|
||||||
|
"""Resume the loaded music. Warn the user you'll stop listening while it
|
||||||
|
plays — they'll need to come back to you to talk again."""
|
||||||
|
return _control("resume")
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def stop_music() -> str:
|
||||||
|
"""Stop playback completely and clear the loaded track."""
|
||||||
|
return _control("stop")
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def skip_song() -> str:
|
||||||
|
"""Skip to the next track in the library (most recently added first).
|
||||||
|
If the library has nothing else, playback stops."""
|
||||||
|
return _control("skip")
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def seek_music(delta_seconds: int) -> str:
|
||||||
|
"""Skip forward or backward in the current track. Positive delta_seconds
|
||||||
|
jumps ahead, negative goes back (e.g. -30 rewinds 30 seconds, 20 skips
|
||||||
|
20 seconds forward)."""
|
||||||
|
try:
|
||||||
|
reported = (_media_get("/media/api/state", {"room": MEDIA_ROOM}).get("reported") or {})
|
||||||
|
position = float(reported.get("position") or 0.0)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
position = 0.0
|
||||||
|
target = max(0.0, position + float(delta_seconds))
|
||||||
|
return _control("seek", position=target)
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def now_playing() -> str:
|
||||||
|
"""Return what is currently loaded/playing in the user's web player and
|
||||||
|
its playback position. Use when the user asks what's playing or what this
|
||||||
|
song is."""
|
||||||
|
try:
|
||||||
|
st = _media_get("/media/api/state", {"room": MEDIA_ROOM})
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return f"Media server unreachable: {exc}"
|
||||||
|
track = st.get("track") or {}
|
||||||
|
if not track:
|
||||||
|
return "Nothing is loaded in the player right now."
|
||||||
|
reported = st.get("reported") or {}
|
||||||
|
state = reported.get("state") or st.get("desired") or "unknown"
|
||||||
|
position = _fmt_duration(reported.get("position"))
|
||||||
|
duration = _fmt_duration(track.get("duration"))
|
||||||
|
bits = [f"'{track.get('title')}'"]
|
||||||
|
if track.get("channel"):
|
||||||
|
bits.append(f"by {track['channel']}")
|
||||||
|
if state == "playing" and position:
|
||||||
|
bits.append(f"at {position}" + (f" of {duration}" if duration else ""))
|
||||||
|
else:
|
||||||
|
bits.append(f"(playback state: {state})")
|
||||||
|
return "Currently loaded: " + " ".join(bits) + "."
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def list_library() -> str:
|
||||||
|
"""List the permanent music library: downloaded YouTube Music tracks and
|
||||||
|
added live streams, ready to play instantly. Use when the user asks what
|
||||||
|
music is available or wants something from their collection."""
|
||||||
|
data = _media_get("/media/api/library")
|
||||||
|
tracks = data.get("tracks", [])
|
||||||
|
if not tracks:
|
||||||
|
return "The library is empty — search for something with search_music first."
|
||||||
|
lines = []
|
||||||
|
for t in tracks[:25]:
|
||||||
|
status = "" if t.get("status") == "ready" else f" ({t.get('status')})"
|
||||||
|
dur = _fmt_duration(t.get("duration"))
|
||||||
|
dur = f" [{dur}]" if dur else ""
|
||||||
|
channel = f" — {t['channel']}" if t.get("channel") else ""
|
||||||
|
lines.append(f"{t['title']}{channel}{dur}{status}")
|
||||||
|
return f"{len(tracks)} tracks in the library:\n" + "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def add_stream(name: str, url: str) -> str:
|
||||||
|
"""Add a live audio stream URL (internet radio station, direct mp3/aac
|
||||||
|
stream) to the permanent library. It then shows up in the web player's
|
||||||
|
library and can be played by name. Pass a short friendly name."""
|
||||||
|
if not url.lower().startswith(("http://", "https://")):
|
||||||
|
return "That does not look like an http(s) stream URL."
|
||||||
|
data = _media_post("/media/api/streams", {"name": name, "url": url})
|
||||||
|
if "error" in data:
|
||||||
|
return f"Could not add the stream: {data['error']}"
|
||||||
|
return f"Added '{name}' to the library as a live stream. You can play it on request."
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
mcp.run(transport="stdio")
|
||||||
@@ -9,10 +9,14 @@ description = "Real-time voice assistant: Azure STT/TTS + Gemma LLM via LiveKit
|
|||||||
requires-python = ">=3.10,<3.15"
|
requires-python = ">=3.10,<3.15"
|
||||||
|
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"fastapi>=0.141.1",
|
||||||
|
"httpx>=0.28.1",
|
||||||
"livekit-agents[mcp]~=1.7",
|
"livekit-agents[mcp]~=1.7",
|
||||||
"livekit-plugins-azure~=1.7",
|
"livekit-plugins-azure~=1.7",
|
||||||
"livekit-plugins-openai~=1.7",
|
"livekit-plugins-openai~=1.7",
|
||||||
"python-dotenv",
|
"python-dotenv",
|
||||||
|
"uvicorn>=0.52.4",
|
||||||
|
"yt-dlp>=2026.8.19",
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
|
|||||||
Generated
+33
@@ -671,6 +671,22 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
|
{ url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "fastapi"
|
||||||
|
version = "0.141.1"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
dependencies = [
|
||||||
|
{ name = "annotated-doc" },
|
||||||
|
{ name = "pydantic" },
|
||||||
|
{ name = "starlette" },
|
||||||
|
{ name = "typing-extensions" },
|
||||||
|
{ name = "typing-inspection" },
|
||||||
|
]
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/8a/02/91e3416a8fdd715abb903a952a6bec7cdd8d14eed55d415fc8595524c319/fastapi-0.141.1.tar.gz", hash = "sha256:e8822fc40db1e1858054d7a949a888695bc9bdce70139178e33bd2871a453ca1", size = 425799, upload-time = "2026-07-29T17:18:05.568Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/cb/03/10388a42375ee7e4ac9b94eb2c5c569c8b5795e377e701c9ac3ad63de890/fastapi-0.141.1-py3-none-any.whl", hash = "sha256:bfb91aa2d334c61cb35ba9a116fc123b3d3df31640b801cf57a7a78ec3f603b3", size = 131954, upload-time = "2026-07-29T17:18:04.364Z" },
|
||||||
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "frozenlist"
|
name = "frozenlist"
|
||||||
version = "1.8.0"
|
version = "1.8.0"
|
||||||
@@ -2744,18 +2760,26 @@ name = "voice-agent"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
source = { editable = "." }
|
source = { editable = "." }
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
{ name = "fastapi" },
|
||||||
|
{ name = "httpx" },
|
||||||
{ name = "livekit-agents", extra = ["mcp"] },
|
{ name = "livekit-agents", extra = ["mcp"] },
|
||||||
{ name = "livekit-plugins-azure" },
|
{ name = "livekit-plugins-azure" },
|
||||||
{ name = "livekit-plugins-openai" },
|
{ name = "livekit-plugins-openai" },
|
||||||
{ name = "python-dotenv" },
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "uvicorn" },
|
||||||
|
{ name = "yt-dlp" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[package.metadata]
|
[package.metadata]
|
||||||
requires-dist = [
|
requires-dist = [
|
||||||
|
{ name = "fastapi", specifier = ">=0.141.1" },
|
||||||
|
{ name = "httpx", specifier = ">=0.28.1" },
|
||||||
{ name = "livekit-agents", extras = ["mcp"], specifier = "~=1.7" },
|
{ name = "livekit-agents", extras = ["mcp"], specifier = "~=1.7" },
|
||||||
{ name = "livekit-plugins-azure", specifier = "~=1.7" },
|
{ name = "livekit-plugins-azure", specifier = "~=1.7" },
|
||||||
{ name = "livekit-plugins-openai", specifier = "~=1.7" },
|
{ name = "livekit-plugins-openai", specifier = "~=1.7" },
|
||||||
{ name = "python-dotenv" },
|
{ name = "python-dotenv" },
|
||||||
|
{ name = "uvicorn", specifier = ">=0.52.4" },
|
||||||
|
{ name = "yt-dlp", specifier = ">=2026.8.19" },
|
||||||
]
|
]
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
@@ -3038,3 +3062,12 @@ wheels = [
|
|||||||
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
|
{ url = "https://files.pythonhosted.org/packages/bf/f4/ed5c402ac8fde4403ed3366c2716bfddc8a6677ebd59f3d62772cc7fe468/yarl-1.24.5-cp314-cp314t-win_arm64.whl", hash = "sha256:cf139c02f5f23ef6532040a30ff662c00a318c952334f211046b8e60b7f17688", size = 97222, upload-time = "2026-07-20T02:07:41.55Z" },
|
||||||
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
|
{ url = "https://files.pythonhosted.org/packages/61/02/962c1cbfc401a30c1d034dc67ff395f64b52302c6d62de556c1fca99acc0/yarl-1.24.5-py3-none-any.whl", hash = "sha256:a33700d13d9b7d84fd10947b09ff69fb9a792e519c8cb9764a3ca70baa6c23a7", size = 58612, upload-time = "2026-07-20T02:07:43.461Z" },
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[[package]]
|
||||||
|
name = "yt-dlp"
|
||||||
|
version = "2026.8.19"
|
||||||
|
source = { registry = "https://pypi.org/simple" }
|
||||||
|
sdist = { url = "https://files.pythonhosted.org/packages/1e/e0/832fa4ca334b766a06933a196066edc3dba37cdb6f14cd98d59bcc69a4b4/yt_dlp-2026.8.19.tar.gz", hash = "sha256:9e213e48cea35c66b378e4447903f118f6392a5fa380a2b6d7070ec86f4e0af1", size = 3052025, upload-time = "2026-08-19T23:48:59.291Z" }
|
||||||
|
wheels = [
|
||||||
|
{ url = "https://files.pythonhosted.org/packages/69/b2/8cd1613f56eed7ceb64fbd4df3f1c01246bfb098e6f398228bafda22b80b/yt_dlp-2026.8.19-py3-none-any.whl", hash = "sha256:1d57897e94c6665a0a6f9bc54b34e584284e32c034ffab3a7df25d8f7b24eedf", size = 3185533, upload-time = "2026-08-19T23:48:56.925Z" },
|
||||||
|
]
|
||||||
@@ -3,7 +3,11 @@ Name: voice-agent
|
|||||||
Version: 0.1.0
|
Version: 0.1.0
|
||||||
Summary: Real-time voice assistant: Azure STT/TTS + Gemma LLM via LiveKit Agents
|
Summary: Real-time voice assistant: Azure STT/TTS + Gemma LLM via LiveKit Agents
|
||||||
Requires-Python: <3.15,>=3.10
|
Requires-Python: <3.15,>=3.10
|
||||||
|
Requires-Dist: fastapi>=0.141.1
|
||||||
|
Requires-Dist: httpx>=0.28.1
|
||||||
Requires-Dist: livekit-agents[mcp]~=1.7
|
Requires-Dist: livekit-agents[mcp]~=1.7
|
||||||
Requires-Dist: livekit-plugins-azure~=1.7
|
Requires-Dist: livekit-plugins-azure~=1.7
|
||||||
Requires-Dist: livekit-plugins-openai~=1.7
|
Requires-Dist: livekit-plugins-openai~=1.7
|
||||||
Requires-Dist: python-dotenv
|
Requires-Dist: python-dotenv
|
||||||
|
Requires-Dist: uvicorn>=0.52.4
|
||||||
|
Requires-Dist: yt-dlp>=2026.8.19
|
||||||
@@ -1,4 +1,8 @@
|
|||||||
|
fastapi>=0.141.1
|
||||||
|
httpx>=0.28.1
|
||||||
livekit-agents[mcp]~=1.7
|
livekit-agents[mcp]~=1.7
|
||||||
livekit-plugins-azure~=1.7
|
livekit-plugins-azure~=1.7
|
||||||
livekit-plugins-openai~=1.7
|
livekit-plugins-openai~=1.7
|
||||||
python-dotenv
|
python-dotenv
|
||||||
|
uvicorn>=0.52.4
|
||||||
|
yt-dlp>=2026.8.19
|
||||||
+5
-1
@@ -18,12 +18,16 @@ services:
|
|||||||
WEB_PORT: "${WEB_PORT:-9178}"
|
WEB_PORT: "${WEB_PORT:-9178}"
|
||||||
VOICE_USERS_FILE: "/etc/hope/users.json"
|
VOICE_USERS_FILE: "/etc/hope/users.json"
|
||||||
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
|
VOICE_DEFAULT_USER: "${VOICE_DEFAULT_USER:-shane}"
|
||||||
|
MEDIA_DIR: "/data/media"
|
||||||
|
MEDIA_SERVER_PORT: "${MEDIA_SERVER_PORT:-8092}"
|
||||||
|
YTDLP_COOKIES: "${YTDLP_COOKIES:-}"
|
||||||
volumes:
|
volumes:
|
||||||
- ./livekit.yaml:/etc/livekit.yaml:ro
|
- ./livekit.yaml:/etc/livekit.yaml:ro
|
||||||
- ./certs:/etc/voice/certs
|
- ./certs:/etc/voice/certs
|
||||||
- ./skills:/skills
|
- ./skills:/skills
|
||||||
|
- ./media:/data/media
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "curl -sk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -s http://localhost:7880/ -o /dev/null"]
|
test: ["CMD-SHELL", "curl -sk https://localhost:$${WEB_PORT:-9178}/ -o /dev/null && curl -s http://localhost:7880/ -o /dev/null && curl -s http://localhost:$${MEDIA_SERVER_PORT:-8092}/media/api/healthz -o /dev/null"]
|
||||||
interval: 15s
|
interval: 15s
|
||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 3
|
retries: 3
|
||||||
|
|||||||
Whitespace-only changes.
+20
@@ -23,6 +23,26 @@ server {
|
|||||||
proxy_send_timeout 3600s;
|
proxy_send_timeout 3600s;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Media server API (library, search, player state) and the proxy for
|
||||||
|
# external live-stream URLs (http:// sources would be blocked as mixed
|
||||||
|
# content in the browser, so they are re-served over HTTPS from here).
|
||||||
|
location /media/api/ {
|
||||||
|
proxy_pass http://127.0.0.1:8092/media/api/;
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Downloaded tracks are plain static files — nginx serves them with
|
||||||
|
# native HTTP Range support, which gives the web player seeking for free.
|
||||||
|
location /media/files/ {
|
||||||
|
alias /data/media/files/;
|
||||||
|
}
|
||||||
|
|
||||||
|
location /media/thumbs/ {
|
||||||
|
alias /data/media/thumbs/;
|
||||||
|
}
|
||||||
|
|
||||||
location / {
|
location / {
|
||||||
try_files $uri $uri/ =404;
|
try_files $uri $uri/ =404;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -56,3 +56,18 @@ environment=
|
|||||||
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
|
LIVEKIT_API_SECRET="%(ENV_LIVEKIT_API_SECRET)s",
|
||||||
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
|
VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s",
|
||||||
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
|
VOICE_DEFAULT_USER="%(ENV_VOICE_DEFAULT_USER)s"
|
||||||
|
|
||||||
|
[program:media-server]
|
||||||
|
; Runs as root because the /data/media bind mount is a host (Unraid) share
|
||||||
|
; whose uid does not match the container's voiceuser.
|
||||||
|
command=/opt/voice-agent/.venv/bin/python /opt/voice/media_server.py
|
||||||
|
user=root
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
stdout_logfile=/dev/stdout
|
||||||
|
stdout_logfile_maxbytes=0
|
||||||
|
redirect_stderr=true
|
||||||
|
environment=
|
||||||
|
MEDIA_DIR="%(ENV_MEDIA_DIR)s",
|
||||||
|
MEDIA_SERVER_PORT="%(ENV_MEDIA_SERVER_PORT)s",
|
||||||
|
YTDLP_COOKIES="%(ENV_YTDLP_COOKIES)s"
|
||||||
@@ -27,6 +27,7 @@ python -m pytest tests/ -v -m "not slow" # skip LLM/network-heavy tests
|
|||||||
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
| `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
|
||||||
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
|
||||||
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
| `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
|
||||||
|
| `test_media.py` | media server API + player UI + media MCP | library/player state flow, Range proxy, static mp3 Range, live YT Music search (`-m slow`), gated full download (`HOPE_RUN_YTDLP_DOWNLOAD=1`) |
|
||||||
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
| `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
|
||||||
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,398 @@
|
|||||||
|
"""Tests for the media player subsystem.
|
||||||
|
|
||||||
|
Covers the media server API (via nginx HTTPS and direct), the stream proxy
|
||||||
|
with HTTP Range passthrough, static file serving of the ./media volume, the
|
||||||
|
web player markup, and Hope's media MCP toolset. Download tests are gated
|
||||||
|
behind HOPE_RUN_YTDLP_DOWNLOAD=1 (network-heavy, occasionally flaky).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from conftest import TOKEN_URL, docker_exec
|
||||||
|
|
||||||
|
MEDIA_API = "/media/api"
|
||||||
|
TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}"
|
||||||
|
|
||||||
|
|
||||||
|
# ── local upstream for stream-proxy tests ───────────────────────────────────
|
||||||
|
|
||||||
|
def _serve_fake_audio(payload: bytes):
|
||||||
|
"""Tiny Range-capable HTTP server on an ephemeral host port.
|
||||||
|
|
||||||
|
The container uses host networking, so the media server inside the
|
||||||
|
container reaches it on 127.0.0.1.
|
||||||
|
"""
|
||||||
|
|
||||||
|
class Handler(BaseHTTPRequestHandler):
|
||||||
|
def do_GET(self):
|
||||||
|
rng = self.headers.get("Range")
|
||||||
|
if rng and rng.startswith("bytes="):
|
||||||
|
start_s, _, end_s = rng[6:].partition("-")
|
||||||
|
start = int(start_s or 0)
|
||||||
|
end = int(end_s) if end_s else len(payload) - 1
|
||||||
|
end = min(end, len(payload) - 1)
|
||||||
|
body = payload[start : end + 1]
|
||||||
|
self.send_response(206)
|
||||||
|
self.send_header(
|
||||||
|
"Content-Range", f"bytes {start}-{end}/{len(payload)}"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
body = payload
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "audio/mpeg")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.send_header("Accept-Ranges", "bytes")
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, *args):
|
||||||
|
pass
|
||||||
|
|
||||||
|
server = HTTPServer(("127.0.0.1", 0), Handler)
|
||||||
|
port = server.server_address[1]
|
||||||
|
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||||
|
return server, f"http://127.0.0.1:{port}/audio.mp3"
|
||||||
|
|
||||||
|
|
||||||
|
# ── health + basic API shape ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_media_server_health_via_nginx(client):
|
||||||
|
resp = client.get(f"{MEDIA_API}/healthz")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_library_endpoint(client):
|
||||||
|
resp = client.get(f"{MEDIA_API}/library")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert isinstance(data["tracks"], list)
|
||||||
|
assert data["version"] >= 1
|
||||||
|
for track in data["tracks"]:
|
||||||
|
assert {"id", "kind", "title", "status"} <= set(track)
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_search_requires_query(client):
|
||||||
|
resp = client.post(f"{MEDIA_API}/search", json={"query": ""})
|
||||||
|
assert resp.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_play_requires_target(client):
|
||||||
|
resp = client.post(f"{MEDIA_API}/play", json={"room": TEST_ROOM})
|
||||||
|
assert resp.status_code == 400
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_unknown_stream_404(client):
|
||||||
|
resp = client.get(f"{MEDIA_API}/stream/does-not-exist")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
# ── player state flow (uses a live stream track as the fixture) ─────────────
|
||||||
|
|
||||||
|
def test_media_player_state_flow(client):
|
||||||
|
"""add stream -> play -> state -> report -> pause -> seek hint -> stop."""
|
||||||
|
upstream, url = _serve_fake_audio(b"\x00" * 64)
|
||||||
|
try:
|
||||||
|
added = client.post(
|
||||||
|
f"{MEDIA_API}/streams", json={"name": "Test FM", "url": url}
|
||||||
|
)
|
||||||
|
assert added.status_code == 200
|
||||||
|
track = added.json()["track"]
|
||||||
|
track_id = track["id"]
|
||||||
|
assert track["kind"] == "stream"
|
||||||
|
assert track["status"] == "ready"
|
||||||
|
assert track["audio_url"].startswith(f"{MEDIA_API}/stream/")
|
||||||
|
|
||||||
|
played = client.post(
|
||||||
|
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||||
|
).json()
|
||||||
|
assert played["desired"] == "playing"
|
||||||
|
assert played["track"]["id"] == track_id
|
||||||
|
|
||||||
|
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||||
|
assert state["track"]["id"] == track_id
|
||||||
|
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/report",
|
||||||
|
json={"room": TEST_ROOM, "track_id": track_id,
|
||||||
|
"state": "playing", "position": 12.5},
|
||||||
|
)
|
||||||
|
state = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()
|
||||||
|
assert abs(state["reported"]["position"] - 12.5) < 0.01
|
||||||
|
|
||||||
|
client.post(f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "pause"})
|
||||||
|
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["desired"] == "paused"
|
||||||
|
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/control",
|
||||||
|
json={"room": TEST_ROOM, "action": "seek", "position": 42},
|
||||||
|
)
|
||||||
|
hint = client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["seek_hint"]
|
||||||
|
assert hint["position"] == 42
|
||||||
|
|
||||||
|
# start_paused variant: scope transfer without starting playback
|
||||||
|
transfer = client.post(
|
||||||
|
f"{MEDIA_API}/play",
|
||||||
|
json={"room": TEST_ROOM + "-b", "track_id": track_id, "start_paused": True},
|
||||||
|
).json()
|
||||||
|
assert transfer["desired"] == "paused"
|
||||||
|
|
||||||
|
# playing clears any open suggestion choices
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/choices",
|
||||||
|
json={"room": TEST_ROOM, "choices": [{"video_id": "x", "title": "X"}]},
|
||||||
|
)
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/play", json={"room": TEST_ROOM, "track_id": track_id}
|
||||||
|
)
|
||||||
|
assert client.get(f"{MEDIA_API}/state", params={"room": TEST_ROOM}).json()["choices"] == []
|
||||||
|
|
||||||
|
stopped = client.post(
|
||||||
|
f"{MEDIA_API}/control", json={"room": TEST_ROOM, "action": "stop"}
|
||||||
|
).json()
|
||||||
|
assert stopped["desired"] == "stopped"
|
||||||
|
finally:
|
||||||
|
upstream.shutdown()
|
||||||
|
# Remove the test stream from the permanent library.
|
||||||
|
try:
|
||||||
|
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||||
|
except (httpx.HTTPError, UnboundLocalError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_stream_proxy_and_range(client):
|
||||||
|
"""External audio URLs must be reachable same-origin (HTTPS) and support
|
||||||
|
Range passthrough so the browser can seek progressive streams."""
|
||||||
|
payload = b"0123456789" * 100
|
||||||
|
upstream, url = _serve_fake_audio(payload)
|
||||||
|
track_id = None
|
||||||
|
try:
|
||||||
|
track_id = client.post(
|
||||||
|
f"{MEDIA_API}/streams", json={"name": "Range FM", "url": url}
|
||||||
|
).json()["track"]["id"]
|
||||||
|
|
||||||
|
full = client.get(f"{MEDIA_API}/stream/{track_id}")
|
||||||
|
assert full.status_code == 200
|
||||||
|
assert full.content == payload
|
||||||
|
|
||||||
|
ranged = client.get(
|
||||||
|
f"{MEDIA_API}/stream/{track_id}", headers={"Range": "bytes=100-199"}
|
||||||
|
)
|
||||||
|
assert ranged.status_code == 206
|
||||||
|
assert ranged.content == payload[100:200]
|
||||||
|
assert ranged.headers["content-range"] == f"bytes 100-199/{len(payload)}"
|
||||||
|
finally:
|
||||||
|
upstream.shutdown()
|
||||||
|
if track_id:
|
||||||
|
try:
|
||||||
|
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||||
|
except httpx.HTTPError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_files_static_range_support(client):
|
||||||
|
"""Downloaded tracks are nginx static files with native Range support."""
|
||||||
|
import pathlib
|
||||||
|
|
||||||
|
marker = uuid.uuid4().hex[:8]
|
||||||
|
host_media = pathlib.Path(__file__).parents[1] / "media" / "files"
|
||||||
|
host_media.mkdir(parents=True, exist_ok=True)
|
||||||
|
fname = f"test-{marker}.mp3"
|
||||||
|
payload = b"IMHEADDATA" + b"\x01" * 2048
|
||||||
|
(host_media / fname).write_bytes(payload)
|
||||||
|
try:
|
||||||
|
full = client.get(f"/media/files/{fname}")
|
||||||
|
assert full.status_code == 200
|
||||||
|
assert full.content == payload
|
||||||
|
|
||||||
|
ranged = client.get(f"/media/files/{fname}", headers={"Range": "bytes=0-9"})
|
||||||
|
assert ranged.status_code == 206
|
||||||
|
assert ranged.content == b"IMHEADDATA"
|
||||||
|
finally:
|
||||||
|
(host_media / fname).unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_library_persists_in_volume(client):
|
||||||
|
"""The library JSON is written to the ./media bind mount (permanent)."""
|
||||||
|
import pathlib
|
||||||
|
|
||||||
|
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||||
|
track_id = None
|
||||||
|
try:
|
||||||
|
track_id = client.post(
|
||||||
|
f"{MEDIA_API}/streams", json={"name": "Persist FM", "url": url}
|
||||||
|
).json()["track"]["id"]
|
||||||
|
library_file = pathlib.Path(__file__).parents[1] / "media" / "library.json"
|
||||||
|
data = json.loads(library_file.read_text())
|
||||||
|
assert track_id in data["tracks"]
|
||||||
|
finally:
|
||||||
|
upstream.shutdown()
|
||||||
|
if track_id:
|
||||||
|
try:
|
||||||
|
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||||
|
except httpx.HTTPError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# ── web player UI ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def test_ui_media_panel_serves(client):
|
||||||
|
resp = client.get("/")
|
||||||
|
body = resp.text
|
||||||
|
assert 'id="mediaBtn"' in body
|
||||||
|
assert 'id="mediaPanel"' in body
|
||||||
|
assert 'id="playPauseBtn"' in body
|
||||||
|
assert 'id="seekBar"' in body
|
||||||
|
assert 'id="mediaAudio"' in body
|
||||||
|
|
||||||
|
|
||||||
|
def test_ui_media_logic_serves(client):
|
||||||
|
resp = client.get("/app.js")
|
||||||
|
body = resp.text
|
||||||
|
assert "reconcileMedia" in body
|
||||||
|
assert "media-control" in body # data-channel topic for the agent
|
||||||
|
assert "media_state" in body # playback state reports
|
||||||
|
assert "/media/api/state" in body # reconciliation polling
|
||||||
|
|
||||||
|
|
||||||
|
def test_ui_media_styles_serve(client):
|
||||||
|
resp = client.get("/style.css")
|
||||||
|
assert ".media-panel" in resp.text
|
||||||
|
assert ".media-row" in resp.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_token_response_includes_room():
|
||||||
|
resp = httpx.post(TOKEN_URL, json={}, timeout=10)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert resp.json()["room"].startswith("voice-")
|
||||||
|
|
||||||
|
|
||||||
|
# ── Hope's media MCP toolset ────────────────────────────────────────────────
|
||||||
|
|
||||||
|
MEDIA_MCP_ENV = {
|
||||||
|
"MEDIA_SERVER_URL": "http://127.0.0.1:8092",
|
||||||
|
"MEDIA_ROOM": "mcp-test-room",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_mcp_media_list_library(mcp_client_factory):
|
||||||
|
text, is_error = mcp_client_factory(
|
||||||
|
"media_mcp.py", "list_library", {}, env=MEDIA_MCP_ENV
|
||||||
|
)
|
||||||
|
assert not is_error
|
||||||
|
assert "library" in text.lower()
|
||||||
|
|
||||||
|
|
||||||
|
def test_mcp_media_add_stream_and_now_playing(mcp_client_factory, client):
|
||||||
|
upstream, url = _serve_fake_audio(b"\x00" * 16)
|
||||||
|
try:
|
||||||
|
text, is_error = mcp_client_factory(
|
||||||
|
"media_mcp.py", "add_stream",
|
||||||
|
{"name": "MCP Test Radio", "url": url}, env=MEDIA_MCP_ENV,
|
||||||
|
)
|
||||||
|
assert not is_error, text
|
||||||
|
assert "Added" in text
|
||||||
|
|
||||||
|
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||||
|
probe = next(t for t in library if t["title"] == "MCP Test Radio")
|
||||||
|
try:
|
||||||
|
text, is_error = mcp_client_factory(
|
||||||
|
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||||
|
)
|
||||||
|
assert not is_error
|
||||||
|
# Nothing loaded in mcp-test-room: the tool must say so plainly.
|
||||||
|
assert "Nothing is loaded" in text
|
||||||
|
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/play", json={"room": "mcp-test-room", "track_id": probe["id"]}
|
||||||
|
)
|
||||||
|
text, is_error = mcp_client_factory(
|
||||||
|
"media_mcp.py", "now_playing", {}, env=MEDIA_MCP_ENV
|
||||||
|
)
|
||||||
|
assert not is_error
|
||||||
|
assert "MCP Test Radio" in text
|
||||||
|
finally:
|
||||||
|
client.delete(f"{MEDIA_API}/tracks/{probe['id']}")
|
||||||
|
finally:
|
||||||
|
upstream.shutdown()
|
||||||
|
|
||||||
|
|
||||||
|
def test_mcp_media_resume_without_load(mcp_client_factory, client):
|
||||||
|
"""Errors from the media server reach the LLM as tool text (not MCP
|
||||||
|
errors) so Hope can explain them naturally."""
|
||||||
|
client.post(
|
||||||
|
f"{MEDIA_API}/control", json={"room": "mcp-empty-room", "action": "stop"}
|
||||||
|
)
|
||||||
|
text, is_error = mcp_client_factory(
|
||||||
|
"media_mcp.py", "resume_music", {}, env={
|
||||||
|
"MEDIA_SERVER_URL": "http://127.0.0.1:8092",
|
||||||
|
"MEDIA_ROOM": "mcp-empty-room",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
assert not is_error
|
||||||
|
assert "nothing loaded" in text.lower()
|
||||||
|
|
||||||
|
|
||||||
|
# ── live YouTube Music (network-dependent) ──────────────────────────────────
|
||||||
|
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_media_search_live(client):
|
||||||
|
resp = client.post(
|
||||||
|
f"{MEDIA_API}/search",
|
||||||
|
json={"query": "blind guardian timeless miracle", "limit": 3},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
results = resp.json()["results"]
|
||||||
|
assert len(results) >= 1
|
||||||
|
for r in results:
|
||||||
|
assert r["video_id"]
|
||||||
|
assert r["title"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.slow
|
||||||
|
def test_media_download_and_stream_end_to_end(client):
|
||||||
|
"""Full pipeline: search -> play -> yt-dlp download + ffmpeg transcode ->
|
||||||
|
static mp3 with Range support in the player. Gated: run with
|
||||||
|
HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
|
||||||
|
if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
|
||||||
|
pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")
|
||||||
|
|
||||||
|
search = client.post(
|
||||||
|
f"{MEDIA_API}/search", json={"query": "kingdom hearts dearly beloved", "limit": 3}
|
||||||
|
)
|
||||||
|
results = search.json()["results"]
|
||||||
|
assert results, "search returned nothing"
|
||||||
|
video = results[0]
|
||||||
|
|
||||||
|
played = client.post(
|
||||||
|
f"{MEDIA_API}/play",
|
||||||
|
json={"room": TEST_ROOM, "video_id": video["video_id"],
|
||||||
|
"title": video["title"], "channel": video["channel"]},
|
||||||
|
).json()
|
||||||
|
track_id = played["track"]["id"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
deadline = time.time() + 240
|
||||||
|
track = played["track"]
|
||||||
|
while time.time() < deadline:
|
||||||
|
library = client.get(f"{MEDIA_API}/library").json()["tracks"]
|
||||||
|
track = next(t for t in library if t["id"] == track_id)
|
||||||
|
if track["status"] in ("ready", "failed"):
|
||||||
|
break
|
||||||
|
time.sleep(3)
|
||||||
|
assert track["status"] == "ready", f"download failed: {track.get('error')}"
|
||||||
|
|
||||||
|
stream = client.get(track["audio_url"], headers={"Range": "bytes=0-1023"})
|
||||||
|
assert stream.status_code in (200, 206)
|
||||||
|
assert len(stream.content) >= 1
|
||||||
|
finally:
|
||||||
|
client.delete(f"{MEDIA_API}/tracks/{track_id}")
|
||||||
+519
-5
@@ -693,6 +693,7 @@ document.addEventListener("keydown", (event) => {
|
|||||||
if (event.key !== "Escape") return;
|
if (event.key !== "Escape") return;
|
||||||
if (settingsSheet.classList.contains("open")) closeSheet();
|
if (settingsSheet.classList.contains("open")) closeSheet();
|
||||||
if (tasksPanel.classList.contains("open")) setTasksOpen(false);
|
if (tasksPanel.classList.contains("open")) setTasksOpen(false);
|
||||||
|
if (mediaPanel.classList.contains("open")) setMediaOpen(false);
|
||||||
});
|
});
|
||||||
// ── Wake lock ───────────────────────────────────────────────────────────────
|
// ── Wake lock ───────────────────────────────────────────────────────────────
|
||||||
async function acquireWakeLock() {
|
async function acquireWakeLock() {
|
||||||
@@ -816,6 +817,8 @@ function createRoom(token) {
|
|||||||
stopBtn.disabled = true;
|
stopBtn.disabled = true;
|
||||||
agentParticipant = null;
|
agentParticipant = null;
|
||||||
agentReady = false;
|
agentReady = false;
|
||||||
|
// Leaving the conversation hands the room back to the music player.
|
||||||
|
resumePlaybackAfterTalk();
|
||||||
});
|
});
|
||||||
|
|
||||||
return newRoom.connect(LIVEKIT_URL, token, {
|
return newRoom.connect(LIVEKIT_URL, token, {
|
||||||
@@ -873,8 +876,9 @@ async function connectConversation() {
|
|||||||
await acquireWakeLock();
|
await acquireWakeLock();
|
||||||
|
|
||||||
// Get a signed access token from the token endpoint.
|
// Get a signed access token from the token endpoint.
|
||||||
const token = await fetchToken(ROOM_NAME);
|
const tokenData = await fetchToken(ROOM_NAME);
|
||||||
room = await createRoom(token);
|
await enterMediaScope(tokenData.room);
|
||||||
|
room = await createRoom(tokenData.token);
|
||||||
|
|
||||||
// The agent is dispatched when we join. If it misses the dispatch
|
// The agent is dispatched when we join. If it misses the dispatch
|
||||||
// (for example during a server restart), reconnect once.
|
// (for example during a server restart), reconnect once.
|
||||||
@@ -886,8 +890,9 @@ async function connectConversation() {
|
|||||||
agentParticipant = null;
|
agentParticipant = null;
|
||||||
agentReady = false;
|
agentReady = false;
|
||||||
await stale.disconnect();
|
await stale.disconnect();
|
||||||
const freshToken = await fetchToken(ROOM_NAME);
|
const freshTokenData = await fetchToken(ROOM_NAME);
|
||||||
room = await createRoom(freshToken);
|
await enterMediaScope(freshTokenData.room);
|
||||||
|
room = await createRoom(freshTokenData.token);
|
||||||
if (!(await waitForAgent(15000))) {
|
if (!(await waitForAgent(15000))) {
|
||||||
throw new Error("Assistant did not join — reload the page and try again");
|
throw new Error("Assistant did not join — reload the page and try again");
|
||||||
}
|
}
|
||||||
@@ -999,6 +1004,8 @@ stopBtn.addEventListener("click", () => {
|
|||||||
muteBtn.disabled = true;
|
muteBtn.disabled = true;
|
||||||
micMuted = false;
|
micMuted = false;
|
||||||
updateMuteIcon();
|
updateMuteIcon();
|
||||||
|
// Ending the conversation resumes music that was paused for the talk.
|
||||||
|
resumePlaybackAfterTalk();
|
||||||
});
|
});
|
||||||
|
|
||||||
// ── Mic mute toggle ─────────────────────────────────────────────────────────
|
// ── Mic mute toggle ─────────────────────────────────────────────────────────
|
||||||
@@ -1030,7 +1037,7 @@ async function fetchToken(roomName) {
|
|||||||
});
|
});
|
||||||
if (!resp.ok) throw new Error(`Token endpoint returned ${resp.status}`);
|
if (!resp.ok) throw new Error(`Token endpoint returned ${resp.status}`);
|
||||||
const data = await resp.json();
|
const data = await resp.json();
|
||||||
return data.token;
|
return { token: data.token, room: data.room || null };
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Helper: send voice selection to the agent via data channel ─────────────
|
// ── Helper: send voice selection to the agent via data channel ─────────────
|
||||||
@@ -1064,3 +1071,510 @@ function sendModelMessage(model, targetRoom) {
|
|||||||
reliable: true,
|
reliable: true,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Media player ────────────────────────────────────────────────────────────
|
||||||
|
// The browser <audio> element is the player. The media server owns library +
|
||||||
|
// per-room desired state; this UI reconciles against GET /media/api/state
|
||||||
|
// (polling) and reports real playback back. Hope drives the same state via
|
||||||
|
// her media tools, so voice commands and panel clicks stay in sync.
|
||||||
|
//
|
||||||
|
// Conversation rule (mirrored in agent.py): while a track plays the agent's
|
||||||
|
// mic is off. Connecting to Hope pauses music; ending the conversation
|
||||||
|
// resumes what was playing.
|
||||||
|
|
||||||
|
const mediaBtn = document.getElementById("mediaBtn");
|
||||||
|
const mediaPanel = document.getElementById("mediaPanel");
|
||||||
|
const closeMediaBtn = document.getElementById("closeMediaBtn");
|
||||||
|
const mediaBadge = document.getElementById("mediaBadge");
|
||||||
|
const mediaArt = document.getElementById("mediaArt");
|
||||||
|
const mediaArtPlaceholder = document.getElementById("mediaArtPlaceholder");
|
||||||
|
const mediaTitleEl = document.getElementById("mediaTitle");
|
||||||
|
const mediaChannelEl = document.getElementById("mediaChannel");
|
||||||
|
const mediaProgressWrap = document.getElementById("mediaProgressWrap");
|
||||||
|
const mediaProgressFill = document.getElementById("mediaProgressFill");
|
||||||
|
const mediaProgressLabel = document.getElementById("mediaProgressLabel");
|
||||||
|
const rewBtn = document.getElementById("rewBtn");
|
||||||
|
const fwdBtn = document.getElementById("fwdBtn");
|
||||||
|
const playPauseBtn = document.getElementById("playPauseBtn");
|
||||||
|
const playIcon = document.getElementById("playIcon");
|
||||||
|
const pauseIcon = document.getElementById("pauseIcon");
|
||||||
|
const stopMediaBtn = document.getElementById("stopMediaBtn");
|
||||||
|
const seekBar = document.getElementById("seekBar");
|
||||||
|
const mediaCurEl = document.getElementById("mediaCur");
|
||||||
|
const mediaDurEl = document.getElementById("mediaDur");
|
||||||
|
const volumeBar = document.getElementById("volumeBar");
|
||||||
|
const mediaSearchForm = document.getElementById("mediaSearchForm");
|
||||||
|
const mediaSearchInput = document.getElementById("mediaSearchInput");
|
||||||
|
const mediaResultsEl = document.getElementById("mediaResults");
|
||||||
|
const mediaChoicesWrap = document.getElementById("mediaChoicesWrap");
|
||||||
|
const mediaChoicesEl = document.getElementById("mediaChoices");
|
||||||
|
const mediaLibraryEl = document.getElementById("mediaLibrary");
|
||||||
|
const streamForm = document.getElementById("streamForm");
|
||||||
|
const streamNameInput = document.getElementById("streamName");
|
||||||
|
const streamUrlInput = document.getElementById("streamUrl");
|
||||||
|
const mediaAudio = document.getElementById("mediaAudio");
|
||||||
|
|
||||||
|
let mediaRoom = null; // LiveKit room name (player state scope)
|
||||||
|
let mediaCurrentTrackId = null; // track id loaded into the audio element
|
||||||
|
let mediaAppliedSeekTs = 0; // last applied seek_hint timestamp
|
||||||
|
let mediaWasPlayingBeforeTalk = false;
|
||||||
|
let mediaPollTimer = null;
|
||||||
|
let mediaLastReport = 0;
|
||||||
|
|
||||||
|
function mediaScope() {
|
||||||
|
return mediaRoom || "default";
|
||||||
|
}
|
||||||
|
|
||||||
|
async function mediaPost(path, body) {
|
||||||
|
const resp = await fetch(`/media/api${path}`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ room: mediaScope(), ...body }),
|
||||||
|
});
|
||||||
|
if (!resp.ok) {
|
||||||
|
const detail = await resp.json().catch(() => ({}));
|
||||||
|
throw new Error(detail.error || `media API ${path} -> ${resp.status}`);
|
||||||
|
}
|
||||||
|
return resp.json();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Called just before joining the LiveKit room: carry whatever the audio
|
||||||
|
// element holds into the room-scoped player state, paused (talking pauses
|
||||||
|
// music). resumePlaybackAfterTalk() reverses this when the talk ends.
|
||||||
|
async function enterMediaScope(newRoom) {
|
||||||
|
if (!newRoom || newRoom === mediaRoom) return;
|
||||||
|
const wasPlaying = !!mediaCurrentTrackId && !mediaAudio.paused;
|
||||||
|
mediaWasPlayingBeforeTalk = wasPlaying;
|
||||||
|
const carriedTrack = mediaCurrentTrackId;
|
||||||
|
mediaRoom = newRoom;
|
||||||
|
if (carriedTrack) {
|
||||||
|
try {
|
||||||
|
await mediaPost("/play", { room: mediaRoom, track_id: carriedTrack, start_paused: true });
|
||||||
|
} catch (e) { console.warn("media scope transfer failed:", e); }
|
||||||
|
mediaAudio.pause();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function resumePlaybackAfterTalk() {
|
||||||
|
if (!mediaWasPlayingBeforeTalk) return;
|
||||||
|
mediaWasPlayingBeforeTalk = false;
|
||||||
|
try { await mediaPost("/control", { action: "resume" }); } catch (e) { /* poll retries */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
function publishMediaState(state) {
|
||||||
|
if (!room || !room.localParticipant) return;
|
||||||
|
const payload = new TextEncoder().encode(
|
||||||
|
JSON.stringify({ type: "media_state", state, track_id: mediaCurrentTrackId })
|
||||||
|
);
|
||||||
|
room.localParticipant.publishData(payload, { topic: "media-control", reliable: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
function reportMediaPlayback(state) {
|
||||||
|
const body = {
|
||||||
|
state,
|
||||||
|
track_id: mediaCurrentTrackId,
|
||||||
|
position: isFinite(mediaAudio.currentTime) ? mediaAudio.currentTime : 0,
|
||||||
|
};
|
||||||
|
if (isFinite(mediaAudio.duration)) body.duration = mediaAudio.duration;
|
||||||
|
mediaPost("/report", body).catch(() => {});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Polling + reconciliation ────────────────────────────────────────────────
|
||||||
|
function startMediaPolling() {
|
||||||
|
if (mediaPollTimer) return;
|
||||||
|
mediaPollTimer = setInterval(() => {
|
||||||
|
pollMediaState();
|
||||||
|
}, 1500);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function pollMediaState() {
|
||||||
|
let s;
|
||||||
|
try {
|
||||||
|
const resp = await fetch(`/media/api/state?room=${encodeURIComponent(mediaScope())}`);
|
||||||
|
if (!resp.ok) return;
|
||||||
|
s = await resp.json();
|
||||||
|
} catch (e) {
|
||||||
|
return; // media server briefly unavailable; try again next tick
|
||||||
|
}
|
||||||
|
reconcileMedia(s);
|
||||||
|
}
|
||||||
|
|
||||||
|
function reconcileMedia(s) {
|
||||||
|
renderMediaChoices(s.choices || []);
|
||||||
|
updateMediaBadge(s);
|
||||||
|
|
||||||
|
const track = s.track;
|
||||||
|
const pendingJob = (s.pending || []).find(j => !track || j.track_id === track.id);
|
||||||
|
updateMediaProgress(track, pendingJob);
|
||||||
|
|
||||||
|
if (track && track.id !== mediaCurrentTrackId) loadMediaTrack(track);
|
||||||
|
if (!track && mediaCurrentTrackId && s.desired === "stopped") clearMediaAudio();
|
||||||
|
|
||||||
|
applyDesiredMediaState(s, track);
|
||||||
|
|
||||||
|
if (s.seek_hint && s.seek_hint.ts !== mediaAppliedSeekTs && isFinite(mediaAudio.duration)) {
|
||||||
|
mediaAppliedSeekTs = s.seek_hint.ts;
|
||||||
|
const target = Math.min(s.seek_hint.position, mediaAudio.duration);
|
||||||
|
if (Number.isFinite(target)) {
|
||||||
|
mediaAudio.currentTime = Math.max(0, target);
|
||||||
|
updateSeekUI();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyDesiredMediaState(s, track) {
|
||||||
|
if (s.desired === "playing") {
|
||||||
|
if (track && track.status === "ready" && mediaAudio.src && mediaAudio.paused) {
|
||||||
|
mediaAudio.play().catch((e) => console.warn("playback blocked:", e));
|
||||||
|
}
|
||||||
|
} else if (!mediaAudio.paused) {
|
||||||
|
mediaAudio.pause();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function loadMediaTrack(track) {
|
||||||
|
mediaCurrentTrackId = track.id;
|
||||||
|
updateMediaNowCard(track);
|
||||||
|
if (track.audio_url && (track.kind === "stream" || track.status === "ready")) {
|
||||||
|
if (mediaAudio.getAttribute("src") !== track.audio_url) {
|
||||||
|
mediaAudio.src = track.audio_url;
|
||||||
|
mediaAudio.load();
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Downloading or failed: no stream yet. The poll flips this to the
|
||||||
|
// real URL once status turns "ready".
|
||||||
|
mediaAudio.removeAttribute("src");
|
||||||
|
mediaAudio.load();
|
||||||
|
updateMediaControls();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function clearMediaAudio() {
|
||||||
|
mediaCurrentTrackId = null;
|
||||||
|
mediaAudio.pause();
|
||||||
|
mediaAudio.removeAttribute("src");
|
||||||
|
mediaAudio.load();
|
||||||
|
mediaTitleEl.textContent = "Nothing loaded";
|
||||||
|
mediaChannelEl.textContent = "";
|
||||||
|
mediaArt.hidden = true;
|
||||||
|
mediaArtPlaceholder.hidden = false;
|
||||||
|
updateMediaControls();
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateMediaNowCard(track) {
|
||||||
|
mediaTitleEl.textContent = track.title || track.id;
|
||||||
|
if (track.status === "failed") {
|
||||||
|
mediaChannelEl.textContent = "Download failed — pick another result or try again.";
|
||||||
|
} else if (track.status === "downloading") {
|
||||||
|
mediaChannelEl.textContent = "Preparing…";
|
||||||
|
} else {
|
||||||
|
mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
|
||||||
|
}
|
||||||
|
if (track.art_url) {
|
||||||
|
mediaArt.src = track.art_url;
|
||||||
|
mediaArt.hidden = false;
|
||||||
|
mediaArtPlaceholder.hidden = true;
|
||||||
|
} else {
|
||||||
|
mediaArt.hidden = true;
|
||||||
|
mediaArtPlaceholder.hidden = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateMediaProgress(track, job) {
|
||||||
|
if (!job) {
|
||||||
|
mediaProgressWrap.hidden = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
mediaProgressWrap.hidden = false;
|
||||||
|
const pct = Math.round(job.progress || 0);
|
||||||
|
mediaProgressFill.style.width = `${pct}%`;
|
||||||
|
const stage = job.stage === "transcoding"
|
||||||
|
? "Re-encoding…"
|
||||||
|
: job.stage === "queued" ? "Queued…" : "Downloading…";
|
||||||
|
mediaProgressLabel.textContent = `${stage} ${pct}%`;
|
||||||
|
if (track) updateMediaNowCard(track);
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateMediaBadge(s) {
|
||||||
|
const pending = (s.pending || []).length;
|
||||||
|
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
||||||
|
if (pending > 0) {
|
||||||
|
mediaBadge.hidden = false;
|
||||||
|
mediaBadge.textContent = String(pending);
|
||||||
|
} else if (playing) {
|
||||||
|
mediaBadge.hidden = false;
|
||||||
|
mediaBadge.innerHTML = "♫";
|
||||||
|
} else {
|
||||||
|
mediaBadge.hidden = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Playback controls ───────────────────────────────────────────────────────
|
||||||
|
function updateMediaControls() {
|
||||||
|
const playing = !mediaAudio.paused && !!mediaCurrentTrackId;
|
||||||
|
playIcon.hidden = playing;
|
||||||
|
pauseIcon.hidden = !playing;
|
||||||
|
playPauseBtn.setAttribute("aria-label", playing ? "Pause" : "Play");
|
||||||
|
updateMediaBadge({ pending: [] });
|
||||||
|
}
|
||||||
|
|
||||||
|
function fmtTime(sec) {
|
||||||
|
if (!isFinite(sec) || sec < 0) return "0:00";
|
||||||
|
const s = Math.floor(sec % 60);
|
||||||
|
const m = Math.floor((sec / 60) % 60);
|
||||||
|
const h = Math.floor(sec / 3600);
|
||||||
|
return h ? `${h}:${String(m).padStart(2, "0")}:${String(s).padStart(2, "0")}` : `${m}:${String(s).padStart(2, "0")}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateSeekUI() {
|
||||||
|
if (!isFinite(mediaAudio.duration) || mediaAudio.duration <= 0) {
|
||||||
|
seekBar.value = 0;
|
||||||
|
mediaDurEl.textContent = "0:00";
|
||||||
|
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
seekBar.max = Math.floor(mediaAudio.duration);
|
||||||
|
seekBar.value = Math.floor(mediaAudio.currentTime);
|
||||||
|
mediaCurEl.textContent = fmtTime(mediaAudio.currentTime);
|
||||||
|
mediaDurEl.textContent = fmtTime(mediaAudio.duration);
|
||||||
|
}
|
||||||
|
|
||||||
|
playPauseBtn.addEventListener("click", () => {
|
||||||
|
if (!mediaCurrentTrackId || !mediaAudio.src) return;
|
||||||
|
if (mediaAudio.paused) {
|
||||||
|
mediaAudio.play().catch((e) => console.warn("play failed:", e));
|
||||||
|
mediaPost("/control", { action: "resume" }).catch(() => {});
|
||||||
|
} else {
|
||||||
|
mediaAudio.pause();
|
||||||
|
mediaPost("/control", { action: "pause" }).catch(() => {});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
rewBtn.addEventListener("click", () => {
|
||||||
|
if (!mediaAudio.src) return;
|
||||||
|
mediaAudio.currentTime = Math.max(0, mediaAudio.currentTime - 10);
|
||||||
|
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||||
|
});
|
||||||
|
|
||||||
|
fwdBtn.addEventListener("click", () => {
|
||||||
|
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
||||||
|
mediaAudio.currentTime = Math.min(mediaAudio.duration - 0.5, mediaAudio.currentTime + 10);
|
||||||
|
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||||
|
});
|
||||||
|
|
||||||
|
stopMediaBtn.addEventListener("click", () => {
|
||||||
|
clearMediaAudio();
|
||||||
|
mediaPost("/control", { action: "stop" }).catch(() => {});
|
||||||
|
publishMediaState("stopped");
|
||||||
|
});
|
||||||
|
|
||||||
|
seekBar.addEventListener("change", () => {
|
||||||
|
if (!mediaAudio.src || !isFinite(mediaAudio.duration)) return;
|
||||||
|
mediaAudio.currentTime = Number(seekBar.value);
|
||||||
|
reportMediaPlayback(!mediaAudio.paused ? "playing" : "paused");
|
||||||
|
});
|
||||||
|
|
||||||
|
volumeBar.addEventListener("input", () => {
|
||||||
|
mediaAudio.volume = Number(volumeBar.value) / 100;
|
||||||
|
try { localStorage.setItem("media_volume", volumeBar.value); } catch (e) { /* ok */ }
|
||||||
|
});
|
||||||
|
mediaAudio.volume = Number(localStorage.getItem("media_volume") ?? 90) / 100;
|
||||||
|
volumeBar.value = Math.round(mediaAudio.volume * 100);
|
||||||
|
|
||||||
|
mediaAudio.addEventListener("play", () => {
|
||||||
|
updateMediaControls();
|
||||||
|
reportMediaPlayback("playing");
|
||||||
|
publishMediaState("playing");
|
||||||
|
});
|
||||||
|
mediaAudio.addEventListener("pause", () => {
|
||||||
|
updateMediaControls();
|
||||||
|
if (mediaCurrentTrackId) {
|
||||||
|
reportMediaPlayback("paused");
|
||||||
|
publishMediaState("paused");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
mediaAudio.addEventListener("ended", () => {
|
||||||
|
updateMediaControls();
|
||||||
|
reportMediaPlayback("stopped");
|
||||||
|
publishMediaState("stopped");
|
||||||
|
mediaPost("/control", { action: "stop" }).catch(() => {});
|
||||||
|
});
|
||||||
|
mediaAudio.addEventListener("timeupdate", () => {
|
||||||
|
updateSeekUI();
|
||||||
|
const now = Date.now();
|
||||||
|
if (!mediaAudio.paused && now - mediaLastReport > 5000) {
|
||||||
|
mediaLastReport = now;
|
||||||
|
reportMediaPlayback("playing");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
mediaAudio.addEventListener("durationchange", updateSeekUI);
|
||||||
|
mediaAudio.addEventListener("error", () => {
|
||||||
|
// An empty src (track still downloading) also fires this — only surface
|
||||||
|
// real playback failures.
|
||||||
|
if (mediaCurrentTrackId && mediaAudio.getAttribute("src")) {
|
||||||
|
mediaChannelEl.textContent = "Playback error — the track may still be preparing.";
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Search / choices / library ──────────────────────────────────────────────
|
||||||
|
function fmtListDuration(sec) {
|
||||||
|
const s = Number(sec);
|
||||||
|
return isFinite(s) && s > 0 ? fmtTime(s) : "";
|
||||||
|
}
|
||||||
|
|
||||||
|
function mediaRowActions() {
|
||||||
|
const play = document.createElement("button");
|
||||||
|
play.className = "btn btn-secondary media-row-play";
|
||||||
|
play.type = "button";
|
||||||
|
play.textContent = "Play";
|
||||||
|
return play;
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderMediaResults(results) {
|
||||||
|
mediaResultsEl.innerHTML = "";
|
||||||
|
if (!results || results.length === 0) {
|
||||||
|
mediaResultsEl.innerHTML = '<div class="media-empty">No results.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const r of results) {
|
||||||
|
const row = document.createElement("div");
|
||||||
|
row.className = "media-row";
|
||||||
|
const info = document.createElement("div");
|
||||||
|
info.className = "media-row-info";
|
||||||
|
const dur = fmtListDuration(r.duration);
|
||||||
|
info.innerHTML = `<strong>${escapeHtml(r.title)}</strong>` +
|
||||||
|
`<span>${escapeHtml(r.channel || "")}${dur ? " · " + dur : ""}` +
|
||||||
|
`${r.in_library ? " · in library" : ""}</span>`;
|
||||||
|
const play = mediaRowActions();
|
||||||
|
play.addEventListener("click", () => playVideoId(r));
|
||||||
|
row.appendChild(info);
|
||||||
|
row.appendChild(play);
|
||||||
|
mediaResultsEl.appendChild(row);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function playVideoId(r) {
|
||||||
|
try {
|
||||||
|
await mediaPost("/play", {
|
||||||
|
video_id: r.video_id,
|
||||||
|
title: r.title,
|
||||||
|
channel: r.channel || "",
|
||||||
|
});
|
||||||
|
} catch (e) {
|
||||||
|
mediaChannelEl.textContent = `Could not start: ${e.message}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mediaSearchForm.addEventListener("submit", async (event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
const query = mediaSearchInput.value.trim();
|
||||||
|
if (!query) return;
|
||||||
|
mediaResultsEl.innerHTML = '<div class="media-empty">Searching YouTube Music…</div>';
|
||||||
|
try {
|
||||||
|
const data = await mediaPost("/search", { query, limit: 6 });
|
||||||
|
renderMediaResults(data.results);
|
||||||
|
} catch (e) {
|
||||||
|
mediaResultsEl.innerHTML = `<div class="media-empty">Search failed: ${escapeHtml(e.message)}</div>`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
function renderMediaChoices(choices) {
|
||||||
|
mediaChoicesWrap.hidden = choices.length === 0;
|
||||||
|
mediaChoicesEl.innerHTML = "";
|
||||||
|
for (const c of choices) {
|
||||||
|
const row = document.createElement("div");
|
||||||
|
row.className = "media-row media-choice";
|
||||||
|
const info = document.createElement("div");
|
||||||
|
info.className = "media-row-info";
|
||||||
|
const dur = fmtListDuration(c.duration);
|
||||||
|
info.innerHTML = `<strong>${escapeHtml(c.title)}</strong>` +
|
||||||
|
`<span>${escapeHtml(c.channel || "")}${dur ? " · " + dur : ""}</span>`;
|
||||||
|
const play = mediaRowActions();
|
||||||
|
play.addEventListener("click", () => playVideoId(c));
|
||||||
|
row.appendChild(info);
|
||||||
|
row.appendChild(play);
|
||||||
|
mediaChoicesEl.appendChild(row);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshLibrary() {
|
||||||
|
let data;
|
||||||
|
try {
|
||||||
|
data = await fetch("/media/api/library").then((r) => r.json());
|
||||||
|
} catch (e) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
mediaLibraryEl.innerHTML = "";
|
||||||
|
const tracks = data.tracks || [];
|
||||||
|
if (tracks.length === 0) {
|
||||||
|
mediaLibraryEl.innerHTML = '<div class="media-empty">Downloaded songs and streams live here.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const t of tracks) {
|
||||||
|
const row = document.createElement("div");
|
||||||
|
row.className = "media-row";
|
||||||
|
const info = document.createElement("div");
|
||||||
|
info.className = "media-row-info";
|
||||||
|
const dur = fmtListDuration(t.duration);
|
||||||
|
const statusTag = t.status === "ready" ? "" : ` · ${t.status}`;
|
||||||
|
info.innerHTML = `<strong>${escapeHtml(t.title)}</strong>` +
|
||||||
|
`<span>${escapeHtml(t.channel || (t.kind === "stream" ? "Live stream" : ""))}` +
|
||||||
|
`${dur ? " · " + dur : ""}${statusTag}</span>`;
|
||||||
|
const play = mediaRowActions();
|
||||||
|
play.disabled = t.status !== "ready";
|
||||||
|
play.addEventListener("click", () => {
|
||||||
|
mediaPost("/play", { track_id: t.id }).catch((e) => {
|
||||||
|
mediaChannelEl.textContent = `Could not play: ${e.message}`;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
const del = document.createElement("button");
|
||||||
|
del.className = "btn btn-icon media-row-delete";
|
||||||
|
del.type = "button";
|
||||||
|
del.setAttribute("aria-label", `Remove ${t.title}`);
|
||||||
|
del.title = "Remove from library";
|
||||||
|
del.innerHTML = '<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>';
|
||||||
|
del.addEventListener("click", async () => {
|
||||||
|
try {
|
||||||
|
await fetch(`/media/api/tracks/${encodeURIComponent(t.id)}`, { method: "DELETE" });
|
||||||
|
if (t.id === mediaCurrentTrackId) clearMediaAudio();
|
||||||
|
refreshLibrary();
|
||||||
|
} catch (e) { /* leave the row; next refresh decides */ }
|
||||||
|
});
|
||||||
|
row.appendChild(info);
|
||||||
|
row.appendChild(play);
|
||||||
|
row.appendChild(del);
|
||||||
|
mediaLibraryEl.appendChild(row);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
streamForm.addEventListener("submit", async (event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
const name = streamNameInput.value.trim() || streamUrlInput.value.trim();
|
||||||
|
const url = streamUrlInput.value.trim();
|
||||||
|
if (!url) return;
|
||||||
|
try {
|
||||||
|
await mediaPost("/streams", { name, url });
|
||||||
|
streamNameInput.value = "";
|
||||||
|
streamUrlInput.value = "";
|
||||||
|
refreshLibrary();
|
||||||
|
} catch (e) {
|
||||||
|
mediaChannelEl.textContent = `Could not add stream: ${e.message}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Panel open/close ────────────────────────────────────────────────────────
|
||||||
|
function setMediaOpen(open) {
|
||||||
|
mediaPanel.classList.toggle("open", open);
|
||||||
|
mediaBtn.setAttribute("aria-expanded", String(open));
|
||||||
|
if (open) {
|
||||||
|
refreshLibrary();
|
||||||
|
pollMediaState();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mediaBtn.setAttribute("aria-expanded", "false");
|
||||||
|
mediaBtn.addEventListener("click", () => setMediaOpen(!mediaPanel.classList.contains("open")));
|
||||||
|
closeMediaBtn.addEventListener("click", () => setMediaOpen(false));
|
||||||
|
|
||||||
|
startMediaPolling();
|
||||||
|
updateMediaControls();
|
||||||
@@ -42,6 +42,11 @@
|
|||||||
<span>Tasks</span>
|
<span>Tasks</span>
|
||||||
<span class="tasks-badge" id="tasksBadge" hidden>0</span>
|
<span class="tasks-badge" id="tasksBadge" hidden>0</span>
|
||||||
</button>
|
</button>
|
||||||
|
<button id="mediaBtn" class="btn btn-secondary tasks-trigger" aria-label="Music player" title="Music player">
|
||||||
|
<svg viewBox="0 0 24 24" width="17" height="17" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
||||||
|
<span>Music</span>
|
||||||
|
<span class="tasks-badge" id="mediaBadge" hidden>♫</span>
|
||||||
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
@@ -149,6 +154,85 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div class="tasks-panel media-panel" id="mediaPanel" aria-label="Music player">
|
||||||
|
<div class="tasks-header">
|
||||||
|
<div>
|
||||||
|
<h2>Music</h2>
|
||||||
|
<p>YouTube Music downloads and live streams, streamed to you.</p>
|
||||||
|
</div>
|
||||||
|
<button id="closeMediaBtn" class="btn btn-icon" aria-label="Close music player">
|
||||||
|
<svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="media-now">
|
||||||
|
<img id="mediaArt" class="media-art" alt="" hidden>
|
||||||
|
<div class="media-art-placeholder" id="mediaArtPlaceholder" aria-hidden="true">
|
||||||
|
<svg viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" aria-hidden="true"><path d="M9 18V5l12-2v13"></path><circle cx="6" cy="18" r="3"></circle><circle cx="18" cy="16" r="3"></circle></svg>
|
||||||
|
</div>
|
||||||
|
<div class="media-now-info">
|
||||||
|
<strong id="mediaTitle">Nothing loaded</strong>
|
||||||
|
<span id="mediaChannel"></span>
|
||||||
|
<div class="media-progress" id="mediaProgressWrap" hidden>
|
||||||
|
<div class="media-progress-fill" id="mediaProgressFill"></div>
|
||||||
|
<span class="media-progress-label" id="mediaProgressLabel">Fetching…</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="media-controls" role="group" aria-label="Playback controls">
|
||||||
|
<button id="rewBtn" class="btn btn-icon" aria-label="Back 10 seconds" title="Back 10s">
|
||||||
|
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 17a5 5 0 1 0-5-5"></path><path d="M11 17H6v-5"></path><path d="m19 5-4 4h-3"></path><path d="m15 9 4-4"></path></svg>
|
||||||
|
</button>
|
||||||
|
<button id="playPauseBtn" class="btn btn-icon media-play" aria-label="Play or pause" title="Play / pause">
|
||||||
|
<svg id="playIcon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true"><path d="M8 5v14l11-7Z"></path></svg>
|
||||||
|
<svg id="pauseIcon" viewBox="0 0 24 24" width="22" height="22" fill="currentColor" aria-hidden="true" hidden><path d="M7 5h4v14H7zM13 5h4v14h-4z"></path></svg>
|
||||||
|
</button>
|
||||||
|
<button id="fwdBtn" class="btn btn-icon" aria-label="Forward 10 seconds" title="Forward 10s">
|
||||||
|
<svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M13 17a5 5 0 1 1 5-5"></path><path d="M13 17h5v-5"></path><path d="m5 5 4 4h3"></path><path d="m9 9 4-4"></path></svg>
|
||||||
|
</button>
|
||||||
|
<button id="stopMediaBtn" class="btn btn-icon" aria-label="Stop" title="Stop">
|
||||||
|
<svg viewBox="0 0 24 24" width="18" height="18" fill="currentColor" aria-hidden="true"><rect x="6" y="6" width="12" height="12" rx="2"></rect></svg>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="media-seek">
|
||||||
|
<span class="media-time" id="mediaCur">0:00</span>
|
||||||
|
<input type="range" id="seekBar" min="0" max="1000" value="0" step="1" aria-label="Seek position">
|
||||||
|
<span class="media-time" id="mediaDur">0:00</span>
|
||||||
|
</div>
|
||||||
|
<div class="media-volume">
|
||||||
|
<svg viewBox="0 0 24 24" width="15" height="15" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M11 5 6 9H2v6h4l5 4V5Z"></path><path d="M15.5 8.5a5 5 0 0 1 0 7"></path></svg>
|
||||||
|
<input type="range" id="volumeBar" min="0" max="100" value="90" aria-label="Volume">
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h3 class="sheet-section-title">Search YouTube Music</h3>
|
||||||
|
<form class="media-search" id="mediaSearchForm">
|
||||||
|
<input id="mediaSearchInput" type="search" placeholder="Song, artist, lyrics…" autocomplete="off">
|
||||||
|
<button class="btn btn-secondary" type="submit">Find</button>
|
||||||
|
</form>
|
||||||
|
<div class="media-list" id="mediaResults"></div>
|
||||||
|
|
||||||
|
<div id="mediaChoicesWrap" hidden>
|
||||||
|
<h3 class="sheet-section-title">Hope’s suggestions</h3>
|
||||||
|
<div class="media-list" id="mediaChoices"></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h3 class="sheet-section-title">Library</h3>
|
||||||
|
<div class="media-list" id="mediaLibrary">
|
||||||
|
<div class="media-empty">Downloaded songs and streams live here.</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h3 class="sheet-section-title">Add live stream</h3>
|
||||||
|
<form class="media-search media-stream-form" id="streamForm">
|
||||||
|
<input id="streamName" type="text" placeholder="Station name" autocomplete="off">
|
||||||
|
<input id="streamUrl" type="url" placeholder="https://… audio stream URL" required>
|
||||||
|
<button class="btn btn-secondary" type="submit">Add</button>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<audio id="mediaAudio" preload="metadata"></audio>
|
||||||
|
|
||||||
<script src="livekit-client.umd.js"></script>
|
<script src="livekit-client.umd.js"></script>
|
||||||
<script src="app.js"></script>
|
<script src="app.js"></script>
|
||||||
</body>
|
</body>
|
||||||
|
|||||||
@@ -0,0 +1,639 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Media server for the Hope voice assistant.
|
||||||
|
|
||||||
|
A small FastAPI service that owns the permanent music library and player
|
||||||
|
coordination:
|
||||||
|
|
||||||
|
- Library: downloaded YouTube Music tracks (mp3, transcoded via yt-dlp +
|
||||||
|
ffmpeg) and user-added live audio stream URLs, persisted in library.json.
|
||||||
|
- Search: yt-dlp ``ytsearch`` (YouTube Music pages share video ids), used by
|
||||||
|
Hope's media MCP toolset and by the web UI search box.
|
||||||
|
- Download jobs: one-at-a-time worker thread; progress is parsed from
|
||||||
|
yt-dlp stdout and surfaced through GET /media/api/state.
|
||||||
|
- Player state: per-room desired state (loaded track, playing/paused/stopped,
|
||||||
|
seek hints). The browser is the actual player and reconciles against this;
|
||||||
|
it reports real playback state back so Hope's voice tools can answer
|
||||||
|
"what's playing".
|
||||||
|
|
||||||
|
Security posture matches the token server: bind 127.0.0.1 only, reachability
|
||||||
|
exclusively through the nginx HTTPS proxy (/media/api/). File bytes for
|
||||||
|
downloaded tracks are served as nginx static files (/media/files/, native
|
||||||
|
HTTP Range support for seeking); this service only proxies external stream
|
||||||
|
URLs (mixed-content: an http:// radio URL would be blocked by the browser,
|
||||||
|
so it is fetched server-side and re-served over HTTPS).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI, HTTPException, Request
|
||||||
|
from fastapi.responses import JSONResponse, RedirectResponse, StreamingResponse
|
||||||
|
|
||||||
|
MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/data/media"))
|
||||||
|
FILES_DIR = MEDIA_DIR / "files"
|
||||||
|
THUMBS_DIR = MEDIA_DIR / "thumbs"
|
||||||
|
JOBS_DIR = MEDIA_DIR / "jobs"
|
||||||
|
LIBRARY_FILE = MEDIA_DIR / "library.json"
|
||||||
|
COOKIES_FILE = os.environ.get("YTDLP_COOKIES", "")
|
||||||
|
MEDIA_HOST = os.environ.get("MEDIA_SERVER_HOST", "127.0.0.1")
|
||||||
|
MEDIA_PORT = int(os.environ.get("MEDIA_SERVER_PORT", "8092"))
|
||||||
|
|
||||||
|
DOWNLOAD_TIMEOUT = int(os.environ.get("MEDIA_DOWNLOAD_TIMEOUT", "600"))
|
||||||
|
SEARCH_TIMEOUT = int(os.environ.get("MEDIA_SEARCH_TIMEOUT", "45"))
|
||||||
|
|
||||||
|
app = FastAPI(title="hope-media", docs_url=None, redoc_url=None, openapi_url=None)
|
||||||
|
|
||||||
|
_lock = threading.Lock() # guards library + rooms below
|
||||||
|
_library: dict[str, Any] = {"tracks": {}, "version": 1}
|
||||||
|
_rooms: dict[str, dict[str, Any]] = {}
|
||||||
|
_jobs: dict[str, dict[str, Any]] = {}
|
||||||
|
_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="media-dl")
|
||||||
|
|
||||||
|
for _d in (FILES_DIR, THUMBS_DIR, JOBS_DIR):
|
||||||
|
_d.mkdir(parents=True, exist_ok=True)
|
||||||
|
# The bind mount may be inspected/cleaned from the host by a non-root
|
||||||
|
# user (Unraid share convention) — keep the dirs world-traversable.
|
||||||
|
try:
|
||||||
|
os.chmod(_d, 0o777)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# ── Library store ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _load_library() -> None:
|
||||||
|
if LIBRARY_FILE.exists():
|
||||||
|
try:
|
||||||
|
data = json.loads(LIBRARY_FILE.read_text(encoding="utf-8"))
|
||||||
|
tracks = data.get("tracks", {})
|
||||||
|
if isinstance(tracks, dict):
|
||||||
|
_library["tracks"] = tracks
|
||||||
|
_library["version"] = int(data.get("version", 1))
|
||||||
|
except (json.JSONDecodeError, OSError):
|
||||||
|
# Corrupt library: keep the broken file aside, start fresh.
|
||||||
|
try:
|
||||||
|
LIBRARY_FILE.replace(LIBRARY_FILE.with_suffix(".json.broken"))
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _save_library() -> None:
|
||||||
|
tmp = LIBRARY_FILE.with_suffix(".json.tmp")
|
||||||
|
tmp.write_text(json.dumps(_library), encoding="utf-8")
|
||||||
|
tmp.replace(LIBRARY_FILE)
|
||||||
|
|
||||||
|
|
||||||
|
def _bump_version() -> int:
|
||||||
|
_library["version"] = int(_library.get("version", 1)) + 1
|
||||||
|
return _library["version"]
|
||||||
|
|
||||||
|
|
||||||
|
def _track_dict(track: dict) -> dict:
|
||||||
|
out = dict(track)
|
||||||
|
if track["kind"] == "youtube":
|
||||||
|
if track.get("status") == "ready":
|
||||||
|
out["audio_url"] = f"/media/files/{track['id']}.mp3"
|
||||||
|
out["art_url"] = (
|
||||||
|
f"/media/thumbs/{track['id']}.jpg"
|
||||||
|
if (THUMBS_DIR / f"{track['id']}.jpg").exists()
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
out["audio_url"] = f"/media/api/stream/{track['id']}"
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def _get_track(track_id: str) -> dict | None:
|
||||||
|
return _library["tracks"].get(track_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _upsert_track(track: dict) -> None:
|
||||||
|
_library["tracks"][track["id"]] = track
|
||||||
|
_bump_version()
|
||||||
|
_save_library()
|
||||||
|
|
||||||
|
|
||||||
|
_load_library()
|
||||||
|
|
||||||
|
|
||||||
|
# ── Player state (per room) ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _room_state(room: str) -> dict:
|
||||||
|
return _rooms.setdefault(
|
||||||
|
room,
|
||||||
|
{"track_id": None, "desired": "stopped", "reported": None,
|
||||||
|
"seek_hint": None, "choices": []},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _room_snapshot(room: str) -> dict:
|
||||||
|
st = _room_state(room)
|
||||||
|
track = _get_track(st["track_id"]) if st["track_id"] else None
|
||||||
|
pending = [
|
||||||
|
{
|
||||||
|
"job_id": j["id"],
|
||||||
|
"track_id": j["track_id"],
|
||||||
|
"stage": j["stage"],
|
||||||
|
"progress": j["progress"],
|
||||||
|
"error": j.get("error"),
|
||||||
|
}
|
||||||
|
for j in _jobs.values()
|
||||||
|
if j["stage"] in ("queued", "downloading", "transcoding")
|
||||||
|
]
|
||||||
|
return {
|
||||||
|
"room": room,
|
||||||
|
"track": _track_dict(track) if track else None,
|
||||||
|
"desired": st["desired"],
|
||||||
|
"reported": st["reported"],
|
||||||
|
"seek_hint": st["seek_hint"],
|
||||||
|
"choices": st["choices"],
|
||||||
|
"pending": pending,
|
||||||
|
"version": _library["version"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── yt-dlp helpers ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
def _ytdlp_base_args() -> list[str]:
|
||||||
|
args = [sys.executable, "-m", "yt_dlp", "--no-playlist", "--no-warnings"]
|
||||||
|
if COOKIES_FILE and Path(COOKIES_FILE).exists():
|
||||||
|
args += ["--cookies", COOKIES_FILE]
|
||||||
|
return args
|
||||||
|
|
||||||
|
|
||||||
|
_PROGRESS_RE = re.compile(r"\[download\]\s+([\d.]+)%")
|
||||||
|
|
||||||
|
|
||||||
|
def _run_search(query: str, limit: int) -> list[dict]:
|
||||||
|
"""Flat ytsearch lookup — cheap, no download, works anonymously."""
|
||||||
|
cmd = _ytdlp_base_args() + [
|
||||||
|
"--flat-playlist", "--dump-json",
|
||||||
|
f"ytsearch{limit}:{query}",
|
||||||
|
]
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(
|
||||||
|
cmd, capture_output=True, text=True, timeout=SEARCH_TIMEOUT, check=False
|
||||||
|
)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
raise HTTPException(504, "YouTube Music search timed out")
|
||||||
|
if proc.returncode != 0 and not proc.stdout.strip():
|
||||||
|
detail = (proc.stderr or "").strip().splitlines()[-1:] or ["unknown error"]
|
||||||
|
raise HTTPException(502, f"Search failed: {detail[0][:300]}")
|
||||||
|
results = []
|
||||||
|
for line in proc.stdout.splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line.startswith("{"):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
entry = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
video_id = entry.get("id")
|
||||||
|
if not video_id:
|
||||||
|
continue
|
||||||
|
existing = _get_track(video_id)
|
||||||
|
results.append({
|
||||||
|
"video_id": video_id,
|
||||||
|
"title": entry.get("title") or video_id,
|
||||||
|
"channel": entry.get("uploader") or entry.get("channel") or "",
|
||||||
|
"duration": entry.get("duration"),
|
||||||
|
"view_count": entry.get("view_count"),
|
||||||
|
"in_library": bool(existing),
|
||||||
|
"track_id": existing["id"] if existing else None,
|
||||||
|
"library_status": existing["status"] if existing else None,
|
||||||
|
})
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _job_dict(job: dict) -> dict:
|
||||||
|
return {k: job.get(k) for k in
|
||||||
|
("id", "track_id", "stage", "progress", "error", "started_at")}
|
||||||
|
|
||||||
|
|
||||||
|
def _persist_job(job: dict) -> None:
|
||||||
|
try:
|
||||||
|
(JOBS_DIR / f"{job['id']}.json").write_text(
|
||||||
|
json.dumps(_job_dict(job)), encoding="utf-8"
|
||||||
|
)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _fail_job(job: dict, track: dict, message: str) -> None:
|
||||||
|
job["stage"] = "error"
|
||||||
|
job["error"] = message[:500]
|
||||||
|
_persist_job(job)
|
||||||
|
with _lock:
|
||||||
|
track["status"] = "failed"
|
||||||
|
track["error"] = message[:500]
|
||||||
|
_bump_version()
|
||||||
|
_save_library()
|
||||||
|
|
||||||
|
|
||||||
|
def _download_worker(job: dict, track: dict, video_id: str) -> None:
|
||||||
|
"""Runs in the single-worker executor: yt-dlp download + mp3 transcode."""
|
||||||
|
target = FILES_DIR / f"{track['id']}.mp3"
|
||||||
|
cmd = _ytdlp_base_args() + [
|
||||||
|
"-f", "bestaudio/best",
|
||||||
|
"-x", "--audio-format", "mp3", "--audio-quality", "192K",
|
||||||
|
"--embed-thumbnail", "--add-metadata",
|
||||||
|
"--write-thumbnail", "--convert-thumbnails", "jpg",
|
||||||
|
"--newline", "--progress",
|
||||||
|
"--print-json",
|
||||||
|
"--retries", "3", "--fragment-retries", "3",
|
||||||
|
"-P", str(FILES_DIR),
|
||||||
|
"-o", f"{track['id']}.%(ext)s",
|
||||||
|
f"https://www.youtube.com/watch?v={video_id}",
|
||||||
|
]
|
||||||
|
job["stage"] = "downloading"
|
||||||
|
info: dict = {}
|
||||||
|
# stderr goes to a temp file: reading it from a PIPE while the stdout
|
||||||
|
# loop blocks could deadlock if yt-dlp ever fills the stderr buffer.
|
||||||
|
with tempfile.TemporaryFile(mode="w+", encoding="utf-8", errors="replace") as errfile:
|
||||||
|
try:
|
||||||
|
proc = subprocess.Popen(
|
||||||
|
cmd, stdout=subprocess.PIPE, stderr=errfile,
|
||||||
|
text=True, errors="replace",
|
||||||
|
)
|
||||||
|
deadline = time.monotonic() + DOWNLOAD_TIMEOUT
|
||||||
|
assert proc.stdout is not None
|
||||||
|
for line in proc.stdout:
|
||||||
|
if time.monotonic() > deadline:
|
||||||
|
proc.kill()
|
||||||
|
proc.wait(timeout=30)
|
||||||
|
raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
|
||||||
|
line = line.strip()
|
||||||
|
m = _PROGRESS_RE.search(line)
|
||||||
|
if m:
|
||||||
|
job["progress"] = min(85.0, float(m.group(1)) * 0.85)
|
||||||
|
_persist_job(job)
|
||||||
|
elif line.startswith(("[ExtractAudio]", "[Metadata]", "[EmbedThumbnail]")):
|
||||||
|
job["stage"] = "transcoding"
|
||||||
|
job["progress"] = max(job["progress"], 86.0)
|
||||||
|
_persist_job(job)
|
||||||
|
elif line.startswith("{"):
|
||||||
|
try:
|
||||||
|
info = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
proc.wait(timeout=30)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
errfile.seek(0)
|
||||||
|
stderr_tail = (errfile.read() or "").strip().splitlines()
|
||||||
|
detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
|
||||||
|
raise RuntimeError(detail)
|
||||||
|
if not target.exists():
|
||||||
|
raise RuntimeError("yt-dlp finished but no mp3 was produced")
|
||||||
|
except Exception as exc: # noqa: BLE001 — any failure marks the track failed
|
||||||
|
_fail_job(job, track, str(exc))
|
||||||
|
return
|
||||||
|
|
||||||
|
thumb_src = FILES_DIR / f"{track['id']}.jpg"
|
||||||
|
try:
|
||||||
|
if thumb_src.exists():
|
||||||
|
thumb_src.replace(THUMBS_DIR / f"{track['id']}.jpg")
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
job["stage"] = "ready"
|
||||||
|
job["progress"] = 100.0
|
||||||
|
_persist_job(job)
|
||||||
|
with _lock:
|
||||||
|
track["status"] = "ready"
|
||||||
|
track["error"] = None
|
||||||
|
if info:
|
||||||
|
if info.get("title"):
|
||||||
|
track["title"] = info["title"]
|
||||||
|
if info.get("uploader") or info.get("channel"):
|
||||||
|
track["channel"] = info.get("uploader") or info.get("channel")
|
||||||
|
if info.get("duration"):
|
||||||
|
track["duration"] = round(float(info["duration"]))
|
||||||
|
_bump_version()
|
||||||
|
_save_library()
|
||||||
|
|
||||||
|
|
||||||
|
def _start_download(track: dict, video_id: str) -> dict:
|
||||||
|
job = {
|
||||||
|
"id": uuid.uuid4().hex[:12],
|
||||||
|
"track_id": track["id"],
|
||||||
|
"stage": "queued",
|
||||||
|
"progress": 0.0,
|
||||||
|
"error": None,
|
||||||
|
"started_at": time.time(),
|
||||||
|
}
|
||||||
|
_jobs[job["id"]] = job
|
||||||
|
_persist_job(job)
|
||||||
|
_executor.submit(_download_worker, job, track, video_id)
|
||||||
|
return job
|
||||||
|
|
||||||
|
|
||||||
|
# ── API: health, library ────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
@app.get("/media/api/healthz")
|
||||||
|
def healthz() -> dict:
|
||||||
|
return {"ok": True, "version": _library["version"]}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/media/api/library")
|
||||||
|
def get_library() -> dict:
|
||||||
|
with _lock:
|
||||||
|
tracks = sorted(
|
||||||
|
(_track_dict(t) for t in _library["tracks"].values()),
|
||||||
|
key=lambda t: t.get("added_at", 0),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
return {"tracks": tracks, "version": _library["version"]}
|
||||||
|
|
||||||
|
|
||||||
|
def _delete_track_files(track: dict) -> None:
|
||||||
|
if track["kind"] != "youtube":
|
||||||
|
return
|
||||||
|
for path in (
|
||||||
|
FILES_DIR / f"{track['id']}.mp3",
|
||||||
|
FILES_DIR / f"{track['id']}.jpg",
|
||||||
|
THUMBS_DIR / f"{track['id']}.jpg",
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
path.unlink(missing_ok=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/media/api/tracks/{track_id}")
|
||||||
|
def delete_track(track_id: str) -> dict:
|
||||||
|
with _lock:
|
||||||
|
track = _get_track(track_id)
|
||||||
|
if not track:
|
||||||
|
raise HTTPException(404, "unknown track")
|
||||||
|
del _library["tracks"][track_id]
|
||||||
|
_bump_version()
|
||||||
|
_save_library()
|
||||||
|
for room, st in _rooms.items():
|
||||||
|
if st["track_id"] == track_id:
|
||||||
|
st["track_id"] = None
|
||||||
|
st["desired"] = "stopped"
|
||||||
|
_delete_track_files(track)
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
|
# ── API: search + play + control ────────────────────────────────────────────
|
||||||
|
|
||||||
|
@app.post("/media/api/search")
|
||||||
|
async def search(request: Request) -> dict:
|
||||||
|
body = await request.json()
|
||||||
|
query = str(body.get("query", "")).strip()
|
||||||
|
if not query:
|
||||||
|
raise HTTPException(400, "query is required")
|
||||||
|
try:
|
||||||
|
limit = max(1, min(int(body.get("limit", 6)), 15))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
raise HTTPException(400, "limit must be an integer")
|
||||||
|
return {"query": query, "results": _run_search(query, limit)}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_play_target(body: dict) -> dict:
|
||||||
|
"""Find or create the track a play request refers to.
|
||||||
|
|
||||||
|
Accepts video_id (YouTube), track_id (existing library entry) or a raw
|
||||||
|
url+name (live stream). Returns the library track dict.
|
||||||
|
"""
|
||||||
|
video_id = str(body.get("video_id") or "").strip()
|
||||||
|
track_id = str(body.get("track_id") or "").strip()
|
||||||
|
url = str(body.get("url") or "").strip()
|
||||||
|
|
||||||
|
if video_id:
|
||||||
|
existing = _get_track(video_id)
|
||||||
|
if existing:
|
||||||
|
return existing
|
||||||
|
track = {
|
||||||
|
"id": video_id,
|
||||||
|
"kind": "youtube",
|
||||||
|
"video_id": video_id,
|
||||||
|
"title": str(body.get("title") or video_id),
|
||||||
|
"channel": str(body.get("channel") or ""),
|
||||||
|
"duration": body.get("duration"),
|
||||||
|
"status": "downloading",
|
||||||
|
"error": None,
|
||||||
|
"added_at": time.time(),
|
||||||
|
}
|
||||||
|
_upsert_track(track)
|
||||||
|
_start_download(track, video_id)
|
||||||
|
return track
|
||||||
|
|
||||||
|
if track_id:
|
||||||
|
track = _get_track(track_id)
|
||||||
|
if not track:
|
||||||
|
raise HTTPException(404, "unknown track")
|
||||||
|
return track
|
||||||
|
|
||||||
|
if url:
|
||||||
|
if not re.match(r"^https?://", url):
|
||||||
|
raise HTTPException(400, "url must be http(s)")
|
||||||
|
track_id = "s" + uuid.uuid4().hex[:10]
|
||||||
|
track = {
|
||||||
|
"id": track_id,
|
||||||
|
"kind": "stream",
|
||||||
|
"video_id": None,
|
||||||
|
"title": str(body.get("name") or url),
|
||||||
|
"channel": str(body.get("channel") or ""),
|
||||||
|
"duration": None,
|
||||||
|
"status": "ready",
|
||||||
|
"error": None,
|
||||||
|
"url": url,
|
||||||
|
"added_at": time.time(),
|
||||||
|
}
|
||||||
|
_upsert_track(track)
|
||||||
|
return track
|
||||||
|
|
||||||
|
raise HTTPException(400, "one of video_id, track_id or url is required")
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/media/api/play")
|
||||||
|
async def play(request: Request) -> dict:
|
||||||
|
body = await request.json()
|
||||||
|
room = str(body.get("room") or "default")
|
||||||
|
track = _resolve_play_target(body)
|
||||||
|
with _lock:
|
||||||
|
st = _room_state(room)
|
||||||
|
st["track_id"] = track["id"]
|
||||||
|
# start_paused carries a track into a new scope without starting it —
|
||||||
|
# the UI uses this when a conversation begins (talking pauses music).
|
||||||
|
st["desired"] = "paused" if body.get("start_paused") else "playing"
|
||||||
|
st["choices"] = [] # a pick resolves any open suggestion list
|
||||||
|
st["seek_hint"] = None
|
||||||
|
snapshot = _room_snapshot(room)
|
||||||
|
return {"ok": True, **snapshot}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/media/api/control")
|
||||||
|
async def control(request: Request) -> dict:
|
||||||
|
body = await request.json()
|
||||||
|
room = str(body.get("room") or "default")
|
||||||
|
action = str(body.get("action") or "")
|
||||||
|
with _lock:
|
||||||
|
st = _room_state(room)
|
||||||
|
if action == "pause":
|
||||||
|
st["desired"] = "paused"
|
||||||
|
elif action == "resume":
|
||||||
|
if not st["track_id"]:
|
||||||
|
raise HTTPException(409, "nothing loaded")
|
||||||
|
st["desired"] = "playing"
|
||||||
|
elif action == "stop":
|
||||||
|
st["desired"] = "stopped"
|
||||||
|
elif action == "seek":
|
||||||
|
try:
|
||||||
|
position = max(0.0, float(body.get("position", 0)))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
raise HTTPException(400, "position must be a number")
|
||||||
|
st["seek_hint"] = {"position": position, "ts": time.time()}
|
||||||
|
elif action == "skip":
|
||||||
|
current = st["track_id"]
|
||||||
|
candidates = sorted(
|
||||||
|
(
|
||||||
|
t for t in _library["tracks"].values()
|
||||||
|
if t.get("status") == "ready" and t["id"] != current
|
||||||
|
),
|
||||||
|
key=lambda t: t.get("added_at", 0),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
if candidates:
|
||||||
|
st["track_id"] = candidates[0]["id"]
|
||||||
|
st["desired"] = "playing"
|
||||||
|
st["seek_hint"] = None
|
||||||
|
else:
|
||||||
|
st["desired"] = "stopped"
|
||||||
|
else:
|
||||||
|
raise HTTPException(400, f"unknown action: {action}")
|
||||||
|
snapshot = _room_snapshot(room)
|
||||||
|
return {"ok": True, **snapshot}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/media/api/choices")
|
||||||
|
async def choices(request: Request) -> dict:
|
||||||
|
"""Publish Hope's song suggestions so the UI can render clickable picks."""
|
||||||
|
body = await request.json()
|
||||||
|
room = str(body.get("room") or "default")
|
||||||
|
raw = body.get("choices") or []
|
||||||
|
cleaned = []
|
||||||
|
for c in raw[:5]:
|
||||||
|
if not isinstance(c, dict) or not c.get("video_id"):
|
||||||
|
continue
|
||||||
|
cleaned.append({
|
||||||
|
"video_id": str(c["video_id"]),
|
||||||
|
"title": str(c.get("title") or c["video_id"]),
|
||||||
|
"channel": str(c.get("channel") or ""),
|
||||||
|
"duration": c.get("duration"),
|
||||||
|
})
|
||||||
|
with _lock:
|
||||||
|
_room_state(room)["choices"] = cleaned
|
||||||
|
return {"ok": True, "choices": cleaned}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/media/api/state")
|
||||||
|
def state(room: str = "default") -> dict:
|
||||||
|
with _lock:
|
||||||
|
return _room_snapshot(room)
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/media/api/report")
|
||||||
|
async def report(request: Request) -> dict:
|
||||||
|
"""The browser reports real playback state (source of truth for the player)."""
|
||||||
|
body = await request.json()
|
||||||
|
room = str(body.get("room") or "default")
|
||||||
|
try:
|
||||||
|
position = float(body.get("position") or 0.0)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
position = 0.0
|
||||||
|
with _lock:
|
||||||
|
st = _room_state(room)
|
||||||
|
st["reported"] = {
|
||||||
|
"track_id": body.get("track_id"),
|
||||||
|
"state": str(body.get("state") or "stopped"),
|
||||||
|
"position": position,
|
||||||
|
"duration": body.get("duration"),
|
||||||
|
"ts": time.time(),
|
||||||
|
}
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/media/api/streams")
|
||||||
|
async def add_stream(request: Request) -> dict:
|
||||||
|
body = await request.json()
|
||||||
|
track = _resolve_play_target({"url": body.get("url"), "name": body.get("name")})
|
||||||
|
return {"ok": True, "track": _track_dict(track)}
|
||||||
|
|
||||||
|
|
||||||
|
# ── API: stream proxy (external http(s) audio over same-origin HTTPS) ───────
|
||||||
|
|
||||||
|
_HOP_HEADERS = ("content-type", "content-range", "accept-ranges", "content-length")
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/media/api/stream/{track_id}")
|
||||||
|
async def stream_proxy(track_id: str, request: Request):
|
||||||
|
with _lock:
|
||||||
|
track = _get_track(track_id)
|
||||||
|
if not track:
|
||||||
|
raise HTTPException(404, "unknown track")
|
||||||
|
if track["kind"] == "youtube":
|
||||||
|
# Downloaded tracks are nginx static files (native Range support).
|
||||||
|
if track.get("status") == "ready":
|
||||||
|
return RedirectResponse(f"/media/files/{track_id}.mp3")
|
||||||
|
raise HTTPException(409, f"track is {track.get('status')}, not ready")
|
||||||
|
|
||||||
|
range_header = request.headers.get("range")
|
||||||
|
headers = {"Range": range_header} if range_header else {}
|
||||||
|
client = httpx.AsyncClient(timeout=httpx.Timeout(30, read=600), follow_redirects=True)
|
||||||
|
try:
|
||||||
|
upstream = await client.send(client.build_request("GET", track["url"], headers=headers), stream=True)
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
await client.aclose()
|
||||||
|
raise HTTPException(502, f"upstream stream failed: {exc}")
|
||||||
|
|
||||||
|
if upstream.status_code >= 400:
|
||||||
|
await upstream.aclose()
|
||||||
|
await client.aclose()
|
||||||
|
raise HTTPException(502, f"upstream returned {upstream.status_code}")
|
||||||
|
|
||||||
|
out_headers = {
|
||||||
|
name: value
|
||||||
|
for name, value in upstream.headers.items()
|
||||||
|
if name.lower() in _HOP_HEADERS
|
||||||
|
}
|
||||||
|
media_type = upstream.headers.get("content-type", "audio/mpeg")
|
||||||
|
|
||||||
|
async def relay():
|
||||||
|
try:
|
||||||
|
async for chunk in upstream.aiter_bytes(64 * 1024):
|
||||||
|
yield chunk
|
||||||
|
finally:
|
||||||
|
await upstream.aclose()
|
||||||
|
await client.aclose()
|
||||||
|
|
||||||
|
return StreamingResponse(
|
||||||
|
relay(),
|
||||||
|
status_code=upstream.status_code,
|
||||||
|
media_type=media_type,
|
||||||
|
headers=out_headers,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@app.exception_handler(HTTPException)
|
||||||
|
async def http_error(_request: Request, exc: HTTPException):
|
||||||
|
return JSONResponse({"error": exc.detail}, status_code=exc.status_code)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
uvicorn.run(app, host=MEDIA_HOST, port=MEDIA_PORT, log_level="info")
|
||||||
+259
-1
@@ -1245,6 +1245,263 @@ body[data-mode="text"] .mic-meter {
|
|||||||
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
|
.task-result { margin-top: 0.5rem; color: #bcebd8; background: rgba(118, 217, 178, 0.1); }
|
||||||
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
|
.task-error { margin-top: 0.5rem; color: #ffc2ba; background: rgba(255, 128, 111, 0.1); }
|
||||||
|
|
||||||
|
/* ── Media player ───────────────────────────────────────────────────────── */
|
||||||
|
/* Reuses .tasks-panel positioning (fixed right, slides in via .open); the
|
||||||
|
content scrolls under a sticky header. */
|
||||||
|
.media-panel {
|
||||||
|
overflow-y: auto;
|
||||||
|
padding-bottom: 1.2rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-panel .tasks-header {
|
||||||
|
position: sticky;
|
||||||
|
top: 0;
|
||||||
|
z-index: 1;
|
||||||
|
background: #0d1929;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-now {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.75rem;
|
||||||
|
margin: 1rem 1.1rem 0;
|
||||||
|
padding: 0.75rem;
|
||||||
|
border-radius: 12px;
|
||||||
|
background: var(--surface-raised);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-art,
|
||||||
|
.media-art-placeholder {
|
||||||
|
width: 56px;
|
||||||
|
height: 56px;
|
||||||
|
flex: 0 0 56px;
|
||||||
|
border-radius: 8px;
|
||||||
|
object-fit: cover;
|
||||||
|
background: rgba(23, 41, 65, 0.7);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-art-placeholder {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
color: var(--ink-muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-now-info {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.15rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-now-info strong {
|
||||||
|
color: var(--ink);
|
||||||
|
font-size: 0.88rem;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-now-info span {
|
||||||
|
color: var(--ink-muted);
|
||||||
|
font-size: 0.72rem;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-progress {
|
||||||
|
position: relative;
|
||||||
|
margin-top: 0.35rem;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.5rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Track bar behind the fill (fill width is driven inline by progress). */
|
||||||
|
.media-progress::before {
|
||||||
|
content: "";
|
||||||
|
position: absolute;
|
||||||
|
left: 0;
|
||||||
|
top: 50%;
|
||||||
|
transform: translateY(-50%);
|
||||||
|
width: 90px;
|
||||||
|
height: 4px;
|
||||||
|
border-radius: 2px;
|
||||||
|
background: rgba(139, 167, 255, 0.18);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-progress-fill {
|
||||||
|
height: 4px;
|
||||||
|
flex: 0 0 auto;
|
||||||
|
width: 0;
|
||||||
|
border-radius: 2px;
|
||||||
|
background: var(--iris);
|
||||||
|
transition: width 400ms ease;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-progress-label {
|
||||||
|
color: var(--ink-muted);
|
||||||
|
font-size: 0.66rem;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-controls {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 0.9rem;
|
||||||
|
margin: 0.9rem 1.1rem 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-controls .btn-icon {
|
||||||
|
color: var(--ink-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-play {
|
||||||
|
width: 52px;
|
||||||
|
height: 52px;
|
||||||
|
border-radius: 50%;
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
color: #0b1523 !important;
|
||||||
|
background: var(--iris) !important;
|
||||||
|
border: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-play:hover:not(:disabled) {
|
||||||
|
filter: brightness(1.1);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-seek {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.55rem;
|
||||||
|
margin: 0.75rem 1.1rem 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-time {
|
||||||
|
color: var(--ink-muted);
|
||||||
|
font-size: 0.68rem;
|
||||||
|
font-variant-numeric: tabular-nums;
|
||||||
|
min-width: 2.6rem;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-seek input[type="range"],
|
||||||
|
.media-volume input[type="range"] {
|
||||||
|
flex: 1;
|
||||||
|
accent-color: var(--iris);
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-volume {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.55rem;
|
||||||
|
margin: 0.55rem 1.1rem 0;
|
||||||
|
color: var(--ink-muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-search {
|
||||||
|
display: flex;
|
||||||
|
gap: 0.45rem;
|
||||||
|
margin: 0 1.1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-search input {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
padding: 0.5rem 0.7rem;
|
||||||
|
border: 1px solid var(--line-strong);
|
||||||
|
border-radius: 9px;
|
||||||
|
color: var(--ink);
|
||||||
|
background: rgba(5, 11, 20, 0.45);
|
||||||
|
font-size: 0.8rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-search input::placeholder {
|
||||||
|
color: var(--ink-muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-stream-form input:first-child {
|
||||||
|
flex: 0 0 38%;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-list {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.4rem;
|
||||||
|
margin: 0.65rem 1.1rem 1rem;
|
||||||
|
max-height: 240px;
|
||||||
|
overflow-y: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row {
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 0.6rem;
|
||||||
|
padding: 0.55rem 0.65rem;
|
||||||
|
border-radius: 9px;
|
||||||
|
background: var(--surface-raised);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-info {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 0.1rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-info strong {
|
||||||
|
color: var(--ink);
|
||||||
|
font-size: 0.78rem;
|
||||||
|
font-weight: 600;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-info span {
|
||||||
|
color: var(--ink-muted);
|
||||||
|
font-size: 0.68rem;
|
||||||
|
overflow: hidden;
|
||||||
|
text-overflow: ellipsis;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-play {
|
||||||
|
flex: 0 0 auto;
|
||||||
|
padding: 0.3rem 0.7rem;
|
||||||
|
font-size: 0.7rem;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-delete {
|
||||||
|
flex: 0 0 auto;
|
||||||
|
color: var(--ink-muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-row-delete:hover {
|
||||||
|
color: #ffc2ba;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-empty {
|
||||||
|
padding: 0.8rem;
|
||||||
|
border-radius: 9px;
|
||||||
|
color: var(--ink-muted);
|
||||||
|
background: var(--surface-raised);
|
||||||
|
font-size: 0.74rem;
|
||||||
|
text-align: center;
|
||||||
|
}
|
||||||
|
|
||||||
|
.media-choice {
|
||||||
|
border: 1px solid rgba(139, 167, 255, 0.35);
|
||||||
|
background: rgba(60, 95, 201, 0.12);
|
||||||
|
}
|
||||||
|
|
||||||
/* ── Responsive ─────────────────────────────────────────────────────────── */
|
/* ── Responsive ─────────────────────────────────────────────────────────── */
|
||||||
@media (max-height: 760px) and (min-width: 881px) {
|
@media (max-height: 760px) and (min-width: 881px) {
|
||||||
.workspace {
|
.workspace {
|
||||||
@@ -1386,7 +1643,8 @@ body[data-mode="text"] .mic-meter {
|
|||||||
padding: 1.25rem 1rem;
|
padding: 1.25rem 1rem;
|
||||||
}
|
}
|
||||||
|
|
||||||
.tasks-panel {
|
.tasks-panel,
|
||||||
|
.media-panel {
|
||||||
width: 100vw;
|
width: 100vw;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -97,6 +97,9 @@ class Handler(BaseHTTPRequestHandler):
|
|||||||
resp = json.dumps(
|
resp = json.dumps(
|
||||||
{
|
{
|
||||||
"token": token,
|
"token": token,
|
||||||
|
# The room name scopes media-server player state; the UI sends
|
||||||
|
# it with every /media/api/ call.
|
||||||
|
"room": room_name,
|
||||||
"user": {
|
"user": {
|
||||||
"id": user_id,
|
"id": user_id,
|
||||||
"display_name": profile.get("display_name") or user_id,
|
"display_name": profile.get("display_name") or user_id,
|
||||||
|
|||||||
Reference in new issue
Block a user