feat: progressive music streaming, library-first search, player fixes

- Stream downloads as they grow: raw best-audio (no mp3 transcode) served
  chunked from the stream endpoint mid-download, then with native Range
  once complete. Songs play within seconds of hitting Play.
- Fix searched songs never starting: reconcileMedia now wires the stream
  URL when a track that was preparing gains one; audio_url is stable for
  the whole lifecycle so the player never reloads mid-playback.
- Progress bars now parse yt-dlp stderr (stdout never carried [download]).
- Prefer downloaded songs: /media/api/search merges ready library matches
  ahead of YouTube results; search_music/play_song tooling and the agent
  prompt tell Hope to always pick in_library=true.
- Mute button turns solid red while the mic is muted.

AGENTS.md

@@ -63,10 +63,13 @@ Key variables:
 ## Media Player
 
 Hope can play music on the web UI's media player: YouTube Music search →
-yt-dlp download → ffmpeg transcode to mp3 → nginx static streaming (HTTP
+yt-dlp raw best-audio download (webm/m4a, no transcode) → progressive
-Range) back to the browser's `<audio>` element. Live audio stream URLs
+streaming from the media server while the file grows → finished file served
-(internet radio) are added to the library and proxied over HTTPS by the
+with native HTTP Range for seeking. Songs start playing within seconds of the
-media server (mixed-content safe).
+download starting and remain in the library for instant offline replay.
+Live audio stream URLs (internet radio) are added to the library and proxied
+over HTTPS by the media server (mixed-content safe). Search results put ready
+library matches first so Hope prefers a downloaded copy over a fresh lookup.
 
 - The permanent library lives in `./media` (bind-mounted to `/data/media`):
   `files/` mp3s, `thumbs/` cover art, `jobs/` download status, `library.json`.
@@ -104,6 +107,45 @@ The LLM is instructed to:
 - Use contractions, natural fillers sparingly
 - Never use markdown, lists, or formatting
 - Spell out numbers and abbreviations
+- **Call tools, don't narrate them** — if a tool matches the request, the
+  tool call must appear in the reply; saying "I'll pause the music" without
+  the call is a failure (nothing happens)
+
+## Tool Calling on Small Local Models
+
+The local 3-4B models (qwen2.5-vl-3b, gemma-4-e4b) drift under real session
+conditions: with the full system prompt (~6.3KB) and the full tool surface
+(~26 tools), they narrate actions in first person ("Sure, I'll pause the
+music for you") **without emitting the tool call**. Measured before the fix:
+qwen emitted tools on 2/8 realistic intents, gemma 5/8 — while single-tool
+probes passed 12/12. Three layers address this:
+
+1. **Intent router** (`agent/intent_router.py`) — deterministic forced tool
+   calls for high-frequency, unambiguous intents (media controls,
+   now_playing/list_library, "remember that …" → memory_save, explicit
+   YouTube Music searches, personal-fact recall → memory_recall /
+   search_notes / skill_recall). It runs in `HopeLLM.chat()` before the LLM
+   (same mechanism as forced dispatch), only fires when the tool is
+   available, and guards against "how do I …" questions. New high-frequency
+   voice intents belong here — with unit tests — not in prompt hopes.
+   Combined forced-or-emitted coverage after the fix: 8/8 on both models.
+2. **Prompt rule** — the "call them, don't narrate them" section in
+   SYSTEM_PROMPT. Alone it lifted gemma to 8/8 and qwen to 4/8 LLM emission.
+3. **Schema sanitizer** (`_sanitize_tool_schema` in agent.py) — llama.cpp
+   derives a GBNF grammar from tool schemas for qwen's Hermes-style template
+   and **aborts on length-style constraints ≥ 2000** (the memory server's
+   `maxLength: 200000` broke all qwen tool calling; gemma's text-based tool
+   path tolerated it). The sanitizer caps minLength/maxLength/minItems/
+   maxItems at 1000 per chat call. Verified working post-fix: `anyOf`
+   nullable fields, `list[dict]` with open items, enums, and regex patterns.
+   Also `_strict_tool_schema=False`: llama.cpp's Gemma template ignores
+   named tool_choice under strict schemas.
+
+**Diagnostic method** (learned the hard way): a failing tool call with a
+small model is almost never the plumbing. Probe with the REAL conditions —
+full tool set, real system prompt, and a prompt that matches the tool
+(mismatched prompts make small models legitimately decline, which looks
+identical to a failure). Isolated single-tool probes will pass and mislead.
 
 ## Testing
 
@@ -111,7 +153,9 @@ The LLM is instructed to:
 via `VOICE_WEB_URL` in tests/conftest.py) — that is the instance users hit
 and the one CI (rebuild.yml) deploys on every push to master. The local dev
 compose instance is for development only; set `VOICE_WEB_URL=https://localhost:9178`
-to target it instead.
+to target it instead. Tests that can only pass on the deployment host
+(repo `./media` bind mount) or that probe the local container via
+`docker exec voice` skip automatically on remote targets.
 
 ```bash
 # Verify the container is running
@@ -122,7 +166,6 @@ python -m pytest tests/ -v -m "not slow"
 
 # Real end-to-end YouTube Music download (network-heavy, gated)
 HOPE_RUN_YTDLP_DOWNLOAD=1 python -m pytest tests/test_media.py -m slow
-```
 
 # Check all logs (single service: "voice")
 docker compose logs -f voice
@@ -153,7 +196,7 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
 ├── supervisord.conf     ← process manager (livekit, agent, nginx, token-server, media-server)
 ├── certs/               ← persisted self-signed cert (volume mount)
 ├── media/               ← permanent music library (bind-mounted to /data/media)
-│   ├── files/           ← transcoded mp3 tracks (nginx-served, HTTP Range)
+│   ├── files/           ← raw audio tracks (webm/m4a; media-server streamed, Range when complete)
 │   ├── thumbs/          ← cover art
 │   ├── jobs/            ← download job status snapshots
 │   └── library.json     ← library metadata (permanent)
@@ -188,7 +231,25 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
 - **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
 - **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
 - **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
-- **Music downloads are mp3, not YouTube streams.** The media server transcodes with yt-dlp (`-x --audio-format mp3 --audio-quality 192K --embed-thumbnail`) into `/data/media/files/{video_id}.mp3`; nginx serves those files as statics with native HTTP Range, which is what makes seeking work in the `<audio>` element. Never point the player at YouTube directly.
+- **Music downloads are raw audio, streamed as they grow.** yt-dlp fetches
+  bestaudio directly (`-f bestaudio/best`, no `-x`/transcode) into
+  `/data/media/files/{video_id}.{ext}`; the stream endpoint
+  (`GET /media/api/stream/{id}`) serves the growing `.part` file chunked
+  (no Content-Length) while the download runs, then serves the finished file
+  with native Range for seeking. `audio_url` is that endpoint for the whole
+  lifecycle — a stable URL means the browser never reloads mid-playback. The
+  mp3 transcode was dropped: it delayed playback by the whole download+encode,
+  and modern browsers play webm/opus and m4a everywhere. Never point the
+  player at YouTube directly.
+- **Search prefers the library.** `/media/api/search` merges ready library
+  matches (title token/substring match) in front of the yt-dlp results, and
+  the search_music tool docstring + system prompt tell Hope to always pick
+  `in_library=true`. Deterministic preference lives in the server merge, not
+  in prompt hopes.
+- **Progress lines come from yt-dlp stderr.** `[download] N%` goes to stderr;
+  only `--print-json` metadata goes to stdout. The download worker drains
+  stderr in a thread (deadlock-safe) and reads stdout for the JSON — parsing
+  progress from stdout silently never matched.
 - **External stream URLs are proxied, never played directly.** An `http://` radio URL would be blocked as mixed content on the HTTPS UI, so `GET /media/api/stream/{id}` re-serves the upstream over HTTPS with Range passthrough. Downloaded tracks on that endpoint just redirect to `/media/files/`.
 - **Player state is per-room desired state, reconciled by the browser.** The media server never streams to anything; it records what SHOULD play per LiveKit room and the browser polls and reports back. Track ids for YouTube tracks are the video ids (dedupe by video_id); stream tracks get `s<uuid>`.
 - **Anonymous yt-dlp first, cookies when flaky.** Search (`ytsearch`) and downloads work anonymously from a residential IP; if YouTube starts bot-blocking, drop a cookies.txt into `./media/cookies.txt` and set `YTDLP_COOKIES=/data/media/cookies.txt`.

agent/agent.py

@@ -213,10 +213,13 @@ SYSTEM_PROMPT = textwrap.dedent("""\
     - Ambiguous results → offer_song_choices with the top 2-3 options, name
       them briefly, and let the user pick by voice or in the player panel.
       Never start playback on a guess.
-    The library is permanent: list_library shows what is already downloaded,
+    Prefer what is already saved: search_music puts downloaded library
-    and those tracks play instantly without searching. If the user names a
+    matches first (in_library=true) — always pick one of those over a fresh
-    library track exactly, just play it. Control playback with pause_music,
+    YouTube result, and list_library shows the whole saved collection which
-    resume_music, skip_song and seek_music; check what's on with now_playing.
+    plays instantly. If the user names a library track exactly, just play it.
+    A brand-new song starts playing within seconds and finishes downloading
+    in the background. Control playback with pause_music, resume_music,
+    skip_song and seek_music; check what's on with now_playing.
     add_stream registers a live radio or direct audio URL as a library track.
     While music is playing you cannot hear the user — they will come back to
     you when they want to talk again, and the music pauses automatically.

agent/media_mcp.py

@@ -69,27 +69,30 @@ def _fmt_duration(seconds) -> str:
 
 @mcp.tool()
 def search_music(query: str, limit: int = 5) -> str:
-    """Search YouTube Music for songs. Call this before playing anything the
+    """Search for songs: already-downloaded library matches come first, then
-    user describes by name, artist or lyrics. Returns JSON results with
+    YouTube Music results. Call this before playing anything the user
-    video_id, title, channel, duration and in_library flags. Pick the result
+    describes by name, artist or lyrics. Results carry video_id, title,
-    that clearly matches what the user asked for; if two or more could be
+    channel, duration and in_library flags. ALWAYS prefer a result with
-    right, use offer_song_choices instead of guessing."""
+    in_library=true — it is already saved and plays instantly. Pick the
+    result that clearly matches what the user asked for; if two or more could
+    be right, use offer_song_choices instead of guessing."""
     data = _media_post("/media/api/search", {"query": query, "limit": max(1, min(limit, 10))})
     if "error" in data:
         return f"Search failed: {data['error']}"
     results = data.get("results", [])
     if not results:
-        return f"No YouTube Music results for '{query}'."
+        return f"No results for '{query}'."
     return json.dumps(results, ensure_ascii=False)
 
 
 @mcp.tool()
 def play_song(video_id: str, title: str = "", channel: str = "") -> str:
     """Play a song in the user's web player. Pass the video_id from
-    search_music (or an in_library track_id). Downloads and transcodes first
+    search_music — for an in_library result this plays the saved copy with no
-    if needed; playback starts automatically once ready. After calling this,
+    download; a new song downloads and starts playing automatically while it
-    tell the user what is playing in one short sentence and stop talking —
+    arrives. After calling this, tell the user what is playing in one short
-    the player takes over and the conversation pauses until they return."""
+    sentence and stop talking — the player takes over and the conversation
+    pauses until they return."""
     data = _media_post("/media/api/play", {
         "room": MEDIA_ROOM, "video_id": video_id, "title": title, "channel": channel,
     })
@@ -100,9 +103,9 @@ def play_song(video_id: str, title: str = "", channel: str = "") -> str:
     name = track.get("title") or title or video_id
     if status == "downloading":
         return (
-            f"'{name}' is downloading and will start playing automatically when "
+            f"'{name}' is downloading and starts playing within seconds — "
-            "ready (usually under a minute). Say it's on the way, then hand over "
+            "audio streams in while the download finishes. Say it's on the "
-            "to the player."
+            "way, then hand over to the player."
         )
     return f"Playing '{name}' now. Hand over to the player with a short sign-off."
 

tests/test_media.py

@@ -18,7 +18,9 @@ from urllib.parse import urlparse
 import httpx
 import pytest
 
-from conftest import TOKEN_URL, docker_exec, IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
+from conftest import (
+    TOKEN_URL, docker_exec, docker_exec_python, IS_LOCAL_DEPLOYMENT, WEB_BASE_URL
+)
 
 MEDIA_API = "/media/api"
 TEST_ROOM = f"media-test-{uuid.uuid4().hex[:8]}"
@@ -346,6 +348,85 @@ needs_local_container = pytest.mark.skipif(
 )
 
 
+# ── media server internals (in-process, inside the container) ───────────────
+
+@needs_local_container
+def test_media_search_prefers_library_matches(client):
+    """Search returns ready library matches ahead of YouTube results, so
+    Hope prefers an already-downloaded copy. Runs the endpoint in-process
+    with _run_search stubbed — fully deterministic, no network."""
+    code = '''
+import sys
+sys.path.insert(0, "/opt/voice")
+import media_server as ms
+from fastapi.testclient import TestClient
+
+ms._library["tracks"]["libpref01"] = {
+    "id": "libpref01", "kind": "youtube", "video_id": "libpref01",
+    "title": "Preference Probe Song", "channel": "Probe Artist",
+    "duration": 120, "status": "ready", "ext": "webm",
+    "error": None, "added_at": 1.0,
+}
+ms._run_search = lambda q, limit: [
+    {"video_id": "remote01", "title": "Preference Probe Song (cover)",
+     "channel": "", "duration": 99, "view_count": 1,
+     "in_library": False, "track_id": None, "library_status": None},
+]
+
+c = TestClient(ms.app)
+r = c.post("/media/api/search", json={"query": "preference probe", "limit": 5})
+assert r.status_code == 200, r.text
+results = r.json()["results"]
+assert results, "no results"
+assert results[0]["video_id"] == "libpref01", results
+assert results[0]["in_library"] is True
+assert any(x["video_id"] == "remote01" for x in results), results
+print("OK")
+'''
+    proc = docker_exec_python(code, timeout=60, check=False)
+    assert proc.returncode == 0, proc.stderr[-800:]
+    assert "OK" in proc.stdout
+
+
+@needs_local_container
+def test_media_stream_endpoint_serves_ready_track(client):
+    """GET /media/api/stream/{id} for a finished youtube track serves the
+    file bytes with native Range support (progressive-playback endpoint)."""
+    code = '''
+import sys
+sys.path.insert(0, "/opt/voice")
+import media_server as ms
+from fastapi.testclient import TestClient
+
+probe = ms.FILES_DIR / "probetrk.mp3"
+probe.write_bytes(b"PROBEDATA" * 100)
+ms._library["tracks"]["probetrk"] = {
+    "id": "probetrk", "kind": "youtube", "video_id": "probetrk",
+    "title": "Probe", "channel": "", "duration": 1,
+    "status": "ready", "ext": "mp3", "error": None, "added_at": 1.0,
+}
+try:
+    c = TestClient(ms.app)
+    full = c.get("/media/api/stream/probetrk")
+    assert full.status_code == 200, full.status_code
+    assert full.content == b"PROBEDATA" * 100
+    part = c.get("/media/api/stream/probetrk", headers={"Range": "bytes=0-8"})
+    assert part.status_code == 206, part.status_code
+    assert part.content == b"PROBEDATA"
+    assert (part.headers.get("content-range") or "").startswith("bytes 0-8/")
+    # track dict exposes a stable stream URL while downloading and ready
+    d = ms._track_dict(ms._library["tracks"]["probetrk"])
+    assert d["audio_url"] == "/media/api/stream/probetrk", d
+    print("OK")
+finally:
+    probe.unlink(missing_ok=True)
+    ms._library["tracks"].pop("probetrk", None)
+'''
+    proc = docker_exec_python(code, timeout=60, check=False)
+    assert proc.returncode == 0, proc.stderr[-800:]
+    assert "OK" in proc.stdout
+
+
 @needs_local_container
 def test_mcp_media_list_library(mcp_client_factory):
     text, is_error = mcp_client_factory(
@@ -425,8 +506,9 @@ def test_media_search_live(client):
 
 @pytest.mark.slow
 def test_media_download_and_stream_end_to_end(client):
-    """Full pipeline: search -> play -> yt-dlp download + ffmpeg transcode ->
+    """Full pipeline: search -> play -> yt-dlp download -> playable stream.
-    static mp3 with Range support in the player. Gated: run with
+    The audio_url stays on /media/api/stream/ for the whole lifecycle; once
+    ready it serves the finished file with Range support. Gated: run with
     HOPE_RUN_YTDLP_DOWNLOAD=1 (takes 1-3 minutes)."""
     if os.environ.get("HOPE_RUN_YTDLP_DOWNLOAD") != "1":
         pytest.skip("set HOPE_RUN_YTDLP_DOWNLOAD=1 to run the real download")

web/app.js

@@ -1207,7 +1207,15 @@ function reconcileMedia(s) {
     const pendingJob = (s.pending || []).find(j => !track || j.track_id === track.id);
     updateMediaProgress(track, pendingJob);
 
-    if (track && track.id !== mediaCurrentTrackId) loadMediaTrack(track);
+    if (track && track.id !== mediaCurrentTrackId) {
+        loadMediaTrack(track);
+    } else if (track && track.audio_url && mediaAudio.getAttribute("src") !== track.audio_url) {
+        // The track was loaded before it had a stream (still downloading) and
+        // now has one — wire it up so playback can start without a reload.
+        mediaAudio.src = track.audio_url;
+        mediaAudio.load();
+        updateMediaNowCard(track);
+    }
     if (!track && mediaCurrentTrackId && s.desired === "stopped") clearMediaAudio();
 
     applyDesiredMediaState(s, track);
@@ -1224,7 +1232,9 @@ function reconcileMedia(s) {
 
 function applyDesiredMediaState(s, track) {
     if (s.desired === "playing") {
-        if (track && track.status === "ready" && mediaAudio.src && mediaAudio.paused) {
+        // audio_url presence means a playable stream exists — including a
+        // still-growing download, which plays progressively.
+        if (track && track.audio_url && mediaAudio.src && mediaAudio.paused) {
             mediaAudio.play().catch((e) => console.warn("playback blocked:", e));
         }
     } else if (!mediaAudio.paused) {
@@ -1235,14 +1245,14 @@ function applyDesiredMediaState(s, track) {
 function loadMediaTrack(track) {
     mediaCurrentTrackId = track.id;
     updateMediaNowCard(track);
-    if (track.audio_url && (track.kind === "stream" || track.status === "ready")) {
+    if (track.audio_url) {
         if (mediaAudio.getAttribute("src") !== track.audio_url) {
             mediaAudio.src = track.audio_url;
             mediaAudio.load();
         }
     } else {
-        // Downloading or failed: no stream yet. The poll flips this to the
+        // Downloading or failed: no stream yet. reconcileMedia wires the
-        // real URL once status turns "ready".
+        // stream URL in via polling as soon as one is available.
         mediaAudio.removeAttribute("src");
         mediaAudio.load();
         updateMediaControls();
@@ -1266,7 +1276,9 @@ function updateMediaNowCard(track) {
     if (track.status === "failed") {
         mediaChannelEl.textContent = "Download failed — pick another result or try again.";
     } else if (track.status === "downloading") {
-        mediaChannelEl.textContent = "Preparing…";
+        // Progressive streaming: the track plays while the download runs.
+        mediaChannelEl.textContent = track.channel
+            ? `${track.channel} · downloading` : "Downloading…";
     } else {
         mediaChannelEl.textContent = track.channel || (track.kind === "stream" ? "Live stream" : "");
     }
@@ -1288,9 +1300,7 @@ function updateMediaProgress(track, job) {
     mediaProgressWrap.hidden = false;
     const pct = Math.round(job.progress || 0);
     mediaProgressFill.style.width = `${pct}%`;
-    const stage = job.stage === "transcoding"
+    const stage = job.stage === "queued" ? "Queued…" : "Downloading…";
-        ? "Re-encoding…"
-        : job.stage === "queued" ? "Queued…" : "Downloading…";
     mediaProgressLabel.textContent = `${stage} ${pct}%`;
     if (track) updateMediaNowCard(track);
 }

web/media_server.py

@@ -4,32 +4,36 @@
 A small FastAPI service that owns the permanent music library and player
 coordination:
 
-- Library: downloaded YouTube Music tracks (mp3, transcoded via yt-dlp +
+- Library: downloaded YouTube Music tracks (raw best-audio files, saved by
-  ffmpeg) and user-added live audio stream URLs, persisted in library.json.
+  yt-dlp with no transcode) and user-added live audio stream URLs, persisted
+  in library.json.
 - Search: yt-dlp ``ytsearch`` (YouTube Music pages share video ids), used by
-  Hope's media MCP toolset and by the web UI search box.
+  Hope's media MCP toolset and by the web UI search box. Ready library tracks
+  that match the query are returned first so Hope prefers a downloaded copy.
 - Download jobs: one-at-a-time worker thread; progress is parsed from
-  yt-dlp stdout and surfaced through GET /media/api/state.
+  yt-dlp stderr and surfaced through GET /media/api/state. Playback starts
+  while the file is still growing (progressive streaming), and the finished
+  file stays in the library for instant replay.
 - Player state: per-room desired state (loaded track, playing/paused/stopped,
   seek hints). The browser is the actual player and reconciles against this;
   it reports real playback state back so Hope's voice tools can answer
   "what's playing".
 
 Security posture matches the token server: bind 127.0.0.1 only, reachability
-exclusively through the nginx HTTPS proxy (/media/api/). File bytes for
+exclusively through the nginx HTTPS proxy (/media/api/). Growing downloads
-downloaded tracks are served as nginx static files (/media/files/, native
+are streamed by this service (chunked, no Content-Length); finished files are
-HTTP Range support for seeking); this service only proxies external stream
+served here too with native Range support for seeking. Live stream URLs are
-URLs (mixed-content: an http:// radio URL would be blocked by the browser,
+proxied (mixed-content: an http:// radio URL would be blocked by the browser,
 so it is fetched server-side and re-served over HTTPS).
 """
 from __future__ import annotations
 
+import asyncio
 import json
 import os
 import re
 import subprocess
 import sys
-import tempfile
 import threading
 import time
 import uuid
@@ -40,7 +44,7 @@ from typing import Any
 import httpx
 import uvicorn
 from fastapi import FastAPI, HTTPException, Request
-from fastapi.responses import JSONResponse, RedirectResponse, StreamingResponse
+from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
 
 MEDIA_DIR = Path(os.environ.get("MEDIA_DIR", "/data/media"))
 FILES_DIR = MEDIA_DIR / "files"
@@ -107,8 +111,11 @@ def _bump_version() -> int:
 def _track_dict(track: dict) -> dict:
     out = dict(track)
     if track["kind"] == "youtube":
-        if track.get("status") == "ready":
+        # The stream endpoint serves the growing file during download and the
-            out["audio_url"] = f"/media/files/{track['id']}.mp3"
+        # finished file afterwards; the URL is stable across the transition
+        # so the browser never has to reload mid-playback.
+        if track.get("status") in ("downloading", "ready"):
+            out["audio_url"] = f"/media/api/stream/{track['id']}"
         out["art_url"] = (
             f"/media/thumbs/{track['id']}.jpg"
             if (THUMBS_DIR / f"{track['id']}.jpg").exists()
@@ -179,6 +186,101 @@ def _ytdlp_base_args() -> list[str]:
 
 _PROGRESS_RE = re.compile(r"\[download\]\s+([\d.]+)%")
 
+# Audio containers yt-dlp can produce for a raw best-audio download.
+_AUDIO_EXTS = {
+    ".webm", ".opus", ".ogg", ".m4a", ".mp4", ".m4b", ".aac",
+    ".mp3", ".mka", ".flac", ".wav",
+}
+_AUDIO_MIME = {
+    ".webm": "audio/webm",
+    ".opus": "audio/opus",
+    ".ogg": "audio/ogg",
+    ".m4a": "audio/mp4",
+    ".mp4": "audio/mp4",
+    ".m4b": "audio/mp4",
+    ".aac": "audio/aac",
+    ".mp3": "audio/mpeg",
+    ".mka": "audio/x-matroska",
+    ".flac": "audio/flac",
+    ".wav": "audio/wav",
+}
+
+
+def _media_type_for(name: str) -> str:
+    if name.endswith(".part"):
+        name = name[:-5]
+    return _AUDIO_MIME.get(Path(name).suffix.lower(), "audio/mpeg")
+
+
+def _youtube_media_file(track_id: str) -> Path | None:
+    """The on-disk audio file for a youtube track, if any.
+
+    During an active download yt-dlp writes ``{id}.{ext}.part`` and renames it
+    to ``{id}.{ext}`` on completion — return whichever currently exists (the
+    partial file wins, thumbnails and other sidecars are ignored).
+    """
+    complete: Path | None = None
+    for p in FILES_DIR.glob(f"{track_id}.*"):
+        suffix = p.suffix.lower()
+        if suffix == ".part":
+            if p.stat().st_size > 0:
+                return p
+        elif suffix in _AUDIO_EXTS:
+            if complete is None or p.name < complete.name:
+                complete = p
+    return complete
+
+
+async def _wait_for_media_file(track_id: str, timeout: float = 15.0) -> Path | None:
+    """Wait for yt-dlp to create the output file after a play request."""
+    deadline = time.monotonic() + timeout
+    while time.monotonic() < deadline:
+        path = _youtube_media_file(track_id)
+        if path is not None:
+            return path
+        await asyncio.sleep(0.5)
+    return None
+
+
+async def _growing_file(track_id: str, job: dict, chunk_size: int = 64 * 1024):
+    """Yield bytes from a file that yt-dlp is still writing.
+
+    The response stays open with no Content-Length until the download
+    finishes, so the browser plays the track as it grows. The open descriptor
+    keeps reading the old inode across yt-dlp's .part -> final rename, and the
+    path is re-resolved if the file disappears between resolution and open.
+    """
+    fd = -1
+    pos = 0
+    stall = 0
+    try:
+        while True:
+            if fd == -1:
+                path = _youtube_media_file(track_id)
+                if path is not None:
+                    try:
+                        fd = os.open(path, os.O_RDONLY)
+                    except FileNotFoundError:
+                        fd = -1  # renamed between glob and open — retry
+            if fd != -1:
+                data = await asyncio.to_thread(os.pread, fd, chunk_size, pos)
+                if data:
+                    pos += len(data)
+                    stall = 0
+                    yield data
+                    continue
+            with _lock:
+                active = job.get("stage") == "downloading"
+            if not active:
+                break
+            stall += 1
+            if stall > 60:  # ~30s with no new bytes — stop waiting
+                break
+            await asyncio.sleep(0.5)
+    finally:
+        if fd != -1:
+            os.close(fd)
+
 
 def _run_search(query: str, limit: int) -> list[dict]:
     """Flat ytsearch lookup — cheap, no download, works anonymously."""
@@ -247,15 +349,18 @@ def _fail_job(job: dict, track: dict, message: str) -> None:
 
 
 def _download_worker(job: dict, track: dict, video_id: str) -> None:
-    """Runs in the single-worker executor: yt-dlp download + mp3 transcode."""
+    """Runs in the single-worker executor: raw best-audio download.
-    target = FILES_DIR / f"{track['id']}.mp3"
+
+    No mp3 transcode — the audio container is streamed straight into the
+    library so playback can begin while the file is still growing (the stream
+    endpoint serves the partial file), and the finished file stays playable
+    offline. Thumbnail handling is unchanged.
+    """
     cmd = _ytdlp_base_args() + [
         "-f", "bestaudio/best",
-        "-x", "--audio-format", "mp3", "--audio-quality", "192K",
-        "--embed-thumbnail", "--add-metadata",
-        "--write-thumbnail", "--convert-thumbnails", "jpg",
         "--newline", "--progress",
         "--print-json",
+        "--write-thumbnail", "--convert-thumbnails", "jpg",
         "--retries", "3", "--fragment-retries", "3",
         "-P", str(FILES_DIR),
         "-o", f"{track['id']}.%(ext)s",
@@ -263,46 +368,52 @@ def _download_worker(job: dict, track: dict, video_id: str) -> None:
     ]
     job["stage"] = "downloading"
     info: dict = {}
-    # stderr goes to a temp file: reading it from a PIPE while the stdout
+    # Progress lines go to stderr, the metadata JSON to stdout. Drain stderr
-    # loop blocks could deadlock if yt-dlp ever fills the stderr buffer.
+    # in a thread so the stdout read loop can never deadlock the pipe; the
-    with tempfile.TemporaryFile(mode="w+", encoding="utf-8", errors="replace") as errfile:
+    # tail is kept for the failure detail.
-        try:
+    stderr_tail: list[str] = []
-            proc = subprocess.Popen(
+    proc = None
-                cmd, stdout=subprocess.PIPE, stderr=errfile,
+
-                text=True, errors="replace",
+    def _drain_stderr() -> None:
-            )
+        assert proc is not None and proc.stderr is not None
-            deadline = time.monotonic() + DOWNLOAD_TIMEOUT
+        for line in proc.stderr:
-            assert proc.stdout is not None
+            stderr_tail.append(line)
-            for line in proc.stdout:
+            if len(stderr_tail) > 200:
-                if time.monotonic() > deadline:
+                stderr_tail.pop(0)
-                    proc.kill()
+            m = _PROGRESS_RE.search(line)
-                    proc.wait(timeout=30)
+            if m:
-                    raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
+                job["progress"] = min(99.0, float(m.group(1)))
-                line = line.strip()
+                _persist_job(job)
-                m = _PROGRESS_RE.search(line)
+
-                if m:
+    try:
-                    job["progress"] = min(85.0, float(m.group(1)) * 0.85)
+        proc = subprocess.Popen(
-                    _persist_job(job)
+            cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                elif line.startswith(("[ExtractAudio]", "[Metadata]", "[EmbedThumbnail]")):
+            text=True, errors="replace",
-                    job["stage"] = "transcoding"
+        )
-                    job["progress"] = max(job["progress"], 86.0)
+        threading.Thread(target=_drain_stderr, daemon=True).start()
-                    _persist_job(job)
+        deadline = time.monotonic() + DOWNLOAD_TIMEOUT
-                elif line.startswith("{"):
+        assert proc.stdout is not None
-                    try:
+        for line in proc.stdout:
-                        info = json.loads(line)
+            if time.monotonic() > deadline:
-                    except json.JSONDecodeError:
+                proc.kill()
-                        pass
+                proc.wait(timeout=30)
-            proc.wait(timeout=30)
+                raise RuntimeError(f"Download exceeded {DOWNLOAD_TIMEOUT}s")
-            if proc.returncode != 0:
+            line = line.strip()
-                errfile.seek(0)
+            if line.startswith("{"):
-                stderr_tail = (errfile.read() or "").strip().splitlines()
+                try:
-                detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
+                    info = json.loads(line)
-                raise RuntimeError(detail)
+                except json.JSONDecodeError:
-            if not target.exists():
+                    pass
-                raise RuntimeError("yt-dlp finished but no mp3 was produced")
+        proc.wait(timeout=30)
-        except Exception as exc:  # noqa: BLE001 — any failure marks the track failed
+        if proc.returncode != 0:
-            _fail_job(job, track, str(exc))
+            detail = stderr_tail[-1] if stderr_tail else f"exit code {proc.returncode}"
-            return
+            raise RuntimeError(detail.strip())
+        media = _youtube_media_file(track["id"])
+        if media is None:
+            raise RuntimeError("yt-dlp finished but no audio file was produced")
+    except Exception as exc:  # noqa: BLE001 — any failure marks the track failed
+        _fail_job(job, track, str(exc))
+        return
 
     thumb_src = FILES_DIR / f"{track['id']}.jpg"
     try:
@@ -317,6 +428,7 @@ def _download_worker(job: dict, track: dict, video_id: str) -> None:
     with _lock:
         track["status"] = "ready"
         track["error"] = None
+        track["ext"] = media.suffix.lstrip(".").lower()
         if info:
             if info.get("title"):
                 track["title"] = info["title"]
@@ -364,11 +476,11 @@ def get_library() -> dict:
 def _delete_track_files(track: dict) -> None:
     if track["kind"] != "youtube":
         return
-    for path in (
+    paths = [
-        FILES_DIR / f"{track['id']}.mp3",
+        FILES_DIR / f"{track['id']}.{ext}" for ext in
-        FILES_DIR / f"{track['id']}.jpg",
+        (s.lstrip(".") for s in _AUDIO_EXTS)
-        THUMBS_DIR / f"{track['id']}.jpg",
+    ] + [FILES_DIR / f"{track['id']}.jpg", THUMBS_DIR / f"{track['id']}.jpg"]
-    ):
+    for path in paths:
         try:
             path.unlink(missing_ok=True)
         except OSError:
@@ -394,6 +506,39 @@ def delete_track(track_id: str) -> dict:
 
 # ── API: search + play + control ────────────────────────────────────────────
 
+def _library_search(query: str, limit: int) -> list[dict]:
+    """Ready library tracks whose title matches the query.
+
+    These are merged in front of the YouTube results so Hope prefers an
+    already-downloaded copy over fetching a new one.
+    """
+    q = query.strip().lower()
+    tokens = {t for t in re.split(r"\W+", q) if len(t) > 2}
+    hits: list[dict] = []
+    if not q:
+        return hits
+    for track in _library["tracks"].values():
+        if track.get("kind") != "youtube" or track.get("status") != "ready":
+            continue
+        title = (track.get("title") or "").strip().lower()
+        if not title:
+            continue
+        title_tokens = {t for t in re.split(r"\W+", title) if t}
+        if not (q in title or title in q or (tokens and tokens <= title_tokens)):
+            continue
+        hits.append({
+            "video_id": track["id"],
+            "track_id": track["id"],
+            "title": track.get("title") or track["id"],
+            "channel": track.get("channel") or "",
+            "duration": track.get("duration"),
+            "view_count": None,
+            "in_library": True,
+            "library_status": "ready",
+        })
+    return hits[:limit]
+
+
 @app.post("/media/api/search")
 async def search(request: Request) -> dict:
     body = await request.json()
@@ -404,7 +549,11 @@ async def search(request: Request) -> dict:
         limit = max(1, min(int(body.get("limit", 6)), 15))
     except (TypeError, ValueError):
         raise HTTPException(400, "limit must be an integer")
-    return {"query": query, "results": _run_search(query, limit)}
+    local = _library_search(query, limit)
+    remote = _run_search(query, limit)
+    seen = {r["video_id"] for r in local}
+    merged = local + [r for r in remote if r["video_id"] not in seen]
+    return {"query": query, "results": merged[:limit]}
 
 
 def _resolve_play_target(body: dict) -> dict:
@@ -588,13 +737,32 @@ _HOP_HEADERS = ("content-type", "content-range", "accept-ranges", "content-lengt
 async def stream_proxy(track_id: str, request: Request):
     with _lock:
         track = _get_track(track_id)
+        job = next(
+            (j for j in _jobs.values()
+             if j["track_id"] == track_id and j["stage"] == "downloading"),
+            None,
+        )
     if not track:
         raise HTTPException(404, "unknown track")
+
     if track["kind"] == "youtube":
-        # Downloaded tracks are nginx static files (native Range support).
+        # Progressive download: an active download is streamed in real time
-        if track.get("status") == "ready":
+        # (chunked, no Content-Length) so playback starts while the file
-            return RedirectResponse(f"/media/files/{track_id}.mp3")
+        # grows; a finished file is served directly with native Range support.
-        raise HTTPException(409, f"track is {track.get('status')}, not ready")
+        path = _youtube_media_file(track_id)
+        if path is None and job is not None:
+            path = await _wait_for_media_file(track_id)
+        if path is None:
+            raise HTTPException(
+                404, f"audio file not available (track is {track.get('status')})"
+            )
+        if job is not None:
+            return StreamingResponse(
+                _growing_file(track_id, job),
+                media_type=_media_type_for(path.name),
+                headers={"Cache-Control": "no-store"},
+            )
+        return FileResponse(path, media_type=_media_type_for(path.name))
 
     range_header = request.headers.get("range")
     headers = {"Range": range_header} if range_header else {}

web/style.css

@@ -617,9 +617,15 @@ button {
 }
 
 .btn-icon.muted {
-    color: #ffd1ca;
+    color: #fff;
-    border-color: rgba(255, 128, 111, 0.5);
+    background: var(--danger);
-    background: rgba(143, 64, 58, 0.4);
+    border-color: var(--danger);
+}
+
+.btn-icon.muted:hover:not(:disabled) {
+    color: #fff;
+    background: var(--danger);
+    border-color: var(--danger);
 }
 
 .btn-ghost {