Commit Graph
7 Commits
Author SHA1 Message Date
snowspeeder 436b3fc8b1 refactor: split music streaming into Hope Music
Rebuild & Deploy / rebuild (push) Successful in 2m7s
Move the media server, downloader, library volume, and browser player to the
standalone hope-media-music product. Keep Hope's music tools as a thin HTTP
client of the LAN service on port 8094.

Use a stable voice-room player scope instead of ephemeral LiveKit room names.
The agent now owns the conversation handshake: fresh reported playback mutes
its input, joining pauses music, and the last participant leaving resumes only
music it paused. Peer failures remain non-fatal.

Also move the voice deploy workflow onto the active Unraid Gitea runner and
remove yt-dlp/ffmpeg/media health and volume dependencies from the image.
2026-09-12 06:35:15 -04:00
snowspeeder 04c4a53594 feat: media player API, web player panel, and Hope music MCP
- web/media_server.py: FastAPI service owning the permanent library
  (library.json + ./media volume), yt-dlp YouTube Music search, queued
  download jobs with mp3 transcode (ffmpeg), per-room player desired
  state, and an HTTPS proxy for external live-stream URLs.
- agent/media_mcp.py: Hope tools — search_music, play_song,
  offer_song_choices, pause/resume/stop/skip/seek, now_playing,
  list_library, add_stream.
- agent/agent.py: media toolset wiring + conversation rule (music
  playing disables the agent mic; re-engaging pauses music).
- web UI: slide-out Music panel (search, suggestions, library, live
  stream add, play/pause, ±10s, seek, volume) with a reconciling poller
  and media_state data-channel reports.
- token server returns the minted room name; nginx serves /media/files
  as static (native Range) and proxies /media/api to the media server.
- supervisord runs media-server; compose mounts ./media and adds
  ffmpeg to the image.
2026-08-29 13:03:12 -04:00
snowspeeder c940923539 feat: connect Shane voice sessions to memory 2026-08-28 16:22:18 -04:00
Shane f80b9ebe3d fix: unique room names per session, JT_ROOM dispatch, dispatch-first prompt
- token_server: generate voice-{uuid8} room name per request so each
  browser session creates a fresh room and triggers agent dispatch
- token claim: add jobType JT_ROOM to roomConfig.agents
- livekit.yaml: revert empty_timeout to default (300s)
- agent.py: system prompt now mandates dispatch_task for all non-trivial
  tasks (not just research); removed inline web_search instructions;
  added tool_call logging in _ReasoningFallbackWrapper
2026-08-23 07:05:03 -04:00
Shane d3f9f2c4ed feat: implement full UPDATE.md review — critical fixes, UI upgrade, infra hardening
Critical frontend bugs:
- Add TrackSubscribed/attach() for agent audio playback
- Fix decodeToString TypeError with TextDecoder
- XSS fix: innerHTML -> textContent in addMessage
- Fresh token on reconnect retry

Agent fixes:
- GemmaLLM subclass with reasoning_content fallback wrapper
- Disable Gemma 4 thinking mode via chat_template_kwargs (6.8s -> 0.5s)
- Remove duplicate session-level LLM
- Replace global _active_session with closure-based handler
- asyncio.create_task instead of deprecated get_event_loop
- Explicit silero VAD, topic filter on voice-control

Infra:
- supervisord: all programs log to /dev/stdout
- Dockerfile: uv sync --frozen with committed uv.lock
- nginx config moved to real file, token_server.py no longer served
- entrypoint.sh: cert persisted, only regenerated on IP change
- compose: healthcheck + cert volume
- token_server: CORS removed, room pinned to voice-room

UI upgrade:
- Orb UI with state machine (idle/connecting/listening/thinking/speaking)
- Streaming transcripts via lk.transcription text streams
- Barge-in hint, thinking chip, audio visualizer
- Glassmorphism, chat bubbles, settings sheet, light mode
- PWA manifest, favicon, wake-lock, safe-area insets
- localStorage conversation history

Docs: AGENTS.md drift fixed
2026-08-22 15:21:59 -04:00
Shane 045ddabac7 feat: web access via MCP (Firecrawl search/scrape) + mcp server attach support
- agent/web_mcp.py: stdio MCP server exposing web_search and web_scrape,
  backed by the self-hosted Firecrawl stack on xNAS (no API key needed)
- agent.py: Agent now attaches mcp_servers built from config; EXTRA_MCP_SERVERS
  env var allows adding arbitrary HTTP/SSE MCP servers as JSON
- Dockerfile: installs livekit-agents[mcp], copies web_mcp.py
- .env.example: WEB_MCP_ENABLED, FIRECRAWL_BASE, EXTRA_MCP_SERVERS documented
2026-08-22 12:29:04 -04:00
Shane 6f2b231938 feat: HTTPS web frontend (self-signed) + server-side token endpoint
- nginx serves the UI over HTTPS on 8090 with a self-signed cert
  (browsers require a secure context for microphone access)
- added /token endpoint (tiny Python HTTP server) that signs LiveKit
  JWTs server-side, keeping the API secret out of the browser
- app.js now fetches a signed token from /token and uses wss:// when
  the page is served over HTTPS
- supervisord runs the token-server as a fourth process
2026-08-22 08:31:19 -04:00