Commit Graph
12 Commits
Author SHA1 Message Date
snowspeeder 04c4a53594 feat: media player API, web player panel, and Hope music MCP
- web/media_server.py: FastAPI service owning the permanent library
  (library.json + ./media volume), yt-dlp YouTube Music search, queued
  download jobs with mp3 transcode (ffmpeg), per-room player desired
  state, and an HTTPS proxy for external live-stream URLs.
- agent/media_mcp.py: Hope tools — search_music, play_song,
  offer_song_choices, pause/resume/stop/skip/seek, now_playing,
  list_library, add_stream.
- agent/agent.py: media toolset wiring + conversation rule (music
  playing disables the agent mic; re-engaging pauses music).
- web UI: slide-out Music panel (search, suggestions, library, live
  stream add, play/pause, ±10s, seek, volume) with a reconciling poller
  and media_state data-channel reports.
- token server returns the minted room name; nginx serves /media/files
  as static (native Range) and proxies /media/api to the media server.
- supervisord runs media-server; compose mounts ./media and adds
  ffmpeg to the image.
2026-08-29 13:03:12 -04:00
snowspeeder a4ae96f01b feat: default to qwen2.5-vl-3b and cap tool schema length constraints
Rebuild & Deploy / rebuild (push) Successful in 54s
- Default LLM_MODEL to qwen2.5-vl-3b across agent, task worker, tests,
  compose, workflow, and docs
- Cap min/max length-style constraints in tool schemas to 1000 before
  LLM requests; llama.cpp rejects grammar repetitions >= 2000 and the
  memory MCP's update_note body maxLength (200000) broke qwen2.5-vl-3b
  with 'Failed to initialize samplers: failed to parse grammar'
2026-08-29 12:46:20 -04:00
snowspeeder 360e3276b0 feat: switch LLM to OpenWebUI with live model selection
Rebuild & Deploy / rebuild (push) Successful in 3m18s
- Point the agent at OpenWebUI's OpenAI-compatible API (LLM_BASE_URL,
  LLM_API_KEY, LLM_MODEL) with GEMMA_* env fallbacks
- Add list_available_models() fetching {base_url}/models, filtering
  non-chat ids (embedding/reranker)
- Rename GemmaLLM -> HopeLLM with update_options(model=...) for runtime
  model switching (mutates _opts.model) and instance-based compaction
- Data channel: agent publishes {type: models} after connect and on
  get_models; set_model switches live and acks model_changed
- Web UI: Brain section in the settings sheet with a model picker that
  persists the choice in localStorage
- Update tests, CI workflow env, and docs for the new endpoint
2026-08-29 12:09:03 -04:00
snowspeeder 3c4c43f983 fix: use valid LiveKit development secret 2026-08-28 16:30:01 -04:00
snowspeeder c940923539 feat: connect Shane voice sessions to memory 2026-08-28 16:22:18 -04:00
snowspeeder 2a2a2ff768 feat: plan per-user memory integration 2026-08-28 15:59:26 -04:00
Shane c815bcb485 feat: replace custom memory MCP with Cognee remote server
- Remove memory_mcp.py (stdio markdown-file memory)
- Add Cognee MCP as remote Streamable HTTP toolset at 192.168.86.2:8003/mcp
- Filter to only remember/recall/forget tools via allowed_tools
- Update system prompt: Memory section moved to top priority with
  explicit 'call recall FIRST' instructions and examples
- Add COGNEE_MCP_URL env var to docker-compose
- Remove /memory volume mount (no longer needed)
- Rewrite memory tests to use Cognee HTTP client fixture
- 28/28 tests passing
2026-08-23 08:10:42 -04:00
Shane ab6b5254ef feat: context compaction + skills system
Compaction:
- GemmaLLM.chat() truncates ChatContext to last 30 items (~15 turns)
  before sending to LLM, preventing context window overflow on long
  conversations. Preserves system prompt and removes orphaned tool calls.

Skills:
- agent/skills_mcp.py: MCP server with skill_save, skill_recall,
  skill_list, skill_update tools backed by .md files in /skills
- skills/ dir bind-mounted into container, git-trackable
- System prompt instructs Hope to save repeatable procedures as skills
  and recall them before performing tasks she's done before
- Distinct from memory (facts) — skills are learned *procedures*
2026-08-22 16:42:16 -04:00
Shane de1a5d8400 feat: markdown memory system — Hope learns from conversations
- agent/memory_mcp.py: MCP server with memory_recall, memory_save, memory_list
  tools backed by .md files in /memory (simple keyword matching for v1)
- memory/ dir bind-mounted into container, persists across rebuilds,
  easily backed up via git
- System prompt instructs Hope to recall on past references and save
  personal info/preferences naturally without announcing it
- Dockerfile: copy memory_mcp.py; compose: ./memory:/memory volume
2026-08-22 16:33:00 -04:00
Shane d3f9f2c4ed feat: implement full UPDATE.md review — critical fixes, UI upgrade, infra hardening
Critical frontend bugs:
- Add TrackSubscribed/attach() for agent audio playback
- Fix decodeToString TypeError with TextDecoder
- XSS fix: innerHTML -> textContent in addMessage
- Fresh token on reconnect retry

Agent fixes:
- GemmaLLM subclass with reasoning_content fallback wrapper
- Disable Gemma 4 thinking mode via chat_template_kwargs (6.8s -> 0.5s)
- Remove duplicate session-level LLM
- Replace global _active_session with closure-based handler
- asyncio.create_task instead of deprecated get_event_loop
- Explicit silero VAD, topic filter on voice-control

Infra:
- supervisord: all programs log to /dev/stdout
- Dockerfile: uv sync --frozen with committed uv.lock
- nginx config moved to real file, token_server.py no longer served
- entrypoint.sh: cert persisted, only regenerated on IP change
- compose: healthcheck + cert volume
- token_server: CORS removed, room pinned to voice-room

UI upgrade:
- Orb UI with state machine (idle/connecting/listening/thinking/speaking)
- Streaming transcripts via lk.transcription text streams
- Barge-in hint, thinking chip, audio visualizer
- Glassmorphism, chat bubbles, settings sheet, light mode
- PWA manifest, favicon, wake-lock, safe-area insets
- localStorage conversation history

Docs: AGENTS.md drift fixed
2026-08-22 15:21:59 -04:00
Shane f7e799c9e9 fix: add WEB_MCP_ENABLED + FIRECRAWL_BASE to docker-compose env 2026-08-22 12:41:16 -04:00
Shane 97633cc585 feat: Dockerfile, docker-compose, livekit.yaml, supervisord.conf 2026-08-22 07:41:58 -04:00