feat: switch LLM to OpenWebUI with live model selection

- Point the agent at OpenWebUI's OpenAI-compatible API (LLM_BASE_URL,
  LLM_API_KEY, LLM_MODEL) with GEMMA_* env fallbacks
- Add list_available_models() fetching {base_url}/models, filtering
  non-chat ids (embedding/reranker)
- Rename GemmaLLM -> HopeLLM with update_options(model=...) for runtime
  model switching (mutates _opts.model) and instance-based compaction
- Data channel: agent publishes {type: models} after connect and on
  get_models; set_model switches live and acks model_changed
- Web UI: Brain section in the settings sheet with a model picker that
  persists the choice in localStorage
- Update tests, CI workflow env, and docs for the new endpoint

.env.example

@@ -5,10 +5,11 @@ AZURE_SPEECH_REGION=eastus
 # Default TTS voice (Azure Dragon HD Omni; supports inline delivery markers)
 AZURE_TTS_VOICE=en-US-Ava:DragonHDOmniLatestNeural
 
-# LLM (Gemma on xNAS via llama.cpp, OpenAI-compatible)
-GEMMA_BASE_URL=http://192.168.86.2:8023/v1
-GEMMA_MODEL=gemma-4-e4b
-GEMMA_API_KEY=not-needed
+# LLM (OpenWebUI on xNAS, OpenAI-compatible; also serves the local llama.cpp
+# models. The model can be switched live from the web UI settings sheet.)
+LLM_BASE_URL=http://192.168.86.2:3005/api
+LLM_MODEL=gemma-4-e4b
+LLM_API_KEY=sk-your-openwebui-api-key
 
 # LiveKit (generated in livekit.yaml, referenced here for the agent)
 LIVEKIT_URL=http://localhost:7880

.gitea/workflows/rebuild.yml

@@ -40,9 +40,9 @@ jobs:
             -e AZURE_SPEECH_KEY="$AZURE_SPEECH_KEY" \
             -e AZURE_SPEECH_REGION=eastus \
             -e AZURE_TTS_VOICE=en-US-Ava:DragonHDOmniLatestNeural \
-            -e GEMMA_BASE_URL=http://192.168.86.2:8023/v1 \
-            -e GEMMA_MODEL=gemma-4-e4b \
-            -e GEMMA_API_KEY=not-needed \
+            -e LLM_BASE_URL=http://192.168.86.2:3005/api \
+            -e LLM_MODEL=gemma-4-e4b \
+            -e LLM_API_KEY=sk-4e9259e3bd56472db57fd40adf0fe567 \
             -e WEB_MCP_ENABLED=true \
             -e FIRECRAWL_BASE=http://192.168.86.2:3002 \
             -e WEB_PORT=9178 \

AGENTS.md

@@ -1,20 +1,20 @@
 # Voice — Real-time Voice Assistant
 
-A single-container voice assistant built on LiveKit Agents. Speaks and listens in real time using Azure Speech (STT + TTS) with a local Gemma LLM for reasoning.
+A single-container voice assistant built on LiveKit Agents. Speaks and listens in real time using Azure Speech (STT + TTS) with models hosted by OpenWebUI (xNAS) for reasoning.
 
 ## Architecture
 
 One Docker container runs three processes via supervisord:
 
 1. **LiveKit server** — open-source WebRTC media transport (port 7880 TCP, 7882 UDP muxed media)
-2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → Gemma LLM → Azure TTS
+2. **Voice agent** — Python LiveKit Agents pipeline: Azure STT → LLM (OpenWebUI) → Azure TTS
 3. **Web frontend** — static HTML served by nginx (port 9178 by default)
 
 ```
 Browser ──WebRTC──► LiveKit Server ──audio──► Agent
    ▲                                              │
    └────────── audio ◄────────────────────────────┘
-                        Azure STT → Gemma LLM → Azure TTS
+                        Azure STT → LLM (OpenWebUI) → Azure TTS
 ```
 
 ## Quick Start
@@ -50,8 +50,9 @@ Key variables:
 - `AZURE_SPEECH_KEY` — Azure Speech resource key
 - `AZURE_SPEECH_REGION` — default `eastus`
 - `AZURE_TTS_VOICE` — default voice (`en-US-Ava:DragonHDOmniLatestNeural`)
-- `GEMMA_BASE_URL` — LLM endpoint (default `http://192.168.86.2:8023/v1`)
-- `GEMMA_MODEL` — model name (default `gemma-4-e4b`)
+- `LLM_BASE_URL` — OpenAI-compatible endpoint (default `http://192.168.86.2:3005/api`, OpenWebUI)
+- `LLM_MODEL` — default model (default `gemma-4-e4b`; switchable live from the web UI)
+- `LLM_API_KEY` — OpenWebUI API key (bears `sk-…`)
 
 ## Voice Selection
 
@@ -128,9 +129,11 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
 - **Mic requires HTTPS.** Browsers block getUserMedia outside a secure context. nginx serves the UI on `WEB_PORT` (9178 by default) over HTTPS with a self-signed cert whose SAN includes the detected LAN IP. The cert is persisted in `./certs/` and the LiveKit WS is proxied through nginx at `/livekit/`.
 - **No CDN dependencies.** livekit-client UMD bundle is vendored into `web/`; LAN devices may have no internet access.
 - **Transcripts flow over the data channel.** The agent publishes `{type: "transcript", role, text}` JSON on topic "transcript"; the UI renders them. Voice changes flow the other way as `{type: "set_voice", voice}` on topic "voice-control".
-- **Gemma is a reasoning model.** It sometimes spends tokens on hidden reasoning before producing content. The agent implements this with `max_completion_tokens=1000` and a `GemmaLLM` subclass that wraps the LLM stream, falling back to `reasoning_content` if `content` is empty.
+- **Models come from OpenWebUI.** The LLM endpoint is OpenWebUI's OpenAI-compatible API at `LLM_BASE_URL` (default `http://192.168.86.2:3005/api`); chat goes to `/api/chat/completions` and the model list comes from `GET {LLM_BASE_URL}/models`. Plain `/v1/*` routes are NOT routed on that instance (they serve the SPA). Pipe models (`opencode_go_messages.*`, `go-chat.*`, `go-responses.*`) currently fail upstream with auth errors; the local llama.cpp/vLLM models (e.g. `gemma-4-e4b`, `qwen3.8-27b`) work.
+- **Reasoning models may hide their answer.** Gemma (and similar) sometimes spends tokens on hidden reasoning before producing content. The agent implements this with `max_completion_tokens=1000` and a `HopeLLM` subclass that wraps the LLM stream, falling back to `reasoning_content` if `content` is empty.
 - **Azure TTS uses SSML, not JSON.** The REST endpoint requires `Content-Type: application/ssml+xml`. The LiveKit Azure plugin handles this internally.
 - **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
+- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
 - **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
 
 ## Git

agent/agent.py

@@ -1,7 +1,7 @@
 """
 Voice Agent — real-time voice assistant.
 
-Pipeline: Azure STT → Gemma LLM (xNAS, OpenAI-compatible) → Azure TTS
+Pipeline: Azure STT → LLM (OpenWebUI, OpenAI-compatible) → Azure TTS
 Runs inside the single Docker container alongside LiveKit server and web frontend.
 """
 
@@ -37,9 +37,23 @@ AZURE_REGION = os.environ.get("AZURE_SPEECH_REGION", "eastus")
 DEFAULT_VOICE = os.environ.get(
     "AZURE_TTS_VOICE", "en-US-Ava:DragonHDOmniLatestNeural"
 )
-GEMMA_BASE_URL = os.environ.get("GEMMA_BASE_URL", "http://192.168.86.2:8023/v1")
-GEMMA_MODEL = os.environ.get("GEMMA_MODEL", "gemma-4-e4b")
-GEMMA_API_KEY = os.environ.get("GEMMA_API_KEY", "not-needed")
+LLM_BASE_URL = (
+    os.environ.get("LLM_BASE_URL")
+    or os.environ.get("GEMMA_BASE_URL")
+    or "http://192.168.86.2:3005/api"
+)
+LLM_MODEL = (
+    os.environ.get("LLM_MODEL")
+    or os.environ.get("GEMMA_MODEL")
+    or "gemma-4-e4b"
+)
+LLM_API_KEY = (
+    os.environ.get("LLM_API_KEY")
+    or os.environ.get("GEMMA_API_KEY")
+    or "not-needed"
+)
+# Model ids that cannot hold a conversation (OpenWebUI hosts these too).
+NON_CHAT_MODEL_ID = re.compile(r"embed|rerank|whisper|tts-|stable-?diffusion|flux", re.I)
 WEB_MCP_ENABLED = os.environ.get("WEB_MCP_ENABLED", "true").lower() in ("1", "true", "yes")
 FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
 SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
@@ -67,6 +81,39 @@ def user_id_from_room(room_name: str) -> str:
     return match.group(1)
 
 
+async def list_available_models() -> list[str]:
+    """Fetch chat-capable model ids from the OpenAI-compatible endpoint.
+
+    Works with OpenWebUI (GET /api/models) and plain OpenAI-compatible
+    servers (GET /v1/models) — both live under {base_url}/models. The
+    configured default model is always first so it survives fetch failures.
+    """
+    ids: list[str] = []
+    try:
+        import httpx
+
+        headers = {}
+        if LLM_API_KEY and LLM_API_KEY != "not-needed":
+            headers["Authorization"] = f"Bearer {LLM_API_KEY}"
+        async with httpx.AsyncClient(timeout=10) as client:
+            resp = await client.get(f"{LLM_BASE_URL}/models", headers=headers)
+            resp.raise_for_status()
+            data = resp.json()
+        ids = [
+            model_id
+            for model in data.get("data", [])
+            if isinstance(model, dict) and (model_id := model.get("id", ""))
+        ]
+    except Exception as e:  # noqa: BLE001
+        logger.warning("Could not list models from %s: %s", LLM_BASE_URL, e)
+
+    chat_models = [m for m in ids if not NON_CHAT_MODEL_ID.search(m)]
+    if LLM_MODEL in chat_models:
+        chat_models.remove(LLM_MODEL)
+    ordered = [LLM_MODEL] + sorted(chat_models)
+    return ordered or [LLM_MODEL]
+
+
 SYSTEM_PROMPT = textwrap.dedent("""\
     You are Hope, a warm, conversational voice assistant speaking with
     {user_display_name}. You are talking TO someone, not writing for them to
@@ -167,14 +214,15 @@ except ImportError:
     from task_router import should_force_background_task
 
 
-# ── Gemma LLM with reasoning_content fallback ───────────────────────────────
+# ── OpenAI-compatible LLM with reasoning_content fallback ───────────────────
 
 
 class _ReasoningFallbackWrapper:
     """Wraps an LLMStream to fall back to ``reasoning_content`` when the model
-    finishes a turn with empty visible content (Gemma spends its whole budget
-    on hidden reasoning). Delegates all iteration to the underlying stream and
-    injects a final content chunk if no real content was produced."""
+    finishes a turn with empty visible content (reasoning models such as Gemma
+    spend their whole budget on hidden reasoning). Delegates all iteration to
+    the underlying stream and injects a final content chunk if no real content
+    was produced."""
 
     def __init__(self, inner: lk_llm.LLMStream) -> None:
         self._inner = inner
@@ -257,25 +305,42 @@ class _ForcedToolCallStream(lk_llm.LLMStream):
         )
 
 
-class GemmaLLM(openai.LLM):
-    """OpenAI-compatible LLM (llama.cpp Gemma 4) with thinking disabled."""
+class HopeLLM(openai.LLM):
+    """OpenAI-compatible LLM (OpenWebUI) with runtime model switching."""
 
     # Context compaction: when the conversation grows past this many items,
-    # ask Gemma to summarize the older messages into a compact recap, then
+    # ask the LLM to summarize the older messages into a compact recap, then
     # replace them with that summary. Keeps recent turns verbatim.
     COMPACTION_THRESHOLD = 24  # ~12 turns of user/agent exchange
     KEEP_RECENT = 10           # last N items kept verbatim after compaction
     IDLE_COMPACTION_SECONDS = 300  # also compact after 5 min of silence
 
-    def __init__(self, *args, **kwargs):
+    def __init__(self, *, model: str, **kwargs):
         # llama.cpp's Gemma tool template ignores named tool_choice when OpenAI
         # strict schemas are enabled. Non-strict schemas make forced dispatch
         # (and ordinary MCP calls) work reliably.
         kwargs.setdefault("_strict_tool_schema", False)
-        super().__init__(*args, **kwargs)
+        self._model = model
+        super().__init__(model=model, **kwargs)
         import time
         self._last_call_time: float = time.monotonic()
 
+    @property
+    def model(self) -> str:
+        """Currently selected model id (openai.LLM exposes the same property)."""
+        return self._model
+
+    def update_options(self, *, model: str | None = None) -> None:
+        """Switch the backing model for all subsequent chat calls.
+
+        The base class re-reads ``self._opts`` on every request, so mutating
+        the opts dataclass is enough — no client rebuild needed.
+        """
+        if model and model != self._model:
+            logger.info("LLM model change: %s -> %s", self._model, model)
+            self._model = model
+            self._opts.model = model
+
     def chat(self, *, chat_ctx, tools=None, conn_options=None, **kwargs):
         if conn_options is None:
             from livekit.agents.types import DEFAULT_API_CONNECT_OPTIONS
@@ -303,7 +368,8 @@ class GemmaLLM(openai.LLM):
             chat_ctx = self._compact_context(chat_ctx, conn_options)
 
         # Force dispatch for clear background-work requests. Without this,
-        # Gemma often says "I'll get on that" but emits no tool call at all.
+        # small local models often say "I'll get on that" but emit no tool
+        # call at all.
         items = list(chat_ctx.items)
         latest = items[-1] if items else None
         latest_role = getattr(latest, "role", None)
@@ -333,7 +399,7 @@ class GemmaLLM(openai.LLM):
         """Summarize older messages via a side LLM call, keep recent ones.
 
         Splits the context into [system | old | recent]. The old portion is
-        summarized by Gemma into a short recap message that replaces it.
+        summarized by the LLM into a short recap message that replaces it.
         Falls back to hard truncation if the summary call fails.
         """
         from livekit.agents import llm as lk_llm
@@ -361,7 +427,7 @@ class GemmaLLM(openai.LLM):
         if not transcript.strip():
             return chat_ctx  # nothing meaningful to summarize
 
-        # Side call to Gemma for the summary (no tools, no streaming needed)
+        # Side call to the LLM for the summary (no tools, no streaming needed)
         try:
             import httpx
 
@@ -371,11 +437,14 @@ class GemmaLLM(openai.LLM):
                 "Be concise — this replaces the full history as context.\n\n"
                 + transcript
             )
+            headers = {"Content-Type": "application/json"}
+            if LLM_API_KEY and LLM_API_KEY != "not-needed":
+                headers["Authorization"] = f"Bearer {LLM_API_KEY}"
             resp = httpx.post(
-                f"{GEMMA_BASE_URL}/chat/completions",
-                headers={"Content-Type": "application/json"},
+                f"{LLM_BASE_URL}/chat/completions",
+                headers=headers,
                 json={
-                    "model": GEMMA_MODEL,
+                    "model": self._model,
                     "messages": [
                         {"role": "system", "content": "You summarize conversations concisely."},
                         {"role": "user", "content": summary_prompt},
@@ -554,10 +623,10 @@ class VoiceAssistant(Agent):
         display_name = user_profile.get("display_name") or user_profile["user_id"]
         instructions = SYSTEM_PROMPT.replace("{user_display_name}", display_name)
         super().__init__(
-            llm=GemmaLLM(
-                model=GEMMA_MODEL,
-                base_url=GEMMA_BASE_URL,
-                api_key=GEMMA_API_KEY,
+            llm=HopeLLM(
+                model=LLM_MODEL,
+                base_url=LLM_BASE_URL,
+                api_key=LLM_API_KEY,
                 max_completion_tokens=1000,
                 extra_body={"chat_template_kwargs": {"enable_thinking": False}},
             ),
@@ -613,6 +682,27 @@ async def handle_job(ctx: JobContext) -> None:
         ),
     )
 
+    # The assistant owns the LLM instance; data-channel handlers below reach
+    # into it for live model switching.
+    voice_assistant = VoiceAssistant(user_profile, task_dir)
+
+    # Publish the chat-capable model list so the web UI can offer a picker.
+    async def _publish_models() -> None:
+        models = await list_available_models()
+        payload = json.dumps(
+            {
+                "type": "models",
+                "models": models,
+                "selected": voice_assistant.llm.model,
+            }
+        )
+        try:
+            await ctx.room.local_participant.publish_data(
+                payload, reliable=True, topic="voice-control"
+            )
+        except Exception as e:  # noqa: BLE001
+            logger.warning("failed to publish model list: %s", e)
+
     # Listen for conversation controls (voice selection and voice/text mode).
     # Defined here so it captures this job's session via closure.
     def _on_room_data(packet) -> None:
@@ -650,6 +740,29 @@ async def handle_job(ctx: JobContext) -> None:
             except Exception as e:  # noqa: BLE001
                 logger.warning("Failed to update voice: %s", e)
 
+        if msg.get("type") == "get_models":
+            asyncio.create_task(_publish_models())
+            return
+
+        if msg.get("type") == "set_model":
+            model = msg.get("model", "")
+            if not model:
+                return
+            try:
+                llm = voice_assistant.llm
+                if hasattr(llm, "update_options"):
+                    llm.update_options(model=model)
+                    ack = json.dumps(
+                        {"type": "model_changed", "model": llm.model}
+                    )
+                    asyncio.create_task(
+                        ctx.room.local_participant.publish_data(
+                            ack, reliable=True, topic="voice-control"
+                        )
+                    )
+            except Exception as e:  # noqa: BLE001
+                logger.warning("Failed to update model: %s", e)
+
     ctx.room.on("data_received", _on_room_data)
 
     # Publish user/agent transcripts to the room so the web UI can render them.
@@ -743,7 +856,7 @@ async def handle_job(ctx: JobContext) -> None:
     ctx.add_shutdown_callback(_stop_task_poller)
 
     await session.start(
-        agent=VoiceAssistant(user_profile, task_dir),
+        agent=voice_assistant,
         room=ctx.room,
         room_options=room_io.RoomOptions(
             # Keep the agent in the room when a participant leaves; it must
@@ -769,6 +882,9 @@ async def handle_job(ctx: JobContext) -> None:
         )
     except Exception as exc:  # noqa: BLE001
         logger.warning("failed to publish agent readiness: %s", exc)
+
+    # Announce which models the UI can switch between (selected one included).
+    await _publish_models()
     logger.info("Agent connected to room %s", ctx.room.name)
 
 

agent/task_worker.py

@@ -1,7 +1,7 @@
 """Background task worker — runs an autonomous LLM loop with tools.
 
 Each dispatched task gets its own TaskWorker instance running in a separate
-asyncio task. The worker calls Gemma iteratively, executing tool calls and
+asyncio task. The worker calls the LLM iteratively, executing tool calls and
 logging each step to the registry so the UI can display live progress.
 """
 from __future__ import annotations
@@ -31,9 +31,21 @@ class TaskWorker:
         except ImportError:
             from task_registry import registry as reg
 
-        base_url = os.environ.get("GEMMA_BASE_URL", "http://192.168.86.2:8023/v1")
-        model = os.environ.get("GEMMA_MODEL", "gemma-4-e4b")
-        api_key = os.environ.get("GEMMA_API_KEY", "not-needed")
+        base_url = (
+            os.environ.get("LLM_BASE_URL")
+            or os.environ.get("GEMMA_BASE_URL")
+            or "http://192.168.86.2:3005/api"
+        )
+        model = (
+            os.environ.get("LLM_MODEL")
+            or os.environ.get("GEMMA_MODEL")
+            or "gemma-4-e4b"
+        )
+        api_key = (
+            os.environ.get("LLM_API_KEY")
+            or os.environ.get("GEMMA_API_KEY")
+            or "not-needed"
+        )
 
         reg.add_step(self.task_id, "thinking", f"Starting: {self.description}")
 

docker-compose.yml

@@ -10,9 +10,9 @@ services:
       AZURE_SPEECH_KEY: "${AZURE_SPEECH_KEY:?Set AZURE_SPEECH_KEY in .env}"
       AZURE_SPEECH_REGION: "${AZURE_SPEECH_REGION:-eastus}"
       AZURE_TTS_VOICE: "${AZURE_TTS_VOICE:-en-US-Ava:DragonHDOmniLatestNeural}"
-      GEMMA_BASE_URL: "${GEMMA_BASE_URL:-http://192.168.86.2:8023/v1}"
-      GEMMA_MODEL: "${GEMMA_MODEL:-gemma-4-e4b}"
-      GEMMA_API_KEY: "${GEMMA_API_KEY:-not-needed}"
+      LLM_BASE_URL: "${LLM_BASE_URL:-http://192.168.86.2:3005/api}"
+      LLM_MODEL: "${LLM_MODEL:-gemma-4-e4b}"
+      LLM_API_KEY: "${LLM_API_KEY:-not-needed}"
       WEB_MCP_ENABLED: "${WEB_MCP_ENABLED:-true}"
       FIRECRAWL_BASE: "${FIRECRAWL_BASE:-http://192.168.86.2:3002}"
       WEB_PORT: "${WEB_PORT:-9178}"

supervisord.conf

@@ -27,9 +27,9 @@ environment=
     AZURE_SPEECH_KEY="%(ENV_AZURE_SPEECH_KEY)s",
     AZURE_SPEECH_REGION="%(ENV_AZURE_SPEECH_REGION)s",
     AZURE_TTS_VOICE="%(ENV_AZURE_TTS_VOICE)s",
-    GEMMA_BASE_URL="%(ENV_GEMMA_BASE_URL)s",
-    GEMMA_MODEL="%(ENV_GEMMA_MODEL)s",
-    GEMMA_API_KEY="%(ENV_GEMMA_API_KEY)s",
+    LLM_BASE_URL="%(ENV_LLM_BASE_URL)s",
+    LLM_MODEL="%(ENV_LLM_MODEL)s",
+    LLM_API_KEY="%(ENV_LLM_API_KEY)s",
     WEB_MCP_ENABLED="%(ENV_WEB_MCP_ENABLED)s",
     FIRECRAWL_BASE="%(ENV_FIRECRAWL_BASE)s",
     VOICE_USERS_FILE="%(ENV_VOICE_USERS_FILE)s"

tests/README.md

@@ -9,7 +9,7 @@ tool servers, and agent process.
 - The container is up: `docker compose up -d` (and healthy)
 - Host deps installed: `pip install -r tests/requirements.txt`
   (or `uv pip install -r tests/requirements.txt`)
-- The LLM at `GEMMA_BASE_URL` (default `http://192.168.86.2:8023/v1`) is reachable
+- The LLM at `LLM_BASE_URL` (default `http://192.168.86.2:3005/api`, OpenWebUI) is reachable
 - Internet access for the weather tool tests (wttr.in)
 
 ## Run
@@ -25,10 +25,10 @@ python -m pytest tests/ -v -m "not slow"   # skip LLM/network-heavy tests
 | File | Target | Notes |
 |------|--------|-------|
 | `test_web_ui.py` | nginx HTTPS UI (:9178) + token endpoint (:8091) | static assets, `/token` (direct + via nginx), `/livekit/` proxy |
-| `test_llm_api.py` | Gemma LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
+| `test_llm_api.py` | LLM API directly | completion, tool calling, `enable_thinking=false` latency, streaming |
 | `test_mcp_tools.py` | MCP servers via `docker exec` | weather/time, authenticated remote memory save/recall/delete, and local skills — probes are cleaned up |
 | `test_agent_integration.py` | running container | agent process alive, worker registered with LiveKit, all 4 supervised processes up, container health, LiveKit HTTP API |
-| `test_compaction.py` | `GemmaLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
+| `test_compaction.py` | `HopeLLM._compact_context` in-container | compaction triggers >24 items, system prompt preserved, short context untouched |
 
 ## Notes
 

tests/conftest.py

@@ -3,7 +3,7 @@
 Tests run on the HOST machine against the live "voice" container:
 - HTTPS web UI  -> https://localhost:9178 (self-signed cert, verify=False)
 - Token server  -> http://127.0.0.1:8091 (host network mode)
-- LLM API       -> GEMMA_BASE_URL (default http://192.168.86.2:8023/v1)
+- LLM API       -> LLM_BASE_URL (default http://192.168.86.2:3005/api)
 - In-container  -> docker exec voice ...
 """
 
@@ -18,8 +18,21 @@ import pytest
 CONTAINER = "voice"
 WEB_BASE_URL = os.environ.get("VOICE_WEB_URL", "https://localhost:9178")
 TOKEN_URL = os.environ.get("VOICE_TOKEN_URL", "http://127.0.0.1:8091/token")
-GEMMA_BASE_URL = os.environ.get("GEMMA_BASE_URL", "http://192.168.86.2:8023/v1").rstrip("/")
-GEMMA_MODEL = os.environ.get("GEMMA_MODEL", "gemma-4-e4b")
+LLM_BASE_URL = (
+    os.environ.get("LLM_BASE_URL")
+    or os.environ.get("GEMMA_BASE_URL")
+    or "http://192.168.86.2:3005/api"
+).rstrip("/")
+LLM_MODEL = (
+    os.environ.get("LLM_MODEL")
+    or os.environ.get("GEMMA_MODEL")
+    or "gemma-4-e4b"
+)
+LLM_API_KEY = (
+    os.environ.get("LLM_API_KEY")
+    or os.environ.get("GEMMA_API_KEY")
+    or "not-needed"
+)
 _users = json.loads((Path(__file__).parents[1] / "config" / "users.json").read_text())
 _shane_memory = _users["users"]["shane"]["memory"]
 MEMORY_MCP_URL = os.environ.get("MEMORY_MCP_URL", _shane_memory["url"])
@@ -88,8 +101,11 @@ def token() -> dict:
 
 @pytest.fixture(scope="session")
 def llm_client():
-    """httpx client pointed at the Gemma LLM API (same one Hope uses)."""
-    with httpx.Client(base_url=GEMMA_BASE_URL, timeout=60) as c:
+    """httpx client pointed at the LLM API (same one Hope uses)."""
+    headers = {}
+    if LLM_API_KEY and LLM_API_KEY != "not-needed":
+        headers["Authorization"] = f"Bearer {LLM_API_KEY}"
+    with httpx.Client(base_url=LLM_BASE_URL, headers=headers, timeout=60) as c:
         yield c
 
 

tests/test_compaction.py

@@ -1,26 +1,27 @@
-"""Tests for GemmaLLM context compaction (agent.GemmaLLM._compact_context).
+"""Tests for HopeLLM context compaction (agent.HopeLLM._compact_context).
 
-Compaction runs a real side LLM call to the xNAS, so these are marked slow.
-The agent module is imported inside the container via docker exec.
+Compaction runs a real side LLM call to the configured endpoint, so these
+are marked slow. The agent module is imported inside the container via
+docker exec.
 """
 
 import pytest
 
-from conftest import GEMMA_BASE_URL, GEMMA_MODEL, docker_exec_python
+from conftest import LLM_BASE_URL, LLM_MODEL, docker_exec_python
 
 # {num_messages} and {do_chat} are filled per test; all other braces are literal.
 TEMPLATE = '''
-import logging, sys
+import logging, os, sys
 from livekit.agents import llm as lk_llm
 sys.path.insert(0, "/opt/voice-agent")
-from agent import GemmaLLM
+from agent import HopeLLM
 
 logging.basicConfig(level=logging.INFO, stream=sys.stdout)
 
-llm = GemmaLLM(
+llm = HopeLLM(
     model="{model}",
     base_url="{base_url}",
-    api_key="not-needed",
+    api_key=os.environ.get("LLM_API_KEY", "not-needed"),
     max_completion_tokens=100,
     extra_body={{"chat_template_kwargs": {{"enable_thinking": False}}}},
 )
@@ -44,7 +45,7 @@ async def main():
     print("AFTER items=" + str(len(compacted.items)))
     for it in compacted.items:
         role = getattr(it, "role", None)
-        content = GemmaLLM._msg_content_text(it)
+        content = HopeLLM._msg_content_text(it)
         print("ITEM " + str(role) + ": " + content[:120])
 
     {do_chat}
@@ -57,8 +58,8 @@ CHAT_SNIPPET = '    stream = llm.chat(chat_ctx=compacted)\n    collected = await
 
 def _run(num_messages: int, *, do_chat: bool = False) -> str:
     code = TEMPLATE.format(
-        model=GEMMA_MODEL,
-        base_url=GEMMA_BASE_URL,
+        model=LLM_MODEL,
+        base_url=LLM_BASE_URL,
         num_messages=num_messages,
         do_chat=CHAT_SNIPPET if do_chat else "pass",
     )

tests/test_llm_api.py

@@ -1,10 +1,10 @@
-"""Tests for the Gemma LLM API — the exact endpoint Hope's agent uses."""
+"""Tests for the LLM API — the exact endpoint Hope's agent uses."""
 
 import time
 
 import pytest
 
-from conftest import GEMMA_BASE_URL, GEMMA_MODEL
+from conftest import LLM_BASE_URL, LLM_MODEL
 
 WEATHER_TOOL = {
     "type": "function",
@@ -22,7 +22,7 @@ WEATHER_TOOL = {
 
 def _chat(llm_client, messages, *, tools=None, stream=False, max_tokens=200):
     payload = {
-        "model": GEMMA_MODEL,
+        "model": LLM_MODEL,
         "messages": messages,
         "max_tokens": max_tokens,
         "temperature": 0.3,

web/app.js

@@ -27,6 +27,7 @@ const VOICES = [
 const HISTORY_KEY = "voice_history";
 const HISTORY_MAX = 50;
 const MODE_KEY = "hope_conversation_mode";
+const MODEL_KEY = "preferred_model";
 
 // ── State ───────────────────────────────────────────────────────────────────
 let room = null;
@@ -40,6 +41,10 @@ let wakeLock = null;
 let conversationMode = localStorage.getItem(MODE_KEY) === "text" ? "text" : "voice";
 let connectionPromise = null;
 
+// ── Model (brain) state — populated from the agent's data channel ──────────
+let availableModels = [];          // ids published by the agent
+let selectedModel = localStorage.getItem(MODEL_KEY) || null;
+
 function escapeHtml(s) {
     const d = document.createElement("div");
     d.textContent = s || "";
@@ -63,6 +68,8 @@ const settingsSheet = document.getElementById("settingsSheet");
 const sheetBackdrop = document.getElementById("sheetBackdrop");
 const closeSheetBtn = document.getElementById("closeSheetBtn");
 const voiceListEl = document.getElementById("voiceList");
+const modelListEl = document.getElementById("modelList");
+const modelNoteEl = document.getElementById("modelNote");
 const clearBtn = document.getElementById("clearBtn");
 const tasksBtn = document.getElementById("tasksBtn");
 const tasksPanel = document.getElementById("tasksPanel");
@@ -547,8 +554,20 @@ function handleDataPacket(payload, participant, topic) {
             }
         } else if (msg.type === "task_event") {
             handleTaskEvent(msg);
+        } else if (msg.type === "models") {
+            availableModels = Array.isArray(msg.models) ? msg.models : [];
+            // Don't clobber a saved preference — the agent acks the applied
+            // model separately via model_changed.
+            if (!selectedModel && msg.selected) selectedModel = msg.selected;
+            renderModelList();
+        } else if (msg.type === "model_changed") {
+            if (msg.model) {
+                selectedModel = msg.model;
+                localStorage.setItem(MODEL_KEY, selectedModel);
+                renderModelList();
+            }
         }
-        // set_voice messages flow the other way; nothing to do client-side.
+        // set_voice/set_model messages flow the other way; nothing else to do client-side.
     } catch (e) {
         console.warn("bad data packet", e);
     }
@@ -598,8 +617,61 @@ function selectVoice(voiceId) {
     }
 }
 
+// ── Model (brain) settings ──────────────────────────────────────────────────
+function currentModelId() {
+    if (availableModels.length === 0) return selectedModel;
+    if (selectedModel && availableModels.includes(selectedModel)) return selectedModel;
+    return availableModels[0];
+}
+
+function renderModelList() {
+    modelListEl.innerHTML = "";
+    if (!availableModels.length) {
+        modelNoteEl.hidden = false;
+        modelNoteEl.textContent = room
+            ? "Loading available models…"
+            : "Start a conversation to load the model list.";
+        return;
+    }
+    modelNoteEl.hidden = true;
+    const active = currentModelId();
+    for (const id of availableModels) {
+        const card = document.createElement("div");
+        card.className = "voice-card" + (id === active ? " active" : "");
+        card.dataset.model = id;
+        card.setAttribute("role", "radio");
+        card.setAttribute("aria-checked", String(id === active));
+        card.tabIndex = 0;
+
+        const name = document.createElement("span");
+        name.className = "voice-name";
+        name.textContent = id;
+
+        card.appendChild(name);
+        card.addEventListener("click", () => selectModel(id));
+        card.addEventListener("keydown", (event) => {
+            if (event.key === "Enter" || event.key === " ") {
+                event.preventDefault();
+                selectModel(id);
+            }
+        });
+        modelListEl.appendChild(card);
+    }
+}
+
+function selectModel(modelId) {
+    selectedModel = modelId;
+    localStorage.setItem(MODEL_KEY, modelId);
+    renderModelList();
+    if (room && agentParticipant) {
+        sendModelMessage(modelId);
+        addMessage("system", `Brain changed to ${modelId}`);
+    }
+}
+
 function openSheet() {
     renderVoiceList();
+    renderModelList();
     sheetBackdrop.hidden = false;
     settingsSheet.classList.add("open");
     settingsBtn.setAttribute("aria-expanded", "true");
@@ -665,8 +737,9 @@ function markAgentFound(rm, participant) {
     statusEl.textContent = conversationMode === "text"
         ? "Hope joined · Text chat ready"
         : "Hope joined · Preparing microphone…";
-    // Synchronize voice and interaction mode when the agent joins.
+    // Synchronize voice, brain and interaction mode when the agent joins.
     sendVoiceMessage(currentVoiceId(), rm);
+    if (selectedModel) sendModelMessage(selectedModel, rm);
     sendModeMessage(conversationMode, rm);
 }
 
@@ -980,3 +1053,14 @@ function sendModeMessage(mode, targetRoom) {
         reliable: true,
     });
 }
+
+// ── Helper: send model selection to the agent via data channel ─────────────
+function sendModelMessage(model, targetRoom) {
+    const r = targetRoom || room;
+    if (!r || !r.localParticipant || !model) return;
+    const payload = new TextEncoder().encode(JSON.stringify({ type: "set_model", model }));
+    r.localParticipant.publishData(payload, {
+        topic: "voice-control",
+        reliable: true,
+    });
+}

web/index.html

@@ -75,7 +75,7 @@
                     <button id="muteBtn" class="btn btn-icon" aria-label="Mute microphone" title="Mute microphone" disabled>
                         <svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M12 2a3 3 0 0 0-3 3v7a3 3 0 0 0 6 0V5a3 3 0 0 0-3-3Z"></path><path d="M19 10v2a7 7 0 0 1-14 0v-2"></path><line x1="8" y1="22" x2="16" y2="22"></line></svg>
                     </button>
-                    <button id="settingsBtn" class="btn btn-icon" aria-label="Voice settings" title="Voice settings">
+                    <button id="settingsBtn" class="btn btn-icon" aria-label="Voice and brain settings" title="Voice and brain settings">
                         <svg viewBox="0 0 24 24" width="19" height="19" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M4 7h10M18 7h2M4 17h2M10 17h10"></path><circle cx="16" cy="7" r="2"></circle><circle cx="8" cy="17" r="2"></circle></svg>
                     </button>
                 </div>
@@ -113,18 +113,22 @@
     </main>
 
     <div class="sheet-backdrop" id="sheetBackdrop" hidden></div>
-    <div class="settings-sheet" id="settingsSheet" role="dialog" aria-modal="true" aria-label="Voice settings">
+    <div class="settings-sheet" id="settingsSheet" role="dialog" aria-modal="true" aria-label="Hope settings">
         <div class="sheet-handle" aria-hidden="true"></div>
         <div class="sheet-header">
             <div>
-                <h2>Choose Hope’s voice</h2>
-                <p>Changes apply to this conversation.</p>
+                <h2>Hope’s settings</h2>
+                <p>Voice and brain — changes apply to this conversation.</p>
             </div>
-            <button id="closeSheetBtn" class="btn btn-icon" aria-label="Close voice settings">
+            <button id="closeSheetBtn" class="btn btn-icon" aria-label="Close settings">
                 <svg viewBox="0 0 24 24" width="18" height="18" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><path d="M18 6 6 18M6 6l12 12"></path></svg>
             </button>
         </div>
+        <h3 class="sheet-section-title">Voice</h3>
         <div class="voice-list" id="voiceList"></div>
+        <h3 class="sheet-section-title">Brain</h3>
+        <div class="voice-list" id="modelList"></div>
+        <p class="sheet-note" id="modelNote" hidden></p>
     </div>
 
     <div class="tasks-panel" id="tasksPanel" aria-label="Background tasks">

web/style.css

@@ -1043,6 +1043,22 @@ body[data-mode="text"] .mic-meter {
     font-size: 0.72rem;
 }
 
+.sheet-section-title {
+    margin: 0.4rem 0 0.55rem;
+    padding: 0 0.2rem;
+    color: var(--ink-muted);
+    font-size: 0.68rem;
+    font-weight: 700;
+    letter-spacing: 0.14em;
+    text-transform: uppercase;
+}
+
+.sheet-note {
+    margin: 0.5rem 0 0;
+    color: var(--ink-muted);
+    font-size: 0.72rem;
+}
+
 .voice-list {
     min-height: 0;
     overflow-y: auto;
@@ -1052,6 +1068,11 @@ body[data-mode="text"] .mic-meter {
     padding: 0.15rem 0.2rem 0 0;
 }
 
+/* Two pickers (voice + brain) share the settings sheet height. */
+.settings-sheet .voice-list {
+    flex: 1 1 0;
+}
+
 .voice-card {
     min-height: 54px;
     display: flex;