fix: background tasks follow the live model selection
TaskWorker resolved its model from LLM_MODEL env (frozen at MCP spawn), so after a live set_model switch in the UI, dispatched background tasks still asked for the old model. On local-vl-models (llama-swap --models-max 1) that forces a swap against the loaded conversation model; a force-kill mid-load surfaces as 400 'model failed to load'. The agent now writes the selected model to <task_dir>/model.txt at session start and on every switch; TaskWorker reads it with the env value as fallback.
This commit is contained in:
1 parent
436b3fc8b1
commit
b01ee6478d
3 files changed
+37
-1
No files matched your search
@@ -216,6 +216,7 @@ curl -s "https://eastus.tts.speech.microsoft.com/cognitiveservices/v1" \
|
||||
- **Azure TTS uses SSML, not JSON.** The REST endpoint requires `Content-Type: application/ssml+xml`. The LiveKit Azure plugin handles this internally.
|
||||
- **Voice changes are live.** The web UI sends a data message to the agent; the agent calls `tts.update_options(voice=...)` without restarting. The agent filters data messages by topic ("voice-control") before processing.
|
||||
- **Model changes are live too.** The agent fetches the chat-capable model list (`HopeLLM` filters out embedding/reranker ids) and publishes `{type: "models", models, selected}`; the UI's settings sheet renders a picker and sends `{type: "set_model", model}`, which calls `HopeLLM.update_options(model=...)` (mutates `self._opts.model`) and acks with `{type: "model_changed", model}`. The choice persists in the browser's localStorage.
|
||||
- **Background tasks follow the live model.** The dispatch MCP subprocess inherits its env at spawn, so it cannot see later `set_model` switches. The agent writes the selected model to `<task_dir>/model.txt` at session start and on every switch; `TaskWorker` reads that file (falling back to `LLM_MODEL` env) when starting a task. Without this, a switched session (e.g. gemma-4-e4b) dispatches tasks against the stale env model (qwen2.5-vl-3b), which llama-swap on `local-vl-models` (`--models-max 1`) cannot load without killing the conversation model mid-swap.
|
||||
- **All supervisord programs log to /dev/stdout** so `docker compose logs -f voice` shows everything. Individual process logs are no longer written to files.
|
||||
- **Hope Music stays external.** `hope-media-music` owns yt-dlp, ffmpeg, the
|
||||
library volume, streaming endpoints, and the player UI. Do not add those
|
||||
|
||||
@@ -750,6 +750,20 @@ def build_mcp_toolsets(user_profile: dict, task_dir: str) -> list[mcp.MCPToolset
|
||||
return toolsets
|
||||
|
||||
|
||||
def _sync_task_model(task_dir: str, model: str) -> None:
|
||||
"""Record the live model choice where background task workers can read it.
|
||||
|
||||
The dispatch MCP subprocess inherits its environment at spawn time, so it
|
||||
can never see later UI model switches. The worker instead reads
|
||||
<task_dir>/model.txt (falling back to LLM_MODEL env) when it starts a task.
|
||||
"""
|
||||
try:
|
||||
os.makedirs(task_dir, exist_ok=True)
|
||||
with open(os.path.join(task_dir, "model.txt"), "w") as f:
|
||||
f.write((model or "").strip())
|
||||
except OSError as e:
|
||||
logger.warning("failed to record selected model for background tasks: %s", e)
|
||||
|
||||
class VoiceAssistant(Agent):
|
||||
"""The conversational agent. LLM is the brain; STT/TTS are senses."""
|
||||
|
||||
@@ -824,6 +838,10 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
# into it for live model switching.
|
||||
voice_assistant = VoiceAssistant(user_profile, task_dir)
|
||||
|
||||
# Background task workers (dispatch MCP subprocess) read this file so
|
||||
# they use the same model as the conversation, even after live switches.
|
||||
_sync_task_model(task_dir, voice_assistant.llm.model)
|
||||
|
||||
# Publish the chat-capable model list so the web UI can offer a picker.
|
||||
async def _publish_models() -> None:
|
||||
models = await list_available_models()
|
||||
@@ -1009,6 +1027,7 @@ async def handle_job(ctx: JobContext) -> None:
|
||||
ack, reliable=True, topic="voice-control"
|
||||
)
|
||||
)
|
||||
_sync_task_model(task_dir, llm.model)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("Failed to update model: %s", e)
|
||||
|
||||
|
||||
+17
-1
@@ -45,6 +45,22 @@ def _html_to_text(html: str) -> str:
|
||||
return text.strip()
|
||||
|
||||
|
||||
def _live_model(default: str) -> str:
|
||||
"""Follow the conversation's live model selection when it is fresher.
|
||||
|
||||
The main agent writes the UI-selected model to the session's task dir
|
||||
(shared with this MCP subprocess) on every set_model. Env is the
|
||||
fallback for tasks dispatched before any switch happened.
|
||||
"""
|
||||
env = os.environ.get("TASKS_DIR", "/tmp/tasks")
|
||||
try:
|
||||
with open(os.path.join(env, "model.txt")) as f:
|
||||
chosen = f.read().strip()
|
||||
except OSError:
|
||||
return default
|
||||
return chosen or default
|
||||
|
||||
|
||||
class TaskWorker:
|
||||
"""Runs a background research/lookup task using the LLM + tools."""
|
||||
|
||||
@@ -65,7 +81,7 @@ class TaskWorker:
|
||||
or os.environ.get("GEMMA_BASE_URL")
|
||||
or "http://192.168.86.2:3005/api"
|
||||
)
|
||||
model = (
|
||||
model = _live_model(
|
||||
os.environ.get("LLM_MODEL")
|
||||
or os.environ.get("GEMMA_MODEL")
|
||||
or "qwen2.5-vl-3b"
|
||||
|
||||
Reference in new issue
Block a user