feat: background task dispatch system with live UI panel

- agent/task_registry.py: file-based JSONL event registry (cross-process)
- agent/task_worker.py: autonomous LLM loop with weather/time/memory/web tools
- agent/dispatch_mcp.py: MCP tool exposing dispatch_task to the main agent
- agent/agent.py: registers dispatch toolset, polls task events → room data
- web: slide-out task panel (FAB button + badge), live step streaming via
  data channel topic 'tasks', status dots (running/completed/failed)
- Dockerfile: copies new task_*.py and dispatch_mcp.py files

The dispatch MCP runs in its own process; events flow through
/tmp/tasks/events.jsonl which the main agent tails every second and
forwards to the browser. Tasks run up to 10 LLM iterations with tool calls.
This commit is contained in:
Shane
2026-08-22 18:06:23 -04:00
parent d372adca7d
commit 44e8d05e2c
8 changed files with 457 additions and 48 deletions
+43
View File
@@ -98,6 +98,14 @@ SYSTEM_PROMPT = textwrap.dedent("""\
successfully complete a multi-step task or learn a new procedure from the user,
use skill_save to record it so you can follow it next time. Be selective: only
save skills for repeatable tasks, not one-off facts (those go in memory).
# Background Tasks (dispatch_task)
When the user asks you to research something, look up news, or do anything
that will take more than a few seconds, CALL dispatch_task with a clear
description of what to investigate. Tell the user "I'll take a look at that
for you" and keep talking. The result comes back automatically — when you
receive it, share the findings naturally. You can have multiple tasks running
at once.
""")
@@ -395,6 +403,20 @@ def build_mcp_toolsets() -> list[mcp.MCPToolset]:
)
logger.info("Skills MCP toolset enabled (%s)", SKILLS_DIR)
dispatch_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "dispatch_mcp.py")
toolsets.append(
mcp.MCPToolset(
id="dispatch",
mcp_server=mcp.MCPServerStdio(
command=python_bin,
args=[dispatch_mcp_script],
env={**os.environ},
client_session_timeout_seconds=60,
),
)
)
logger.info("Dispatch MCP toolset enabled")
extra = os.environ.get("EXTRA_MCP_SERVERS", "")
if extra:
try:
@@ -531,6 +553,27 @@ async def handle_job(ctx: JobContext) -> None:
# audio is arriving from the participant.
logger.info("user state -> %s", ev.new_state)
# ── Background task events → room data channel ─────────────────────────
from agent.task_registry import registry as task_registry
async def _publish_task_event(task_id: str, event: dict) -> None:
payload = json.dumps({"type": "task_event", "task_id": task_id, **event})
try:
await ctx.room.local_participant.publish_data(
payload, reliable=True, topic="tasks"
)
except Exception as e: # noqa: BLE001
logger.warning("failed to publish task event: %s", e)
async def _poll_tasks() -> None:
while True:
await asyncio.sleep(1.0)
events = await task_registry.poll_events()
for task_id, event in events:
await _publish_task_event(task_id, event)
asyncio.create_task(_poll_tasks())
await session.start(
agent=VoiceAssistant(),
room=ctx.room,