fix: force tool calls for drifted intents and harden tool-use prompt

Small local models (qwen2.5-vl-3b, gemma-4-e4b) narrate actions ('Sure,
I'll pause the music') without emitting the tool call once the full
system prompt and 26-tool set are in context. Measured: qwen emitted
tools in 2/8 realistic intents, gemma 5/8; single-tool tests pass 100%,
so this is context-induced drift, not plumbing.

- agent/intent_router.py: deterministic forced-call router for
  high-frequency intents — media controls, now_playing/list_library,
  'remember that ...' memory saves, explicit YouTube Music searches,
  and personal-fact recall (memory_recall/search_notes/skill_recall,
  excluded for weather/time questions). Fires only when the tool is
  available; guards against how-do-I questions.
- agent.py: generalize _ForcedToolCallStream to any tool; route through
  intent_router after the dispatch check. System prompt adds a hard
  'call, don't narrate' rule and reconditions 'answer directly'.
- tests/test_intent_router.py: 30 unit tests pinning router behavior.

Verified against the live LLM with the full fidelity harness: combined
forced-or-emitted coverage 8/8 on qwen2.5-vl-3b (was 2/8) and 8/8 on
gemma-4-e4b (was 5/8). Full suite: 79 passed.
This commit is contained in:
snowspeeder committed 2026-08-29 15:38:35 -04:00
1 parent 48b92585c6
commit 8a7cca456f
4 files changed
+464 -14

No files matched your search

+59 -14
View File
@@ -127,11 +127,19 @@ SYSTEM_PROMPT = textwrap.dedent("""\
no markdown, and no formatting except the Azure delivery markers below.
- Spell out numbers when it sounds more natural ("twenty twenty-six"
instead of "2026").
- Make reasonable assumptions and answer directly. Ask a follow-up only
when it is genuinely impossible to proceed without one.
- Make reasonable assumptions and answer directly once any relevant tool
call has been made. Ask a follow-up only when it is genuinely
impossible to proceed without one.
- Be warm and direct. Don't be sycophantic or overly formal.
- If you don't know something, say so briefly and move on.
# Tools: call them, don't narrate them
When a request matches one of your tools, you MUST emit that tool call
in this reply. Never only say "I'll pause the music", "let me check", or
"I'm saving that now" without the tool call — narrating an action is a
failure because nothing actually happens. After the tool result comes
back, confirm briefly in your normal voice.
# Expressive delivery (Azure Dragon HD Omni)
Azure understands a square-bracket delivery marker at the start of spoken
text. Use zero to two markers per response, only when they genuinely improve
@@ -223,12 +231,14 @@ SYSTEM_PROMPT = textwrap.dedent("""\
""")
# ── Background-task routing ─────────────────────────────────────────────────
# ── Intent routing ──────────────────────────────────────────────────────────
try:
from agent.task_router import should_force_background_task
from agent.intent_router import detect_forced_call
except ImportError:
from task_router import should_force_background_task
from intent_router import detect_forced_call
# ── OpenAI-compatible LLM with reasoning_content fallback ───────────────────
@@ -320,23 +330,39 @@ class _ReasoningFallbackWrapper:
class _ForcedToolCallStream(lk_llm.LLMStream):
"""Emit a dispatch_task call without asking the small LLM to route it."""
"""Emit a tool call without asking the small LLM to produce it.
def __init__(self, llm, *, chat_ctx, tools, conn_options, description: str):
self._description = description
Two callers: the task router (dispatch_task for background work) and the
intent router (deterministic intents like "pause the music" or "remember
that ..."), where small models otherwise narrate the action without ever
emitting the call.
"""
def __init__(
self,
llm,
*,
chat_ctx,
tools,
conn_options,
tool_name: str = "dispatch_task",
arguments: dict | None = None,
):
self._tool_name = tool_name
self._arguments = arguments if arguments is not None else {}
super().__init__(llm, chat_ctx=chat_ctx, tools=tools, conn_options=conn_options)
async def _run(self) -> None:
self._event_ch.send_nowait(
lk_llm.ChatChunk(
id=f"forced-dispatch-{uuid.uuid4().hex}",
id=f"forced-{uuid.uuid4().hex}",
delta=lk_llm.ChoiceDelta(
role="assistant",
tool_calls=[
lk_llm.FunctionToolCall(
name="dispatch_task",
arguments=json.dumps({"description": self._description}),
call_id=f"dispatch-{uuid.uuid4().hex}",
name=self._tool_name,
arguments=json.dumps(self._arguments),
call_id=f"forced-{uuid.uuid4().hex}",
)
],
),
@@ -415,9 +441,11 @@ class HopeLLM(openai.LLM):
)
chat_ctx = self._compact_context(chat_ctx, conn_options)
# Force dispatch for clear background-work requests. Without this,
# small local models often say "I'll get on that" but emit no tool
# call at all.
# Force calls for intents small models drift on. Without this they
# say "I'll pause the music" or "I'll get on that" but emit no tool
# call at all. Task router handles background work; the intent router
# handles deterministic in-conversation intents (media controls,
# memory saves, music searches).
items = list(chat_ctx.items)
latest = items[-1] if items else None
latest_role = getattr(latest, "role", None)
@@ -432,7 +460,24 @@ class HopeLLM(openai.LLM):
chat_ctx=chat_ctx,
tools=tools or [],
conn_options=conn_options,
description=latest_text,
tool_name="dispatch_task",
arguments={"description": latest_text},
)
forced = detect_forced_call(latest_text, tools or []) if latest_text else None
if forced:
tool_name, arguments = forced
logger.info(
"Routing request to forced %s(%s): %.120s",
tool_name, arguments, latest_text,
)
return _ForcedToolCallStream(
self,
chat_ctx=chat_ctx,
tools=tools or [],
conn_options=conn_options,
tool_name=tool_name,
arguments=arguments,
)
stream = super().chat(