snowspeeder/hope-voice-api
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: replace custom memory MCP with Cognee remote server

Browse Source 
- Remove memory_mcp.py (stdio markdown-file memory)
- Add Cognee MCP as remote Streamable HTTP toolset at 192.168.86.2:8003/mcp
- Filter to only remember/recall/forget tools via allowed_tools
- Update system prompt: Memory section moved to top priority with
  explicit 'call recall FIRST' instructions and examples
- Add COGNEE_MCP_URL env var to docker-compose
- Remove /memory volume mount (no longer needed)
- Rewrite memory tests to use Cognee HTTP client fixture
- 28/28 tests passing
This commit is contained in:
Shane
2026-08-23 08:10:42 -04:00
parent f80b9ebe3d
commit c815bcb485
6 changed files with 89 additions and 226 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Dockerfile
+1 -1
View File
@@ -42,7 +42,7 @@ COPY --from=build /app/agent/.venv /opt/voice-agent/.venv
COPY agent/agent.py /opt/voice-agent/agent.py
COPY agent/web_mcp.py /opt/voice-agent/web_mcp.py
COPY agent/weather_mcp.py /opt/voice-agent/weather_mcp.py
COPY agent/memory_mcp.py /opt/voice-agent/memory_mcp.py
COPY agent/skills_mcp.py /opt/voice-agent/skills_mcp.py
COPY agent/task_registry.py /opt/voice-agent/task_registry.py
COPY agent/task_worker.py /opt/voice-agent/task_worker.py
agent/agent.py
+28 -30
View File
@@ -38,7 +38,6 @@ GEMMA_MODEL = os.environ.get("GEMMA_MODEL", "gemma-4-e4b")
GEMMA_API_KEY = os.environ.get("GEMMA_API_KEY", "not-needed")
WEB_MCP_ENABLED = os.environ.get("WEB_MCP_ENABLED", "true").lower() in ("1", "true", "yes")
FIRECRAWL_BASE = os.environ.get("FIRECRAWL_BASE", "http://192.168.86.2:3002")
MEMORY_DIR = os.environ.get("MEMORY_DIR", "/memory")
SKILLS_DIR = os.environ.get("SKILLS_DIR", "/skills")
SYSTEM_PROMPT = textwrap.dedent("""\
@@ -63,6 +62,20 @@ SYSTEM_PROMPT = textwrap.dedent("""\
- Never write more than three sentences in a row.
- Never read back URLs, file paths, or technical identifiers.
# Memory (CRITICAL — ALWAYS call these tools directly, never dispatch)
When the user asks about personal information, past conversations, or
anything you might have stored, you MUST call recall FIRST before
answering. Do NOT say "I don't have that info" without calling recall.
When the user tells you something to remember or shares a preference,
you MUST call remember. Do NOT just say "okay I'll remember that."
- User says "what's my name?" → CALL recall with query="my name"
- User says "do you remember who I am?" → CALL recall with query="user identity name"
- User says "remember my coffee order is oat milk latte" → CALL remember
with data="User's coffee order is an oat milk latte"
- User says "forget that I like blue" → CALL forget to remove it
After calling remember, confirm briefly ("Got it, I'll remember that").
If recall returns nothing, then say you don't have that info yet.
# Weather & Time
You can check the weather for any location. Use get_weather when the user
asks about current conditions, temperature, or forecasts. It returns a
@@ -72,38 +85,23 @@ SYSTEM_PROMPT = textwrap.dedent("""\
# Background Tasks (dispatch_task) — USE FOR ALL NON-TRIVIAL TASKS
For ANY task that is not a simple one-sentence answer from your own
knowledge, you MUST call dispatch_task. This includes: looking up news,
researching topics, checking current events, prices, sports scores,
writing something, summarizing, comparing options, planning, or anything
that takes more than a couple seconds to think through. Do NOT use
web_search or web_scrape directly; always dispatch instead.
The only things you answer inline are: weather (get_weather), time
(get_time), memory operations, and trivial facts you already know.
knowledge and NOT a memory operation, you MUST call dispatch_task.
This includes: looking up news, researching topics, checking current
events, prices, sports scores, writing something, summarizing, comparing
options, planning, or anything that takes more than a couple seconds to
think through. Do NOT use web_search or web_scrape directly; always
dispatch instead. Memory recall/remember is NEVER a dispatch task —
call those tools directly.
Examples:
- User says "what's the latest news?" → CALL dispatch_task with
description="Find the top 5 international news headlines today"
- User says "look up the price of a PS5" → CALL dispatch_task with
description="Find the current retail price of a PlayStation 5"
- User says "plan a weekend trip to Denver" → CALL dispatch_task with
description="Plan a two-day weekend trip to Denver including activities"
After calling dispatch_task, tell the user "I'll get on that for you"
and keep the conversation going. The result comes back automatically — when
you receive it, share the findings naturally in your conversational style.
You can have multiple tasks running at once.
# Memory (CRITICAL — always use these tools)
You MUST call memory_save whenever the user tells you something to remember,
shares a preference, or says "remember that...". Do NOT just say "okay I'll
remember that" without actually calling the tool. Always call it.
- User says "remember my coffee order is oat milk latte" → CALL memory_save
with topic="coffee order", content="oat milk latte"
- User says "what did I tell you about my birthday?" → CALL memory_recall
with query="birthday"
- User says "list everything you remember about me" → CALL memory_list
After calling memory_save, confirm briefly ("Got it, I'll remember that")
but do NOT say "I saved it to my memory file" or similar.
If a recall returns nothing, just say you don't have that info yet.
# Skills
You have a skill library of learned procedures. Use skill_recall when you need
to perform a task you've done before — it will give you the steps. When you
@@ -383,19 +381,19 @@ def build_mcp_toolsets() -> list[mcp.MCPToolset]:
)
logger.info("Weather MCP toolset enabled (wttr.in)")
memory_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "memory_mcp.py")
cognee_url = os.environ.get("COGNEE_MCP_URL", "http://192.168.86.2:8003/mcp")
toolsets.append(
mcp.MCPToolset(
id="memory",
mcp_server=mcp.MCPServerStdio(
command=python_bin,
args=[memory_mcp_script],
env={**os.environ, "MEMORY_DIR": "/memory"},
client_session_timeout_seconds=30,
mcp_server=mcp.MCPServerHTTP(
url=cognee_url,
transport_type="streamable_http",
allowed_tools=["remember", "recall", "forget"],
client_session_timeout_seconds=60,
),
)
)
logger.info("Memory MCP toolset enabled (%s)", MEMORY_DIR)
logger.info("Memory MCP toolset enabled (Cognee at %s)", cognee_url)
skills_mcp_script = os.path.join(os.path.dirname(os.path.abspath(__file__)), "skills_mcp.py")
toolsets.append(
agent/memory_mcp.py
-163
View File
@@ -1,163 +0,0 @@
"""Memory MCP server — persistent markdown-based memory for Hope.
Runs over stdio inside the voice container. The agent attaches it via
MCPServerStdio, so the LLM can recall and save memories during a
conversation. Memories live as plain markdown files in MEMORY_DIR
(default /memory), one file per topic, mounted from the host so they
survive rebuilds and are backed up via git.
Tools:
- memory_recall(query) -> relevant passages from all .md files
- memory_save(topic, content) -> append a timestamped note to {topic}.md
- memory_list() -> topics with their first line
"""
from __future__ import annotations
import os
import re
from datetime import datetime, timezone
from mcp.server.fastmcp import FastMCP
MEMORY_DIR = os.environ.get("MEMORY_DIR", "/memory")
mcp = FastMCP("memory")
def _slugify(topic: str) -> str:
"""Turn a topic into a safe filename slug."""
slug = re.sub(r"[^a-z0-9]+", "-", topic.lower()).strip("-")
return slug or "misc"
def _read_files() -> list[tuple[str, str]]:
"""Return (filename, content) for every .md file in MEMORY_DIR."""
files: list[tuple[str, str]] = []
if not os.path.isdir(MEMORY_DIR):
return files
for name in sorted(os.listdir(MEMORY_DIR)):
if not name.endswith(".md"):
continue
path = os.path.join(MEMORY_DIR, name)
try:
with open(path, encoding="utf-8") as f:
files.append((name, f.read()))
except OSError:
continue
return files
def _first_line(content: str) -> str:
for line in content.splitlines():
if line.strip():
return line.strip()
return "(empty)"
@mcp.tool()
def memory_recall(query: str) -> str:
"""Search saved memories for information relevant to a query.
Use this at the start of a conversation, or whenever the user
references past information ("what did I tell you about...",
"remember when..."). Returns matching passages with their topic
names, or a message saying nothing was found.
"""
q = (query or "").strip().lower()
if not q:
return "I don't have any memory of that."
words = [w for w in re.split(r"\W+", q) if len(w) > 2]
files = _read_files()
if not files:
return "I don't have any memory of that."
# Strip markdown heading markers and timestamp lines so snippets are clean prose.
def _clean(text: str) -> str:
text = text.replace("\n", " ")
text = re.sub(r"#+\s*", "", text)
text = re.sub(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2} UTC\s*", "", text)
return re.sub(r"\s{2,}", " ", text).strip()
scored: list[tuple[int, str, list[str]]] = []
for name, content in files:
lower = content.lower()
score = 0
matches: list[str] = []
if q in lower:
score += len(q)
seen: set[int] = set()
for w in words:
start = 0
while True:
idx = lower.find(w, start)
if idx == -1:
break
score += 1
key = idx // 200 # one snippet per ~200-char window
if key not in seen:
seen.add(key)
s = max(0, idx - 80)
e = min(len(content), idx + len(w) + 120)
snippet = _clean(content[s:e])
if snippet and snippet not in matches:
matches.append(snippet)
start = idx + len(w)
if score > 0:
scored.append((score, name, matches[:3]))
if not scored:
return "I don't have any memory of that."
scored.sort(key=lambda s: s[0], reverse=True)
parts = []
for _, name, snippets in scored[:5]:
topic = name.removesuffix(".md")
parts.append(f"[{topic}] " + " ".join(snippets))
return "\n".join(parts)
@mcp.tool()
def memory_save(topic: str, content: str) -> str:
"""Save a fact or preference to persistent memory.
Use this when the user shares personal information worth keeping:
names, birthdays, preferences, projects, important facts. Creates
or appends to a markdown file named after the topic. Returns a
short confirmation.
"""
topic = (topic or "").strip()
content = (content or "").strip()
if not topic or not content:
return "Nothing saved — both a topic and some content are needed."
os.makedirs(MEMORY_DIR, exist_ok=True)
path = os.path.join(MEMORY_DIR, f"{_slugify(topic)}.md")
is_new = not os.path.exists(path)
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
with open(path, "a", encoding="utf-8") as f:
if is_new:
f.write(f"# {topic}\n\n")
f.write(f"## {stamp}\n\n{content}\n\n")
return f"Saved to memory under '{topic}'."
@mcp.tool()
def memory_list() -> str:
"""List all saved memories with a brief description of each.
Returns the topic name and first line of every memory file, or a
message saying no memories exist yet.
"""
files = _read_files()
if not files:
return "No memories saved yet."
lines = [f"{name.removesuffix('.md')}: {_first_line(content)}" for name, content in files]
return "\n".join(lines)
if __name__ == "__main__":
mcp.run(transport="stdio")
docker-compose.yml
+1 -1
View File
@@ -15,10 +15,10 @@ services:
GEMMA_API_KEY: "${GEMMA_API_KEY:-not-needed}"
WEB_MCP_ENABLED: "${WEB_MCP_ENABLED:-true}"
FIRECRAWL_BASE: "${FIRECRAWL_BASE:-http://192.168.86.2:3002}"
COGNEE_MCP_URL: "${COGNEE_MCP_URL:-http://192.168.86.2:8003/mcp}"
volumes:
- ./livekit.yaml:/etc/livekit.yaml:ro
- ./certs:/etc/voice/certs
- ./memory:/memory
- ./skills:/skills
healthcheck:
test: ["CMD-SHELL", "curl -sk https://localhost:8090/ -o /dev/null && curl -s http://localhost:7880/ -o /dev/null"]
tests/conftest.py
+40
View File
@@ -19,6 +19,7 @@ WEB_BASE_URL = os.environ.get("VOICE_WEB_URL", "https://localhost:8090")
TOKEN_URL = os.environ.get("VOICE_TOKEN_URL", "http://127.0.0.1:8091/token")
GEMMA_BASE_URL = os.environ.get("GEMMA_BASE_URL", "http://192.168.86.2:8023/v1").rstrip("/")
GEMMA_MODEL = os.environ.get("GEMMA_MODEL", "gemma-4-e4b")
COGNEE_MCP_URL = os.environ.get("COGNEE_MCP_URL", "http://192.168.86.2:8003/mcp")
# Path to the agent's venv python inside the container (has mcp, livekit.agents)
CONTAINER_PYTHON = "/opt/voice-agent/.venv/bin/python"
@@ -134,3 +135,42 @@ asyncio.run(main())
return data["text"], data["isError"]
return _call
@pytest.fixture(scope="session")
def cognee_client():
"""Factory that calls tools on the remote Cognee MCP server (Streamable HTTP).
Returns (result_text, is_error). Uses the mcp client library from the
agent venv inside the container.
"""
def _call(tool_name: str, args: dict | None = None) -> tuple[str, bool]:
code = f"""
import asyncio, json
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
async def main():
async with streamablehttp_client({COGNEE_MCP_URL!r}) as (read, write, _):
async with ClientSession(read, write) as session:
await session.initialize()
result = await session.call_tool({tool_name!r}, {json.dumps(args or {})})
text = ""
for block in result.content:
if getattr(block, "type", None) == "text":
text += block.text
print(json.dumps({{"text": text, "isError": bool(result.isError)}}))
asyncio.run(main())
"""
proc = docker_exec_python(code, timeout=120, check=False)
if proc.returncode != 0:
raise AssertionError(
f"Cognee MCP call {tool_name} failed (rc={proc.returncode}): "
f"stderr={proc.stderr[-800:]}"
)
data = json.loads(proc.stdout.strip().splitlines()[-1])
return data["text"], data["isError"]
return _call
tests/test_mcp_tools.py
+19 -31
View File
@@ -46,42 +46,30 @@ def test_time_get(mcp_client_factory):
@pytest.mark.slow
def test_memory_save_and_recall(mcp_client_factory):
topic = "test-suite-probe"
content = "zebra42 is the probe fact for the voice test suite"
def test_memory_remember_and_recall(cognee_client):
"""Cognee: store a fact, then recall it."""
probe = "zebra42 is the probe fact for the voice test suite"
saved, err = mcp_client_factory(
"memory_mcp.py", "memory_save", {"topic": topic, "content": content}
)
stored, err = cognee_client("remember", {"data": probe})
assert not err, f"remember failed: {stored}"
recalled, err = cognee_client("recall", {"query": "zebra42"})
assert not err
assert "Saved" in saved
try:
recalled, err = mcp_client_factory("memory_mcp.py", "memory_recall", {"query": "zebra42"})
assert not err
assert "zebra42" in recalled
assert topic in recalled # recall output is tagged with the topic name
finally:
# Clean up: remove the probe file so repeated runs stay green.
from conftest import docker_exec
docker_exec("rm", "-f", "/memory/test-suite-probe.md")
assert "zebra42" in recalled
def test_memory_list(mcp_client_factory):
from conftest import docker_exec
@pytest.mark.slow
def test_memory_forget(cognee_client):
"""Cognee: store a fact, verify recall, then forget it."""
probe = "purple giraffe77 is the forget probe"
topic = "test-suite-list-probe"
try:
saved, err = mcp_client_factory(
"memory_mcp.py", "memory_save", {"topic": topic, "content": "listing probe"}
)
assert not err
listed, err = mcp_client_factory("memory_mcp.py", "memory_list", {})
assert not err
assert topic in listed
finally:
docker_exec("rm", "-f", "/memory/test-suite-list-probe.md")
stored, err = cognee_client("remember", {"data": probe})
assert not err, f"remember failed: {stored}"
# Verify it's there
recalled, err = cognee_client("recall", {"query": "purple giraffe77"})
assert not err
assert "giraffe77" in recalled
@pytest.mark.slow