feat: provision private Hermes assistants for authenticated Hope accounts

This commit is contained in:
snowspeeder committed 2026-10-01 13:30:18 -04:00
1 parent da65da68b3
commit 2aedea84b5
33 files changed
+3063 -1743

No files matched your search

+109
View File
@@ -0,0 +1,109 @@
# Account control plane
Run `python -m control` with FastAPI and uvicorn installed. Defaults to
`127.0.0.1:8092`; configure `CONTROL_HOST` / `CONTROL_PORT` explicitly if needed.
Run one control process against a database (lifecycle scheduling is process-local).
No Docker logic lives in the account API: production imports
`control.provisioner.Provisioner()` (locally aliased HermesProvisioner) lazily
on startup.
## Runtime settings
- `CONTROL_DB_PATH`: SQLite path, default `/data/control/accounts.sqlite3`.
Mount persistent private storage. The DB contains hashes and Hermes keys;
protect the directory and its WAL/SHM files, backups, and host access.
- `CONTROL_ADMIN_USERNAME`: initial administrator lowercase slug, default `admin`.
- `CONTROL_ADMIN_PASSWORD`: **required when DB is empty**, 12–256 characters;
no built-in password. Ignored once existing accounts are present. Do not log it.
- `CONTROL_SERVICE_SECRET`: >=32-character secret for trusted internal callers.
Missing/short secret disables all internal access; send as `X-Control-Secret`.
- `CONTROL_ALLOWED_ORIGINS`: exact comma-separated HTTPS browser origins.
**Set this behind nginx**; no forwarded host/proto headers are trusted.
Native login sends `Content-Type: application/json` and no browser Origin.
Browser login requires its actual Origin to match. Cookie mutations require a
matching Origin; bearer-authenticated requests are exempt. Login is throttled
per validated client IP and username (10 failed attempts / 5 minutes; successes
release their reservation). Only a direct loopback peer may supply `X-Real-IP`,
which nginx must overwrite with `$remote_addr`. Do not expose the control port
or preserve untrusted client-supplied identity headers in another proxy.
Opaque sessions last 24 hours, are hashed at rest and revoked by logout,
password reset or deletion. Cookies are Secure, HttpOnly, SameSite=Strict.
## Public API
- `POST /api/auth/login` JSON `{username,password}` →
`{token,user,expires_at}` (expiry Unix seconds) plus `hope_session` cookie.
- `POST /api/auth/logout` → `{ok:true}`; invalid sessions return 401.
- `GET /api/auth/me` → `{user}`.
- Administrator-only `GET /api/admin/users` → `{users:[...]}`.
- Administrator-only `POST /api/admin/users` JSON
`{username,display_name,password,role?:"user"}` → 201 `{user}`.
- Administrator-only `DELETE /api/admin/users/{id}` → 202 `{user}` in
deleting state. Sessions/rooms revoked synchronously before return; container
deletion is background, durable and retriable. Last administrator is protected.
- Administrator-only `POST /api/admin/users/{id}/password` JSON `{password}` →
`{ok:true}`; revokes all sessions and room grants.
- Administrator-only `POST /api/admin/users/{id}/retry` → 202 `{user}`;
retries failed provisioning or removal, preserving intended lifecycle.
- `GET /health` → `{status:"ok"}` (control HTTP liveness, not Hermes readiness).
Public user is exactly `{id,username,display_name,role,status,container_name,error}`.
IDs are stable UUID hex. Usernames match `[a-z][a-z0-9]*(?:-[a-z0-9]+)*`,
1–48 characters. Status is `provisioning`, `ready`, `error` or `deleting`.
Deleting failures remain deleting with a sanitized visible error, never become
login-eligible. Background reconcile runs on startup and every 30 seconds.
## Trusted speech boundary
Every internal route requires `X-Control-Secret`, including room lookups.
- `POST /internal/resolve-session` JSON `{token}` validates the exact cookie or
bearer token; returns `{user,hermes_url,hermes_api_key}` only for ready user.
- `GET /internal/users/{id}` returns the same ready-user metadata.
- `POST /internal/rooms` JSON `{user_id,room_name,session_token,ttl?:3600}`
validates that session belongs to the user and persists a unique server-minted
room grant, returning 201 `{room_name,user_id}`. TTL 1–86400 sec, capped at
original session expiry.
- `GET /internal/rooms/{room_name}` resolves that grant only while unexpired
and user still active/ready. Returns the same metadata. Deletion/reset revoke
grants immediately; speech agents should poll to stop active conversations.
401 means absent/invalid auth; 403 forbidden role/Origin; 404 missing/deleted
user or invalid room; 503 means not yet ready. Metadata is never in public APIs.
Do not proxy internal endpoints publicly. Token service resolves client sessions,
mints an opaque room name, persists its grant before issuing LiveKit credentials,
and never accepts browser-supplied account IDs, room ownership or Hermes URLs.
For colocated FastAPI integration use `Depends(control.current_user)` and
`Depends(control.administrator)`; these return public user dicts and require
this application's `app.state.accounts`/settings. Separate services use the
secret-authenticated internal endpoints instead.
## Provisioner interface
`create_app(settings: Settings, provisioner=...)` supports offline injection.
The adapter receives a **private trusted dict**, including `id`, `username`,
`container_name`, `desired`, and stored `hermes_url` / `hermes_api_key`.
- `async ensure(user) -> dict`: idempotently create/recover owned container;
return `{container_name,hermes_url,hermes_api_key}`. Container name must match
`hope-hermes-<username>-<id[:8]>`; metadata is persisted before readiness check.
- `async ready(user) -> bool`: owned service health, not existence alone.
- `async remove(user) -> None`: verify management/user ownership before any
removal; retain private data by default; failure raises and remains retriable.
- Optional `async aclose()`: close adapter resources during shutdown.
Async adapters offload blocking Docker/HTTP calls themselves; synchronous
adapters are supported using a worker thread. Account SQLite and scrypt work
are offloaded from the ASGI loop. Slow provision/remove run separate tasks;
request handlers do not await container startup. Exception text is not logged or
returned because provider failures may contain credentials.
Offline tests:
```sh
python -m pytest tests/unit/test_accounts.py --confcutdir=tests/unit -q
```
`--confcutdir` avoids this repository's deployment-oriented top-level fixtures.
Use a temporary DB and fake adapter; no Docker or provider calls are made.
+5
View File
@@ -0,0 +1,5 @@
"""Trusted account and per-user Hermes control plane."""
from .app import Settings, create_app, current_user, administrator
__all__ = ["Settings", "create_app", "current_user", "administrator"]
+12
View File
@@ -0,0 +1,12 @@
"""Run `python -m control`; never exposes the control plane on LAN by default."""
import os
import uvicorn
from .app import create_app
if __name__ == '__main__':
# Access logs include path identifiers; disable to avoid accidental secret
# disclosure if a malformed client includes credentials in a URL.
uvicorn.run(create_app(), host=os.getenv('CONTROL_HOST', '127.0.0.1'),
port=int(os.getenv('CONTROL_PORT', '8092')), access_log=False,
proxy_headers=False)
+204
View File
@@ -0,0 +1,204 @@
"""SQLite persistence. No credential-bearing row is ever a public API schema."""
from __future__ import annotations
import hashlib
import hmac
import os
import re
import secrets
import sqlite3
import threading
import time
import uuid
from pathlib import Path
PUBLIC_FIELDS = ('id', 'username', 'display_name', 'role', 'status', 'container_name', 'error')
SLUG = re.compile(r'[a-z][a-z0-9]*(?:-[a-z0-9]+)*\Z')
def public_user(user):
return {key: user[key] for key in PUBLIC_FIELDS}
def validate_username(value):
if not isinstance(value, str) or not 1 <= len(value) <= 48 or not SLUG.fullmatch(value):
raise ValueError('Username must be a lowercase slug (1–48 characters)')
return value
def validate_password(value):
if not isinstance(value, str) or not 12 <= len(value) <= 256 or len(value.encode('utf-8')) > 1024:
raise ValueError('Password must contain 12–256 characters')
return value
def password_hash(password):
salt = secrets.token_bytes(16)
digest = hashlib.scrypt(password.encode(), salt=salt, n=16384, r=8, p=1, dklen=32)
return f'scrypt$16384$8$1${salt.hex()}${digest.hex()}'
def password_matches(password, encoded):
try:
_, n, r, p, salt, digest = encoded.split('$')
actual = hashlib.scrypt(password.encode(), salt=bytes.fromhex(salt), n=int(n), r=int(r), p=int(p), dklen=32)
return hmac.compare_digest(actual.hex(), digest)
except (ValueError, TypeError):
return False
def token_hash(token):
return hashlib.sha256(token.encode()).hexdigest()
class Accounts:
def __init__(self, path):
if path != ':memory:':
Path(path).parent.mkdir(parents=True, exist_ok=True, mode=0o700)
self.lock = threading.RLock()
if path != ':memory:':
# Create privately before SQLite can create journal companions.
fd = os.open(path, os.O_CREAT | os.O_RDWR, 0o600)
os.close(fd)
os.chmod(path, 0o600)
self.db = sqlite3.connect(path, check_same_thread=False)
self.db.row_factory = sqlite3.Row
self.db.execute('PRAGMA foreign_keys=ON')
self.db.execute('PRAGMA busy_timeout=5000')
self.db.execute('PRAGMA journal_mode=WAL')
self.db.executescript('''
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY, username TEXT NOT NULL UNIQUE,
display_name TEXT NOT NULL, role TEXT NOT NULL,
status TEXT NOT NULL, container_name TEXT NOT NULL,
error TEXT, password_hash TEXT NOT NULL,
hermes_url TEXT, hermes_api_key TEXT, desired TEXT NOT NULL DEFAULT 'ready'
);
CREATE TABLE IF NOT EXISTS sessions (
hash TEXT PRIMARY KEY, user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at REAL NOT NULL
);
CREATE TABLE IF NOT EXISTS rooms (
room_name TEXT PRIMARY KEY, user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
expires_at REAL NOT NULL, session_hash TEXT
);
''')
if 'session_hash' not in {r[1] for r in self.db.execute('PRAGMA table_info(rooms)')}:
self.db.execute('ALTER TABLE rooms ADD COLUMN session_hash TEXT')
self.db.commit()
if path != ':memory:':
os.chmod(path, 0o600)
for suffix in ('-wal', '-shm'):
companion = Path(path + suffix)
if companion.exists():
os.chmod(companion, 0o600)
def close(self):
self.db.close()
def users(self):
with self.lock:
return [dict(r) for r in self.db.execute('SELECT * FROM users ORDER BY username')]
def get(self, user_id):
with self.lock:
row = self.db.execute('SELECT * FROM users WHERE id=?', (user_id,)).fetchone()
return dict(row) if row else None
def by_username(self, username):
with self.lock:
row = self.db.execute('SELECT * FROM users WHERE username=?', (username,)).fetchone()
return dict(row) if row else None
def create(self, username, display_name, role, password):
validate_username(username)
validate_password(password)
if role not in ('admin', 'user'):
raise ValueError('Invalid role')
if not isinstance(display_name, str) or not 1 <= len(display_name.strip()) <= 100:
raise ValueError('Display name must contain 1–100 characters')
user_id = uuid.uuid4().hex
hashed = password_hash(password)
with self.lock, self.db:
self.db.execute('INSERT INTO users(id,username,display_name,role,status,container_name,password_hash) VALUES(?,?,?,?,?,?,?)',
(user_id, username, display_name.strip(), role, 'provisioning', f'hope-hermes-{username}-{user_id[:8]}', hashed))
return self.get(user_id)
def update(self, user_id, **values):
allowed = {'status', 'error', 'hermes_url', 'hermes_api_key', 'container_name'}
if not values or not set(values) <= allowed:
raise ValueError('Invalid account update')
with self.lock, self.db:
guard = " AND desired!='deleted'" if values.get('status') in ('ready', 'provisioning', 'error') else ''
self.db.execute('UPDATE users SET ' + ','.join(f'{k}=?' for k in values) + ' WHERE id=?' + guard, (*values.values(), user_id))
def begin_delete(self, user_id):
with self.lock, self.db:
user = self.get(user_id)
if user is None:
raise KeyError(user_id)
if user['role'] == 'admin' and user['desired'] != 'deleted':
count = self.db.execute("SELECT count(*) FROM users WHERE role='admin' AND desired!='deleted'").fetchone()[0]
if count <= 1:
raise ValueError('Cannot delete the last administrator')
self.db.execute("UPDATE users SET desired='deleted',status='deleting',error=NULL WHERE id=?", (user_id,))
self.db.execute('DELETE FROM sessions WHERE user_id=?', (user_id,))
self.db.execute('DELETE FROM rooms WHERE user_id=?', (user_id,))
return self.get(user_id)
def finish_delete(self, user_id):
with self.lock, self.db:
self.db.execute("DELETE FROM users WHERE id=? AND desired='deleted'", (user_id,))
def reset_password(self, user_id, password):
hashed = password_hash(validate_password(password))
with self.lock, self.db:
user = self.get(user_id)
if user is None or user['desired'] == 'deleted':
raise KeyError(user_id)
self.db.execute('UPDATE users SET password_hash=? WHERE id=?', (hashed, user_id))
self.db.execute('DELETE FROM sessions WHERE user_id=?', (user_id,))
self.db.execute('DELETE FROM rooms WHERE user_id=?', (user_id,))
def issue_session(self, user_id, ttl, expected_password_hash):
token = secrets.token_urlsafe(32)
expires = time.time() + ttl
with self.lock, self.db:
self.db.execute('DELETE FROM sessions WHERE expires_at<=?', (time.time(),))
# Single SQL statement atomically compares the verified credential
# and inserts; reset/delete cannot interleave with this check.
cursor = self.db.execute("INSERT INTO sessions(hash,user_id,expires_at) SELECT ?,id,? FROM users WHERE id=? AND desired!='deleted' AND password_hash=?",
(token_hash(token), expires, user_id, expected_password_hash))
if cursor.rowcount != 1:
raise ValueError('Account credentials changed or unavailable')
return token, expires
def resolve_session(self, token):
if not isinstance(token, str) or not 20 <= len(token) <= 256:
return None
with self.lock:
row = self.db.execute("SELECT u.* FROM sessions s JOIN users u ON u.id=s.user_id WHERE s.hash=? AND s.expires_at>? AND u.desired!='deleted'", (token_hash(token), time.time())).fetchone()
return dict(row) if row else None
def revoke(self, token):
with self.lock, self.db:
self.db.execute('DELETE FROM sessions WHERE hash=?', (token_hash(token),))
def grant_room(self, user_id, room_name, ttl, session_token):
with self.lock, self.db:
user = self.get(user_id)
if not user or user['desired'] == 'deleted' or user['status'] != 'ready':
raise ValueError('Account not ready')
if not isinstance(session_token, str) or not 20 <= len(session_token) <= 256:
raise ValueError('Invalid session')
hashed = token_hash(session_token)
session = self.db.execute('SELECT expires_at FROM sessions WHERE hash=? AND user_id=? AND expires_at>?', (hashed, user_id, time.time())).fetchone()
if session is None:
raise ValueError('Invalid session')
self.db.execute('DELETE FROM rooms WHERE expires_at<=?', (time.time(),))
self.db.execute('INSERT INTO rooms(room_name,user_id,expires_at,session_hash) VALUES(?,?,?,?)', (room_name, user_id, min(time.time() + ttl, session['expires_at']), hashed))
def resolve_room(self, room_name):
with self.lock:
row = self.db.execute("SELECT u.* FROM rooms r JOIN users u ON r.user_id=u.id JOIN sessions s ON s.hash=r.session_hash AND s.user_id=u.id WHERE r.room_name=? AND r.expires_at>? AND s.expires_at>? AND u.desired!='deleted' AND u.status='ready'", (room_name, time.time(), time.time())).fetchone()
return dict(row) if row else None
+408
View File
@@ -0,0 +1,408 @@
"""Account HTTP API; trusted adapters alone hold Hermes connection secrets."""
from __future__ import annotations
import asyncio
import hmac
import inspect
import ipaddress
import json
import os
import sqlite3
import time
from collections import defaultdict, deque
from contextlib import asynccontextmanager
from dataclasses import dataclass, field
from typing import Protocol
from urllib.parse import urlsplit
from fastapi import Depends, FastAPI, HTTPException, Request, Response
from .accounts import Accounts, password_hash, password_matches, public_user, validate_password, validate_username
COOKIE = 'hope_session'
MAX_BODY = 8192
class Provisioner(Protocol):
"""All methods async. remove MUST verify Hope/user ownership before deletion.
ensure is idempotent and returns {container_name, hermes_url, hermes_api_key}.
ready checks the owned Hermes API health, not just container existence.
Private home/workspace must be retained by remove unless explicitly erased.
"""
async def ensure(self, user: dict) -> dict: ...
async def remove(self, user: dict) -> None: ...
async def ready(self, user: dict) -> bool: ...
@dataclass
class Settings:
db_path: str = field(default_factory=lambda: os.getenv('CONTROL_DB_PATH', '/data/control/accounts.sqlite3'))
admin_username: str = field(default_factory=lambda: os.getenv('CONTROL_ADMIN_USERNAME', 'admin'))
admin_password: str | None = field(default_factory=lambda: os.getenv('CONTROL_ADMIN_PASSWORD'))
service_secret: str | None = field(default_factory=lambda: os.getenv('CONTROL_SERVICE_SECRET'))
allowed_origins: tuple[str, ...] = field(default_factory=lambda: tuple(x.strip().rstrip('/') for x in os.getenv('CONTROL_ALLOWED_ORIGINS', '').split(',') if x.strip()))
session_ttl: int = 86400
reconcile_interval: float = 30
login_limit: int = 10
login_window: int = 300
async def bounded_json(request):
if request.headers.get('content-type', '').split(';', 1)[0].strip().lower() != 'application/json':
raise HTTPException(415, 'Content-Type application/json required')
data = bytearray()
async for chunk in request.stream():
data.extend(chunk)
if len(data) > MAX_BODY:
raise HTTPException(413, 'Request body too large')
try:
value = json.loads(data)
except (ValueError, UnicodeDecodeError):
raise HTTPException(400, 'Invalid JSON') from None
if not isinstance(value, dict):
raise HTTPException(400, 'JSON object required')
return value
def bearer_token(request):
authorization = request.headers.get('authorization')
if authorization is None:
return None
parts = authorization.split()
if len(parts) != 2 or parts[0].lower() != 'bearer' or not 20 <= len(parts[1]) <= 256:
raise HTTPException(401, 'Invalid authorization')
return parts[1]
def check_origin(request, *, login=False):
"""No forwarded-header trust; explicit origins required behind TLS proxy.
Native bearer clients are exempt. Login without Origin is allowed only for
JSON native clients with no session cookie and no browser Fetch Metadata.
"""
if bearer_token(request) is not None:
return
origin = request.headers.get('origin')
if not origin:
if login and not request.cookies.get(COOKIE) and not request.headers.get('sec-fetch-site'):
return
raise HTTPException(403, 'Origin required')
settings = request.app.state.settings
allowed = settings.allowed_origins or (str(request.base_url).rstrip('/'),)
if origin not in allowed or origin == 'null':
raise HTTPException(403, 'Origin not allowed')
async def current_user(request: Request):
token = bearer_token(request) or request.cookies.get(COOKIE)
if not token:
raise HTTPException(401, 'Authentication required')
user = await asyncio.to_thread(request.app.state.accounts.resolve_session, token)
if user is None:
raise HTTPException(401, 'Session invalid or expired')
if request.method not in ('GET', 'HEAD', 'OPTIONS'):
check_origin(request)
request.state.session_token = token
return public_user(user)
async def administrator(user=Depends(current_user)):
if user['role'] != 'admin':
raise HTTPException(403, 'Administrator required')
return user
def internal_authorized(request: Request):
secret = request.app.state.settings.service_secret
supplied = request.headers.get('x-control-secret', '')
if not secret or len(secret) < 32 or not supplied or not hmac.compare_digest(secret.encode('utf-8'), supplied.encode('utf-8')):
raise HTTPException(401, 'Internal authorization required')
class Lifecycle:
def __init__(self, accounts, provisioner):
self.accounts = accounts
self.provisioner = provisioner
self.tasks = {}
self.closed = False
def schedule(self, user_id):
if not self.closed and (user_id not in self.tasks or self.tasks[user_id].done()):
self.tasks[user_id] = asyncio.create_task(self.run(user_id))
async def call(self, method, user):
# Native async adapters must offload their own blocking Docker/HTTP calls.
# Also tolerate synchronous test/legacy adapters without blocking ASGI.
func = getattr(self.provisioner, method)
if inspect.iscoroutinefunction(func):
return await func(user)
result = await asyncio.to_thread(func, user)
return await result if inspect.isawaitable(result) else result
async def run(self, user_id):
user = await asyncio.to_thread(self.accounts.get, user_id)
if user is None:
return
try:
if user['desired'] == 'deleted':
await self.call('remove', user)
await asyncio.to_thread(self.accounts.finish_delete, user_id)
return
await asyncio.to_thread(self.accounts.update, user_id, status='provisioning', error=None)
metadata = await self.call('ensure', user)
if not isinstance(metadata, dict):
raise ValueError('Invalid provisioning metadata')
url = metadata.get('hermes_url')
key = metadata.get('hermes_api_key')
parsed = urlsplit(url or '')
if parsed.scheme not in ('http', 'https') or not parsed.hostname or not isinstance(key, str) or not key:
raise ValueError('Invalid provisioning metadata')
# Deterministic name cannot be overridden by adapter metadata.
if metadata.get('container_name', user['container_name']) != user['container_name']:
raise ValueError('Unexpected container ownership')
await asyncio.to_thread(self.accounts.update, user_id, hermes_url=url, hermes_api_key=key)
user.update(hermes_url=url, hermes_api_key=key)
if not await self.call('ready', user):
raise RuntimeError('Hermes not healthy')
current = await asyncio.to_thread(self.accounts.get, user_id)
if current and current['desired'] == 'deleted':
await self.call('remove', current)
await asyncio.to_thread(self.accounts.finish_delete, user_id)
else:
await asyncio.to_thread(self.accounts.update, user_id, status='ready', error=None)
except asyncio.CancelledError:
raise
except Exception:
# Exception messages can contain upstream URLs, keys or Docker env.
current = await asyncio.to_thread(self.accounts.get, user_id)
if current:
deleting = current['desired'] == 'deleted'
await asyncio.to_thread(self.accounts.update, user_id, status='deleting' if deleting else 'error', error='Container removal failed; retry pending' if deleting else 'Hermes provisioning failed; retry pending')
async def reconcile(self):
for user in await asyncio.to_thread(self.accounts.users):
if user['desired'] == 'deleted' or user['status'] != 'ready':
self.schedule(user['id'])
elif user['id'] not in self.tasks or self.tasks[user['id']].done():
try:
healthy = await self.call('ready', user)
except Exception:
healthy = False
if not healthy:
await asyncio.to_thread(self.accounts.update, user['id'], status='error', error='Hermes health check failed; retry pending')
self.schedule(user['id'])
async def loop(self, interval):
while True:
await self.reconcile()
await asyncio.sleep(interval)
async def stop(self):
self.closed = True
tasks = list(self.tasks.values())
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
def create_app(settings=None, provisioner=None):
settings = settings or Settings()
@asynccontextmanager
async def lifespan(app):
store = Accounts(settings.db_path)
app.state.accounts = store
try:
if not store.users():
if not settings.admin_password:
raise RuntimeError('CONTROL_ADMIN_PASSWORD required to bootstrap an empty account database')
await asyncio.to_thread(store.create, settings.admin_username, 'Administrator', 'admin', settings.admin_password)
adapter = provisioner
if adapter is None:
from .provisioner import Provisioner as HermesProvisioner
adapter = HermesProvisioner()
app.state.lifecycle = Lifecycle(store, adapter)
# Revalidate even previously-ready containers after control restart.
for user in store.users():
app.state.lifecycle.schedule(user['id'])
app.state.dummy_hash = await asyncio.to_thread(password_hash, 'timing-only-password')
loop = asyncio.create_task(app.state.lifecycle.loop(settings.reconcile_interval))
try:
yield
finally:
loop.cancel()
await asyncio.gather(loop, return_exceptions=True)
await app.state.lifecycle.stop()
if hasattr(adapter, 'aclose'):
await adapter.aclose()
finally:
store.close()
app = FastAPI(title='Hope account control', lifespan=lifespan, docs_url=None, redoc_url=None, openapi_url=None)
app.state.settings = settings
attempts = defaultdict(deque)
@app.middleware('http')
async def no_store(request, call_next):
response = await call_next(request)
response.headers['Cache-Control'] = 'no-store'
return response
@app.post('/api/auth/login')
async def login(request: Request, response: Response):
check_origin(request, login=True)
body = await bounded_json(request)
try:
username = validate_username(body.get('username'))
password = validate_password(body.get('password'))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
peer = request.client.host if request.client else 'unknown'
# nginx /api/ overwrites X-Real-IP with $remote_addr. Only its direct
# loopback connection is trusted; never trust X-Forwarded-For.
try:
if ipaddress.ip_address(peer).is_loopback:
forwarded = request.headers.get('x-real-ip')
if forwarded:
peer = str(ipaddress.ip_address(forwarded))
except ValueError:
pass
now = time.monotonic()
for key in list(attempts):
bucket = attempts[key]
while bucket and bucket[0] <= now - settings.login_window:
bucket.popleft()
if not bucket:
del attempts[key]
scope = (peer, username)
bucket = attempts.get(scope)
if bucket is not None and len(bucket) >= settings.login_limit:
raise HTTPException(429, 'Too many login attempts', headers={'Retry-After': str(settings.login_window)})
if bucket is None:
if len(attempts) >= 4096:
raise HTTPException(429, 'Too many login attempts')
bucket = attempts[scope]
# Reserve a slot while scrypt is in flight to prevent concurrent bypass.
# Successful requests release their own slot; only failures persist.
slot = time.monotonic()
bucket.append(slot)
store = app.state.accounts
user = await asyncio.to_thread(store.by_username, username)
valid = await asyncio.to_thread(password_matches, password, user['password_hash'] if user else app.state.dummy_hash)
if not valid or not user or user['desired'] == 'deleted':
raise HTTPException(401, 'Invalid username or password')
try:
token, expires = await asyncio.to_thread(store.issue_session, user['id'], settings.session_ttl, user['password_hash'])
except ValueError:
raise HTTPException(401, 'Account unavailable') from None
try:
bucket.remove(slot)
except ValueError:
pass # Expired while the request was in flight.
if not bucket and attempts.get(scope) is bucket:
attempts.pop(scope, None)
response.set_cookie(COOKIE, token, max_age=settings.session_ttl, httponly=True, secure=True, samesite='strict', path='/')
return {'token': token, 'user': public_user(user), 'expires_at': expires}
@app.post('/api/auth/logout')
async def logout(request: Request, response: Response, user=Depends(current_user)):
await asyncio.to_thread(app.state.accounts.revoke, request.state.session_token)
response.delete_cookie(COOKIE, path='/', secure=True, httponly=True, samesite='strict')
return {'ok': True}
@app.get('/api/auth/me')
async def me(user=Depends(current_user)):
return {'user': user}
@app.get('/api/admin/users')
async def list_users(user=Depends(administrator)):
return {'users': [public_user(u) for u in await asyncio.to_thread(app.state.accounts.users)]}
@app.post('/api/admin/users', status_code=201)
async def create_user(request: Request, admin=Depends(administrator)):
body = await bounded_json(request)
try:
user = await asyncio.to_thread(app.state.accounts.create, body.get('username'), body.get('display_name'), body.get('role', 'user'), body.get('password'))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
except sqlite3.IntegrityError:
raise HTTPException(409, 'Username already exists') from None
app.state.lifecycle.schedule(user['id'])
return {'user': public_user(user)}
@app.delete('/api/admin/users/{user_id}', status_code=202)
async def delete_user(user_id: str, admin=Depends(administrator)):
try:
user = await asyncio.to_thread(app.state.accounts.begin_delete, user_id)
except KeyError:
raise HTTPException(404, 'User not found') from None
except ValueError as exc:
raise HTTPException(409, str(exc)) from None
app.state.lifecycle.schedule(user_id)
return {'user': public_user(user)}
@app.post('/api/admin/users/{user_id}/password')
async def reset_password(user_id: str, request: Request, admin=Depends(administrator)):
body = await bounded_json(request)
try:
await asyncio.to_thread(app.state.accounts.reset_password, user_id, body.get('password'))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
except KeyError:
raise HTTPException(404, 'User not found') from None
return {'ok': True}
@app.post('/api/admin/users/{user_id}/retry', status_code=202)
async def retry(user_id: str, admin=Depends(administrator)):
user = await asyncio.to_thread(app.state.accounts.get, user_id)
if not user:
raise HTTPException(404, 'User not found')
app.state.lifecycle.schedule(user_id)
return {'user': public_user(user)}
def connection(user):
if not user or user['desired'] == 'deleted':
raise HTTPException(404, 'User not found')
if user['status'] != 'ready' or not user['hermes_url'] or not user['hermes_api_key']:
raise HTTPException(503, 'User Hermes is not ready')
return {'user': public_user(user), 'hermes_url': user['hermes_url'], 'hermes_api_key': user['hermes_api_key']}
@app.get('/internal/users/{user_id}', dependencies=[Depends(internal_authorized)])
async def internal_user(user_id: str):
return connection(await asyncio.to_thread(app.state.accounts.get, user_id))
@app.post('/internal/resolve-session', dependencies=[Depends(internal_authorized)])
async def resolve_session(request: Request):
body = await bounded_json(request)
user = await asyncio.to_thread(app.state.accounts.resolve_session, body.get('token'))
if user is None:
raise HTTPException(401, 'Session invalid or expired')
return connection(user)
@app.post('/internal/rooms', dependencies=[Depends(internal_authorized)], status_code=201)
async def create_room(request: Request):
body = await bounded_json(request)
user_id, room = body.get('user_id'), body.get('room_name')
ttl = body.get('ttl', 3600)
if not isinstance(user_id, str) or not isinstance(room, str) or not 1 <= len(room) <= 128 or not all(c.isascii() and (c.isalnum() or c in '-_') for c in room):
raise HTTPException(400, 'Invalid room grant')
if type(ttl) is not int or not 1 <= ttl <= 86400:
raise HTTPException(400, 'Invalid room expiry')
try:
await asyncio.to_thread(app.state.accounts.grant_room, user_id, room, ttl, body.get('session_token'))
except ValueError:
raise HTTPException(409, 'Account not ready') from None
except sqlite3.IntegrityError:
raise HTTPException(409, 'Room already granted') from None
return {'room_name': room, 'user_id': user_id}
@app.get('/internal/rooms/{room_name}', dependencies=[Depends(internal_authorized)])
async def resolve_room(room_name: str):
return connection(await asyncio.to_thread(app.state.accounts.resolve_room, room_name))
@app.get('/health')
async def health():
return {'status': 'ok'}
return app
+313
View File
@@ -0,0 +1,313 @@
"""Trusted Docker-UDS control plane for independent Hermes accounts.
Only this service may receive the host Docker socket. Bind-source paths must be
identical in the control service and Docker host (mount the runtime root there).
The official image's root/s6 boot is deliberately bypassed: a nonroot Python
bootstrap seeds bundled skills, then execs the verified .venv Hermes CLI. Image
compatibility still needs deployment qualification before an image pin is promoted.
"""
from __future__ import annotations
import asyncio
import json
import os
from pathlib import Path
import re
import secrets
import tempfile
import httpx
class ProvisioningError(RuntimeError):
"""Safe-to-display provisioning failure (never includes Docker env/secrets)."""
class OwnershipError(ProvisioningError):
"""An existing resource is not owned by this account."""
# Run as the image's hermes UID without executing root-only s6 startup. No source
# checkout/profile is copied: only the image's public bundled skills are seeded.
_BOOTSTRAP = """
import os, pathlib, shutil
home = pathlib.Path('/opt/data')
for name in ('home', 'sessions', 'memories', 'skills', 'cron', 'hooks', 'logs', 'cache'):
(home / name).mkdir(mode=0o700, parents=True, exist_ok=True)
source = pathlib.Path('/opt/hermes/skills')
if source.is_dir() and not (home / '.hope-skills-seeded').exists():
shutil.copytree(source, home / 'skills', dirs_exist_ok=True)
(home / '.hope-skills-seeded').touch(mode=0o600)
os.makedirs('/tmp/hermes-runtime', mode=0o700, exist_ok=True)
os.execv('/opt/hermes/.venv/bin/hermes', ['hermes', 'gateway', 'run'])
"""
def container_name(user: dict) -> str:
uid, username = user.get('id'), user.get('username')
if not isinstance(uid, str) or not re.fullmatch(r'[0-9a-f]{32}', uid):
raise ValueError('Account id must be a lowercase UUID hex string')
if not isinstance(username, str) or not re.fullmatch(r'[a-z][a-z0-9]*(?:-[a-z0-9]+)*', username) or len(username) > 48:
raise ValueError('Account username must be a lowercase slug')
return f'hope-hermes-{username}-{uid[:8]}'
class Provisioner:
def __init__(self, state_dir=None, *, docker_socket=None, image=None,
upstream_api_key=None, docker_client=None, health_client=None,
ready_timeout=60, poll_interval=.5, runtime_uid=10000,
runtime_gid=10000, memory_bytes=2 * 1024 ** 3, nano_cpus=2_000_000_000):
self.state_dir = Path(state_dir or os.environ.get('HOPE_HERMES_DATA_DIR', '.runtime/hermes')).absolute()
self.image = image or os.environ.get('HERMES_IMAGE', 'nousresearch/hermes-agent@sha256:d4da4a40cd7a28aba983775d9fd31d94cbf153eeb0cb9e844d6d0f612b7c24db')
self.upstream_api_key = upstream_api_key if upstream_api_key is not None else os.environ.get('HERMES_UPSTREAM_API_KEY', '')
self.ready_timeout, self.poll_interval = ready_timeout, poll_interval
self.runtime_uid, self.runtime_gid = runtime_uid, runtime_gid
if runtime_uid <= 0 or runtime_gid <= 0:
raise ValueError('Hermes must run nonroot')
self.memory_bytes, self.nano_cpus = memory_bytes, nano_cpus
self._own_docker, self._own_health = docker_client is None, health_client is None
# No TCP Docker endpoint, DOCKER_HOST, subprocess, or proxy inheritance.
self.docker = docker_client or httpx.AsyncClient(
transport=httpx.AsyncHTTPTransport(uds=docker_socket or os.environ.get('DOCKER_SOCKET', '/var/run/docker.sock')),
base_url='http://docker', timeout=20, trust_env=False)
self.health = health_client or httpx.AsyncClient(timeout=3, trust_env=False, follow_redirects=False)
self._locks = {}
async def aclose(self):
if self._own_docker:
await self.docker.aclose()
if self._own_health:
await self.health.aclose()
def _lock(self, user):
container_name(user)
return self._locks.setdefault(user['id'], asyncio.Lock())
async def _request(self, method, path, *, allowed=(200,), **kwargs):
try:
response = await self.docker.request(method, path, **kwargs)
except httpx.HTTPError as exc:
raise ProvisioningError('Docker control socket unavailable; retry provisioning') from exc
if response.status_code not in allowed:
raise ProvisioningError(f'Docker operation failed (HTTP {response.status_code}); retry provisioning')
return response
async def _inspect(self, name):
response = await self._request('GET', f'/containers/{name}/json', allowed=(200, 404))
return response.json() if response.status_code == 200 else None
@staticmethod
def _owned(resource, user):
labels = resource.get('Config', {}).get('Labels') or resource.get('Labels') or {}
if labels.get('hope.managed') != 'true' or labels.get('hope.user_id') != user['id']:
raise OwnershipError('Refusing resource whose Hope ownership labels do not match')
@staticmethod
def _directory(path):
# Do not follow a user-created symlink into another account's state.
if path.is_symlink():
raise ProvisioningError('Private runtime directory cannot be a symlink')
path.mkdir(mode=0o700, parents=True, exist_ok=True)
os.chmod(path, 0o700)
@staticmethod
def _write_private(path, value):
if path.is_symlink():
raise ProvisioningError('Private runtime file cannot be a symlink')
fd, temp = tempfile.mkstemp(prefix='.hope-', dir=path.parent)
try:
with os.fdopen(fd, 'w') as stream:
stream.write(value)
stream.flush()
os.fsync(stream.fileno())
os.replace(temp, path)
finally:
if os.path.exists(temp):
temporary = Path(temp)
if temporary.resolve().parent != path.parent.resolve() or not temporary.name.startswith('.hope-'):
raise ProvisioningError('Refusing unexpected temporary-file cleanup path')
temporary.unlink()
def _paths(self, user):
container_name(user)
# Every ancestor under the configured root is control-owned, not mounted
# into Hermes. HOME cannot access the sibling API credential file.
return self.state_dir / user['id'], self.state_dir / user['id'] / 'home', self.state_dir / user['id'] / 'workspace'
def _read_key(self, user):
private, _, _ = self._paths(user)
path = private / 'api-key'
if path.is_symlink():
raise ProvisioningError('Private runtime file cannot be a symlink')
try:
key = path.read_text().strip()
except FileNotFoundError:
return None
if len(key) < 32 or '\n' in key:
raise ProvisioningError('Invalid persisted Hermes API credential')
return key
def _prepare(self, user, exists):
private, home, workspace = self._paths(user)
self._directory(self.state_dir)
for path in (private, home, workspace):
self._directory(path)
key = self._read_key(user)
if not key:
if exists:
raise ProvisioningError('Owned container has no persisted API credential; refusing unsafe adoption')
key = secrets.token_urlsafe(48)
self._write_private(private / 'api-key', key + '\n')
config = home / 'config.yaml'
if not config.exists():
# JSON is a YAML subset; no credentials are placed in configuration.
self._write_private(config, json.dumps(self.config(), indent=2) + '\n')
elif config.is_symlink():
raise ProvisioningError('Hermes config cannot be a symlink')
if os.geteuid() == 0:
# Only seeded directory roots/config, never recursive retained data.
for path in (home, workspace, config):
os.chown(path, self.runtime_uid, self.runtime_gid)
elif home.stat().st_uid != self.runtime_uid:
raise ProvisioningError('Control service must prepare private directories for the Hermes runtime UID')
return key, home, workspace
@staticmethod
def config():
model = os.environ.get('HERMES_UPSTREAM_MODEL', 'auto')
return {
'model': {'default': model, 'provider': 'custom:hope-upstream'},
'providers': {'hope-upstream': {'api': os.environ.get('HERMES_UPSTREAM_URL', 'http://192.168.86.5:2099/v1'),
'key_env': 'HERMES_UPSTREAM_API_KEY', 'transport': 'codex_responses', 'default_model': model,
'models': [model], 'discover_models': False}},
'agent': {'reasoning_effort': 'none'},
'terminal': {'backend': 'local', 'cwd': '/workspace'},
'platform_toolsets': {'api_server': ['terminal', 'file', 'web', 'browser', 'memory', 'skills', 'delegation']},
'gateway': {'multiplex_profiles': False,
'api_server': {'enabled': True, 'host': '0.0.0.0', 'port': 8642,
'max_concurrent_runs': 1, 'tool_progress_events': True}},
}
async def _network(self, user):
name = 'hope-hermes-net-' + user['id']
response = await self._request('GET', f'/networks/{name}', allowed=(200, 404))
if response.status_code == 404:
await self._request('POST', '/networks/create', allowed=(201, 409), json={
'Name': name, 'Driver': 'bridge', 'CheckDuplicate': True,
'Labels': {'hope.managed': 'true', 'hope.user_id': user['id']}})
response = await self._request('GET', f'/networks/{name}')
network = response.json()
self._owned(network, user)
if network.get('Driver') != 'bridge' or network.get('Internal'):
raise ProvisioningError('Owned Hermes network is not an outbound-capable dedicated bridge')
return name
def _spec(self, user, key, home, workspace, network):
return {
'Image': self.image, 'User': f'{self.runtime_uid}:{self.runtime_gid}',
'Entrypoint': ['/opt/hermes/.venv/bin/python'], 'Cmd': ['-c', _BOOTSTRAP],
'WorkingDir': '/workspace',
'Labels': {'hope.managed': 'true', 'hope.user_id': user['id']},
'Env': ['HOME=/opt/data/home', 'HERMES_HOME=/opt/data',
'HERMES_GATEWAY_NO_SUPERVISE=1', 'API_SERVER_ENABLED=true',
'API_SERVER_HOST=0.0.0.0', 'API_SERVER_PORT=8642', f'API_SERVER_KEY={key}',
f'HERMES_UPSTREAM_API_KEY={self.upstream_api_key}'],
'ExposedPorts': {'8642/tcp': {}},
'HostConfig': {
'Binds': [f'{home}:/opt/data:rw', f'{workspace}:/workspace:rw'],
'NetworkMode': network, 'PortBindings': {'8642/tcp': [{'HostIp': '127.0.0.1', 'HostPort': ''}]},
'Init': True, 'Privileged': False, 'CapDrop': ['ALL'],
'SecurityOpt': ['no-new-privileges:true'], 'PidsLimit': 256,
'Memory': self.memory_bytes, 'MemorySwap': self.memory_bytes, 'NanoCpus': self.nano_cpus,
'ShmSize': 256 * 1024 ** 2,
'RestartPolicy': {'Name': 'unless-stopped'},
'LogConfig': {'Type': 'json-file', 'Config': {'max-size': '10m', 'max-file': '3'}},
},
}
def _connection(self, user, info, key):
self._owned(info, user)
env = info.get('Config', {}).get('Env') or []
if f'API_SERVER_KEY={key}' not in env:
raise ProvisioningError('Owned container API credential does not match private state')
bindings = info.get('NetworkSettings', {}).get('Ports', {}).get('8642/tcp') or []
if len(bindings) != 1 or bindings[0].get('HostIp') != '127.0.0.1':
raise ProvisioningError('Hermes API must have exactly one loopback-only publication')
port = str(bindings[0].get('HostPort', ''))
if not port.isdigit() or not 1 <= int(port) <= 65535:
raise ProvisioningError('Hermes API has no allocated host port')
return {'container_name': container_name(user), 'hermes_url': f'http://127.0.0.1:{port}', 'hermes_api_key': key}
async def _healthy(self, connection):
try:
response = await self.health.get(connection['hermes_url'] + '/v1/models',
headers={'Authorization': 'Bearer ' + connection['hermes_api_key']})
return response.status_code == 200
except httpx.HTTPError:
return False
async def ensure(self, user: dict) -> dict:
async with self._lock(user):
if not self.upstream_api_key:
raise ProvisioningError('HERMES_UPSTREAM_API_KEY must be provided at runtime')
name = container_name(user)
info = await self._inspect(name)
if info:
self._owned(info, user)
key, home, workspace = self._prepare(user, info is not None)
if info is None:
network = await self._network(user)
await self._request('POST', '/containers/create', params={'name': name},
json=self._spec(user, key, home, workspace, network), allowed=(201, 409))
info = await self._inspect(name)
if info is None:
raise ProvisioningError('Docker container creation did not persist; retry')
self._owned(info, user)
if not info.get('State', {}).get('Running'):
await self._request('POST', f'/containers/{info["Id"]}/start', allowed=(204, 304))
# One overall deadline also bounds Docker inspect / health I/O; no
# deletion on failed health, so a retry starts/adopts the same state.
try:
async with asyncio.timeout(self.ready_timeout):
while True:
info = await self._inspect(name)
if info is None:
raise ProvisioningError('Hermes container disappeared; retry')
self._owned(info, user)
if info.get('State', {}).get('Running'):
connection = self._connection(user, info, key)
if await self._healthy(connection):
return connection
await asyncio.sleep(self.poll_interval)
except TimeoutError as exc:
raise ProvisioningError('Hermes API readiness timed out; retained container and data for retry') from exc
async def ready(self, user: dict) -> bool:
try:
info = await self._inspect(container_name(user))
if not info or not info.get('State', {}).get('Running'):
return False
key = self._read_key(user)
return bool(key) and await self._healthy(self._connection(user, info, key))
except (ProvisioningError, ValueError, OSError):
return False
async def remove(self, user: dict):
async with self._lock(user):
info = await self._inspect(container_name(user))
if info:
self._owned(info, user)
# Operate on inspected immutable ID, never a raced name. Labels
# are immutable for the lifetime of that Docker container ID.
await self._request('DELETE', f'/containers/{info["Id"]}',
params={'force': 'true', 'v': 'false'}, allowed=(204, 404))
name = 'hope-hermes-net-' + user['id']
response = await self._request('GET', f'/networks/{name}', allowed=(200, 404))
if response.status_code == 200:
network = response.json()
self._owned(network, user)
await self._request('DELETE', f'/networks/{network["Id"]}', allowed=(204, 404))
# Intentionally retain private credential, home, workspace and memory.
DockerProvisioner = Provisioner
+3
View File
@@ -0,0 +1,3 @@
fastapi>=0.115,<1
uvicorn>=0.30,<1
httpx>=0.27,<1