Archived
feat: add local machine registry foundation
This commit is contained in:
Marc Kassubeck
+120
-19
@@ -98,8 +98,24 @@ Initial storage file:
|
||||
$PI_WEB_DATA_DIR/machines.json
|
||||
```
|
||||
|
||||
Allow tests and advanced deployments to override it with:
|
||||
|
||||
```text
|
||||
PI_WEB_MACHINES_FILE=/path/to/machines.json
|
||||
```
|
||||
|
||||
The `local` machine is synthesized by the service, not persisted. The stored file contains remote machines only. This keeps the default local endpoint stable, prevents accidental deletion/corruption of the built-in machine, and allows a fresh install with no `machines.json` to behave exactly like current Pi Web.
|
||||
|
||||
Default behavior when no file exists:
|
||||
|
||||
```json
|
||||
{
|
||||
"machines": []
|
||||
}
|
||||
```
|
||||
|
||||
API responses still include the synthesized local machine first:
|
||||
|
||||
```json
|
||||
{
|
||||
"machines": [
|
||||
@@ -139,10 +155,12 @@ Example create request:
|
||||
|
||||
Rules:
|
||||
|
||||
- `local` machine cannot be deleted.
|
||||
- `local` machine cannot be created, patched, or deleted through the registry because it is synthesized.
|
||||
- Remote `baseUrl` must be `http:` or `https:`.
|
||||
- Remote `baseUrl` must not include username/password, query, or hash components.
|
||||
- Normalize `baseUrl` by trimming trailing slash.
|
||||
- Do not return `token` in normal responses.
|
||||
- Treat machine registry credentials as gateway-to-remote Pi Web credentials, not model-provider credentials.
|
||||
|
||||
### Machine-scoped project/workspace/file/git routes
|
||||
|
||||
@@ -195,6 +213,14 @@ Compatibility aliases keep using local machine:
|
||||
/api/sessions...
|
||||
```
|
||||
|
||||
Remote auth policy for first remote implementation:
|
||||
|
||||
- Machine registry `token`/`headers` authenticate the gateway to the remote Pi Web instance.
|
||||
- Model-provider API keys and OAuth state remain owned by each target machine/session daemon.
|
||||
- API-key provider configuration may be proxied once the normal remote HTTP proxy is working.
|
||||
- OAuth flows should not be fully proxied in the first remote phase. The UI should offer to open the selected remote Pi Web directly for OAuth login/logout until callback origin behavior is explicitly designed and tested.
|
||||
- If a remote auth endpoint is unavailable or intentionally unsupported, return a clear error telling the user to configure auth on the remote machine.
|
||||
|
||||
### Machine-scoped WebSockets
|
||||
|
||||
Canonical new routes:
|
||||
@@ -231,8 +257,9 @@ src/server/machines/machineProxyRoutes.ts
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Read/write `$PI_WEB_DATA_DIR/machines.json`.
|
||||
- Return default local machine if file is missing.
|
||||
- Read/write `$PI_WEB_DATA_DIR/machines.json`, or `PI_WEB_MACHINES_FILE` when configured.
|
||||
- Store remote machine records only. Do not persist the synthesized `local` machine.
|
||||
- Return an empty remote list if the file is missing.
|
||||
- Validate JSON shape.
|
||||
- Generate stable IDs for new remote machines.
|
||||
|
||||
@@ -240,8 +267,9 @@ Responsibilities:
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- CRUD machine records.
|
||||
- Prevent deleting `local`.
|
||||
- CRUD remote machine records.
|
||||
- Synthesize the built-in `local` machine in list/get responses.
|
||||
- Prevent creating, patching, or deleting `local`.
|
||||
- Resolve a machine by ID.
|
||||
- Create an appropriate gateway target:
|
||||
- local target: existing services and local session daemon client;
|
||||
@@ -259,16 +287,20 @@ Responsibilities:
|
||||
Pseudo-interface:
|
||||
|
||||
```ts
|
||||
interface MachineHttpResponse {
|
||||
statusCode: number;
|
||||
headers: Record<string, string | string[] | undefined>;
|
||||
body: string | Buffer | NodeJS.ReadableStream;
|
||||
}
|
||||
|
||||
interface MachineClient {
|
||||
request(method: string, path: string, body?: unknown): Promise<{
|
||||
statusCode: number;
|
||||
headers: Record<string, string>;
|
||||
body: string;
|
||||
}>;
|
||||
request(method: string, path: string, body?: unknown): Promise<MachineHttpResponse>;
|
||||
connectWebSocket(path: string): WebSocket;
|
||||
}
|
||||
```
|
||||
|
||||
The interface must support streaming/binary responses because file previews and future downloads cannot safely be represented as JSON strings.
|
||||
|
||||
For `local`, this can be backed by direct local services where practical or by existing local route handlers/session daemon clients. For first implementation, keep local code paths mostly unchanged and add route wrappers.
|
||||
|
||||
### Route implementation strategy
|
||||
@@ -279,7 +311,15 @@ For `local`, this can be backed by direct local services where practical or by e
|
||||
- If `machineId === "local"`, call current local services.
|
||||
- Else proxy equivalent path to remote machine without the `/api/machines/:machineId` prefix.
|
||||
|
||||
Example remote mapping:
|
||||
Path translation must be explicit and tested:
|
||||
|
||||
```text
|
||||
/api/machines/:machineId/<compat-path>
|
||||
-> /api/<compat-path> for remote Pi Web HTTP/WebSocket routes
|
||||
-> /<compat-path> for local sessiond routes where sessiond expects non-/api paths
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
GET /api/machines/devbox/projects
|
||||
@@ -287,10 +327,33 @@ GET /api/machines/devbox/projects
|
||||
|
||||
WS /api/machines/devbox/sessions/abc/events
|
||||
-> WS wss://devbox.example.ts.net/api/sessions/abc/events
|
||||
|
||||
GET /api/machines/local/sessions/abc/status
|
||||
-> local sessiond GET /sessions/abc/status
|
||||
```
|
||||
|
||||
This lets remote machines run unmodified Pi Web at first. Later, when remote Pi Web also supports machine-scoped APIs, the gateway can still target the compatibility aliases on that remote.
|
||||
|
||||
Proxy response handling rules:
|
||||
|
||||
- Preserve query strings exactly after the machine prefix is stripped.
|
||||
- Pass through successful JSON responses using normal API parsers.
|
||||
- Pass through binary/streaming responses such as file previews without buffering into strings.
|
||||
- Forward only safe response headers such as `content-type`, `content-length`, `cache-control`, `last-modified`, and `etag`.
|
||||
- Strip hop-by-hop headers such as `connection`, `transfer-encoding`, `upgrade`, `keep-alive`, and `proxy-authenticate`.
|
||||
- Apply short request timeouts for health checks and bounded timeouts for normal HTTP proxy requests.
|
||||
- Normalize remote unreachable/timeouts to gateway errors (`502`/`504`) with clear messages.
|
||||
|
||||
Proxy security rules:
|
||||
|
||||
- Never ignore TLS certificate errors by default.
|
||||
- Do not follow redirects for proxied API requests unless there is a specific, reviewed need.
|
||||
- Do not forward browser credentials/cookies to remote machines by default.
|
||||
- Only attach credentials configured on the machine record, and block configured headers that would override transport semantics such as `host`, `connection`, `upgrade`, `transfer-encoding`, `content-length`, or `authorization` unless the field is the explicit token/auth mechanism.
|
||||
- Use request body size limits consistent with the existing local API.
|
||||
- Use response size limits for JSON endpoints where practical; streaming/binary endpoints should stream with timeout/backpressure rather than unbounded buffering.
|
||||
- Private network URLs are allowed because Tailscale/WireGuard/SSH tunnels are a primary use case, but the UI and docs should warn that registering a machine gives the local Pi Web server permission to contact that endpoint.
|
||||
|
||||
## Client architecture
|
||||
|
||||
### State changes
|
||||
@@ -314,6 +377,31 @@ workspaces // workspaces for selectedProject on selectedMachine
|
||||
sessions // sessions for selectedWorkspace on selectedMachine
|
||||
```
|
||||
|
||||
### Cross-machine identity and cache keys
|
||||
|
||||
Server APIs should keep returning the target machine's native IDs. The client must namespace any state, cache, route restoration, or lookup table that can contain entities from more than one machine.
|
||||
|
||||
Use helper functions rather than ad hoc string concatenation:
|
||||
|
||||
```ts
|
||||
const machineProjectKey = (machineId: string, projectId: string) => `${machineId}:${projectId}`;
|
||||
const machineWorkspaceKey = (machineId: string, projectId: string, workspaceId: string) => `${machineId}:${projectId}:${workspaceId}`;
|
||||
const machineSessionKey = (machineId: string, sessionId: string) => `${machineId}:${sessionId}`;
|
||||
```
|
||||
|
||||
At minimum, namespace:
|
||||
|
||||
- `workspacesByProjectId` or its replacement;
|
||||
- `sessionStatuses`;
|
||||
- `sessionActivities`;
|
||||
- `workspaceActivities`;
|
||||
- chat transcript caches;
|
||||
- prompt draft storage;
|
||||
- any cached new-session or session-restoration state;
|
||||
- terminal socket state if more than one machine can be active at a time.
|
||||
|
||||
Flat selected-machine views are still fine for rendering, but persisted and long-lived maps should never assume project, workspace, session, or terminal IDs are globally unique.
|
||||
|
||||
### API client changes
|
||||
|
||||
In `src/client/src/api/clients.ts`, add:
|
||||
@@ -473,13 +561,16 @@ src/client/src/route.test.ts
|
||||
|
||||
Cover:
|
||||
|
||||
- default local machine when no machines file exists;
|
||||
- default local machine is synthesized when no machines file exists;
|
||||
- `machines.json` stores remote machines only and does not persist `local`;
|
||||
- `PI_WEB_MACHINES_FILE` overrides the default store path;
|
||||
- add remote machine;
|
||||
- reject invalid base URLs;
|
||||
- reject invalid base URLs, including username/password, query, and hash components;
|
||||
- do not expose token in response;
|
||||
- cannot delete local machine;
|
||||
- cannot create, patch, or delete local machine;
|
||||
- route read/write with and without `machine`;
|
||||
- switching machine clears project/workspace/session state;
|
||||
- machine-scoped cache key helpers avoid collisions;
|
||||
- missing route machine falls back to local.
|
||||
|
||||
### Integration tests
|
||||
@@ -487,8 +578,12 @@ Cover:
|
||||
Add server route tests with mocked remote machine client:
|
||||
|
||||
- `GET /api/machines/remote/projects` proxies to `/api/projects` on remote.
|
||||
- local sessiond path mapping strips `/api/machines/local` and forwards `/sessions...`, `/auth...`, and `/activity` correctly.
|
||||
- Remote non-2xx status passes through reasonably.
|
||||
- Remote unreachable returns 502 with useful error.
|
||||
- Remote timeout returns 504 with useful error.
|
||||
- Binary/streaming responses such as file previews are not coerced into strings.
|
||||
- Hop-by-hop headers are stripped and safe response headers are preserved.
|
||||
- WebSocket path mapping uses `ws:`/`wss:` correctly.
|
||||
|
||||
### Manual test matrix
|
||||
@@ -533,12 +628,17 @@ npm test
|
||||
|
||||
### Phase 1: Local machine registry only
|
||||
|
||||
Deliverable: Pi Web has a Machines list, but only `local` exists and all existing behavior works.
|
||||
Deliverable: Pi Web has a Machines list, but only synthesized `local` exists and all existing behavior works.
|
||||
|
||||
This can be split into two PRs if review size matters:
|
||||
|
||||
- Phase 1a: shared `Machine` types, remote-only `MachineStore`, `MachineService`, `/api/machines` routes, and tests.
|
||||
- Phase 1b: client `machinesApi`, `MachineController`, selected-machine state, route support, and Local-only UI.
|
||||
|
||||
Tasks:
|
||||
|
||||
- Add `Machine` shared types.
|
||||
- Add `MachineStore`, `MachineService`, and `/api/machines` routes.
|
||||
- Add remote-only `MachineStore`, `MachineService`, and `/api/machines` routes that synthesize `local`.
|
||||
- Add client `machinesApi`.
|
||||
- Add `MachineController`.
|
||||
- Add `selectedMachine` to app state.
|
||||
@@ -548,7 +648,7 @@ Tasks:
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Fresh UI shows `Local` under Machines.
|
||||
- Fresh UI shows synthesized `Local` under Machines.
|
||||
- Current project/workspace/session workflows unchanged.
|
||||
- Current URLs continue to work.
|
||||
|
||||
@@ -578,7 +678,8 @@ Tasks:
|
||||
- Add remote `MachineClient`.
|
||||
- Add `GET /api/machines/:id/health`.
|
||||
- Proxy machine-scoped HTTP routes for remote machines to remote compatibility routes.
|
||||
- Add token/header support.
|
||||
- Add token/header support for gateway-to-remote authentication.
|
||||
- Keep OAuth provider login/logout flows remote-direct unless callback origin behavior is explicitly implemented.
|
||||
- Add UI for add/remove remote machines.
|
||||
|
||||
Acceptance:
|
||||
@@ -673,7 +774,7 @@ docs/machines.md or docs/federation.md
|
||||
## Open questions
|
||||
|
||||
1. Should remote machine auth be a bearer token, arbitrary headers, or both?
|
||||
2. Should `machines.json` store secrets directly, or should it use a separate secret store later?
|
||||
2. Should remote machine secrets in `machines.json` stay inline for v1, or should they use a separate secret store later?
|
||||
3. Should central Pi Web allow adding projects to remote machines, or only list existing remote projects at first?
|
||||
4. Should activity be subscribed only for selected machine, or for all machines with active health polling?
|
||||
5. Should machine IDs be user-chosen slugs or generated UUIDs with editable names?
|
||||
|
||||
Reference in New Issue
Block a user