From 102c2db2648c82796d1e5f349783715976ce5b0e Mon Sep 17 00:00:00 2001 From: Federico Jaramillo Martinez Date: Fri, 8 May 2026 08:43:36 +0200 Subject: [PATCH] docs: document testing approach --- .ai-work/testing-approach.md | 78 ++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 .ai-work/testing-approach.md diff --git a/.ai-work/testing-approach.md b/.ai-work/testing-approach.md new file mode 100644 index 0000000..6f4174d --- /dev/null +++ b/.ai-work/testing-approach.md @@ -0,0 +1,78 @@ +# Testing approach + +Pi Web should be tested as a set of stable contracts, not as a collection of implementation details. The goal is confidence that users can safely manage projects, workspaces, sessions, and realtime agent interactions as the app evolves. + +## Priorities + +1. **Safety boundaries** + - Workspace path normalization and traversal prevention. + - File read/tree/suggestion APIs never escape the workspace. + - Git/file operations behave correctly for missing, binary, symlinked, large, and non-directory paths. + +2. **Server contracts** + - Fastify routes return stable status codes and response shapes. + - Project/workspace/git/session services are covered with real temporary directories and repositories where practical. + - Session daemon client/proxy behavior is tested separately from the browser-facing API process. + +3. **Session lifecycle and realtime behavior** + - Starting, listing, resuming, archiving/restoring, stopping, and aborting sessions should be tested with fakes at the Pi SDK boundary. + - WebSocket/event hub tests should verify event routing, global events, disconnect cleanup, and reconnect-safe state. + - Do not require real Pi credentials in normal CI; use fake runtimes/services for deterministic tests. + +4. **Client state and UX-critical flows** + - Test pure state helpers heavily: routes, chat history merging, transcript/message grouping, input modes, formatting, and API parsing. + - Browser tests should cover UX behavior, not styling: selecting project/workspace/session, sending a prompt, streaming status, file/git panels, and command-picker flows. + - Prefer durable `data-testid` selectors for Playwright instead of CSS/style assertions. + +## Test layers + +### Unit tests + +Use Vitest for pure functions and small state transitions. These tests should be fast, deterministic, and edge-case heavy. Good targets include: + +- `src/server/workspaces/pathSafety.ts` +- `src/client/src/chatHistoryCache.ts` +- `src/client/src/chatGroups.ts` +- `src/client/src/chatMessages.ts` +- `src/client/src/chatTranscript.ts` +- `src/client/src/inputModes.ts` +- `src/client/src/route.ts` +- `src/client/src/api/parsers.ts` +- formatting and URL helpers + +### Service tests + +Use real temporary directories via `mkdtemp()` and cleanup after each test. Prefer real filesystem/git behavior over mocks for workspace, file, and git services. Keep tests isolated and avoid depending on the repository checkout. + +### API integration tests + +Construct Fastify apps in tests, register routes with fake services, and use `app.inject()`. Assert public HTTP contracts: validation, status code, and response body. + +### Daemon/protocol tests + +Test the split-process session architecture with fake session services and temporary Unix sockets. Critical behavior: daemon unavailable errors, request/response framing, WebSocket proxying, and browser/API disconnects not stopping daemon-owned sessions. + +### Browser UX tests + +Use Playwright for a small set of happy-path and recovery flows. Do not test colors/layout. Test user-visible behavior: loading empty states, adding/selecting projects, starting sessions, chat updates, file/git panels, and command dialogs. + +## Coverage guidance + +Coverage numbers are a guardrail, not the goal. Start with moderate global thresholds and raise them as meaningful tests are added. Critical modules should receive targeted tests even if global coverage is already high. + +Recommended initial gates once coverage tooling is enabled: + +- statements: 70% +- branches: 60% +- functions: 70% +- lines: 70% + +## Conventions for future agents + +- Add tests near the code under test as `*.test.ts`. +- Prefer testing public functions/contracts over private implementation details. +- When a module is hard to test, first add a small composable seam instead of using broad mocks. +- Use fakes at external boundaries: Pi SDK, daemon process, network, and browser WebSockets. +- Use real temp directories for filesystem behavior. +- Run `npm run verify` before committing. +- Make progressive commits: one commit for test infrastructure/docs, then focused commits for each tested area.