From d2b55207858fd3652d2ee9230df08bf75f633e03 Mon Sep 17 00:00:00 2001 From: Federico Jaramillo Martinez Date: Fri, 3 Jul 2026 10:36:59 +0200 Subject: [PATCH] docs(relay): include leg identity in handoffs --- skills/relay/SKILL.md | 19 +++++++++++-------- skills/relay/evals/evals.json | 10 +++++----- skills/relay/evals/live-behavior-testing.md | 5 ++++- 3 files changed, 20 insertions(+), 14 deletions(-) diff --git a/skills/relay/SKILL.md b/skills/relay/SKILL.md index d10d7c4..f043205 100644 --- a/skills/relay/SKILL.md +++ b/skills/relay/SKILL.md @@ -32,7 +32,7 @@ Every relay has these three core files: - **Goal / finish line.** A concrete, achievable end state. Without this the relay runs forever — this is non-negotiable. - **Sizing.** How much is *one leg*? This is project- and plan-specific; the charter defines it (a task, a slice, a time/scope budget — whatever fits). The skill does not decide this for you. - **Task selection policy.** How a runner chooses the next task when `status.md` does not name one explicitly. -- **Handover.** How a runner hands off: what the spawn prompt should say and what the next runner must read. A normal handoff points at `charter.md` and `status.md`, not the full log. +- **Handover.** How a runner hands off: what the spawn prompt should say and what the next runner must read. A normal handoff starts with a natural header containing the relay name and next leg number, then points at `charter.md` and `status.md`, not the full log. - **Intervention signal.** When and how a runner must stop and get the human, and how that is made visible. The charter must define this; the skill does not define it for you. - **Reading discipline.** The files a runner should read to orient, and any files that should not be read defensively. @@ -42,11 +42,12 @@ The charter *can* be edited, but it should rarely *need* to be. If it is changin - **Current position.** Where the relay is now. - **Current or next task.** The next leg if known; otherwise enough information to apply the charter's task selection policy. +- **Leg tracking.** The last completed leg and the next leg to run. Keep this explicit so runners do not have to infer whether “current leg” means the leg just finished or the leg being handed off, and so new PI-WEB sessions can distinguish relay legs from the first line of their prompt without naming instructions. - **Relevant context.** Only the files, sections, commands, artifacts, or specific log entries needed for the next leg. - **Progress documentation.** Where this runner must write progress: update `status.md`, append `log.md`, update artifacts, commit, etc. - **Blockers / intervention state.** Current risks, open decisions, or active reasons to stop. -Think of `status.md` as the thing passed from runner to runner. If it grows into a history dump, compress it back into current state plus pointers. +Think of `status.md` as the thing passed from runner to runner. If it grows into a history dump, compress it back into current state plus pointers. If an older relay lacks leg tracking, repair it when you update status; prefer the leg number from the prompt or status, and do not read `log.md` end-to-end just to count prior legs. **Log** (`log.md`) — append-only history. Each leg appends a concise entry recording what it did, decisions made and why, durable artifacts changed, status updates made, and blockers. The log preserves auditability, but it is **not** orientation memory. @@ -70,19 +71,21 @@ If `status.md` is insufficient, fix the baton rather than compensating by readin This is the loop you run when you are dispatched into a relay. -1. **Orient from the packet.** Read `charter.md` and `status.md`. Confirm the relay name/root, goal, sizing, handoff protocol, intervention signal, and current/next task. If you are not sure you are in a relay, the prompt or `.pi-web/relays/` is your clue — and reading this skill means you are. +1. **Orient from the packet.** Read `charter.md` and `status.md`. Confirm the relay name/root, goal, sizing, handoff protocol, last completed leg, next leg to run, intervention signal, and current/next task. If you are not sure you are in a relay, the prompt or `.pi-web/relays/` is your clue — and reading this skill means you are. 2. **Choose the leg.** Prefer the explicit current/next task in `status.md`. If none is named, apply the charter's task selection policy. If that still requires context, inspect only the referenced plan/backlog/artifact sections. If the next task is still ambiguous or would materially change direction, stop and involve the human. 3. **Re-anchor to the goal.** Does the goal still make sense given the status and what you now see? If reality has diverged from the charter, that is often an intervention moment — don't quietly redefine the task. 4. **Run one leg.** Do exactly one well-sized slice, per the charter's sizing. Resist doing "just a bit more" — extra scope bloats context and breaks the containment that makes Relay work. -5. **Document progress.** Make all work durable. Update `status.md` with the new current state, next task or task-selection pointer, relevant context for the next runner, and blockers. Append a concise `log.md` entry with what you did, why, decisions made, artifacts changed, and whether you are handing off or stopping. +5. **Document progress.** Make all work durable. Update `status.md` with the new current state, last completed leg, next leg to run (if any), next task or task-selection pointer, relevant context for the next runner, and blockers. Append a concise `log.md` entry with what you did, why, decisions made, artifacts changed, and whether you are handing off or stopping. 6. **Decide: hand off, or stop.** - - **Hand off** if there is a clear next leg and you are on track. Use `spawn_session` once, with a prompt that names the Relay method and points the next runner at `charter.md` and `status.md` (so this skill loads and they can orient cheaply). Then you are done. Handoff is deliberately fire-and-forget: `spawn_session` starts an independent session you will not see and cannot steer — do not reach for a tracked subsession to keep an eye on it. Letting go is the point. The next runner is trusted to run their own leg, and the relay packet is the only thread between you; if you feel the need to watch downstream work, that usually means the leg wasn't sized or handed off cleanly, or an intervention signal should have fired. + - **Hand off** if there is a clear next leg and you are on track. Use `spawn_session` once, with a prompt whose first line is a natural task header containing the relay name and next leg number (for example, `Relay "" leg begins now.`), followed by the Relay method and pointers to `charter.md` and `status.md` (so this skill loads and they can orient cheaply). Then you are done. Handoff is deliberately fire-and-forget: `spawn_session` starts an independent session you will not see and cannot steer — do not reach for a tracked subsession to keep an eye on it. Letting go is the point. The next runner is trusted to run their own leg, and the relay packet is the only thread between you; if you feel the need to watch downstream work, that usually means the leg wasn't sized or handed off cleanly, or an intervention signal should have fired. - **Stop — do not spawn —** if the goal is reached, or you are blocked, or the charter's intervention signal fires. Update `status.md`, append a clear note in `log.md`, and raise the intervention signal so the watching human sees exactly what happened and what they need to decide. A stalled relay that stopped cleanly with a clear blocker is a success; a relay that spawned a confused next runner is a failure. -A good handoff prompt is short and explicit: +A good handoff prompt is short and explicit. Put the relay identity and leg number at the very beginning so PI-WEB's session title generator sees useful distinguishing context without any naming instruction: ```text -You are continuing Relay "". +Relay "" leg begins now. + +You are the next runner in this Relay method chain. Read: - .pi-web/relays//charter.md @@ -95,7 +98,7 @@ Run one leg according to the charter. Before handing off, update status.md, appe ## Planning a relay -When the user asks to set up a relay, your job is to produce the relay packet: `charter.md`, `status.md`, and `log.md`. The charter must have the required slots filled: relay identity, goal, sizing, task selection policy, handover, intervention signal, and reading discipline. The initial status must give the first runner a compact baton: current position, first task or task selection pointer, relevant context, documentation expectations, and known blockers. The log may start empty or with a short seed entry explaining that the relay was created. +When the user asks to set up a relay, your job is to produce the relay packet: `charter.md`, `status.md`, and `log.md`. The charter must have the required slots filled: relay identity, goal, sizing, task selection policy, handover, intervention signal, and reading discipline. The initial status must give the first runner a compact baton: current position, leg tracking (usually last completed leg 0 and next leg to run 1 for a new relay), first task or task selection pointer, relevant context, documentation expectations, and known blockers. The log may start empty or with a short seed entry explaining that the relay was created. Draw the required choices out from the user rather than inventing them: ask what the finish line is, how much should be one leg, how runners pick tasks, how runners hand off, what they should read, and when they must stop and get the human. Sizing, task selection, and the intervention signal especially are the user's to decide — propose options if it helps them think, but do not quietly settle them yourself. diff --git a/skills/relay/evals/evals.json b/skills/relay/evals/evals.json index 5214ded..44b04e8 100644 --- a/skills/relay/evals/evals.json +++ b/skills/relay/evals/evals.json @@ -6,17 +6,17 @@ "id": 0, "name": "plan-a-relay", "prompt": "I want to migrate all our REST endpoints to the new validation layer — there are around 40 of them across src/server/routes. I won't be able to babysit this. Set it up as a relay so an agent can grind through it across sessions and only pull me in when it actually needs me.", - "expected_output": "Produces a relay packet (default .pi-web/relays//) with charter.md, status.md, and log.md. The charter has all required slots present: relay identity/root, goal/finish-line, sizing, task selection policy, handover, intervention signal, and reading discipline. The initial status is a compact baton with current position, first task or task-selection pointer, relevant context, progress documentation expectations, and known blockers. The agent asks the user to make sizing, task selection, reading discipline, and the intervention signal concrete rather than inventing strict rules. It does not prescribe what a 'good' leg size or cadence is. It may dispatch the first leg only after the packet is agreed.", + "expected_output": "Produces a relay packet (default .pi-web/relays//) with charter.md, status.md, and log.md. The charter has all required slots present: relay identity/root, goal/finish-line, sizing, task selection policy, handover, intervention signal, and reading discipline. The handover guidance says the spawn prompt starts with the relay name and leg number before pointing at charter.md and status.md. The initial status is a compact baton with current position, leg tracking for the last completed leg and next leg to run, first task or task-selection pointer, relevant context, progress documentation expectations, and known blockers. The agent asks the user to make sizing, task selection, reading discipline, and the intervention signal concrete rather than inventing strict rules. It does not prescribe what a 'good' leg size or cadence is. It may dispatch the first leg only after the packet is agreed.", "files": [], "assertions": [ { "name": "packet-created", "text": "A relay packet is created with charter.md, status.md, and log.md under the relay location (default .pi-web/relays// unless specified).", "type": "script" }, { "name": "goal-slot-present", "text": "The charter defines a concrete, achievable finish line / goal.", "type": "judgment" }, { "name": "sizing-slot-present", "text": "The charter states how much work is one leg (sizing), rather than leaving it undefined.", "type": "judgment" }, { "name": "task-selection-slot-present", "text": "The charter states how a runner chooses the next task when status.md does not name one explicitly.", "type": "judgment" }, - { "name": "handover-slot-present", "text": "The charter states the handover mechanism, including that the next runner reads charter.md and status.md.", "type": "judgment" }, + { "name": "handover-slot-present", "text": "The charter states the handover mechanism, including that the handoff prompt starts with the relay name and next leg number and that the next runner reads charter.md and status.md.", "type": "judgment" }, { "name": "intervention-slot-present", "text": "The charter defines an intervention signal: when/how a runner stops and gets the human.", "type": "judgment" }, { "name": "reading-discipline-present", "text": "The charter states the reading discipline, including not reading log.md end-to-end by default.", "type": "judgment" }, - { "name": "status-seeded", "text": "status.md is seeded as a compact baton with current position, first task or task-selection pointer, relevant context, documentation expectations, and known blockers.", "type": "judgment" }, + { "name": "status-seeded", "text": "status.md is seeded as a compact baton with current position, leg tracking for the last completed leg and next leg to run, first task or task-selection pointer, relevant context, documentation expectations, and known blockers.", "type": "judgment" }, { "name": "asks-not-prescribes", "text": "For sizing, task selection, reading discipline, and the intervention signal, the agent asks the user to make them concrete instead of imposing its own strict rules/cadence.", "type": "judgment" }, { "name": "no-premature-spawn", "text": "The agent does not spawn the first leg before the relay packet is agreed with the user.", "type": "script" } ] @@ -25,7 +25,7 @@ "id": 1, "name": "run-one-leg-and-hand-off", "prompt": "You're working under the Relay framework. Read .pi-web/relays//charter.md and .pi-web/relays//status.md, continue the plan, then dispatch the next agent.", - "expected_output": "Loads the relay skill (handoff prompt names the framework). Orients by reading charter.md and status.md, not the full log. Re-anchors to the goal, chooses the next task from status.md or the charter's task-selection policy, does exactly ONE well-sized leg per the charter's sizing, updates status.md as a compact baton, appends a concise log.md entry, makes work durable (saves files, commits if the charter calls for it), then calls spawn_session exactly once with a handoff prompt that names Relay and points at charter.md and status.md. Does not do extra legs, does not spawn more than once, and does not tell the next runner to read log.md end-to-end.", + "expected_output": "Loads the relay skill (handoff prompt names the framework). Orients by reading charter.md and status.md, not the full log. Re-anchors to the goal, chooses the next task from status.md or the charter's task-selection policy, does exactly ONE well-sized leg per the charter's sizing, updates status.md as a compact baton, appends a concise log.md entry, makes work durable (saves files, commits if the charter calls for it), then calls spawn_session exactly once with a handoff prompt that starts with the relay name and next leg number, names Relay, and points at charter.md and status.md. Does not do extra legs, does not spawn more than once, and does not tell the next runner to read log.md end-to-end.", "files": [], "assertions": [ { "name": "skill-loads-from-handoff", "text": "The agent recognizes it is in a relay and loads/consults the relay skill from the handoff prompt.", "type": "judgment" }, @@ -37,7 +37,7 @@ { "name": "log-appended", "text": "A concise log.md entry is appended recording what was done, decisions, artifacts changed, status updates made, and any blocker.", "type": "script" }, { "name": "work-durable-before-handoff", "text": "Work is saved (and committed if the charter requires it) before spawn_session is called.", "type": "script" }, { "name": "spawn-exactly-once", "text": "spawn_session is called exactly once.", "type": "script" }, - { "name": "handoff-names-relay-and-status", "text": "The spawn prompt names the Relay framework and points the next runner at charter.md and status.md, not the full log, so the skill loads downstream with bounded context.", "type": "judgment" } + { "name": "handoff-names-relay-leg-and-status", "text": "The spawn prompt starts with the relay name and next leg number, names the Relay framework, and points the next runner at charter.md and status.md, not the full log, so the skill loads downstream with bounded context and PI-WEB can generate a distinguishable session name.", "type": "judgment" } ] }, { diff --git a/skills/relay/evals/live-behavior-testing.md b/skills/relay/evals/live-behavior-testing.md index b23e799..586f52c 100644 --- a/skills/relay/evals/live-behavior-testing.md +++ b/skills/relay/evals/live-behavior-testing.md @@ -75,7 +75,10 @@ So when testing or running a relay whose packet lives outside the repo, keep `cw spawn_session cwd: Prompt: -You are continuing Relay "sandbox". +Relay "sandbox" leg 2 begins now. + +You are the next runner in this Relay method chain. + Read: - /tmp/pi-web-relay-live-evals/.../sandbox/.pi-web/relays/sandbox/charter.md - /tmp/pi-web-relay-live-evals/.../sandbox/.pi-web/relays/sandbox/status.md