1402 lines
71 KiB
Markdown
1402 lines
71 KiB
Markdown
---
|
||||
|
|
name: verify
|
|||
|
|
description: >
|
|||
|
|
Use ONLY after all work items in an iteration have passed code review. The
|
|||
|
|
Verifier (Worker) runs the iteration's DoD matrix, integration tests, NFR
|
|||
|
|
validation, and regression checks. The iteration is not Done until all DoD
|
|||
|
|
items pass.
|
|||
|
|
stage: verify
|
|||
|
|
triggers:
|
|||
|
|
- verify iteration
|
|||
|
|
- 验证迭代
|
|||
|
|
- iteration verification
|
|||
|
|
- 检查迭代完成
|
|||
|
|
- validate iteration
|
|||
|
|
- DoD check
|
|||
|
|
role: Verifier
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
> Core 中立版(Increment 6a 改写,原 deferHard verbatimDir)。机制、结构与 frontmatter 保持;实例术语(工具名、路径、工单号)按 `core/adapters/TERMINOLOGY.md` 绑定到具体实例。
|
|||
|
|
|
|||
|
|
# Iteration Verification
|
|||
|
|
|
|||
|
|
Systematically verify that all work items in the current iteration integrate
|
|||
|
|
correctly and satisfy every Definition-of-Done criterion. This is the
|
|||
|
|
iteration-level quality gate — individual work items may have passed code
|
|||
|
|
review, but the iteration is not complete until integration is verified.
|
|||
|
|
|
|||
|
|
## Process Overview
|
|||
|
|
|
|||
|
|
Every diamond is a gate the Verifier rationalizes waiving — most often by confusing "nothing failed yet" with "verified passing".
|
|||
|
|
|
|||
|
|
```dot
|
|||
|
|
digraph verify {
|
|||
|
|
rankdir=TB;
|
|||
|
|
node [shape=box, fontname="Helvetica"];
|
|||
|
|
|
|||
|
|
pre [shape=diamond, label="Reviews converged?\n(code + review-dag for DAG runs)"];
|
|||
|
|
abort [label="ABORT: list un-converged\nreviews / missing artifacts"];
|
|||
|
|
exec [label="Phase 2: Execute\n(full suite + typecheck + lint\n+ perf + security + regression)"];
|
|||
|
|
dod [shape=diamond, label="Any DoD criterion\nFAIL?"];
|
|||
|
|
nfr [shape=diamond, label="Any NFR threshold\nviolated?"];
|
|||
|
|
td [shape=diamond, label="Every tech-debt trigger\nobjectively verifiable?"];
|
|||
|
|
tdgap [label="Not tech debt →\niteration verdict = FAIL"];
|
|||
|
|
commit [shape=diamond, label="Commits exist\nfor this iteration?"];
|
|||
|
|
done [shape=doublecircle, label="Iteration Done"];
|
|||
|
|
|
|||
|
|
pre -> abort [label="no"];
|
|||
|
|
pre -> exec [label="yes"];
|
|||
|
|
exec -> dod;
|
|||
|
|
dod -> exec [label="yes: route to Developer"];
|
|||
|
|
dod -> nfr [label="all PASS"];
|
|||
|
|
nfr -> exec [label="yes"];
|
|||
|
|
nfr -> td [label="all PASS"];
|
|||
|
|
td -> tdgap [label="no"];
|
|||
|
|
td -> commit [label="yes"];
|
|||
|
|
commit -> done [label="yes"];
|
|||
|
|
commit -> abort [label="no: must commit first"];
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Agent Role
|
|||
|
|
|
|||
|
|
> **Progressive disclosure ([org-internal #3722], [org-internal #3734], [org-internal #3745])**: when dispatching a Verifier
|
|||
|
|
> scoped to one phase, pass `skills: ["verify:phase-2"]` (preamble + that
|
|||
|
|
> phase only) instead of the full body; multi-phase dispatches may union.
|
|||
|
|
> Phase 5 is subdivided ([org-internal #3734]): `phase-5` covers the report template only;
|
|||
|
|
> each mandatory filing subsection carries its own marker (`phase-5.5` tech
|
|||
|
|
> debt, `phase-5.55` baseline failures, `phase-5.56` flaky tests,
|
|||
|
|
> `phase-5.6` checklist sync, `phase-5.7` CI sync). Dispatch recipes:
|
|||
|
|
> FAIL verdict → `"verify:phase-5,phase-5.5,phase-5.6"` (union `phase-5.55`
|
|||
|
|
> / `phase-5.56` when baseline failures / flaky tests are present); PASS
|
|||
|
|
> close-out →
|
|||
|
|
> `"verify:phase-5,phase-5.5,phase-5.55,phase-5.56,phase-5.6,phase-5.7,phase-6"`.
|
|||
|
|
> The DAG branch is separately marked ([org-internal #3745]): `phase-6` narrows to
|
|||
|
|
> Approval + Commit Gate, and the tail References rides `dag-branch` (same
|
|||
|
|
> tail rule as review-code's phase-e). Non-DAG dispatches thereby drop the
|
|||
|
|
> ~175-line DAG branch; DAG milestone/task verify → union `dag-branch` onto
|
|||
|
|
> the close-out family (`"…,phase-6,dag-branch"`) — the branch's override
|
|||
|
|
> clauses take precedence over the unioned phase bodies.
|
|||
|
|
> Sections not listed here (Process Overview, Preconditions, …) are unmarked
|
|||
|
|
> preamble and always inject. Full-body remains the default when no selector
|
|||
|
|
> is passed.
|
|||
|
|
|
|||
|
|
The verification is owned and executed by the **Verifier** (Worker). The Verifier is a dedicated sub-agent that
|
|||
|
|
validates — it does not write code.
|
|||
|
|
|
|||
|
|
The Verifier is responsible for:
|
|||
|
|
|
|||
|
|
- Reading the iteration's DoD matrix from the plan.
|
|||
|
|
- Running the full test suite (unit + integration) for the iteration.
|
|||
|
|
- Validating non-functional requirements against quantified thresholds.
|
|||
|
|
- Running regression tests to confirm no prior iteration is broken.
|
|||
|
|
- Producing a pass/fail verdict per DoD criterion with evidence.
|
|||
|
|
- Reporting failures back for the Developer to fix.
|
|||
|
|
|
|||
|
|
The Orchestrator's role is to present the verification result and, on failure,
|
|||
|
|
route findings back to a new implementation cycle. The Verifier MUST NOT
|
|||
|
|
modify code, tests, or configuration files — only validate and report.
|
|||
|
|
|
|||
|
|
**Context compaction**: verify is a stage boundary in the main session's
|
|||
|
|
pipeline. The orchestrator compacts at this clean boundary ONLY when a
|
|||
|
|
capacity/projection trigger holds, per the L1 rule `core/rules/compact.md`
|
|||
|
|
§"Stage-boundary compaction" (long multi-stage runs — DAG Epic orchestration — keep the legacy
|
|||
|
|
every-boundary compaction; short runs — bugfix / DAG task — and standalone
|
|||
|
|
runs default to NOT compacting). The
|
|||
|
|
Verifier sub-agent itself is single-phase: it persists its verdict to
|
|||
|
|
`{slug}/05-verify-iteration-{N}` as it goes, so a mid-run compaction loses
|
|||
|
|
nothing — re-read the DoD matrix and the verify wiki page to resume (see the
|
|||
|
|
Verifier role's `compact.preserve` list in
|
|||
|
|
`core/skills/_shared/roles/verifier.yaml`).
|
|||
|
|
|
|||
|
|
## Greenfield vs. Brownfield
|
|||
|
|
|
|||
|
|
**Greenfield** (new project): Verify the current iteration's DoD matrix only.
|
|||
|
|
No prior iterations exist.
|
|||
|
|
|
|||
|
|
**Brownfield** (existing project + new feature):
|
|||
|
|
|
|||
|
|
- **Regression is PRIMARY.** Run the FULL test suite (all prior iterations +
|
|||
|
|
new code). A regression failure blocks the iteration regardless of new
|
|||
|
|
feature DoD status.
|
|||
|
|
- If no prior verification report exists (check Gitea wiki for
|
|||
|
|
`{slug}/05-verify-iteration-{N}` pages — this replaces the deprecated
|
|||
|
|
`.artifacts/{slug}/verification/` directory),
|
|||
|
|
run the full test suite first to establish a baseline BEFORE verifying the
|
|||
|
|
new iteration. Baseline failures that pre-date this iteration are **filed as
|
|||
|
|
`baseline-failure` Gitea issues in Phase 5.55** — they do not block the
|
|||
|
|
iteration verdict, but the iteration is NOT Done until every pre-existing
|
|||
|
|
failure is registered (an un-filed baseline failure is a process gap). Only
|
|||
|
|
new regressions (caused by this iteration) block the verdict.
|
|||
|
|
- New NFR thresholds must not degrade existing system metrics. If a new feature
|
|||
|
|
increases P99 latency from 150ms to 180ms (still under 200ms threshold),
|
|||
|
|
log it as a warning — the trend is toward the limit.
|
|||
|
|
- In brownfield mode the Phase 2.1 full-suite run executes FIRST (before DoD
|
|||
|
|
evaluation) so its output doubles as the regression baseline for Phase 2.6 —
|
|||
|
|
still a single execution per tree state (2.6 reuses it; see Phase 2.1).
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Preconditions
|
|||
|
|
|
|||
|
|
Before starting verification, confirm:
|
|||
|
|
|
|||
|
|
- [ ] All work items in the current iteration have passed code review
|
|||
|
|
AND code review has **converged**. Run `octopus review status --stage code`
|
|||
|
|
— the `pipeline/review-code` commit status must show state `success`.
|
|||
|
|
Auto-approve does not guarantee convergence; a review that found
|
|||
|
|
BLOCKERs and stopped before resolution is not converged.
|
|||
|
|
- [ ] Code review final report exists at `{slug}/reviews/code/final/report`
|
|||
|
|
(Gitea wiki page; deprecated: `.artifacts/{slug}/reviews/code/final/report.md`)
|
|||
|
|
and final verdict is PASS (or user-approved WARN with no unresolved
|
|||
|
|
BLOCKERs).
|
|||
|
|
- [ ] The work spec exists. DAG task/milestone mode (the live path — see the
|
|||
|
|
DAG branch below): the frozen DAG copy `{epic-slug}/dag` supplies the
|
|||
|
|
node spec, its `acceptance_criteria` + `test_id` declarations, and the
|
|||
|
|
DoD slice. (Legacy tier rule — `{slug}/04-plan-*` wiki pages / issue
|
|||
|
|
comment, and the design-space + iteration-plan convergence checks that
|
|||
|
|
used to follow here — was archived 2026-08-21, [org-internal #3072] phase 3; historical
|
|||
|
|
plan/AC pages stay readable via `_shared/gitea-read-patterns.md`.)
|
|||
|
|
- [ ] `core/checklists/verification.md` is accessible.
|
|||
|
|
- [ ] `slug` matches the run's slug (DAG: `{epic-slug}`).
|
|||
|
|
- [ ] Upstream dependencies are complete. DAG task mode: the node's
|
|||
|
|
cross-session in-edge source nodes are at terminal state (task `done` /
|
|||
|
|
milestone `green`). If any dependency is not complete → abort, listing
|
|||
|
|
the blocked nodes.
|
|||
|
|
- [ ] Carried items from the previous iteration are closed: every carried
|
|||
|
|
item must have an explicit Target Iteration and must have been handled
|
|||
|
|
in the current iteration or have a fresh deferral record. If any
|
|||
|
|
carried item is not closed → warn and list the open items.
|
|||
|
|
- [ ] Carried risks from the previous iteration are updated: each carried
|
|||
|
|
risk recorded by the prior iteration (historically the plan's Carried
|
|||
|
|
Risks table) is marked `open` or `closed` with a current status. A risk
|
|||
|
|
left `open` without an explicit reactivation note is a gap → warn and
|
|||
|
|
list it.
|
|||
|
|
|
|||
|
|
If the user asks to verify an iteration that still has unreviewed work items,
|
|||
|
|
warn and ask via the `question` tool, passing `stage: "verify"`:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Work items {list} have not passed code review.
|
|||
|
|
All work items in an iteration must be code-reviewed before integration
|
|||
|
|
verification.
|
|||
|
|
→ Run code review first? (yes / no)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The `question` tool auto-approves per `core/rules/auto-approve.md` (stage `"verify"`).
|
|||
|
|
|
|||
|
|
For any other precondition failure (missing artifacts, un-converged reviews,
|
|||
|
|
incomplete dependency chunks, open carried items), abort and inform the user.
|
|||
|
|
Refer to `core/checklists/pipeline-gate.md` — both the Failure Protocol
|
|||
|
|
and the Recovery Protocol — to determine which skill to run next to fix
|
|||
|
|
each missing item.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-1 -->
|
|||
|
|
## Phase 1 — Load Verification Context
|
|||
|
|
|
|||
|
|
> **Pipeline stage**: if the source issue exists, move it to the `verify`
|
|||
|
|
> column on the Pipeline Stages board per `_shared/gitea-write-patterns.md`
|
|||
|
|
> Pattern 7.5. Skip if no source issue exists.
|
|||
|
|
|
|||
|
|
Read the upstream artifacts. DAG task/milestone mode (the live path — see
|
|||
|
|
the DAG branch below for its read map):
|
|||
|
|
|
|||
|
|
1. **Work scope** — the node(s) under verification in the frozen DAG copy
|
|||
|
|
`{epic-slug}/dag` (+ `{epic-slug}/dag-nodes/{node-id}` subpages when AC
|
|||
|
|
detail is sunk) and the node ticket's issue body:
|
|||
|
|
- Node ACs, `size_attrs`, and the milestone DoD slice (milestone mode).
|
|||
|
|
|
|||
|
|
2. **DoD matrix** — the node `acceptance_criteria` with each AC's declared
|
|||
|
|
`test_id` (`{file-path}::{test-name}` / `MANUAL` / `BENCH:<script>` /
|
|||
|
|
`BROWSER:{pack-ref}` for UI-claim browser-evidence rows — see Phase 2.7
|
|||
|
|
and Phase 3, contract `browser-evidence-4486/shared/evidence-ref-v1`);
|
|||
|
|
milestone mode uses the fan-in slice (spec-05 §5).
|
|||
|
|
|
|||
|
|
3. **Requirements** (for reference) — the DAG's requirement registry rows
|
|||
|
|
(`req_refs`) and any `NFR:` entries on the node ACs.
|
|||
|
|
|
|||
|
|
4. **Non-functional targets** — the node's `NFR:` AC entries (product NFRs
|
|||
|
|
live in node AC; there is no separate NFR design page on the DAG route).
|
|||
|
|
|
|||
|
|
(The legacy `Size/*`-tiered locations — `{slug}/04-plan-*`,
|
|||
|
|
`{slug}/02-requirements-index`, `{slug}/02-03-req-design`,
|
|||
|
|
`{slug}/03-design-06-non-functional-design` — were archived 2026-08-21,
|
|||
|
|
[org-internal #3072] phase 3; historical pages stay readable for legacy-run verification
|
|||
|
|
via `_shared/gitea-read-patterns.md`.)
|
|||
|
|
|
|||
|
|
5. **Code review final report** — `{slug}/reviews/code/final/report`
|
|||
|
|
(Gitea wiki page; deprecated: `.artifacts/{slug}/reviews/code/final/report.md`):
|
|||
|
|
- Confirm all issues are resolved and code review verdict is PASS.
|
|||
|
|
|
|||
|
|
**Output**: internal. Produce a verification plan summary:
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
## Verification Plan: Iteration {N}
|
|||
|
|
|
|||
|
|
**DoD entries**: {K} total
|
|||
|
|
|
|||
|
|
- {U} unit-test verifiable
|
|||
|
|
- {I} integration-test verifiable
|
|||
|
|
- {P} performance-test verifiable
|
|||
|
|
- {S} security-scan verifiable
|
|||
|
|
- {M} manual verifiable
|
|||
|
|
|
|||
|
|
**Non-functional targets**: {NF} from plan
|
|||
|
|
**Regression scope**: iterations 0–{N-1}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-2 -->
|
|||
|
|
## Phase 2 — Execute Verification
|
|||
|
|
|
|||
|
|
Run verification commands from the relevant package directory (per `rules/testing.md` — do NOT run tests from repo root). Capture all output.
|
|||
|
|
|
|||
|
|
### 2.0 — Reuse Same-SHA CI Results (fail-closed)
|
|||
|
|
|
|||
|
|
Mechanical checks Gitea CI has already gated on this exact code state must
|
|||
|
|
not be re-run locally. If a PR exists for the iteration branch:
|
|||
|
|
|
|||
|
|
1. Resolve the PR head SHA and read that SHA's commit statuses (the read
|
|||
|
|
counterpart of `_shared/gitea-write-patterns.md` Pattern 8:
|
|||
|
|
`GET /api/v1/repos/Octopus/octopus/commits/{sha}/statuses`).
|
|||
|
|
2. For Phase 2.2 (typecheck), 2.3 (lint), and 2.5 (security scan): if the
|
|||
|
|
corresponding same-SHA CI status/job is `success` (e.g. the `CI` workflow's
|
|||
|
|
`check` job, the SAST workflow), cite it as the DoD evidence (record the
|
|||
|
|
job name + SHA in the report) and SKIP the local re-run.
|
|||
|
|
3. A `pending`, `failure`, or missing status is NOT evidence — run the check
|
|||
|
|
locally instead. Do not block waiting for a pending CI run.
|
|||
|
|
4. Phase 2.1 (full suite) is never skipped this way: PR CI runs `test:changed`
|
|||
|
|
(incremental), so the full `test:parallel` execution stays local unless a
|
|||
|
|
same-SHA `run-full-tests` run exists.
|
|||
|
|
|
|||
|
|
If no PR exists (or statuses are stale/missing), run everything locally as
|
|||
|
|
before. This section saves redundant typecheck/lint/security re-runs; it
|
|||
|
|
never weakens a gate.
|
|||
|
|
|
|||
|
|
**Second evidence source — the review-code precondition-gate cache
|
|||
|
|
(eff-gate-cache [org-internal #2598] / [org-internal #2990])**. The mechanical checks this phase would re-run
|
|||
|
|
(oxlint / typecheck / `test:changed`) are exactly the precondition-gate's
|
|||
|
|
check-set, and the gate caches GREEN results keyed on a tree fingerprint
|
|||
|
|
(HEAD + tracked diff + untracked contents) in
|
|||
|
|
`<runs-root>/<slug>/reviews/code/precondition-gate.jsonl` (Tier 1, same
|
|||
|
|
worktree). Consume it by RUNNING the gate script — it computes the
|
|||
|
|
fingerprint, reuses the green record on a cache hit, and appends an audit
|
|||
|
|
record; never re-derive the fingerprint by hand:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
bun core/skills/review-code/scripts/precondition-gate.ts <slug>
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
5. Exit 0 with `GREEN (cache hit …)` — the identical tree was already proven
|
|||
|
|
mechanically green by review-code's gate. Cite the record (slug +
|
|||
|
|
fingerprint timestamp from the output) as the DoD evidence for Phase 2.2
|
|||
|
|
(typecheck) and 2.3 (lint) and skip their local re-runs — the same
|
|||
|
|
fail-closed reuse as the CI clauses above.
|
|||
|
|
6. Exit 0 with a FRESH green (fingerprint differs — the tree changed after
|
|||
|
|
review-code, e.g. review fixes landed): this run IS the single local
|
|||
|
|
mechanical execution for the new tree state; its record now serves any
|
|||
|
|
later consumer. Cite it the same way.
|
|||
|
|
7. Exit 1 (RED) or the gate log absent (different worktree — Tier 1 is
|
|||
|
|
worktree-local): NOT evidence — follow the normal local path above. After
|
|||
|
|
an environment change (bun version, fresh node_modules) pass `--no-cache`
|
|||
|
|
so a stale green can never be reused across environments.
|
|||
|
|
|
|||
|
|
Phase 2.1 (full suite) is never skipped by the gate cache either — the gate
|
|||
|
|
deliberately excludes `test:parallel` (the full suite belongs to verify). The
|
|||
|
|
gate fingerprint is keyed on the working tree (SHA + uncommitted state +
|
|||
|
|
untracked), so it covers pre-push/headless states the same-SHA CI clauses
|
|||
|
|
cannot.
|
|||
|
|
|
|||
|
|
### 2.1 Full Test Suite
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
bun run test:parallel # or project-equivalent
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- **Single execution per tree state**: run the full suite ONCE. The captured
|
|||
|
|
output serves Phase 2.6 (regression classification) as well — do not re-run
|
|||
|
|
the suite there. Re-run only if code changed since this run (fixes landed
|
|||
|
|
mid-verification).
|
|||
|
|
- All tests must pass. Any failure is a DoD failure.
|
|||
|
|
- If the project uses separate unit and integration test commands, run both.
|
|||
|
|
- **Detect flaky tests.** Re-run ONLY the tests that failed or look
|
|||
|
|
timing-sensitive (targeted reruns via
|
|||
|
|
`cd <harness-package> && bun test --timeout 120000 <file>`,
|
|||
|
|
per checklist item 2.4's isolated-serial allowance) — not the whole suite.
|
|||
|
|
Checklist 2.4: modules using `git worktree add/remove` may hang
|
|||
|
|
`test:parallel` on git internal file locks (parallel git contention); the
|
|||
|
|
isolated/serial rerun above with `--timeout 120000` is the accepted
|
|||
|
|
substitute and must be annotated TST-GIT-CONTENTION. Record each flaky
|
|||
|
|
test name; new flaky tests introduced this iteration are routed to the
|
|||
|
|
Developer for root-cause, and every flaky test NOT fixed in-iteration is
|
|||
|
|
filed in Phase 5.56.
|
|||
|
|
|
|||
|
|
### 2.2 Typecheck
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
bun typecheck
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- Zero type errors. Type errors after code review indicate a merge or
|
|||
|
|
integration issue.
|
|||
|
|
|
|||
|
|
### 2.3 Lint
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
bun oxlint --deny-warnings
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- Zero lint errors. The canonical invocation is `bun oxlint --deny-warnings`
|
|||
|
|
at the repo root, mirroring ci.yml — `bun lint` is the package-script alias.
|
|||
|
|
Warnings are noted but do not block DoD unless the project gates on them.
|
|||
|
|
|
|||
|
|
### 2.4 Performance Validation
|
|||
|
|
|
|||
|
|
For each DoD entry with verification method "Performance test":
|
|||
|
|
|
|||
|
|
1. **Baseline**: if prior iteration performance data exists (brownfield mode),
|
|||
|
|
load it as the baseline. If no baseline exists (greenfield), run the
|
|||
|
|
performance test suite twice and use the average as the baseline for
|
|||
|
|
future iterations.
|
|||
|
|
2. **Run performance tests**: execute the project's performance test suite.
|
|||
|
|
Capture P50, P95, P99, and throughput for each critical path.
|
|||
|
|
3. **Compare**: compare against the quantified targets from the design
|
|||
|
|
document AND against the baseline. A regression below the threshold but
|
|||
|
|
trending toward it (>10% degradation) is a ⚠️ WARNING, not a FAIL.
|
|||
|
|
4. **Profile if failing**: if a performance target is exceeded, run a profiler
|
|||
|
|
(e.g. `clinic`, `0x`, `py-spy`, `pprof`) on the critical path and identify
|
|||
|
|
the bottleneck. Include the profile hotspot in the failure report.
|
|||
|
|
5. If no automated performance tests exist, identify the gap and mark the
|
|||
|
|
criterion as `⚠️ UNVERIFIABLE — no automated test`. Suggest a specific
|
|||
|
|
performance test framework or benchmark library for the project's
|
|||
|
|
technology stack.
|
|||
|
|
|
|||
|
|
### 2.5 Security Validation
|
|||
|
|
|
|||
|
|
For each DoD entry with verification method "Security scan":
|
|||
|
|
|
|||
|
|
1. Run the project's security scanner (e.g. `npm audit`, SAST tool).
|
|||
|
|
2. Check for new vulnerabilities introduced in this iteration.
|
|||
|
|
3. If no automated security scan exists, mark as `⚠️ UNVERIFIABLE`.
|
|||
|
|
|
|||
|
|
### 2.6 Regression Check
|
|||
|
|
|
|||
|
|
1. Reuse the Phase 2.1 full-suite output (same tree, same run) — the suite
|
|||
|
|
covers tests from all prior iterations by construction. Re-run the full
|
|||
|
|
suite ONLY if code changed since Phase 2.1 (fixes landed mid-verification).
|
|||
|
|
2. Confirm zero regressions — no previously passing test now fails.
|
|||
|
|
3. If regressions are found, report each with the failing test name and
|
|||
|
|
likely cause (from git diff since last successful run).
|
|||
|
|
4. **Classify each failure as regression vs pre-existing (brownfield).** For
|
|||
|
|
every failing test, run it on the last known-green commit (or the base
|
|||
|
|
branch) in isolation. **Never use `git stash` for this** — the stash stack
|
|||
|
|
is shared across all worktrees of this repo and a parallel session can
|
|||
|
|
overwrite your entry mid-flight (issue [org-internal #1655]). Instead, use the
|
|||
|
|
lifecycle-managed scratch-worktree script ([org-internal #2790]) — it registers the
|
|||
|
|
worktree for the sweep backstop, symlinks `node_modules` from the source
|
|||
|
|
checkout when `bun.lock` matches (dropping the per-instance cost from
|
|||
|
|
~170k inodes to ~7k), and deregisters on removal:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
bash script/scratch-worktree.sh create <slug>-base <base> # base / last-green commit
|
|||
|
|
(cd /tmp/octopus/<slug>-base/<harness-package> && bun test --timeout 120000 <failing-test>)
|
|||
|
|
bash script/scratch-worktree.sh remove <slug>-base # cleanup when done
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
When `create` symlinks `node_modules` (its output says so) the test run
|
|||
|
|
reuses the source checkout's install directly — do NOT run `bun install`
|
|||
|
|
in the scratch worktree (it would write through the symlink into the
|
|||
|
|
shared `node_modules`). Only if `create` printed the "bun.lock differs"
|
|||
|
|
note run a real `bun install --silent` there first.
|
|||
|
|
|
|||
|
|
The current iteration worktree stays dirty and untouched; the scratch
|
|||
|
|
worktree runs the baseline in isolation. If the failing test needs
|
|||
|
|
uncommitted changes from the iteration branch to even _run_, WIP-commit
|
|||
|
|
them on a throwaway branch (`git switch -c wip/<slug>-base-check && git
|
|||
|
|
commit -am "wip: baseline probe"`) and cherry-pick that commit into the
|
|||
|
|
scratch worktree instead (create it with `--base` pointing at a throwaway
|
|||
|
|
branch the commit is cherry-picked onto).
|
|||
|
|
- **Fails on base too** → **pre-existing / baseline failure (BF)**. Does
|
|||
|
|
NOT block the verdict; MUST be filed in Phase 5.55.
|
|||
|
|
- **Passes on base, fails here** → **regression** (caused by this
|
|||
|
|
iteration). BLOCKS the verdict; route to the Developer.
|
|||
|
|
|
|||
|
|
Abandoned scratch worktrees (session died before `remove`) are reclaimed
|
|||
|
|
by the `sweep` backstop — see `script/scratch-worktree.sh` header and
|
|||
|
|
`deploy/systemd/octopus-scratch-sweep.service`.
|
|||
|
|
|
|||
|
|
Never silently drop a pre-existing failure as "someone else's problem" —
|
|||
|
|
every BF becomes a tracked `baseline-failure` issue or the iteration is
|
|||
|
|
not Done.
|
|||
|
|
|
|||
|
|
### 2.7 — Browser Evidence Reference Validation (UI-touching PR)
|
|||
|
|
|
|||
|
|
For DoD entries of the「UI 主张 ↔ 可复现浏览器证据」type — contract
|
|||
|
|
`browser-evidence-4486/shared/evidence-ref-v1` (frozen 2026-09-10, [org-internal #4498]).
|
|||
|
|
Applicable ONLY to UI-touching PRs (changes under `packages/app`,
|
|||
|
|
`packages/ui`, or `packages/console/*` frontend surfaces):
|
|||
|
|
|
|||
|
|
1. Validate each `BROWSER:{pack-ref}` reference mechanically
|
|||
|
|
(`pack-ref` = `<runs-root>/{slug}/browser/{session-id}`, the manifest's
|
|||
|
|
directory):
|
|||
|
|
`bun <harness-package>/scripts/browser-evidence-ref.ts {pack-ref}`.
|
|||
|
|
Exit 0 / `status: ok` = manifest parses with the pack-manifest-v1 shape,
|
|||
|
|
the pack is REGISTERED in {slug}/index.json (browser-evidence row), and it
|
|||
|
|
carries a trace (replayable). Every other status — `unregistered`
|
|||
|
|
(no such dir, not a .../browser/{session-id} layout, or no index row),
|
|||
|
|
`manifest-missing`, `manifest-corrupt`, `not-replayable` (no trace),
|
|||
|
|
`over-limit` — is an explicit failure: the
|
|||
|
|
DoD row is ⚠️ UNVERIFIABLE at best (❌ FAIL when the evidence itself was
|
|||
|
|
the deliverable), never silently green.
|
|||
|
|
2. Summary line for the DoD Evidence column / PR description (contract §2):
|
|||
|
|
`UI 主张 {claim} ↔ {pack-ref}(replay {succeeded}/{attempted},env {mode.resolved})`
|
|||
|
|
— rendered from the pack manifest only; the publisher never touches raw
|
|||
|
|
captures (sanitize boundary is upstream, `shared/pack-manifest-v1` §5).
|
|||
|
|
3. Tier-2 visibility: at most 5 screenshots / 200KB total may be published
|
|||
|
|
to the wiki evidence namespace (`{slug}/verify/evidence/{name}`, existing
|
|||
|
|
convention); the pack itself stays Tier 1 and rides archived-at-close
|
|||
|
|
into git.
|
|||
|
|
4. A pack without a trace is not replayable and MUST NOT produce a green
|
|||
|
|
DoD row for a replay claim.
|
|||
|
|
5. Non-UI-touching PRs do not carry this entry type at all — a BROWSER row
|
|||
|
|
on a backend-only change is a DoD-matrix error.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-3 -->
|
|||
|
|
## Phase 3 — DoD Matrix Evaluation
|
|||
|
|
|
|||
|
|
For every DoD entry, produce a verdict:
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
| # | REQ ID | Acceptance Criterion | Verification Method | Verdict | Evidence |
|
|||
|
|
| --- | ---------- | ------------------------------- | ------------------- | --------------- | ---------------------------------------------------------- |
|
|||
|
|
| 1 | REQ-F-001 | User can register with email+pw | Integration test | ✅ PASS | `test/auth/register.test.ts:42` passes |
|
|||
|
|
| 2 | REQ-F-001 | Invalid email returns 400 | Unit test | ✅ PASS | `test/auth/register.test.ts:67` passes |
|
|||
|
|
| 3 | REQ-NF-005 | Login P99 < 200ms | Performance test | ❌ FAIL | P99 measured at 312ms (target: 200ms) |
|
|||
|
|
| 4 | REQ-F-003 | JWT expiry at 1h | Unit test | ⚠️ UNVERIFIABLE | No test found; code review confirmed but no automated test |
|
|||
|
|
| 5 | REQ-F-002 | 登录页视觉回归可回放证据 | Browser evidence | ✅ PASS | UI 主张 登录页视觉回归 ↔ <runs-root>/{slug}/browser/{session-id}(replay 3/3,env headed)|
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Verdict Rules
|
|||
|
|
|
|||
|
|
| Verdict | Meaning | Blocks Iteration? |
|
|||
|
|
| --------------- | ---------------------------------------- | ---------------------------- |
|
|||
|
|
| ✅ PASS | Criterion verified with passing evidence | No |
|
|||
|
|
| ❌ FAIL | Criterion verified and fails | YES |
|
|||
|
|
| ⚠️ UNVERIFIABLE | No automated test exists to verify | No (but logged as tech debt) |
|
|||
|
|
|
|||
|
|
Browser-evidence rows (UI-touching PRs only, Phase 2.7): ✅ PASS requires a
|
|||
|
|
mechanically validated, replayable pack-ref; `unregistered` / `manifest-missing` /
|
|||
|
|
`manifest-corrupt` / `not-replayable` / `over-limit` references are
|
|||
|
|
⚠️ UNVERIFIABLE at best — the reference channel fails explicitly, never
|
|||
|
|
silently green ([org-internal #4498] AC-4).
|
|||
|
|
|
|||
|
|
### Iteration Verdict
|
|||
|
|
|
|||
|
|
The iteration verdict is computed from the DoD matrix:
|
|||
|
|
|
|||
|
|
- **PASS**: 100% of verifiable criteria are ✅ PASS. UNVERIFIABLE entries do
|
|||
|
|
not block.
|
|||
|
|
- **FAIL**: One or more criteria are ❌ FAIL.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-4 -->
|
|||
|
|
## Phase 4 — Non-Functional Validation
|
|||
|
|
|
|||
|
|
For each non-functional requirement target in the current iteration:
|
|||
|
|
|
|||
|
|
````markdown
|
|||
|
|
| NFR ID | Target | Measured | Threshold | Verdict | Notes |
|
|||
|
|
| ---------- | ------------------------ | -------- | --------- | ------- | --------------------- |
|
|||
|
|
| REQ-NF-005 | Login P99 latency | 312ms | < 200ms | ❌ FAIL | Exceeds target by 56% |
|
|||
|
|
| REQ-NF-007 | Registration P99 latency | 145ms | < 200ms | ✅ PASS | |
|
|||
|
|
| REQ-NF-010 | Test coverage | 87% | ≥ 80% | ✅ PASS | |
|
|||
|
|
````
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-5 -->
|
|||
|
|
## Phase 5 — Report
|
|||
|
|
|
|||
|
|
Produce the verification report:
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
# Iteration Verification Report
|
|||
|
|
|
|||
|
|
**Project**: {slug}
|
|||
|
|
**Iteration**: {N}: {Goal}
|
|||
|
|
**Date**: YYYY-MM-DD
|
|||
|
|
|
|||
|
|
## Verdict: {PASS | FAIL}
|
|||
|
|
|
|||
|
|
## DoD Matrix Summary
|
|||
|
|
|
|||
|
|
| Status | Count |
|
|||
|
|
| --------------- | ----- |
|
|||
|
|
| ✅ PASS | {N} |
|
|||
|
|
| ❌ FAIL | {N} |
|
|||
|
|
| ⚠️ UNVERIFIABLE | {N} |
|
|||
|
|
| **Total** | {N} |
|
|||
|
|
|
|||
|
|
## Failed Criteria
|
|||
|
|
|
|||
|
|
| # | REQ ID | Criterion | Measured | Expected | Gap |
|
|||
|
|
| --- | ---------- | ----------------- | -------- | -------- | ---- |
|
|||
|
|
| 3 | REQ-NF-005 | Login P99 < 200ms | 312ms | 200ms | +56% |
|
|||
|
|
|
|||
|
|
## NFR Summary
|
|||
|
|
|
|||
|
|
- {N}/{M} thresholds passed
|
|||
|
|
- {F} thresholds violated
|
|||
|
|
|
|||
|
|
## Unverifiable Criteria (Tech Debt)
|
|||
|
|
|
|||
|
|
| REQ ID | Criterion | Reason | Reactivation Trigger | Issue |
|
|||
|
|
| --------- | ---------------- | ------------------------ | -------------------- | ----- |
|
|||
|
|
| REQ-F-003 | JWT expiry at 1h | No automated test exists | {可客观判断的条件} | #NNNN |
|
|||
|
|
|
|||
|
|
> Every row is registered as a `## TD 登记` registry row in Phase 5.5
|
|||
|
|
> (`core/rules/ticket-lifecycle.md`). The `Issue` column records the
|
|||
|
|
> mapping produced there: `TD-NNN → 登记行(source #NNNN)` while unpromoted,
|
|||
|
|
> `TD-NNN → #NNNN` once promoted.
|
|||
|
|
|
|||
|
|
## Regression Status
|
|||
|
|
|
|||
|
|
- Prior iteration tests: {N} passing, {M} failing (regressions)
|
|||
|
|
- Non-regression check: {PASS | FAIL — {list}}
|
|||
|
|
|
|||
|
|
## Baseline Failures (Pre-existing)
|
|||
|
|
|
|||
|
|
| # | Test | BF ID | Issue | Severity | Root cause (if known) |
|
|||
|
|
| --- | ---------------------------- | ------ | ----- | -------- | ---------------------------------------------- |
|
|||
|
|
| 1 | `test/foo/bar.test.ts::name` | BF-NNN | #NNNN | MEDIUM | pre-dates this iteration; fails on base commit |
|
|||
|
|
|
|||
|
|
> Every pre-existing failure classified in Phase 2.6 is filed against (or
|
|||
|
|
> into) a root-cause family umbrella in Phase 5.55. The `BF-NNN → #NNNN`
|
|||
|
|
> mapping names the FAMILY umbrella; sibling occurrences attach as evidence
|
|||
|
|
> comments on it. If this section is empty, record "0 baseline failures".
|
|||
|
|
|
|||
|
|
## Flaky Tests (Intermittent)
|
|||
|
|
|
|||
|
|
| # | Test | FT ID | Issue | Severity | Status |
|
|||
|
|
| --- | ---------------------------- | ------ | ----- | -------- | ----------------------------- |
|
|||
|
|
| 1 | `test/foo/bar.test.ts::name` | FT-NNN | #NNNN | MEDIUM | intermittent (3/10 runs fail) |
|
|||
|
|
|
|||
|
|
> Every flaky test not fixed in-iteration is filed against (or into) a
|
|||
|
|
> root-cause family umbrella in Phase 5.56. If this section is empty, record
|
|||
|
|
> "0 flaky tests".
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Verification Commands Executed
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
bun run test:parallel → N passed, M failed
|
|||
|
|
bun typecheck → {pass/fail}
|
|||
|
|
bun oxlint --deny-warnings → {pass/fail}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Next Steps
|
|||
|
|
|
|||
|
|
- If PASS: iteration is complete. Ready for next iteration.
|
|||
|
|
- If FAIL: route failed criteria to Developer. Re-run `review-code` on fixes,
|
|||
|
|
then re-run verification.
|
|||
|
|
|
|||
|
|
Write the report as a Gitea wiki page using `wiki 读写 API(见 TERMINOLOGY)` with
|
|||
|
|
title `{slug}/05-verify-iteration-{N}` (replaces the deprecated
|
|||
|
|
`.artifacts/{slug}/verification/iteration-{N}/report.md` path).
|
|||
|
|
|
|||
|
|
Then update the `## 工件索引` comment on the source issue per
|
|||
|
|
`_shared/gitea-write-patterns.md` Pattern 10: add the `VERIFY-{N}` row
|
|||
|
|
(位置 = the report page, 重读 = ARCHIVE — closed-iteration reports are never
|
|||
|
|
re-read for context recovery). Per-row ARCHIVE here marks this iteration's
|
|||
|
|
report; the **archive-at-close action** (banner + all rows → ARCHIVE at
|
|||
|
|
issue close) is performed later in Phase 5.6 — see
|
|||
|
|
`rules/issue-checklist-sync.md` § 归档动作.
|
|||
|
|
|
|||
|
|
<!-- section:phase-5.5 -->
|
|||
|
|
### Phase 5.5 — Tech Debt Registry Filing (mandatory before PASS)
|
|||
|
|
|
|||
|
|
Tech debt is filed **registry-first** (`core/rules/ticket-lifecycle.md`,
|
|||
|
|
[org-internal #3002] G1): new items become ROWS in the source ticket's `## TD 登记` comment;
|
|||
|
|
an independent ticket is created only at **promotion** (claim/schedule). There
|
|||
|
|
is no `tech-debt.md` file and no `{slug}/tech-debt` wiki page — both remain
|
|||
|
|
retired. The registry preserves the three guarantees of the old
|
|||
|
|
one-issue-per-item model: visible (the comment lives on a tracker ticket),
|
|||
|
|
individually closeable (row-level checkbox), de-duplicated across sessions
|
|||
|
|
(the TD index is stronger than title-keyword matching).
|
|||
|
|
|
|||
|
|
**Sprint-mode override ([org-internal #3002] G5/[org-internal #3007])**: read `filing.sprint-mode` from
|
|||
|
|
`<instance-root>/workflow-routing.yaml` at the START of this phase (no caching). When
|
|||
|
|
`true`, ALL severities file registry-only — registration rows are written as
|
|||
|
|
normal (severity included, nothing lost) but promotion to standalone tickets
|
|||
|
|
is FROZEN until the switch returns to `false`. Rationale: the 2026-08-12
|
|||
|
|
sprint zeroed fix capacity while filing kept running (the 125-fragment
|
|||
|
|
backlog's direct trigger); an explicit debt-side downshift beats hoping for
|
|||
|
|
discipline. BF/FT phases 5.55/5.56 are NOT covered by this switch — failure
|
|||
|
|
evidence must stay immediately visible. Flipping the switch is a human
|
|||
|
|
decision recorded on the sprint tracker; semantics in
|
|||
|
|
`rules/workflow-routing.md` §立案降档.
|
|||
|
|
|
|||
|
|
**Adjacency quota ([org-internal #3002] G3/[org-internal #3005])**: while collecting inputs, also check the
|
|||
|
|
AREAS this iteration touched (packages/modules with commits this iteration).
|
|||
|
|
If the source tickets for those areas carry applicable `## TD 登记` rows, at
|
|||
|
|
least ONE row must be taken along and disposed this iteration (fixed, or
|
|||
|
|
explicitly re-deferred with a reason — silence is not a disposition). Record
|
|||
|
|
`consumed X / applicable Y` (or `0 applicable`) in the TD report section;
|
|||
|
|
verification checklist §8.4 cross-checks it against the run's debt-quota
|
|||
|
|
record (historically the roadmap's per-chunk debt-quota line). This gives debt digestion a steady capacity source tied to
|
|||
|
|
touch, not to cleanup campaigns.
|
|||
|
|
|
|||
|
|
**Promotion quota + pull event ([org-internal #3061])**: promotion to a standalone ticket
|
|||
|
|
requires (a) a **pull event** — the row's Reactivation Trigger fired (state
|
|||
|
|
the observed event) OR a named claimer/schedule; "it seemed important" is not
|
|||
|
|
a pull event — recorded in the promotion body's `## 拉动事件 / Pull Event`
|
|||
|
|
section; and (b) the origin module's OPEN promoted-TD count is below the
|
|||
|
|
quota (**default 10** — counted **per origin module**: resolve the row's
|
|||
|
|
module from the ticket body's `Origin` field, then count that module's OPEN
|
|||
|
|
`tech-debt` tickets; retro §2.10 reports breaches). At or above quota →
|
|||
|
|
register-only until the count drops; before each Phase 5.5 promotion
|
|||
|
|
re-check the owning module's count, and a row whose module is at or above
|
|||
|
|
quota is NOT promoted — defer it and annotate the registry row with
|
|||
|
|
`quota-held` (re-checked on the next verify run, retro 2026-08-21). A row
|
|||
|
|
marked `[COLD]` by retrospective §2.10
|
|||
|
|
(unclaimed >3 retro cycles) promotes only after the pull event clears the
|
|||
|
|
mark.
|
|||
|
|
|
|||
|
|
**Collect inputs** — every tech-debt item to register this iteration:
|
|||
|
|
|
|||
|
|
1. Every UNVERIFIABLE row in the DoD matrix (Phase 3), with its Reactivation
|
|||
|
|
Trigger.
|
|||
|
|
2. Every ACCEPTED_RISK finding in the code-review final report
|
|||
|
|
(`{slug}/reviews/code/final/report`), with its Recommended Reactivation
|
|||
|
|
Trigger.
|
|||
|
|
3. Every `[OPEN: <short-id>]` marker the Developer left in code this iteration
|
|||
|
|
(per implementation checklist POST-6.2).
|
|||
|
|
|
|||
|
|
If the code-review final report does not exist, warn the user — ACCEPTED_RISK
|
|||
|
|
items cannot be captured. This is itself a process gap that must be flagged.
|
|||
|
|
|
|||
|
|
**Step 1 — De-duplicate against the registry index and promoted issues.**
|
|||
|
|
|
|||
|
|
1. Read the source ticket's `## TD 登记` comment (create the comment if this
|
|||
|
|
is its first row) and match each candidate against existing rows by
|
|||
|
|
`TD-NNN` and reactivation trigger. Same-source re-reporting is the
|
|||
|
|
anti-pattern this step exists to prevent (the visual-cluster incident: 18
|
|||
|
|
fragments from one source under the old keyword matching).
|
|||
|
|
2. Query OPEN promoted tech-debt issues for cross-source duplicates:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
工单 API 检索(见 TERMINOLOGY)
|
|||
|
|
(owner/repo 按实例绑定,labels="tech-debt", state="open")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Step 2 — Assign `TD-NNN` via the ledger mutex ([org-internal #3322]).** One script call
|
|||
|
|
per batch: `bash script/td-alloc.sh <count = new items> <source-slug>
|
|||
|
|
[--ticket N]` returns a contiguous range from the TD allocation ledger
|
|||
|
|
(append-only comments, earliest-comment-id wins; the seed comment pins the
|
|||
|
|
historical floor). Split the returned range across the Step 3 rows in order.
|
|||
|
|
**Any allocation — manual included — must land a `**td-alloc**` comment on
|
|||
|
|
the ledger before the registry row is written; hand-computed `max + 1` is
|
|||
|
|
forbidden** (the 2026-08-24 three-way collision). On exit 5 (retry
|
|||
|
|
exhaustion) abort this phase's filing and retry — never hand-assign. The id is **reserved for life**: promotion reuses it, never
|
|||
|
|
renumbers. Run `--check` (reconciliation) here if the last check predates
|
|||
|
|
this batch.
|
|||
|
|
|
|||
|
|
**Step 3 — Append one registry row per new item.** Append to the source
|
|||
|
|
ticket's `## TD 登记` comment (no source ticket → the owning Epic's tracker
|
|||
|
|
ticket; no Epic → the standing `TD 冷存档` tracker):
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
- [ ] TD-{NNN} · {UNVERIFIABLE | ACCEPTED_RISK | DEFERRED} · {HIGH | MEDIUM | LOW}
|
|||
|
|
· {origin chunk-id/iter-N} — {one-line summary}
|
|||
|
|
· trigger: {objectively verifiable reactivation condition — never "when ready";
|
|||
|
|
if no trigger can be defined this is NOT tech debt and the iteration verdict
|
|||
|
|
becomes FAIL}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
**Step 4 — Cross-link (adapted).** The registry comment already lives on the
|
|||
|
|
source issue — the source → debt navigation the old body-edit provided. When
|
|||
|
|
filing went to an Epic tracker or the 冷存档 (no source ticket), reference the
|
|||
|
|
originating slug in each row so the trail survives.
|
|||
|
|
|
|||
|
|
**Step 5 — Record the mapping in the verification report.** The report's
|
|||
|
|
"Tech Debt" section records `TD-NNN → registry row` (source `#NNNN` +
|
|||
|
|
`## TD 登记` comment); promoted items record `TD-NNN → #NNNN` as before.
|
|||
|
|
|
|||
|
|
**Promotion (认领升票) — when a row is scheduled or picked up.** Create the
|
|||
|
|
independent ticket THEN, carrying the reserved `TD-NNN`:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
gitea_issue__create(owner="Octopus", repo="octopus",
|
|||
|
|
title="[{origin}] Tech debt: {short description} (TD-{NNN})",
|
|||
|
|
body=<promotion template below>,
|
|||
|
|
labels=[tech_debt_label_id, severity_label_id])
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- **Severity → label** (reuse the existing exclusive Priority set):
|
|||
|
|
|
|||
|
|
| Severity | Label |
|
|||
|
|
| -------- | ----------------- |
|
|||
|
|
| HIGH | `Priority/High` |
|
|||
|
|
| MEDIUM | `Priority/Medium` |
|
|||
|
|
| LOW | `Priority/Low` |
|
|||
|
|
|
|||
|
|
- **Promotion body template** (the pre-[org-internal #3002] issue template, now used only at
|
|||
|
|
promotion):
|
|||
|
|
|
|||
|
|
```markdown
|
|||
|
|
## Parent / 父级
|
|||
|
|
|
|||
|
|
{#source_issue carrying the ## TD 登记 row}
|
|||
|
|
|
|||
|
|
## Summary
|
|||
|
|
|
|||
|
|
{one-paragraph description of the debt}
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
| -------- | ------------------------------------------- |
|
|||
|
|
| TD ID | TD-{NNN} (reserved at registration) |
|
|||
|
|
| Type | {UNVERIFIABLE \| ACCEPTED_RISK \| DEFERRED} |
|
|||
|
|
| Origin | {chunk-id/iter-N/phase or slug} |
|
|||
|
|
| Severity | {HIGH \| MEDIUM \| LOW} |
|
|||
|
|
|
|||
|
|
## Reactivation Trigger
|
|||
|
|
|
|||
|
|
{from the registry row}
|
|||
|
|
|
|||
|
|
## 拉动事件 / Pull Event
|
|||
|
|
|
|||
|
|
{trigger fired: {observed event} | claimed by {who} for {when}}
|
|||
|
|
|
|||
|
|
## Acceptance Criteria (how to close this issue)
|
|||
|
|
|
|||
|
|
- [ ] {specific action that resolves the debt}
|
|||
|
|
|
|||
|
|
## Context
|
|||
|
|
|
|||
|
|
- Verification report: `{slug}/05-verify-iteration-{N}`
|
|||
|
|
- Code review report: `{slug}/reviews/code/final/report` (if ACCEPTED_RISK)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- After creating the promoted ticket: tick the registry row (`- [x]`) and
|
|||
|
|
append `→ #NNNN` to it.
|
|||
|
|
|
|||
|
|
If no new tech-debt items were found this iteration, the phase is a no-op —
|
|||
|
|
record "0 tech-debt items" in the report and proceed.
|
|||
|
|
|
|||
|
|
<!-- section:phase-5.55 -->
|
|||
|
|
### Phase 5.55 — Baseline-Failure Issue Registration (mandatory before PASS when pre-existing failures exist)
|
|||
|
|
|
|||
|
|
Pre-existing test failures (BF) are tracked as **root-cause family umbrella
|
|||
|
|
tickets** (`core/rules/ticket-lifecycle.md`, [org-internal #3002] G1; precedents [org-internal #2585]
|
|||
|
|
umbrella + [org-internal #2925] cluster) — one `BF-NNN` ticket per failure-signature family,
|
|||
|
|
labeled `baseline-failure`, with each occurrence's evidence attached as
|
|||
|
|
comments. There is no `baseline-failures.md` file and no
|
|||
|
|
`{slug}/baseline-failures` wiki page. A pre-existing failure does not block
|
|||
|
|
the iteration that _discovered_ it (it pre-dates that work), but it MUST be
|
|||
|
|
filed against (or into) a family; an un-filed baseline failure is a process
|
|||
|
|
gap and the iteration is NOT Done.
|
|||
|
|
|
|||
|
|
**Birth triage SLA ([org-internal #3061])**: a BF umbrella is born with `Kind/Bug` (it IS a
|
|||
|
|
bug — of the baseline) and must be triaged the SAME DAY it is filed:
|
|||
|
|
assigned, scheduled, or explicitly deferred with a reason on the ticket.
|
|||
|
|
Untriaged-next-day BF umbrellas are named individually in retro §2.10
|
|||
|
|
(compliance %), not averaged away.
|
|||
|
|
|
|||
|
|
**Collect inputs** — every pre-existing failure to register this iteration:
|
|||
|
|
|
|||
|
|
1. Every failure classified **pre-existing** in Phase 2.6 (fails on the base
|
|||
|
|
branch / last-green commit too).
|
|||
|
|
2. Any baseline-run failures noted in the brownfield pre-check (Greenfield vs.
|
|||
|
|
Brownfield section).
|
|||
|
|
|
|||
|
|
**Step 1 — De-duplicate by failure SIGNATURE against existing families (open
|
|||
|
|
AND closed), per `core/rules/testing.md`** (de-duplicate BEFORE filing,
|
|||
|
|
retro [org-internal #2696]). Query `baseline-failure` issues in ALL states:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
gitea_issue__list(owner="Octopus", repo="octopus",
|
|||
|
|
labels="baseline-failure", state="all")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Match each candidate by **failure signature** — the error/assertion signature
|
|||
|
|
plus the affected surface, not just `file::test-name` (one root-cause family
|
|||
|
|
may span several tests; several tests failing with the same assertion shape
|
|||
|
|
are ONE family). On a signature hit, do NOT create a new ticket: attach the
|
|||
|
|
new run evidence (run, commit, failing output, sibling tests observed) as a
|
|||
|
|
comment on the FAMILY umbrella ticket. Filing a second ticket inside a known
|
|||
|
|
root-cause family is the anti-pattern this phase exists to prevent. If a
|
|||
|
|
tracked family is now FIXED by this iteration's work, close its umbrella with
|
|||
|
|
a comment citing the fixing commit (a bonus, not a requirement). If the query
|
|||
|
|
itself errors (API failure — not "no matches"), retry once; on the second
|
|||
|
|
failure proceed to file with the literal marker `dedup-unverified` in the
|
|||
|
|
umbrella's first evidence comment (query-failure ruling, [org-internal #3383] — fail-open,
|
|||
|
|
visible; rationale in `core/rules/ticket-lifecycle.md`).
|
|||
|
|
|
|||
|
|
**Step 2 — Assign the next `BF-NNN`** per `core/rules/testing.md`
|
|||
|
|
§ "Pre-existing test failures must be filed, not silently logged" (the L1 rule
|
|||
|
|
owns the repo-wide sequential numbering invariant — `max + 1`, duplicate ids
|
|||
|
|
forbidden incl. closed tickets). `BF-NNN` names the FAMILY; the Gitea issue
|
|||
|
|
number `#NNNN` is the tracker handle.
|
|||
|
|
|
|||
|
|
**Step 3 — Create one umbrella ticket per NEW root-cause family.** Only when
|
|||
|
|
no existing family matches the signature:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
gitea_issue__create(owner="Octopus", repo="octopus",
|
|||
|
|
title="[{origin}] Baseline failure: {test identifier} (BF-{NNN})",
|
|||
|
|
body=<issue body template below>,
|
|||
|
|
labels=[baseline_failure_label_id, severity_label_id, kind_bug_label_id])
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- **Title**: `[{origin}] Baseline failure: {signature summary} (BF-{NNN})`
|
|||
|
|
where `{origin}` is `chunk-{id}/iter-{N}` or the bugfix slug, and
|
|||
|
|
`{signature summary}` is the family's error/assertion shape (not a single
|
|||
|
|
test name — sibling tests observed later join this umbrella as evidence
|
|||
|
|
comments).
|
|||
|
|
- **Severity → label** (reuse the existing exclusive Priority set; a failing
|
|||
|
|
test defaults to at least MEDIUM, HIGH if on a critical path):
|
|||
|
|
|
|||
|
|
| Severity | Label |
|
|||
|
|
| -------- | ----------------- |
|
|||
|
|
| HIGH | `Priority/High` |
|
|||
|
|
| MEDIUM | `Priority/Medium` |
|
|||
|
|
| LOW | `Priority/Low` |
|
|||
|
|
|
|||
|
|
- Also attach `Kind/Bug` so the failure is discoverable in the bug tracker
|
|||
|
|
alongside manually-reported bugs.
|
|||
|
|
- **Issue body template**:
|
|||
|
|
|
|||
|
|
````markdown
|
|||
|
|
## Parent / 父级
|
|||
|
|
|
|||
|
|
{#source_issue, or "no source issue — discovered during iteration baseline"}
|
|||
|
|
|
|||
|
|
## Summary
|
|||
|
|
|
|||
|
|
A test was already failing before this iteration's work began. It does not
|
|||
|
|
block the discovering iteration but must be tracked to closure.
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
| -------------- | ---------------------------------------------- |
|
|||
|
|
| BF ID | BF-{NNN} (family) |
|
|||
|
|
| Signature | {error/assertion shape} |
|
|||
|
|
| Tests affected | `{file::test-name}`, … (initial members) |
|
|||
|
|
| Origin | {chunk-id/iter-N or slug} |
|
|||
|
|
| Severity | {HIGH \| MEDIUM \| LOW} |
|
|||
|
|
| First observed | {base commit / date the failure reproduces on} |
|
|||
|
|
|
|||
|
|
## Reproduction
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
git checkout {base commit}
|
|||
|
|
(cd <harness-package> && bun test --timeout 120000 {test file}) # or project-equivalent
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Failing output:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
{paste the relevant assertion / error}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Acceptance Criteria (how to close this issue)
|
|||
|
|
|
|||
|
|
- [ ] The named test passes on a clean checkout of the target branch.
|
|||
|
|
- [ ] If the test is deleted/renamed, the replacement covers the same
|
|||
|
|
behavior and passes.
|
|||
|
|
|
|||
|
|
## Context
|
|||
|
|
|
|||
|
|
- Verification report: `{slug}/05-verify-iteration-{N}`
|
|||
|
|
````
|
|||
|
|
|
|||
|
|
**Step 4 — Cross-link to the source issue (mandatory).** Per the
|
|||
|
|
issue-cross-linking rule, every baseline-failure issue MUST link back to its
|
|||
|
|
source:
|
|||
|
|
|
|||
|
|
- The issue body already carries `## Parent` (above).
|
|||
|
|
- Update the source/parent issue body to reference each new baseline-failure
|
|||
|
|
issue by `#NNNN` in its task list (e.g.
|
|||
|
|
`- [ ] #NNNN — BF-{NNN} {test identifier}`), so reviewers can navigate from
|
|||
|
|
source → failure.
|
|||
|
|
|
|||
|
|
**Step 5 — Record the mapping in the verification report.** The
|
|||
|
|
`BF-NNN → #NNNN` mapping is written ONLY in the Phase 5 verification report's
|
|||
|
|
"Baseline Failures" section (and in the issue title itself). There is no local
|
|||
|
|
file to update.
|
|||
|
|
|
|||
|
|
If no new baseline failures were found this iteration, the phase is a no-op —
|
|||
|
|
record "0 baseline failures" in the report and proceed.
|
|||
|
|
|
|||
|
|
<!-- section:phase-5.56 -->
|
|||
|
|
### Phase 5.56 — Flaky-Test Issue Registration (mandatory before PASS when flaky tests are unfixed)
|
|||
|
|
|
|||
|
|
Flaky tests (intermittent pass/fail) are tracked as **root-cause family
|
|||
|
|
umbrella tickets** (`core/rules/ticket-lifecycle.md`, [org-internal #3002] G1) — one
|
|||
|
|
`FT-NNN` ticket per failure-signature family, labeled `flaky-test`, with each
|
|||
|
|
occurrence's evidence (run count, failure rate, commit) attached as comments.
|
|||
|
|
A flaky test that is not fixed within this iteration MUST be filed against
|
|||
|
|
(or into) a family; otherwise it evaporates at iteration end and the
|
|||
|
|
"compare flaky count against baseline" check (TST-2.5) has no persistent
|
|||
|
|
register to compare against. Filing does not block the verdict, but an unfixed
|
|||
|
|
flaky test left un-filed is a process gap and the iteration is NOT Done.
|
|||
|
|
|
|||
|
|
**Expiry SLA ([org-internal #3061])**: an FT umbrella is born with `Kind/Testing` and a
|
|||
|
|
**fix-or-isolate window** (default 7 days) recorded in its body (isolating =
|
|||
|
|
quarantining/skipping the test with the family id in the skip reason, which
|
|||
|
|
keeps the register comparable). Past the window without fix or isolation, the
|
|||
|
|
umbrella is escalated on the ticket (comment naming the owner); retro §2.10
|
|||
|
|
reports fix-or-isolate compliance per cycle.
|
|||
|
|
|
|||
|
|
**Collect inputs** — every flaky test to register this iteration:
|
|||
|
|
|
|||
|
|
1. Every flaky test surfaced in Phase 2.1 (intermittent pass/fail across
|
|||
|
|
reruns) that was NOT fixed by the Developer this iteration.
|
|||
|
|
2. Any flaky test carried over from a prior iteration's report that is still
|
|||
|
|
intermittently failing.
|
|||
|
|
|
|||
|
|
**Step 1 — De-duplicate by failure SIGNATURE against existing families (open
|
|||
|
|
AND closed), per `core/rules/testing.md`** (de-duplicate BEFORE filing,
|
|||
|
|
retro [org-internal #2696]). Query `flaky-test` issues in ALL states:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
gitea_issue__list(owner="Octopus", repo="octopus",
|
|||
|
|
labels="flaky-test", state="all")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Match each candidate by **failure signature** — the flakiness mode (timeout /
|
|||
|
|
race / order-dependence / env-leak) plus the affected surface, not just
|
|||
|
|
`file::test-name` (one root cause may flake several tests). On a signature
|
|||
|
|
hit, do NOT create a new ticket: attach the new run evidence (run count,
|
|||
|
|
failure rate, commit, sibling tests) as a comment on the FAMILY umbrella
|
|||
|
|
ticket. Filing a second ticket inside a known root-cause family is the
|
|||
|
|
anti-pattern this phase exists to prevent. If a tracked family now passes
|
|||
|
|
deterministically, close its umbrella citing the fixing commit. If the query
|
|||
|
|
itself errors (API failure — not "no matches"), retry once; on the second
|
|||
|
|
failure proceed to file with the literal marker `dedup-unverified` in the
|
|||
|
|
umbrella's first evidence comment (query-failure ruling, [org-internal #3383] — fail-open,
|
|||
|
|
visible; rationale in `core/rules/ticket-lifecycle.md`).
|
|||
|
|
|
|||
|
|
**Step 2 — Assign the next `FT-NNN`** per `core/rules/testing.md`
|
|||
|
|
§ "Flaky tests must be filed, not silently tolerated" (the L1 rule owns the
|
|||
|
|
repo-wide sequential numbering invariant — `max + 1`, duplicate ids forbidden
|
|||
|
|
incl. closed tickets). `FT-NNN` names the FAMILY.
|
|||
|
|
|
|||
|
|
**Step 3 — Create one umbrella ticket per NEW root-cause family.** Only when
|
|||
|
|
no existing family matches the signature:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
gitea_issue__create(owner="Octopus", repo="octopus",
|
|||
|
|
title="[{origin}] Flaky test: {test identifier} (FT-{NNN})",
|
|||
|
|
body=<issue body template below>,
|
|||
|
|
labels=[flaky_test_label_id, severity_label_id, kind_testing_label_id])
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
- **Title**: `[{origin}] Flaky test: {signature summary} (FT-{NNN})` — the
|
|||
|
|
family's flakiness mode + surface (sibling tests join later as evidence
|
|||
|
|
comments, not new tickets).
|
|||
|
|
- **Severity** → a flaky test defaults to MEDIUM (HIGH if on a critical path
|
|||
|
|
or blocking CI); reuse the exclusive Priority set.
|
|||
|
|
- Also attach `Kind/Testing` so it is discoverable in the testing tracker.
|
|||
|
|
- **Issue body template**:
|
|||
|
|
|
|||
|
|
````markdown
|
|||
|
|
## Parent / 父级
|
|||
|
|
|
|||
|
|
{#source_issue, or "no source issue — discovered during iteration verification"}
|
|||
|
|
|
|||
|
|
## Summary
|
|||
|
|
|
|||
|
|
A test passes and fails intermittently across runs. It undermines the
|
|||
|
|
reliability of the suite and must be stabilized or quarantined.
|
|||
|
|
|
|||
|
|
| Field | Value |
|
|||
|
|
| ------------ | ------------------------- |
|
|||
|
|
| FT ID | FT-{NNN} (family) |
|
|||
|
|
| Signature | {flakiness mode + surface} |
|
|||
|
|
| Tests affected | `{file::test-name}`, … (initial members) |
|
|||
|
|
| Origin | {chunk-id/iter-N or slug} |
|
|||
|
|
| Severity | {HIGH \| MEDIUM \| LOW} |
|
|||
|
|
| Failure rate | {e.g. 3/10 runs} |
|
|||
|
|
|
|||
|
|
## Reproduction
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
for i in $(seq 1 10); do (cd <harness-package> && bun test --timeout 120000 {test file}) || echo "run $i FAILED"; done
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Acceptance Criteria (how to close this issue)
|
|||
|
|
|
|||
|
|
- [ ] The test passes deterministically across 10 consecutive runs.
|
|||
|
|
- [ ] If it cannot be stabilized, it is quarantined (`test.skip` /
|
|||
|
|
`test.todo` with a comment citing FT-{NNN}) and a stable replacement
|
|||
|
|
is filed as tech-debt (TD-NNN).
|
|||
|
|
|
|||
|
|
## Context
|
|||
|
|
|
|||
|
|
- Verification report: `{slug}/05-verify-iteration-{N}`
|
|||
|
|
````
|
|||
|
|
|
|||
|
|
**Step 4 — Cross-link to the source issue (mandatory).** Every flaky-test
|
|||
|
|
issue links back to its source via `## Parent`, and the source/parent issue
|
|||
|
|
task list references each by `#NNNN`
|
|||
|
|
(`- [ ] #NNNN — FT-{NNN} {test identifier}`).
|
|||
|
|
|
|||
|
|
**Step 5 — Record the mapping in the verification report.** The
|
|||
|
|
`FT-NNN → #NNNN` mapping is written ONLY in the Phase 5 report's "Flaky Tests"
|
|||
|
|
section (and the issue title). No local file.
|
|||
|
|
|
|||
|
|
If no unfixed flaky tests exist this iteration, the phase is a no-op — record
|
|||
|
|
"0 flaky tests" in the report and proceed.
|
|||
|
|
|
|||
|
|
<!-- section:phase-5.6 -->
|
|||
|
|
### Phase 5.6 — Issue Checklist Sync (final sweep, mandatory before PASS)
|
|||
|
|
|
|||
|
|
This is the **final** checklist sync — the last of the progressive sync
|
|||
|
|
points (requirements exit, design exit, plan exit, each iteration commit, PR
|
|||
|
|
creation, review convergence, CI transitions, and here — per the sync-point
|
|||
|
|
table in `core/rules/issue-checklist-sync.md`). Earlier syncs may have
|
|||
|
|
already checked off completed items; this phase catches everything remaining
|
|||
|
|
and ensures no `- [ ]` item is left without an annotation.
|
|||
|
|
|
|||
|
|
1. **Identify the source issue** per `core/rules/issue-checklist-sync.md`
|
|||
|
|
§ "How to sync (each point)" step 1 (find `Closes #N` / `Fixes #N` in the
|
|||
|
|
PR body or commit messages). If no source issue exists (e.g. standalone
|
|||
|
|
bugfix with no tracking issue), skip this phase.
|
|||
|
|
2. **Fetch the issue body** via `工单 API(见 TERMINOLOGY)get` (rule step 2).
|
|||
|
|
3. **Map every remaining `- [ ]` checklist item** to one of three states:
|
|||
|
|
- **Done** (missed by earlier syncs) → mark `- [x]` and append
|
|||
|
|
`_(PR #NNN: <file/component>)_`.
|
|||
|
|
- **Deferred** → keep `- [ ]` and append `_(Deferred: TD-NNN <reason>)_`.
|
|||
|
|
- **Pending** → keep `- [ ]` and append `_(Pending: <blocker>)_`.
|
|||
|
|
4. **Fold the `## 当前状态` live-status section** (if present). Rows that have
|
|||
|
|
a checklist counterpart (review, process-ACs) are already reflected as
|
|||
|
|
checklist annotations at their sync points. Rows with no checklist
|
|||
|
|
counterpart in a non-Epic issue (PR opened/merged) are folded as follow-up
|
|||
|
|
evidence on the issue narrative (e.g. `_(PR #NNN merged: <summary>)_` on the
|
|||
|
|
item it delivered, or in the report). Once every row is accounted for,
|
|||
|
|
remove the section from the issue body.
|
|||
|
|
5. **archive-at-close: mark the `## 工件索引` comment archived** (归档动作 — [org-internal #1929]). Issue
|
|||
|
|
closing IS the archive-at-close action; it is explicit, not implicit. Edit the index
|
|||
|
|
comment in place per `_shared/gitea-write-patterns.md` Pattern 10 Archive-at-close
|
|||
|
|
(mechanism — banner + every row's 重读 → ARCHIVE + invariants — is defined there,
|
|||
|
|
do not restate; sequence/actor per `rules/issue-checklist-sync.md` § 归档动作).
|
|||
|
|
- Skip if no source issue exists.
|
|||
|
|
- **Tier 1 run archive** ([org-internal #1988] two-tier artifacts — only if a Tier 1 run
|
|||
|
|
workspace exists): if `<runs-root>/{slug}/` exists (a stage in this run
|
|||
|
|
produced Tier 1 local artifacts, e.g. review-code findings), archive the
|
|||
|
|
run workspace per `core/rules/two-tier-artifacts.md` § Archive-at-close
|
|||
|
|
(the L1 rule is authoritative — do NOT restate its steps here).
|
|||
|
|
The archived bundle must pass
|
|||
|
|
`core/schemas/runs-bundle.schema.json` validation (the generator
|
|||
|
|
validates by default; a red exit blocks close).
|
|||
|
|
Verify deltas only: skip in milestone/task DAG mode (the aggregator agent
|
|||
|
|
performs it at Epic/task-ticket close — see the DAG branch below).
|
|||
|
|
6. **Update the issue body** via `工单 API(见 TERMINOLOGY)update` with the annotated
|
|||
|
|
checklist. Preserve all non-checklist content unchanged.
|
|||
|
|
7. **Verify**: re-read the issue and confirm no `- [ ]` item lacks an
|
|||
|
|
annotation. Every unchecked item MUST have a `(Deferred: ...)` or
|
|||
|
|
`(Pending: ...)` note explaining why it is not done.
|
|||
|
|
8. **Close the issue** (completes the archive-at-close loop): if the flow has
|
|||
|
|
no PR (no `Closes #N` on merge), the Verifier closes the issue after PASS
|
|||
|
|
approval via `工单 API(见 TERMINOLOGY)update(state="closed")` — the banner date is
|
|||
|
|
corrected to the actual close date if it differs from today. If a PR carries
|
|||
|
|
`Closes #N`, the merge closes it; no further action.
|
|||
|
|
|
|||
|
|
If a checklist item cannot be mapped to evidence, deferral, or pending
|
|||
|
|
status, the iteration is NOT Done — the item represents untracked scope.
|
|||
|
|
|
|||
|
|
<!-- section:phase-5.7 -->
|
|||
|
|
### Phase 5.7 — CI State Transition Sync (progressive)
|
|||
|
|
|
|||
|
|
Process-type acceptance criteria (e.g. "ci.yml passes N consecutive runs",
|
|||
|
|
AC#3 in [org-internal #1668]) have no single moment of completion — their progress lives on
|
|||
|
|
the issue. The `CI` row in `## 当前状态` is written by the `status-sync` poller
|
|||
|
|
(`.gitea/scripts/status-sync-poll.ts`), NOT this skill — do NOT manually
|
|||
|
|
`工单 API(见 TERMINOLOGY)update` it. When verification observes a CI state transition
|
|||
|
|
(first green, a failure, or the cumulative goal being reached), update only the
|
|||
|
|
process-AC's progress note in place via `工单 API(见 TERMINOLOGY)update` (per the
|
|||
|
|
`issue-checklist-sync` L1 rule § Automated sync — the poller owns the CI row;
|
|||
|
|
the checklist annotation remains manual). Do NOT wait for Phase 5.6: a
|
|||
|
|
transition left unsynced for tens of minutes is exactly the staleness [org-internal #1689]
|
|||
|
|
targets. Skip if no source issue exists.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:phase-6 -->
|
|||
|
|
## Phase 6 — Approval
|
|||
|
|
|
|||
|
|
Present the verification result to the user and obtain approval via the
|
|||
|
|
`question` tool, passing `stage: "verify"`:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
[Iteration {N} Verification: {PASS | FAIL}]
|
|||
|
|
|
|||
|
|
DoD: {passed}/{total} passed, {failed} failed
|
|||
|
|
|
|||
|
|
## Failed items:
|
|||
|
|
| REQ ID | Criterion | Gap |
|
|||
|
|
|--------|-----------|-----|
|
|||
|
|
| REQ-NF-005 | Login P99 < 200ms | 312ms (+56%) |
|
|||
|
|
|
|||
|
|
## Tech Debt (Gitea issues created):
|
|||
|
|
| TD ID | Issue | Severity | Description |
|
|||
|
|
|-------|-------|----------|-------------|
|
|||
|
|
| TD-NNN | #NNNN | MEDIUM | ... |
|
|||
|
|
|
|||
|
|
## Baseline Failures (Gitea issues created):
|
|||
|
|
| BF ID | Issue | Severity | Test |
|
|||
|
|
|-------|-------|----------|------|
|
|||
|
|
| BF-NNN | #NNNN | MEDIUM | `file::test-name` |
|
|||
|
|
|
|||
|
|
## Flaky Tests (Gitea issues created):
|
|||
|
|
| FT ID | Issue | Severity | Test |
|
|||
|
|
|-------|-------|----------|------|
|
|||
|
|
| FT-NNN | #NNNN | MEDIUM | `file::test-name` |
|
|||
|
|
|
|||
|
|
→ If FAIL: route to Developer for fixes? (yes / no)
|
|||
|
|
→ If PASS: iteration complete. {N} tech-debt (labeled `tech-debt`), {M} baseline-failure (labeled `baseline-failure`), and {K} flaky-test (labeled `flaky-test`) issues created. Start next iteration? (yes / no)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The `question` tool auto-approves per `core/rules/auto-approve.md` (stage `"verify"`).
|
|||
|
|
|
|||
|
|
Do NOT advance without approval (auto or explicit).
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Phase 6.5 — Commit Gate (mandatory after PASS)
|
|||
|
|
|
|||
|
|
After verification PASS, confirm that git commits exist for the iteration
|
|||
|
|
before declaring Done. Verification PASS without committed code is a process
|
|||
|
|
gap — the feature exists only as uncommitted working tree changes.
|
|||
|
|
|
|||
|
|
### Check
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
git log <base-branch>..HEAD --oneline --no-merges
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
If zero commits exist on the current branch (all work uncommitted), ask via the
|
|||
|
|
`question` tool, passing `stage: "verify"`:
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
⚠️ Verification PASS but 0 commits found on this branch.
|
|||
|
|
|
|||
|
|
The iteration code must be committed before Done can be declared.
|
|||
|
|
→ Commit the work with per-iteration granularity now? (yes / no)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
The `question` tool auto-approves per `core/rules/auto-approve.md` (stage `"verify"`).
|
|||
|
|
|
|||
|
|
If the user confirms, invoke the commit workflow per
|
|||
|
|
`core/skills/implement/SKILL.md` Phase 4.5 (iteration completion commit).
|
|||
|
|
|
|||
|
|
### Gate Rule
|
|||
|
|
|
|||
|
|
| Condition | Verdict | Action |
|
|||
|
|
| --------------------------- | -------- | ----------------------- |
|
|||
|
|
| Commits exist for this iter | ✅ PASS | Proceed to Done |
|
|||
|
|
| No commits | ❌ BLOCK | Must commit before Done |
|
|||
|
|
|
|||
|
|
An iteration is NOT Done until all code is committed in git history.
|
|||
|
|
|
|||
|
|
After the commit gate passes, move the source issue to the `done` column on
|
|||
|
|
the Pipeline Stages board per `_shared/gitea-write-patterns.md` Pattern 7.5.
|
|||
|
|
Skip if no source issue exists. `done` is a stage column marker; issue
|
|||
|
|
closing follows the lifecycle per `issue-checklist-sync.md` 归档动作.
|
|||
|
|
|
|||
|
|
To notify workflow completion, call the `signal_stage_done` tool.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
<!-- section:dag-branch -->
|
|||
|
|
## DAG branch (milestone + task modes)
|
|||
|
|
|
|||
|
|
> This branch OVERRIDES the Preconditions table, Phase 1 read inputs, Phase 5
|
|||
|
|
> report index, Phase 5.5 ACCEPTED_RISK source, Phase 5.6 final sweep, and
|
|||
|
|
> Phase 6.5 commit gate for a DAG-ticket-pipeline Epic. It applies when the
|
|||
|
|
> routed Epic is DAG-shaped: a frozen DAG copy exists at `{epic-slug}/dag` and
|
|||
|
|
> the DAG's `task_route.keep_gates` includes `verify`. Legacy iteration verify
|
|||
|
|
> (the sections above) remains authoritative for non-DAG iterations. Spec:
|
|||
|
|
> `dag-pipeline/spec-05` §5. commit status stays
|
|||
|
|
> `pipeline/verify` in both modes.
|
|||
|
|
|
|||
|
|
### Milestone mode (verify runs on a milestone M when `all_incoming_done_and_green`)
|
|||
|
|
|
|||
|
|
A milestone node (`type: milestone`) has no implementation work and no ticket;
|
|||
|
|
its verify is the integration verification welded at a DAG convergence point
|
|||
|
|
(`cross_session_in ≥ 2`). Trigger: the aggregator agent (main
|
|||
|
|
session/Orchestrator) detects `all_incoming_done_and_green` — every fan-in
|
|||
|
|
source node at its terminal state (task source `done` / milestone source
|
|||
|
|
`green`) — and initiates this verify run; the milestone never self-triggers.
|
|||
|
|
|
|||
|
|
**Precondition replacements** (vs legacy iteration verify):
|
|||
|
|
|
|||
|
|
| Legacy verify precondition | Milestone-mode replacement |
|
|||
|
|
| ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
|||
|
|
| design-space review converged (`review status --stage design-space` success) | review-dag converged (`review status --stage review-dag` success) |
|
|||
|
|
| iteration-plan review converged | REMOVED (DAG has no plan review; the single review-dag gate replaces it) |
|
|||
|
|
| legacy iteration plan at `{slug}/04-plan-index` | DAG artifact + milestone fan-in slice (no `04-plan-*` page) |
|
|||
|
|
| legacy NFR targets from `{slug}/03-design-06-non-functional-design` | REMOVED (product NFRs live in node AC, handled by per-task review-code) |
|
|||
|
|
| upstream roadmap chunk done+verified | all milestone fan-in source nodes at terminal state (task source `done` / milestone source `green`, i.e. `all_incoming_done_and_green`) |
|
|||
|
|
| code review converged (`pipeline/review-code` success) | ALL fan-in node tickets review-code PASS + PR merged (implied by `done`) |
|
|||
|
|
| code review final report exists | REMOVED (no Epic-level final report; per-node review-code reports live on task tickets) |
|
|||
|
|
| carried items from previous iteration closed | REMOVED (vacuous — no iteration) |
|
|||
|
|
| `core/checklists/verification.md` accessible | KEEP (unconditional) |
|
|||
|
|
| Acceptance criteria exist | from DAG-slice-generated DoD matrix (fan-in node AC + integration/contract-conformance/regression/NFR-coverage items) |
|
|||
|
|
| `slug` matches the plan slug | N/A (output naming frozen to `{epic-slug}`) |
|
|||
|
|
|
|||
|
|
**Phase 1 read inputs** → the DAG slice: read the milestone's fan-in subgraph
|
|||
|
|
(fan-in nodes + their cross-session edges + contracts) from the frozen DAG copy
|
|||
|
|
`{epic-slug}/dag`, and generate the DoD matrix from it (integration items,
|
|||
|
|
contract-conformance items, regression items, NFR-coverage items) instead of
|
|||
|
|
reading `04-plan-*` pages.
|
|||
|
|
|
|||
|
|
**Output naming**: `{epic-slug}/05-verify-milestone-{M-id}` (distinct from
|
|||
|
|
iteration verify `05-verify-iteration-{N}`).
|
|||
|
|
|
|||
|
|
**Phase 5 report index**: append a `VERIFY-M-{M-id}` row to the `## 工件索引`
|
|||
|
|
comment with 重读 = `ON-DEMAND` (NOT `ARCHIVE`). The milestone verify does NOT
|
|||
|
|
edit the comment directly — it reports the row-payload to the aggregator agent
|
|||
|
|
(main session/Orchestrator), which is the SINGLE WRITER appending in place
|
|||
|
|
(`工单评论 API(见 TERMINOLOGY)`). `ARCHIVE` is never written by milestone verify
|
|||
|
|
(Epic close is the sole archive point).
|
|||
|
|
|
|||
|
|
**Phase 5.5 ACCEPTED_RISK source**: fan-in node task-ticket review-code reports
|
|||
|
|
(located in each task ticket's review annotations / `## 当前状态` review row),
|
|||
|
|
not an Epic-level final report. A missing Epic-level final report does NOT
|
|||
|
|
trigger the "code-review final report does not exist" warn (that warn is
|
|||
|
|
legacy-iteration-only). tech-debt / baseline-failure / flaky-test registration
|
|||
|
|
proceeds as usual, sourced from the milestone DoD matrix's regression /
|
|||
|
|
baseline-classification results.
|
|||
|
|
|
|||
|
|
**Phase 5.6 milestone mode** (restricted semantics — skip archive-at-close and
|
|||
|
|
issue close):
|
|||
|
|
|
|||
|
|
1. **Checklist final sweep** only — object = each fan-in task ticket's row 6
|
|||
|
|
(`verify里程碑`) + the milestone DoD matrix, NOT the Epic body checklist.
|
|||
|
|
Verify each fan-in node's row 6 is checked or annotated
|
|||
|
|
`_(Pending: M-{M-id} 未 green)_`. Do NOT flip a still-Pending row 6 — that
|
|||
|
|
flip belongs to the aggregator agent (main session/Orchestrator), which
|
|||
|
|
performs the node-ticket close action. Write the milestone DoD matrix final
|
|||
|
|
sweep to `{epic-slug}/05-verify-milestone-{M-id}`.
|
|||
|
|
2. **Do NOT fold/remove any Epic-body `## 当前状态` / `## DAG 状态` section** —
|
|||
|
|
milestone verify is read-only on Epic body state.
|
|||
|
|
3. **SKIP archive-at-close** (the `## 工件索引` archive banner + row-level
|
|||
|
|
`ARCHIVE`) and **SKIP issue close** — both belong to the aggregator agent at
|
|||
|
|
Epic close.
|
|||
|
|
4. **SKIP Tier 1 run archive** (Phase 5.6 step 5, Tier 1 run archive sub-bullet) — the aggregator agent
|
|||
|
|
performs it at Epic close.
|
|||
|
|
|
|||
|
|
**Milestone verify execution context (pinned)**: run the
|
|||
|
|
integration/regression/contract-conformance/NFR suite on a checkout CONTAINING
|
|||
|
|
all fan-in nodes' merged code — the aggregator agent syncs main (containing the
|
|||
|
|
merged fan-in PRs) into the execution worktree, or builds a temp verify worktree
|
|||
|
|
from main HEAD, before triggering; the verify report
|
|||
|
|
`{epic-slug}/05-verify-milestone-{M-id}` records the actual execution commit
|
|||
|
|
SHA.
|
|||
|
|
|
|||
|
|
**M→green status transition**: NOT done by milestone verify (read-only on Epic
|
|||
|
|
body). The aggregator agent (main session/Orchestrator) reads the verify report
|
|||
|
|
and sets the aggregator table M row 状态 = `green`, 重读 = `ON-DEMAND`, and
|
|||
|
|
syncs the frozen DAG copy `{epic-slug}/dag` node `status` to `green`.
|
|||
|
|
|
|||
|
|
**Phase 6.5 milestone mode**: vacuous skip — a milestone has no implementation
|
|||
|
|
commit. Commit existence is evidenced by the fan-in node verify PASS + PR
|
|||
|
|
merged (already implied by `all_incoming_done_and_green`).
|
|||
|
|
|
|||
|
|
**Milestone verify FAIL recovery**: FAIL → M `status` = `blocked`. Fix
|
|||
|
|
responsibility = the fan-in node owning the failing integration/regression item
|
|||
|
|
(the last-done fan-in node's session) or the Epic-level session (DAG author) if
|
|||
|
|
un-attributable. Re-run is triggered ONLY by a "fix-landed event" (fix commit
|
|||
|
|
merged + CI green, or an explicit re-done report from the owning session) —
|
|||
|
|
NOT status polling (done nodes are never reset; `all_incoming_done_and_green`
|
|||
|
|
stays true after FAIL). M stays `blocked` until a fix event, then returns to
|
|||
|
|
`pending` and verify re-runs. No review-dag re-run unless the fix changes a
|
|||
|
|
cross-session contract or the node/edge set.
|
|||
|
|
|
|||
|
|
### Task mode (verify on a task node ticket, because `task_route.keep_gates` includes `verify`)
|
|||
|
|
|
|||
|
|
Each task ticket closes only after its own verify run, but legacy verify
|
|||
|
|
preconditions do not hold in DAG mode — this node-level branch reuses the
|
|||
|
|
`verify` skill with the following node-scoped semantics (so the `verify` gate is
|
|||
|
|
not silently downgraded to CI-only).
|
|||
|
|
|
|||
|
|
**Verification object**: the node's own `acceptance_criteria` → the `test_id`
|
|||
|
|
suite (`file-path::test-name` / `MANUAL` / `BENCH:`) all passing fresh + that
|
|||
|
|
node's review-code PASS recorded + CI green. NOT milestone fan-in
|
|||
|
|
integration/contract/regression items (those are milestone verify's). Spec
|
|||
|
|
authority = the node in the frozen DAG copy `{epic-slug}/dag`.
|
|||
|
|
|
|||
|
|
**Precondition replacements** (task mode — mirror the milestone table, scoped to
|
|||
|
|
the node):
|
|||
|
|
|
|||
|
|
| Legacy verify precondition | Task-mode replacement |
|
|||
|
|
| ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- |
|
|||
|
|
| design-space review converged | review-dag converged (this Epic already passed the single gate; do NOT re-run it) |
|
|||
|
|
| iteration-plan review converged | REMOVED (DAG has no plan review) |
|
|||
|
|
| legacy iteration plan at `{slug}/04-plan-index` | the node in the frozen DAG copy `{epic-slug}/dag` (no `04-plan-*` page) |
|
|||
|
|
| legacy NFR targets from `{slug}/03-design-06-non-functional-design` | REMOVED (product NFRs live in node AC, handled by per-task review-code) |
|
|||
|
|
| upstream roadmap chunk done+verified | this node's cross-session in-edge source nodes at terminal state (task source `done` / milestone source `green`) |
|
|||
|
|
| code review converged (`pipeline/review-code` success) | this node's ticket review-code PASS + PR merged |
|
|||
|
|
| code review final report exists | REMOVED (this node's review-code report lives on the task ticket) |
|
|||
|
|
| carried items from previous iteration closed | REMOVED (vacuous — no iteration) |
|
|||
|
|
| `core/checklists/verification.md` accessible | KEEP |
|
|||
|
|
| Acceptance criteria exist | from node AC + `test_id` suite |
|
|||
|
|
| `slug` matches the plan slug | N/A (Tier 1 run slug = `{epic-slug}-task-{node-id}` — the same node-task run namespace review-code runs in and this mode's archive bundle uses below; the Phase 2 precondition-gate cache read MUST use this slug. Only the wiki output naming stays Epic-scoped: `{epic-slug}/05-verify-task-{node-id}`) |
|
|||
|
|
|
|||
|
|
**Phase 1 read inputs** → the node's AC + `test_id` suite from the frozen DAG
|
|||
|
|
copy `{epic-slug}/dag`, generating the DoD matrix.
|
|||
|
|
|
|||
|
|
**Output naming**: `{epic-slug}/05-verify-task-{node-id}`.
|
|||
|
|
|
|||
|
|
**Phase 5 report index**: append a `VERIFY-TASK-{node-id}` row, 重读 =
|
|||
|
|
`ON-DEMAND` (NOT `ARCHIVE`), via the aggregator agent as single writer (task
|
|||
|
|
verify does not edit the comment directly).
|
|||
|
|
|
|||
|
|
**Phase 5.5 task mode**: ACCEPTED_RISK source = this node's task-ticket
|
|||
|
|
review-code report (no Epic-level final report → no "does not exist" warn).
|
|||
|
|
tech-debt / baseline-failure / flaky-test registration proceeds as usual.
|
|||
|
|
|
|||
|
|
**Phase 5.6 task mode**: only checklist final sweep — object = this ticket's
|
|||
|
|
5-row checklist (NOT row 6 `verify里程碑` — row 6 is flipped ONLY by the
|
|||
|
|
aggregator agent). SKIP archive-at-close (`## 工件索引` row-level `ARCHIVE`) and
|
|||
|
|
issue close (both belong to the aggregator agent). SKIP Tier 1 run archive — the
|
|||
|
|
aggregator agent collects it cross-worktree at task-ticket close (archive to
|
|||
|
|
`<runs-root>/archive/{epic-slug}-task-{node-id}.json` and commit to the
|
|||
|
|
workflow branch). Do NOT fold Epic body status sections.
|
|||
|
|
|
|||
|
|
**Phase 6.5 task mode**: object = this node ticket's commit; semantics = "node
|
|||
|
|
implementation commit exists". Baseline = the task worktree's creation-commit
|
|||
|
|
SHA (`git log {creation-SHA}..HEAD`), NOT a generic base-branch (post-merge
|
|||
|
|
there are no new commits vs base). The task worktree is retained until ticket
|
|||
|
|
close (the aggregator agent removes it after collecting task-level Tier 1
|
|||
|
|
archive); the task session must NOT remove its own worktree.
|
|||
|
|
|
|||
|
|
**commit status**: still `pipeline/verify` in both modes.
|
|||
|
|
|
|||
|
|
### No-milestone DAG Epic-level verify gate
|
|||
|
|
|
|||
|
|
When the routed Epic's DAG has NO milestone (no `cross_session_in ≥ 2`
|
|||
|
|
convergence point), the Epic-level `keep_gates: verify` is satisfied
|
|||
|
|
AGGREGATELY by all task-ticket verify runs (each with commit status
|
|||
|
|
`pipeline/verify` success) — NO independent Epic-level verify run, NO
|
|||
|
|
`{epic-slug}/05-verify-milestone-{M-id}` artifact, NO `VERIFY-M-{M-id}` index
|
|||
|
|
row. The Epic close condition degenerates to "all task nodes `done`".
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## References
|
|||
|
|
|
|||
|
|
- `core/checklists/verification.md` — Verification self-check checklist
|
|||
|
|
- `core/skills/implement/SKILL.md` — Implementation (precedes verification)
|
|||
|
|
- `core/skills/review-code/SKILL.md` — Code review (precedes verification)
|
|||
|
|
- `core/skills/browser-debug/SKILL.md` — Interactive browser evidence capture (use for UI/E2E DoD entries; save screenshots as Gitea wiki pages under `{slug}/verify/evidence/{name}` using `wiki 读写 API(见 TERMINOLOGY)` — replaces the deprecated `.artifacts/{slug}/verify/evidence/` directory)
|
|||
|
|
- `core/rules/` and L2 wiki pages — Engineering conventions (test commands, typecheck; style-guide, effect-rules, database conventions per the "L2 on-demand reference" section of AGENTS.md)
|
|||
|
|
- ISO/IEC/IEEE 29119:2022 — Software testing
|