Initial publish v0.1.0: standalone workflow core (corpus + examples + guards)
This commit is contained in:
octopus
@@ -0,0 +1,762 @@
|
||||
---
|
||||
name: retrospective
|
||||
description: >
|
||||
Use ONLY when running a retrospective at the end of any work cycle
|
||||
(release, feature, bugfix, port). The Retrospective Lead (Worker) inspects project
|
||||
facts — git log, file churn, test history, commit patterns — and generates
|
||||
actionable improvement items. No pipeline artifacts required.
|
||||
triggers:
|
||||
- retrospective
|
||||
- 复盘
|
||||
- lessons learned
|
||||
- post-mortem
|
||||
- postmortem
|
||||
role: Producer
|
||||
---
|
||||
|
||||
> Core 中立版（Increment 6a 改写，原 deferHard verbatimDir）。机制、结构与 frontmatter 保持；实例术语（工具名、路径、工单号）按 `core/adapters/TERMINOLOGY.md` 绑定到具体实例。
|
||||
|
||||
# Retrospective
|
||||
|
||||
Inspect the project's recent work cycle to extract patterns and generate
|
||||
concrete action items. Based purely on project facts: git history, code churn,
|
||||
commit quality, test results, and build health.
|
||||
|
||||
**Purpose**: Continuous improvement of BOTH the project AND the SDLC pipeline
|
||||
itself. Every retrospective may modify templates, checklists, or SKILLs.
|
||||
|
||||
## Agent Role
|
||||
|
||||
The retrospective is owned and executed by the **Retrospective Lead** (Worker).
|
||||
|
||||
**Context compaction**: retrospective is a pipeline stage boundary. The main
|
||||
session compacts at this clean boundary ONLY when a capacity/projection trigger
|
||||
holds, per `core/rules/compact.md` §"Stage-boundary compaction" (long multi-stage runs
|
||||
— DAG Epic orchestration — keep the legacy every-boundary compaction; short
|
||||
runs — bugfix / DAG task — and standalone runs default to NOT compacting). The Retrospective Lead itself is single-phase and
|
||||
data-driven: all findings are written to the retrospective report as they are
|
||||
produced, so a mid-run compaction loses nothing — re-read the report artifact
|
||||
to resume.
|
||||
|
||||
---
|
||||
|
||||
## Preconditions
|
||||
|
||||
Before starting the retrospective:
|
||||
|
||||
- [ ] A work cycle has been completed (release was cut, feature merged,
|
||||
bug fixed, or port landed).
|
||||
- [ ] The project is a git repository with recent commits.
|
||||
- [ ] `core/checklists/retrospective.md` is accessible.
|
||||
|
||||
No `.artifacts/{slug}/` files are required. (**Deprecated**: `.artifacts/` file-system access is being migrated to Gitea wiki. SDLC artifacts now live as wiki pages under `{slug}/`; use `wiki 读写 API（见 TERMINOLOGY）` and `wiki 读写 API（见 TERMINOLOGY）` to read them.) The retrospective reads the project's own state and git history.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Scope the Cycle
|
||||
|
||||
Determine the time range to analyze:
|
||||
|
||||
1. If a **release was just done**: use `git log <last-tag>..<new-tag>`.
|
||||
2. If **no tag exists**: prompt the user for a time range or revision range
|
||||
(e.g. `HEAD~20..HEAD`, or `--since="last work cycle"`).
|
||||
3. If the user specifies a range, use that.
|
||||
|
||||
```markdown
|
||||
## Cycle Scope
|
||||
|
||||
**Range**: {commit range or "last work cycle"}
|
||||
**Date**: {start} → {end}
|
||||
**Commits analyzed**: {N}
|
||||
**Authors**: {names}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Collect Data
|
||||
|
||||
Run project-inspection commands and summarize findings:
|
||||
|
||||
### 2.1 Commit patterns
|
||||
|
||||
```bash
|
||||
git log {range} --oneline --no-merges
|
||||
git log {range} --format='%s' | grep -cE '^(BREAKING|feat|fix|perf|refactor|docs|chore)'
|
||||
```
|
||||
|
||||
- Count commits by type (BREAKING, feat, fix, refactor, etc.).
|
||||
- Flag commits with no conventional prefix — these are opaque and hard to
|
||||
trace → log as `[SIGNAL: low commit hygiene]`.
|
||||
- Count revert commits (`git log {range} --grep="Revert" --oneline`).
|
||||
- High revert count → likely insufficient testing or review before merge.
|
||||
|
||||
### 2.2 File churn
|
||||
|
||||
```bash
|
||||
git diff --stat {range}
|
||||
git diff --numstat {range} | sort -k1 -rn | head -20
|
||||
```
|
||||
|
||||
- Identify the most-churned files (top 10 by lines added + deleted).
|
||||
- High churn in a single file (> 200 lines in one cycle) → possible
|
||||
monolithic module, design issue, or scope creep.
|
||||
- List new files vs. deleted files.
|
||||
|
||||
### 2.3 Test health
|
||||
|
||||
1. Run the test suite: confirm pass/fail count and duration.
|
||||
2. Check if any test files changed during the cycle:
|
||||
```bash
|
||||
git diff --name-only {range} | grep -E 'test|spec|__tests__'
|
||||
```
|
||||
3. If test files were NOT changed but source files were → `[SIGNAL: untested changes]`.
|
||||
**Architecture-A exemption**: If the cycle touches ONLY `<instance-root>/` and
|
||||
`.gitea/` files (config, skills, templates, rules, checklists) with zero
|
||||
`packages/*` source changes, the "untested changes" signal does NOT apply
|
||||
— `<instance-root>/` files are validated by the audit-process review gate, not
|
||||
by unit tests. Record this as `[NOTE: Architecture A — config-only cycle,
|
||||
unit-test exemption applies]` in the test summary.
|
||||
4. Check for skipped/flaky tests if the framework reports them.
|
||||
|
||||
### 2.4 Build health
|
||||
|
||||
1. Run `bun typecheck` (or project equivalent). Note first-time errors.
|
||||
2. Run `bun oxlint --deny-warnings` (repo root; `bun lint` is the package-script alias). Note any first-time warnings.
|
||||
3. If the project has CI, check the latest run status.
|
||||
|
||||
### 2.5 Dependency health
|
||||
|
||||
1. Run `bun audit` (or equivalent). Flag any new HIGH/CRITICAL CVEs.
|
||||
2. Check if any dependency was added/removed/upgraded:
|
||||
```bash
|
||||
git diff {range} -- package.json bun.lockb
|
||||
```
|
||||
|
||||
### 2.6 Process quality
|
||||
|
||||
Inspect the SDLC infrastructure around the project — not just the code, but
|
||||
the factory that produces it. These checks are all file-existence and
|
||||
configuration reads; they require zero prior pipeline artifacts.
|
||||
|
||||
1. **Pre-commit guards**:
|
||||
- Does the project have pre-commit hooks? (Check `.husky/`, `lefthook.yml`,
|
||||
`.pre-commit-config.yaml`, `package.json` `"lint-staged"` key.)
|
||||
- If absent → `[GAP: no pre-commit guard — bad code can land]`.
|
||||
- If present → what commands do they run? (lint? typecheck? test?)
|
||||
|
||||
2. **CI/CD pipeline**:
|
||||
- Does CI exist? (Check `.gitea/workflows/` — this repo's CI location —
|
||||
then `.github/workflows/`, `.gitlab-ci.yml`, `Jenkinsfile`, etc.)
|
||||
- If absent → `[GAP: no CI — no automated gate before merge]`.
|
||||
- If present → does it block merge on failure? Check branch protection
|
||||
(GitHub: `gh api repos/{owner}/{repo}/branches/main/protection`).
|
||||
|
||||
3. **Code review practice**:
|
||||
- Does the project have a review template or guideline? (Check
|
||||
`.github/PULL_REQUEST_TEMPLATE.md`, `CONTRIBUTING.md`, etc.)
|
||||
- Are PRs used? Look at merge commits: do they reference PR numbers?
|
||||
- If direct pushes to main → `[SIGNAL: no review gate]`.
|
||||
|
||||
4. **SDLC artifacts**:
|
||||
- Use `wiki 读写 API（见 TERMINOLOGY）(owner="Octopus", repo="octopus")` to check
|
||||
for `{slug}/` wiki pages (`.artifacts/` file-system access is retired —
|
||||
SDLC artifacts now live as wiki pages under `{slug}/`).
|
||||
- If no `{slug}/` pages exist → `[NOTE: no structured SDLC artifacts — decisions not traceable]`.
|
||||
- If present but stale → `[SIGNAL: artifacts not maintained — process drift]`.
|
||||
|
||||
5. **Tooling consistency**:
|
||||
- Does the project have a unified formatter config? (`.prettierrc`,
|
||||
`biome.json`, etc.)
|
||||
- Does it have a unified typecheck/lint/test command? (Check `package.json`
|
||||
scripts.)
|
||||
- Are there multiple competing tools for the same concern? (e.g. both
|
||||
prettier AND biome, both jest AND vitest.)
|
||||
|
||||
6. **Documentation health**:
|
||||
- Does `README.md` include setup, build, and test instructions?
|
||||
- Does `AGENTS.md` or `<instance-root>/AGENTS.md` exist?
|
||||
- Are there any outdated docs? (Check for files referencing removed
|
||||
commands or directories.)
|
||||
|
||||
### 2.7 Token telemetry
|
||||
|
||||
Collect LLM token-consumption signals to assess workflow quality. Five
|
||||
metrics: M1 from an inline check; M2–M5 from the token-telemetry probe.
|
||||
|
||||
1. **M1 — Review convergence.** The number of rounds each review dimension
|
||||
needed to converge is the strongest signal of upstream-stage quality.
|
||||
|
||||
Discover review rounds via `wiki 读写 API（见 TERMINOLOGY）(owner="Octopus", repo="octopus")`
|
||||
with prefix `{slug}/reviews/` (`.artifacts/` file-system listing is retired).
|
||||
For each review, `max(roundN)` is its convergence round count.
|
||||
- 🟢 1–2 / 🟡 3–4 / 🔴 ≥5.
|
||||
- ≥5 rounds → `[SIGNAL: review convergence ≥5 — upstream stage quality
|
||||
insufficient, rework deferred to review]`.
|
||||
|
||||
The remaining four metrics come from a single probe run:
|
||||
|
||||
```bash
|
||||
bun run core/skills/retrospective/scripts/token-telemetry.ts
|
||||
```
|
||||
|
||||
It scans `~/.local/share/octopus/octopus-*.db` (token usage) and review
|
||||
rounds from two sources — the Tier 1 review-status file
|
||||
`<runs-root>/{slug}/reviews/{stage}/status.json` (canonical — reads the
|
||||
`history[]` field (legacy alias `rounds[]`) and `current_round`) with the
|
||||
legacy `.artifacts/**/reviews/*/status.json` tree fallback, plus the
|
||||
committed archive bundles `<runs-root>/archive/{slug}.json` for closed
|
||||
runs — and prints M2–M5.
|
||||
|
||||
2. **M2 — Stage distribution.** Token spend per pipeline stage (design,
|
||||
review, implement, …). The probe reconstructs a per-session stage
|
||||
timeline from `~/.local/share/octopus/token-stage-ledger.jsonl` —
|
||||
written by the auto-discovered `<instance-root>/plugin/token-stage-ledger.ts`
|
||||
plugin, which hooks `tool.execute.after` on the `skill` tool — and
|
||||
attributes each message's tokens to the stage active at its creation.
|
||||
- **Ledger-gated.** If the plugin was not active during the cycle the
|
||||
ledger is absent and the probe prints
|
||||
`[NOTE: token-stage-ledger.jsonl absent — M2 skipped]`. That is
|
||||
"unavailable", not "failed" — proceed.
|
||||
- Review-stage share: 🟢 <35% / 🟡 35–60% / 🔴 >60%.
|
||||
- > 60% → `[SIGNAL: review stage >60% of token spend — over-reviewing,
|
||||
review findings not actionable upstream]`.
|
||||
|
||||
3. **M3 — Review rework.** Rework fraction — review rounds beyond the first
|
||||
as a share of total review rounds.
|
||||
The probe merges two sources, deduped by `{slug}/reviews/{stage}` (the
|
||||
ACTIVE status.json wins): (a) the Tier 1 review-status file
|
||||
`<runs-root>/{slug}/reviews/{stage}/status.json` (canonical — `history[]`
|
||||
field, legacy alias `rounds[]`, and `current_round`; skips `_archive/`)
|
||||
with the legacy `.artifacts/**/reviews/*/status.json` tree fallback —
|
||||
in-flight runs only, since the active workspace is deleted at
|
||||
archive-at-close; (b) the committed archive bundles
|
||||
`<runs-root>/archive/{slug}.json`, where per-review round counts are
|
||||
reconstructed from the documented `reviews/{stage}/round{N}/` layout in
|
||||
`index.artifacts[].path` ([org-internal #2591] — the archive source is what makes M3
|
||||
durable instead of structurally emptying as runs close). Available once
|
||||
any run has closed.
|
||||
- Rework fraction: 🟢 <15% / 🟡 15–30% / 🔴 >30%.
|
||||
- > 30% → `[SIGNAL: rework fraction >30% — review findings not actionable
|
||||
or upstream design unclear]`.
|
||||
|
||||
4. **M4 — Context hygiene.** Per-session input-token growth and cache
|
||||
efficiency, read from the octopus session database. The cache
|
||||
read/input ratio measures context reuse.
|
||||
- 🟢 cache_read/input > 10:1 / 🟡 3–10:1 / 🔴 < 3:1.
|
||||
- < 3:1 → `[SIGNAL: cache hit <3:1 — context re-read, code-graph-first
|
||||
not followed]`.
|
||||
- **Compact frequency ([org-internal #2601] pilot data)**: report the token-telemetry
|
||||
"Compactor Activity" line (compactor messages/tokens in window) and the
|
||||
zero-compact share of short runs (bugfix / DAG task) — the capacity-driven compaction
|
||||
pilot metrics. A rising zero-compact share with NO late-stage
|
||||
degradation signal is the evidence that retires the pilot gate in
|
||||
`rules/compact.md` § Stage-boundary compaction.
|
||||
|
||||
5. **M5 — Explore/execute ratio.** Token spend by agent type
|
||||
(`data.agent` per message). A low ratio means workers are doing
|
||||
explorers' job — context-gathering that should be delegated.
|
||||
- 🟢 > 2:1 / 🟡 1–2:1 / 🔴 < 1:1.
|
||||
- < 1:1 → `[SIGNAL: explore/execute <1:1 — workers doing explorers' job,
|
||||
exploration skipped or under-delegated]`.
|
||||
|
||||
### 2.8 Gate defect-escape analysis (single metric, [org-internal #3061])
|
||||
|
||||
Gates earn their cost by what they CATCH, and the honest test is what slips
|
||||
past them: a gate that runs clean while the same defect resurfaces downstream
|
||||
is under-powered — the opposite of redundant. This probe computes ONE metric
|
||||
per `gate_id` (the legacy per-(Kind × Size × gate) keying is retired with the
|
||||
sizing subsystem — sub-5 cell counts produced noise, not evidence):
|
||||
|
||||
**`escape_rate` = clean runs with a downstream escape / clean runs**
|
||||
|
||||
- **Clean run**: the gate ran on a cycle ticket and passed round 1 with zero
|
||||
INFO-or-worse findings.
|
||||
- **Downstream escape**: within 14 days after the clean gate, either
|
||||
(a) a verify FAIL/WARN finding in the same area (module/dimension) on the
|
||||
same ticket, or (b) a post-merge `Kind/Bug` ticket whose body references
|
||||
the area the gate covered.
|
||||
|
||||
**Data sources** (all read-only):
|
||||
|
||||
- **Review status**: commit-status context `pipeline/{review_type}`
|
||||
(`_shared/gitea-write-patterns.md` Pattern 8), read via `octopus review
|
||||
status` CLI or commit-status inspection. Final reports at
|
||||
`{slug}/reviews/{stage}/final/report` record round count and highest
|
||||
severity.
|
||||
- **Downstream (a)**: verify reports `{slug}/05-verify-iteration-{N}` for the
|
||||
same slug.
|
||||
- **Downstream (b)**: `工单 API（见 TERMINOLOGY）list(labels="Kind/Bug", state=closed)`
|
||||
within the window; match by module/path references in the body.
|
||||
- **slug ↔ issue map**: the issue's `## 工件索引` comment (legacy:
|
||||
`## Pipeline 工件追踪表`) or `Closes #N` in the PR body.
|
||||
|
||||
**Procedure**:
|
||||
|
||||
1. Enumerate closed tickets in the cycle with any review run; resolve slugs.
|
||||
2. Per gate run: classify clean / not-clean; for clean runs, search both
|
||||
downstream sources for an escape.
|
||||
3. Tally per `gate_id`: `clean_runs`, `escapes`, `escape_rate`.
|
||||
4. Reading (thresholds are this skill's own policy — the `gate_trim:` block
|
||||
they once deferred to was retired 2026-08-21, [org-internal #3072] phase 3; nothing
|
||||
trims gates anymore, so the trim-candidate branch below is gone):
|
||||
- `escape_rate ≥ 0.3` → the gate is
|
||||
**UNDER-POWERED**: route the escape causes into §2.9 pre-flight
|
||||
(producer-side self-checks) and note the gate in the report. Do NOT
|
||||
propose trimming it.
|
||||
- `escape_rate < 0.3` → healthy; report the numbers, no further verdict.
|
||||
|
||||
**DAG single-gate exclusion ([org-internal #2267])**: `review-dag` is structurally
|
||||
non-trimmable (`dag.route.single_gate.never_trim: true` — self-contained in
|
||||
the `dag:` block); `verify` and `merge-pr` were `never_trim` in the retired
|
||||
`gate_trim` block and stay untouchable by convention.
|
||||
|
||||
**keep_gates exclusion (TD-390, [org-internal #3061]) — RETIRED with gate_trim**: the
|
||||
rule that a trim candidate's gate must not be a member of the target route's
|
||||
`keep_gates` guarded a landing field (`sizing.tiers.*.additional_skip`)
|
||||
that no longer exists. Historical record: wiki `rules/gate-trim`.
|
||||
|
||||
**Sample-sufficiency gate** (structural, not optional): fewer than 5 clean
|
||||
runs for a gate → report the raw numbers with
|
||||
`[NOTE: insufficient sample for gate {id}]` and emit no verdict for it.
|
||||
|
||||
```markdown
|
||||
### Gate defect-escape (M6)
|
||||
|
||||
| gate | clean runs | escapes | escape_rate | verdict |
|
||||
| ----------------------------------- | ---------- | ------- | ----------- | ------------------------- |
|
||||
| review-code | 12 | 1 | 0.08 | healthy |
|
||||
| review-design-space (sticky legacy) | 6 | 3 | 0.50 | UNDER-POWERED → feed §2.9 |
|
||||
| ... | ... | ... | ... | ... |
|
||||
|
||||
- [NOTE: insufficient sample for gate X] where clean runs < 5.
|
||||
- UNDER-POWERED gates (escape_rate ≥ 0.3) feed §2.9 pre-flight — never a
|
||||
unilateral trim: the only exit is a furlough entry in
|
||||
`<instance-root>/gate-ledger.yaml` (evidence + reopen condition, [org-internal #3607]).
|
||||
```
|
||||
|
||||
### 2.9 Pre-flight defect-prevention analysis ([org-internal #2599])
|
||||
|
||||
First-round FAIL/WARN findings are the pipeline's cost multiplier: every
|
||||
multi-round review pays for them twice (review round + revision round).
|
||||
This probe tallies their root causes and proposes producer-side self-checks —
|
||||
the defensive complement of the retired gate-trim meta-process: instead of
|
||||
removing a gate that never catches anything, inject the causes that keep
|
||||
costing rounds as a pre-flight checklist the Producer verifies BEFORE writing
|
||||
code.
|
||||
|
||||
**Data sources** (read-only, same as 2.8): review status pages / final
|
||||
reports for the cycle's tickets (`{slug}/reviews/{stage}/final/report` —
|
||||
round-1 findings with severity FAIL/WARN, their dimension codes, and the
|
||||
finding text for root-cause categorization) + the source issues'
|
||||
`Kind/*` labels. For DAG-routed Epics ([org-internal #2905] 方案 3), two further
|
||||
read-only sources feed the `REQ × late-discovery` root cause (step 1):
|
||||
(a) **DAG oversize-signal events** — `node_split` / registry-row additions
|
||||
recorded on the Epic's issue timeline / `## DAG 状态` whose trigger is a NEW
|
||||
requirement rather than a refactor; (b) **demo-period feedback** —
|
||||
stakeholder comments naming functionality the frozen DAG never covered. For browser-evidence
|
||||
cycles ([org-internal #4499], contract `browser-evidence-4486/shared/evidence-ref-v1` §3), a
|
||||
third read-only
|
||||
source: (c) **browser session rows** — per browser-debug session
|
||||
`{session_id, outcome: evidence-captured | env-unavailable | replay-failed,
|
||||
replay summary, env.mode}`, derived from Tier-1 pack manifests (sanitized
|
||||
transitively by the N-03 write boundary — this probe never touches raw
|
||||
captures). Replay failure attribution rides the FIXED
|
||||
`ReplayFailureCategory` enum (`browser-evidence-4486/shared/pack-manifest-v1`
|
||||
§4); `env-unavailable` sessions default to the enum's own environment slot
|
||||
(`env-binary-missing`) unless the row carries an explicit attribution;
|
||||
browser-class
|
||||
causes enter the step-2 route-class tally as `BROWSER × {category}` and a
|
||||
qualifying cause becomes a PRE-FLIGHT PROPOSAL candidate landing on
|
||||
`routes.{Kind}.preflight` (human-landed in Phase 6, never auto-applied).
|
||||
Rows are currently Task-DAG cycle products by default (the helper's
|
||||
`--route-class` override / row-level `route_class` field re-keys a future
|
||||
Bug- or Feature-cycle session to its own landing slot). The replay
|
||||
success-rate baseline (N-03 BENCH) is exposed as a consumable metric — a SEEDED CONVENTION (fixture-authored outcomes, channel health),
|
||||
NOT a live-browser trend baseline. Mechanical helper:
|
||||
`<harness-package>/scripts/browser-retro-tally.ts`. Boundary rules ([org-internal #4499]
|
||||
AC-3): out-of-enum or empty categories fall into an explicit `other` bucket
|
||||
with provenance (recorded, never silently swallowed, never a crash); rows
|
||||
missing replay/attribution fields yield structured errors in the tally
|
||||
output; a cycle with no browser session rows keeps this probe's current
|
||||
behavior (empty tally, no synthetic proposal rows).
|
||||
|
||||
**Procedure**:
|
||||
|
||||
1. For each ticket with a multi-round review in the cycle, read the round-1
|
||||
findings with severity FAIL or WARN. Categorize each by
|
||||
(dimension × root-cause) — the dimension code is checklist-qualified
|
||||
(checklist-local namespace; e.g. `code-review.md TST × missing-boundary-test`,
|
||||
`code-review.md COR × unhandled-null`,
|
||||
`code-review.md STY × wrong-import-order`).
|
||||
Canonical root-cause category for DAG-routed Epics: `REQ ×
|
||||
late-discovery` — 冻结后才被发现的需求 (a requirement that surfaced only
|
||||
after the DAG froze). A late discovery usually manifests as a round-1
|
||||
REQMAP FAIL on the re-run review, but the underlying event is an
|
||||
oversize signal or demo comment — count it from sources (a)/(b) above,
|
||||
not only from review findings.
|
||||
2. Tally per **route-class** — `Bug`, `Feature-DAG`, `Epic-DAG-route`,
|
||||
`Task-DAG` (Kind/Feature routes to the DAG pipeline since [org-internal #3061] Phase 2 /
|
||||
TD-388 — `Feature-legacy` survives only as a historical bucket for tickets
|
||||
closed before 2026-08-20; the Size dimension is retired from retro keying,
|
||||
[org-internal #3061]: DAG-routed tickets derive depth instead of carrying `Size/*`
|
||||
(spec-06), and `Kind/Bug` carries no ladder). First-round hit count
|
||||
and distinct-ticket count per cause per class.
|
||||
3. A cause is a **pre-flight candidate** when ALL hold (thresholds from
|
||||
`workflow-routing.yaml` `preflight` — do NOT restate values here):
|
||||
- hit count ≥ `preflight.min_sample`
|
||||
- distinct tickets ≥ `preflight.consecutive_recur`
|
||||
A qualifying `Epic × DAG-route` cause (e.g. `REQ × late-discovery`) lands
|
||||
on `<instance-root>/workflow-routing.yaml` `dag.route.preflight` — NOT
|
||||
`routes.Kind/Epic.preflight` (legacy-roadmap-era landing; the legacy route
|
||||
was archived [org-internal #3072] phase 3) ([org-internal #2905] 方案 3). The analyze-dag skill reads
|
||||
`dag.route.preflight` before decomposition.
|
||||
4. Aging: for causes ALREADY landed in `routes.{Kind}.preflight` or
|
||||
`dag.route.preflight`, count
|
||||
consecutive clean first rounds since landing (from this and prior retro
|
||||
data); at ≥ `preflight.aging_consecutive_clean`, emit a REMOVAL row.
|
||||
|
||||
Sample-sufficiency guard mirrors 2.8: zero qualifying tickets → emit
|
||||
`[NOTE: insufficient sample for pre-flight proposals]` and skip this probe.
|
||||
|
||||
```markdown
|
||||
### Pre-flight defect prevention (M7)
|
||||
|
||||
| route-class | Cause (checklist-qualified dim × root-cause) | 1st-round hits | tickets | qualify? |
|
||||
| -------------- | ------------------------------------------- | -------------- | --------------------------- | -------------------------------- |
|
||||
| Bug | code-review.md COR × unhandled-null | 5 | 4 ([org-internal #2400] [org-internal #2429] [org-internal #2471] [org-internal #2488]) | ✅ PROPOSE |
|
||||
| Bug | code-review.md STY × wrong-import-order | 6 | 2 | ⛔ tickets <3 |
|
||||
| Feature-DAG | code-review.md TST × missing-boundary-test | 3 | 3 | ⛔ hits <5 |
|
||||
| Epic-DAG-route | REQ × late-discovery | 5 | 3 (#27xx #28xx #29xx) | ✅ PROPOSE → dag.route.preflight |
|
||||
|
||||
- [NOTE: no pre-flight proposals] if the table is empty or nothing clears threshold.
|
||||
- ✅ PROPOSE rows feed Phase 5 PRE-FLIGHT PROPOSAL action items.
|
||||
- Landed-item aging: `cor-unhandled-null clean streak 5 ≥ aging 5` → REMOVAL row.
|
||||
```
|
||||
|
||||
### 2.10 Derived-ticket health ([org-internal #3061])
|
||||
|
||||
Derived tickets (TD promotions, BF umbrellas, FT tickets) are the pipeline's
|
||||
exhaust. Unmanaged they accumulate into flush cycles (2026-08 evidence: 122
|
||||
open tech-debt tickets, ~9.4/day creation, zero pre-August closures, a
|
||||
115-ticket bulk flush). Under the registry-first regime (verify Phase 5.5)
|
||||
this probe checks whether the system DIGESTS what it records:
|
||||
|
||||
**Metrics** (cycle window):
|
||||
|
||||
- **TD flow**: registry rows created / rows promoted to tickets / promoted
|
||||
tickets closed-as-fixed vs closed-as-wontfix.
|
||||
- **Median open age**: open TD rows + promoted tickets, by module/origin.
|
||||
- **BF triage compliance**: % of verify-Phase-5.55 BF umbrellas triaged
|
||||
(assigned or scheduled) within 1 day of filing.
|
||||
- **FT expiry compliance**: % of verify-Phase-5.56 FT tickets fixed-or-
|
||||
isolated within their N-day window.
|
||||
- **Per-module open count** vs the promotion quota (verify Phase 5.5).
|
||||
|
||||
**Actions**:
|
||||
|
||||
- A category (module/origin) with creation ≥ fix across ≥2 consecutive
|
||||
retros → mark it **register-only**: verify Phase 5.5 stops promoting rows
|
||||
to tickets there until one retro shows net-negative backlog.
|
||||
- Open TD rows unclaimed for >3 retro cycles → mark the row `[COLD]`
|
||||
(revivable — a pull event clears the mark). Cold rows are excluded from
|
||||
adjacency-quota pressure and do not count against the module quota.
|
||||
- BF/FT compliance < 100% → name the untriaged/expired items in the report
|
||||
(they are SLA breaches, not statistics).
|
||||
|
||||
```markdown
|
||||
### Derived-ticket health (M8)
|
||||
|
||||
| metric | this cycle | last cycle | trend |
|
||||
| ----------------------------------------- | ---------- | ---------- | ----- |
|
||||
| TD rows created / promoted / closed-fixed | | | |
|
||||
| median open-TD age (days) | | | |
|
||||
| BF same-day triage % | | | |
|
||||
| FT fix-or-isolate % | | | |
|
||||
|
||||
- register-only categories: {list or "none"}
|
||||
- rows marked [COLD] this cycle: {list or "none"}
|
||||
```
|
||||
|
||||
### 2.11 Threshold calibration tally ([org-internal #3380])
|
||||
|
||||
Process prose constants (round caps, D1–D4 thresholds, preflight knobs,
|
||||
escape-rate cutoffs, quotas — full index:
|
||||
`docs/workflow-refactor/thresholds-ledger.md`) are only as good as the data
|
||||
behind them. This probe keeps them honest, mirroring §2.9's propose-and-human-
|
||||
lands pattern:
|
||||
|
||||
1. **Round-cap distribution** (per stage): read `{slug}/reviews/*/final/report`
|
||||
headers (`**Rounds completed**: N`) for the cycle's slugs; tally per stage
|
||||
(code / dag), report n / p50 / p90 / p95 / max, and how often the cap was
|
||||
the binding stop (`Rounds completed` == cap with `Converged: false`).
|
||||
2. **CALIBRATION PROPOSAL rows**: for any ledger constant whose data source
|
||||
this retro tallied, emit a proposal row — `constant | current | observed |
|
||||
proposed | evidence` — when the data contradicts the current value (cap
|
||||
never binding AND p95 ≪ cap → propose lowering; cap binding with
|
||||
converged-improving runs → propose raising). Proposals are **never
|
||||
auto-landed**: a human lands them by editing the definition site AND the
|
||||
ledger row in one PR citing this retro (preflight evidence-field pattern).
|
||||
3. **Ledger sync**: if any constant's definition site changed since the last
|
||||
retro (value or location), update the ledger row — drift between the two
|
||||
is a TD-480-class double-source failure.
|
||||
|
||||
```markdown
|
||||
### Threshold calibration (M9)
|
||||
|
||||
| stage | n | p50 | p90 | p95 | max | cap-binding runs |
|
||||
| --------------------------------- | ------- | -------- | -------- | ------------------- | --- | ---------------- |
|
||||
| code | | | | | | |
|
||||
| dag | | | | | | |
|
||||
| - CALIBRATION PROPOSAL: {constant | current | observed | proposed | evidence} or "none" |
|
||||
|
||||
- ledger sync: {rows updated or "none"}
|
||||
```
|
||||
|
||||
```markdown
|
||||
## Data Summary
|
||||
|
||||
### Commits
|
||||
|
||||
| Type | Count |
|
||||
| ------------- | ------ |
|
||||
| BREAKING | {N} |
|
||||
| feat | {N} |
|
||||
| fix | {N} |
|
||||
| refactor | {N} |
|
||||
| docs/chore | {N} |
|
||||
| **no prefix** | {N} ⚠️ |
|
||||
| **reverts** | {N} |
|
||||
|
||||
### Churn Top 10
|
||||
|
||||
| File | +lines | -lines | Concern |
|
||||
| --------------------- | ------ | ------ | ----------- |
|
||||
| src/{module}/large.ts | 200 | 150 | Monolithic? |
|
||||
| ... | ... | ... | ... |
|
||||
|
||||
### Tests
|
||||
|
||||
- Suite: {N} passed, {M} failed, {S} skipped — {duration}s
|
||||
- Test files changed: {N} / {M}
|
||||
- Untested source changes: {list or "none"}
|
||||
|
||||
### Build
|
||||
|
||||
- Typecheck: ✅ / ❌ (N errors)
|
||||
- Lint: ✅ / ⚠️ (N warnings)
|
||||
|
||||
### Dependencies
|
||||
|
||||
- Added: {list}
|
||||
- Removed: {list}
|
||||
- Upgraded: {list}
|
||||
- Audit: ✅ / ⚠️ N CVEs
|
||||
|
||||
### Process
|
||||
|
||||
| Check | Status | Detail |
|
||||
| ------------------- | ------------ | ---------------------------------- |
|
||||
| Pre-commit hooks | ✅ / ❌ | {what runs / "none"} |
|
||||
| CI/CD | ✅ / ❌ | {provider / "none"} |
|
||||
| PR / review gate | ✅ / ❌ | {PR # pattern / direct push} |
|
||||
| SDLC artifacts | ✅ / ⚠️ / ❌ | {present & fresh / stale / absent} |
|
||||
| Formatter config | ✅ / ❌ | {tool / "none"} |
|
||||
| Typecheck+Lint+Test | ✅ / ⚠️ | {unified scripts?} |
|
||||
| README / AGENTS.md | ✅ / ⚠️ | {present / stale / absent} |
|
||||
|
||||
### Token
|
||||
|
||||
- M1 review convergence: max {N} rounds ({review name}) — 🟢/🟡/🔴
|
||||
- M2 stage distribution: design {X}% / review {X}% / implement {X}% (review share {X}% — 🟢/🟡/🔴)
|
||||
- M3 rework: {X}% rework fraction ({rework}/{total} rounds across {N} reviews) — 🟢/🟡/🔴
|
||||
- M4 context: p50={N} / p90={N} / max={N} input tokens; cache {ratio}:1 — 🟢/🟡/🔴
|
||||
- M5 explore/execute: {ratio}:1 (explorer {N} / worker {M} tokens) — 🟢/🟡/🔴
|
||||
- M6 gate defect-escape: {N} gates scanned, {M} UNDER-POWERED (escape_rate at/above threshold, top: {gate}) — 🟢 all healthy / 🟡 {M} under-powered → feed §2.9 / 🔴 broad under-powering across gates
|
||||
- [NOTE: M2 skipped if token-stage-ledger.jsonl absent]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — What Went Well?
|
||||
|
||||
Identify positive patterns — things to repeat or amplify:
|
||||
|
||||
1. **High-quality commits**: commits with clear prefixes, atomic scope, good
|
||||
descriptions → name specific examples.
|
||||
2. **Low-churn modules**: files that were changed but had low +/− counts
|
||||
(well-factored, easy to modify).
|
||||
3. **Tests that caught bugs**: if any test was added before the fix commit,
|
||||
that's TDD → highlight it.
|
||||
4. **Fast turnaround**: if any commit → production cycle was unusually fast.
|
||||
5. **Process safeguards active**: pre-commit hooks catching errors before push,
|
||||
CI blocking broken builds, PR review catching design issues early.
|
||||
6. **Fresh documentation**: README and AGENTS.md are up to date and referenceable.
|
||||
|
||||
```markdown
|
||||
## What Went Well
|
||||
|
||||
1. {finding} — {evidence from data} — {why it worked, do again}
|
||||
2. {finding}
|
||||
3. {finding}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — What Went Wrong?
|
||||
|
||||
Identify problems — focus on patterns in the data, not blame:
|
||||
|
||||
| Data signal | Root cause pattern | Example |
|
||||
| ------------------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
||||
| Revert commits > 0 | Bug slipped through review / testing | "Revert 'fix auth' — original fix broke login" |
|
||||
| High churn in a single file (> 300) | Monolithic module, hard to change safely | `src/handler.ts` +400/-350 in one cycle |
|
||||
| feat commits without test changes | New features landed untested | 3 feat commits, 0 test files changed |
|
||||
| No conventional commit prefix | Low commit discipline, harder to auto-changelog | 8 of 12 commits have no prefix |
|
||||
| Test suite growing slower | Test debt accumulating | Source +200 lines, tests +10 lines |
|
||||
| Dependency added without audit check | Supply chain risk | New dep added, `audit` not run |
|
||||
| Typecheck broke mid-cycle | No pre-commit / pre-push hooks | Type error landed on main, fixed later |
|
||||
| No pre-commit hooks | Every developer must remember to run checks manually | Type errors and lint violations land on main |
|
||||
| No CI/CD | No automated gate before merge | Broken build merged, discovered later |
|
||||
| No PR template / direct pushes | No structured review process | Design flaws not caught until production |
|
||||
| Stale SDLC artifacts | Process was followed once then abandoned | `.artifacts/` exists but empty for last 3 cycles (**deprecated**: check Gitea wiki `{slug}/` pages instead) |
|
||||
| Multiple formatters / test frameworks | Tooling inconsistency slows onboarding | Both prettier and biome configured |
|
||||
| Review rounds ≥5 | Upstream stage quality low — rework deferred to review | design converged only at round 5 |
|
||||
| cache_read/input < 3:1 | Context re-read repeatedly — code-graph-first not followed | 2.8:1 across design stage |
|
||||
| explore/execute < 1:1 | Workers doing explorers' job — exploration skipped or under-delegated | 0.7:1 — worker tokens exceed explorer |
|
||||
| review stage >60% of token spend | Over-reviewing — review findings not actionable upstream | design 20% / review 65% / implement 15% |
|
||||
| rework fraction >30% | Review findings not actionable or upstream design unclear | 62% rework — 30 of 48 rounds beyond first |
|
||||
| retrospective skill modified in cycle range | Retrospective reviewing its own modification — potential self-review bias | `core/skills/retrospective/SKILL.md` changed in `git diff {range}` |
|
||||
|
||||
```markdown
|
||||
## What Went Wrong
|
||||
|
||||
1. **{signal}** — {root cause} — {impact: what broke / slowed down}
|
||||
2. **{signal}** — {root cause} — {impact}
|
||||
3. **{signal}** — {root cause} — {impact}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Action Items
|
||||
|
||||
For each "what went wrong", generate a concrete, measurable action item.
|
||||
Each item MUST specify:
|
||||
|
||||
- **What** — the change to make.
|
||||
- **Where** — which template, checklist, SKILL, or project config to modify.
|
||||
- **Who** — which role or agent is responsible.
|
||||
- **When** — effective immediately or next cycle.
|
||||
|
||||
```markdown
|
||||
## Action Items
|
||||
|
||||
| # | What | Where | Who | When |
|
||||
| --- | -------------------------------------------------- | --------------------------------------- | ------------------ | ---------- |
|
||||
| 1 | Add pre-commit hook: typecheck + lint on staged | `.husky/pre-commit` | Developer | immediate |
|
||||
| 2 | Require test file changes for every feat commit | `core/checklists/implementation.md` | Retrospective Lead | next cycle |
|
||||
| 3 | Add commit message template (conventional commits) | `.gitmessage` or `CONTRIBUTING.md` | Retrospective Lead | next cycle |
|
||||
```
|
||||
|
||||
**Rules**:
|
||||
|
||||
- Maximum 5 action items per retrospective. If you have more, group by root
|
||||
cause and take the top 5.
|
||||
- Every action item MUST reference a specific file to modify.
|
||||
- Action items that modify SKILLs or templates are applied immediately (the
|
||||
Retrospective Lead can edit `<instance-root>/` files).
|
||||
- Action items MUST NOT be vague ("try harder", "be more careful").
|
||||
- **Layering question** (injection-budget ruling item 3, [org-internal #3547]): every action
|
||||
item that encodes a lesson as a process constraint MUST answer **"which
|
||||
layer carries this lesson?"** — options in ascending per-turn cost, pick
|
||||
the cheapest that actually enforces it:
|
||||
1. **L0 tool-enforced** — hook / bash guard / CI validation (zero prompt
|
||||
bytes);
|
||||
2. **Skill step** — inline in the phase skill that executes the work
|
||||
(bytes paid only when that skill is loaded);
|
||||
3. **L2 on-demand** — wiki / rule doc fetched when a task needs it;
|
||||
4. **L1 per-turn injection** — an `core/rules/*.md` whitelist entry
|
||||
(bytes paid by EVERY role on EVERY turn; item must cite the current
|
||||
corpus bytes vs `bun run check:rule-budget` cap headroom).
|
||||
Option 4 is **default-deny**: choosing it requires stating why 1–3 cannot
|
||||
carry the lesson (2026-08 batch-1 cut builder -41% / explorer -69% per-turn
|
||||
corpus — do not casually re-grow it; the rule-GC report in that same check
|
||||
flags aging L1 rules for L2 retirement).
|
||||
|
||||
**TRIM PROPOSAL rows — RETIRED ([org-internal #3072] phase 3, 2026-08-21)**: the
|
||||
gate-trim landing machinery (`gate_trim.action` →
|
||||
`sizing.tiers.{Size}.additional_skip` / `routes.{Kind}.skip` /
|
||||
`auto_approve.stages`) was removed with the `gate_trim:` and `sizing:`
|
||||
blocks — nothing trims gates anymore, so Phase 5 emits NO trim proposals.
|
||||
Gate health findings flow exclusively through §2.8's escape-rate verdict
|
||||
(UNDER-POWERED → §2.9 pre-flight) and ordinary action items. Historical
|
||||
spec: wiki `rules/gate-trim` (L2).
|
||||
|
||||
**PRE-FLIGHT PROPOSAL rows** (from Phase 2.9 defect-prevention, [org-internal #2599]): format
|
||||
the What cell as
|
||||
`Add pre-flight '{id}' to {Kind} ({hits} first-round FAIL/WARN hits, {tickets} tickets — {evidence})`
|
||||
and the Where cell as
|
||||
`<instance-root>/workflow-routing.yaml` `routes.{Kind}.preflight` — or, for causes
|
||||
keyed `Epic × DAG-route` (e.g. `REQ × late-discovery`, [org-internal #2905] 方案 3),
|
||||
`<instance-root>/workflow-routing.yaml` `dag.route.preflight` (consumed by
|
||||
analyze-dag before decomposition).
|
||||
Aging removal rows: `Remove pre-flight '{id}' from {Kind} (clean streak {N} ≥ aging threshold)`.
|
||||
|
||||
Same doctrine the retired TRIM proposals used: a PRE-FLIGHT PROPOSAL is a
|
||||
recommendation, not an
|
||||
auto-apply — landing (and removal) happens in Phase 6 only when the action
|
||||
item survives the retrospective's own review. Landed checklists must stay
|
||||
within `preflight.max_items` entries per route (drop-oldest by `added_cycle`).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Apply Improvements
|
||||
|
||||
For each action item that modifies a `<instance-root>/` or project-config file:
|
||||
|
||||
1. Read the current file.
|
||||
2. Apply the change.
|
||||
3. Note the change in the retrospective report.
|
||||
|
||||
```markdown
|
||||
## Applied Improvements
|
||||
|
||||
1. Modified `{.file}`: {what was changed} — {commit hash}
|
||||
2. Modified `{.file}`: {what was changed} — {commit hash}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Report
|
||||
|
||||
Publish the retrospective report as a Gitea wiki page `_retrospectives/{cycle-name}` via `wiki 读写 API（见 TERMINOLOGY）`. The `_retrospectives/` namespace is an intentional cross-cycle, slug-less exception to the `{slug}/...` artifact-path convention (retrospectives aggregate multiple slugs and outlive any one pipeline run) — analogous to the audit `audit/{date}/` date-slug exception documented in the NAM 4.4 checklist item.
|
||||
|
||||
```markdown
|
||||
# Retrospective: {cycle description}
|
||||
|
||||
**Date**: {YYYY-MM-DD}
|
||||
**Range**: {commit range or time range}
|
||||
**Commits**: {N}
|
||||
**Overall**: 🟢 GREEN / 🟡 YELLOW / 🔴 RED
|
||||
|
||||
## What Went Well
|
||||
|
||||
{list}
|
||||
|
||||
## What Went Wrong
|
||||
|
||||
{list}
|
||||
|
||||
## Action Items
|
||||
|
||||
{table}
|
||||
|
||||
## Applied Improvements
|
||||
|
||||
{list}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- `core/checklists/retrospective.md` — Retrospective self-check
|
||||
@@ -0,0 +1,830 @@
|
||||
import { Database } from "bun:sqlite"
|
||||
import { readdirSync, readFileSync, existsSync, statSync } from "node:fs"
|
||||
import { join, relative } from "node:path"
|
||||
import { homedir } from "node:os"
|
||||
|
||||
const DBS_DIR = join(homedir(), ".local/share/octopus")
|
||||
const LEDGER_PATH = join(DBS_DIR, "token-stage-ledger.jsonl")
|
||||
|
||||
// #2628: workspace containers (octopus-ws-*) persist their session store in
|
||||
// named volumes octopus-sessions-<id> (bind added in
|
||||
// packages/containers/src/runtime/docker.ts). Scan those alongside the host
|
||||
// dir so telemetry no longer depends on which machine/container ran a session.
|
||||
const SESSION_VOLUMES_ROOT = "/data/docker/volumes"
|
||||
|
||||
function collectSessionVolumeDbs(): string[] {
|
||||
let vols: string[] = []
|
||||
try {
|
||||
vols = readdirSync(SESSION_VOLUMES_ROOT).filter((d) => d.startsWith("octopus-sessions-"))
|
||||
} catch {
|
||||
return [] // not on the docker host (e.g. a dev workstation) — fine
|
||||
}
|
||||
const out: string[] = []
|
||||
for (const v of vols) {
|
||||
const dir = join(SESSION_VOLUMES_ROOT, v, "_data")
|
||||
try {
|
||||
for (const e of readdirSync(dir, { withFileTypes: true })) {
|
||||
if (e.isFile() && e.name.endsWith(".db") && (e.name === "octopus.db" || e.name.startsWith("octopus-"))) {
|
||||
out.push(join(dir, e.name))
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// unreadable volume — skip it
|
||||
}
|
||||
}
|
||||
return out.sort()
|
||||
}
|
||||
|
||||
function dbLabel(dbPath: string): string {
|
||||
const base = dbPath.split(/[/\\]/).pop()!.replace(".db", "")
|
||||
const vol = dbPath.match(/octopus-sessions-([a-zA-Z0-9-]+)[/\\]_data/)
|
||||
return vol ? `${base}@${vol[1]!.slice(0, 8)}` : base
|
||||
}
|
||||
|
||||
// SINCE_DAYS=<N> env var scopes queries to messages from the last N days,
|
||||
// avoiding full-table scans on multi-GB databases.
|
||||
// message.time_created is MILLISECONDS (verified: raw values ~1.78e12). Keep
|
||||
// `since` in ms — a seconds-based value is always smaller than every ms
|
||||
// timestamp, so the filter would silently match everything (#2599).
|
||||
const sinceDays = Number(process.env.SINCE_DAYS ?? "30")
|
||||
const since = sinceDays > 0 ? Date.now() - sinceDays * 86_400_000 : 0
|
||||
|
||||
// Cycle-window filters (retro #4034 quick-wins): `--since <ISO-date>` and
|
||||
// `--slug <slug>` constrain the cycle-window metrics (M2/M3/M5) to the
|
||||
// window / matching run. M1/M4 keep the SINCE_DAYS env semantics. A filter
|
||||
// that yields no data prints an explicit "no data in window" line for the
|
||||
// metric — never a silent fallback to all-time numbers.
|
||||
const argv = process.argv.slice(2)
|
||||
const arg = (name: string): string | undefined => {
|
||||
const i = argv.indexOf(`--${name}`)
|
||||
return i >= 0 ? argv[i + 1] : undefined
|
||||
}
|
||||
const sinceArg = arg("since")
|
||||
const slugArg = arg("slug")
|
||||
const windowSince = sinceArg !== undefined ? Date.parse(sinceArg) : undefined
|
||||
if (sinceArg !== undefined && Number.isNaN(windowSince)) {
|
||||
console.error(`invalid --since "${sinceArg}" — use an ISO date (e.g. 2026-09-02)`)
|
||||
process.exit(1)
|
||||
}
|
||||
const filtersActive = sinceArg !== undefined || slugArg !== undefined
|
||||
const windowOrSince = windowSince ?? since
|
||||
const m5Filter = filtersActive ? { since: windowOrSince, slug: slugArg } : undefined
|
||||
|
||||
function percentile(sorted: number[], p: number): number {
|
||||
if (sorted.length === 0) return 0
|
||||
const idx = Math.min(Math.floor((sorted.length * p) / 100), sorted.length - 1)
|
||||
return sorted[idx] ?? 0
|
||||
}
|
||||
|
||||
type AgentStats = Map<string, { msgCount: number; totalTokens: number }>
|
||||
|
||||
type SessionRow = {
|
||||
id: string
|
||||
parent_id: string | null
|
||||
slug: string | null
|
||||
title: string | null
|
||||
directory: string | null
|
||||
}
|
||||
|
||||
function loadSessionRows(db: Database): Map<string, SessionRow> | null {
|
||||
const hasTable = db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='session'").get()
|
||||
if (!hasTable) return null
|
||||
const rows = db.prepare("SELECT id, parent_id, slug, title, directory FROM session").all() as SessionRow[]
|
||||
return new Map(rows.map((r) => [r.id, r]))
|
||||
}
|
||||
|
||||
// Session→run matching for --slug: a session matches when its slug, title,
|
||||
// or directory contains the run slug (workflow sessions live in worktrees
|
||||
// named after the run). A match propagates to the whole subtree (subagents),
|
||||
// so every message of the run's sessions is included.
|
||||
function slugSessionIncludeSet(sessions: Map<string, SessionRow>, slug: string): Set<string> {
|
||||
const needle = slug.toLowerCase()
|
||||
const children = new Map<string, string[]>()
|
||||
for (const r of sessions.values()) {
|
||||
if (!r.parent_id) continue
|
||||
const arr = children.get(r.parent_id) ?? []
|
||||
arr.push(r.id)
|
||||
children.set(r.parent_id, arr)
|
||||
}
|
||||
const include = new Set<string>()
|
||||
const markSubtree = (id: string) => {
|
||||
if (include.has(id)) return
|
||||
include.add(id)
|
||||
for (const c of children.get(id) ?? []) markSubtree(c)
|
||||
}
|
||||
for (const r of sessions.values()) {
|
||||
const hay = [r.slug, r.title, r.directory].filter((x): x is string => typeof x === "string")
|
||||
if (hay.some((x) => x.toLowerCase().includes(needle))) markSubtree(r.id)
|
||||
}
|
||||
return include
|
||||
}
|
||||
|
||||
function processDb(dbPath: string, since = 0, m5Filter?: { since: number; slug?: string }) {
|
||||
const dbName = dbLabel(dbPath)
|
||||
const sessions: {
|
||||
sessionId: string
|
||||
msgCount: number
|
||||
inputs: number[]
|
||||
totalInput: number
|
||||
totalCacheRead: number
|
||||
}[] = []
|
||||
const agents: AgentStats = new Map()
|
||||
|
||||
let db: Database | null = null
|
||||
try {
|
||||
db = new Database(dbPath, { readonly: true })
|
||||
db.exec("PRAGMA busy_timeout = 5000")
|
||||
} catch {
|
||||
return { dbName, sessionCount: 0, messageCount: 0, sessions, agents }
|
||||
}
|
||||
|
||||
try {
|
||||
const hasTable = db.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='message'").get()
|
||||
if (!hasTable) return { dbName, sessionCount: 0, messageCount: 0, sessions, agents }
|
||||
|
||||
// Single query replaces the former N+1 pattern (one query per session).
|
||||
// Grouping in JS avoids N full-table scans with json_extract.
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
session_id,
|
||||
CAST(json_extract(data, '$.tokens.input') AS REAL) as inp,
|
||||
CAST(json_extract(data, '$.tokens.cache.read') AS REAL) as cr
|
||||
FROM message
|
||||
WHERE json_extract(data, '$.role') = 'assistant'
|
||||
AND json_extract(data, '$.tokens.input') IS NOT NULL
|
||||
${since > 0 ? "AND time_created >= ?" : ""}
|
||||
ORDER BY session_id, time_created
|
||||
`,
|
||||
)
|
||||
.all(...(since > 0 ? [since] : [])) as {
|
||||
session_id: string
|
||||
inp: number | null
|
||||
cr: number | null
|
||||
}[]
|
||||
|
||||
const sessionMap = new Map<string, { inputs: number[]; totalInput: number; totalCacheRead: number }>()
|
||||
|
||||
for (const r of rows) {
|
||||
const inp = Number(r.inp ?? 0)
|
||||
const cr = Number(r.cr ?? 0)
|
||||
if (inp <= 0) continue
|
||||
let s = sessionMap.get(r.session_id)
|
||||
if (!s) {
|
||||
s = { inputs: [], totalInput: 0, totalCacheRead: 0 }
|
||||
sessionMap.set(r.session_id, s)
|
||||
}
|
||||
s.inputs.push(inp)
|
||||
s.totalInput += inp
|
||||
s.totalCacheRead += cr
|
||||
}
|
||||
|
||||
for (const [sessionId, s] of sessionMap) {
|
||||
if (s.inputs.length === 0) continue
|
||||
sessions.push({
|
||||
sessionId,
|
||||
msgCount: s.inputs.length,
|
||||
inputs: s.inputs,
|
||||
totalInput: s.totalInput,
|
||||
totalCacheRead: s.totalCacheRead,
|
||||
})
|
||||
}
|
||||
|
||||
if (m5Filter) {
|
||||
let include: Set<string> | undefined
|
||||
if (m5Filter.slug !== undefined) {
|
||||
const sessions = loadSessionRows(db)
|
||||
include = sessions ? slugSessionIncludeSet(sessions, m5Filter.slug) : new Set<string>()
|
||||
}
|
||||
// include.size === 0 (or a slug-less filter) → contribute nothing;
|
||||
// the merged M5 output prints the explicit "no data in window" line.
|
||||
if (include === undefined || include.size > 0) {
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
session_id,
|
||||
COALESCE(json_extract(data, '$.agent'), 'unknown') as agent,
|
||||
CAST(json_extract(data, '$.tokens.input') AS REAL) as inp,
|
||||
CAST(json_extract(data, '$.tokens.output') AS REAL) as outp,
|
||||
CAST(json_extract(data, '$.tokens.reasoning') AS REAL) as rea
|
||||
FROM message
|
||||
WHERE json_extract(data, '$.role') = 'assistant'
|
||||
AND json_extract(data, '$.tokens.input') IS NOT NULL
|
||||
${m5Filter.since > 0 ? "AND time_created >= ?" : ""}
|
||||
`,
|
||||
)
|
||||
.all(...(m5Filter.since > 0 ? [m5Filter.since] : [])) as {
|
||||
session_id: string
|
||||
agent: string
|
||||
inp: number | null
|
||||
outp: number | null
|
||||
rea: number | null
|
||||
}[]
|
||||
for (const r of rows) {
|
||||
if (include !== undefined && !include.has(r.session_id)) continue
|
||||
const cur = agents.get(r.agent) ?? { msgCount: 0, totalTokens: 0 }
|
||||
cur.msgCount++
|
||||
cur.totalTokens += Number(r.inp ?? 0) + Number(r.outp ?? 0) + Number(r.rea ?? 0)
|
||||
agents.set(r.agent, cur)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
const agentRows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT
|
||||
COALESCE(json_extract(data, '$.agent'), 'unknown') as agent,
|
||||
COUNT(*) as msg_count,
|
||||
CAST(TOTAL(json_extract(data, '$.tokens.input')) AS REAL) as inp,
|
||||
CAST(TOTAL(json_extract(data, '$.tokens.output')) AS REAL) as outp,
|
||||
CAST(TOTAL(json_extract(data, '$.tokens.reasoning')) AS REAL) as rea
|
||||
FROM message
|
||||
WHERE json_extract(data, '$.role') = 'assistant'
|
||||
AND json_extract(data, '$.tokens.input') IS NOT NULL
|
||||
${since > 0 ? "AND time_created >= ?" : ""}
|
||||
GROUP BY agent
|
||||
`,
|
||||
)
|
||||
.all(...(since > 0 ? [since] : [])) as {
|
||||
agent: string
|
||||
msg_count: number
|
||||
inp: number
|
||||
outp: number
|
||||
rea: number
|
||||
}[]
|
||||
|
||||
for (const r of agentRows) {
|
||||
agents.set(r.agent, {
|
||||
msgCount: Number(r.msg_count ?? 0),
|
||||
totalTokens: Number(r.inp ?? 0) + Number(r.outp ?? 0) + Number(r.rea ?? 0),
|
||||
})
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
|
||||
const messageCount = sessions.reduce((s, se) => s + se.msgCount, 0)
|
||||
return { dbName, sessionCount: sessions.length, messageCount, sessions, agents }
|
||||
}
|
||||
|
||||
function dbSummary(db: ReturnType<typeof processDb>) {
|
||||
if (db.sessionCount === 0) return null
|
||||
return ` ${db.dbName}: ${db.sessionCount} sessions, ${db.messageCount.toLocaleString()} messages`
|
||||
}
|
||||
|
||||
function computeSessionMedians(dbs: ReturnType<typeof processDb>[]) {
|
||||
const medians: number[] = []
|
||||
for (const db of dbs) {
|
||||
for (const s of db.sessions) {
|
||||
const sorted = [...s.inputs].sort((a, b) => a - b)
|
||||
medians.push(percentile(sorted, 50))
|
||||
}
|
||||
}
|
||||
return medians
|
||||
}
|
||||
|
||||
function computeSessionMaxes(dbs: ReturnType<typeof processDb>[]) {
|
||||
const maxes: number[] = []
|
||||
for (const db of dbs) {
|
||||
for (const s of db.sessions) {
|
||||
maxes.push(Math.max(...s.inputs))
|
||||
}
|
||||
}
|
||||
return maxes
|
||||
}
|
||||
|
||||
function computeSessionP90s(dbs: ReturnType<typeof processDb>[]) {
|
||||
const p90s: number[] = []
|
||||
for (const db of dbs) {
|
||||
for (const s of db.sessions) {
|
||||
const sorted = [...s.inputs].sort((a, b) => a - b)
|
||||
p90s.push(percentile(sorted, 90))
|
||||
}
|
||||
}
|
||||
return p90s
|
||||
}
|
||||
|
||||
function computeOverallCacheRatio(dbs: ReturnType<typeof processDb>[]) {
|
||||
let totalInput = 0
|
||||
let totalCacheRead = 0
|
||||
for (const db of dbs) {
|
||||
for (const s of db.sessions) {
|
||||
totalInput += s.totalInput
|
||||
totalCacheRead += s.totalCacheRead
|
||||
}
|
||||
}
|
||||
return totalInput > 0 ? totalCacheRead / totalInput : 0
|
||||
}
|
||||
|
||||
function trafficLight(ratio: number): string {
|
||||
if (ratio > 10) return "🟢"
|
||||
if (ratio >= 3) return "🟡"
|
||||
return "🔴"
|
||||
}
|
||||
|
||||
const dbFiles = [
|
||||
...readdirSync(DBS_DIR, { withFileTypes: true })
|
||||
.filter((e) => e.isFile() && e.name.endsWith(".db") && (e.name === "octopus.db" || e.name.startsWith("octopus-")))
|
||||
.map((e) => join(DBS_DIR, e.name)),
|
||||
...collectSessionVolumeDbs(), // #2628
|
||||
].sort()
|
||||
|
||||
for (const f of dbFiles) {
|
||||
const size = statSync(f).size
|
||||
if (size > 1_000_000_000 && since === 0) {
|
||||
console.error(
|
||||
`[WARN] ${f.split("/").pop()} is ${(size / 1e9).toFixed(1)} GB — query may be slow. Set SINCE_DAYS=<N> to scope to recent sessions.`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const allDbs = dbFiles.map((p) => processDb(p, since, m5Filter))
|
||||
|
||||
console.log("# Token Telemetry: Context Hygiene (M4)")
|
||||
console.log()
|
||||
if (filtersActive) {
|
||||
const parts = [sinceArg ? `--since ${sinceArg}` : "", slugArg ? `--slug ${slugArg}` : ""].filter(Boolean)
|
||||
console.log(`Cycle-window filters active (${parts.join(" ")}): M2/M3/M5 constrained to the window; M1/M4 unchanged`)
|
||||
console.log()
|
||||
}
|
||||
|
||||
console.log("## Per-Database Summaries")
|
||||
console.log()
|
||||
for (const db of allDbs) {
|
||||
const s = dbSummary(db)
|
||||
if (s) console.log(s)
|
||||
}
|
||||
|
||||
const totalSessions = allDbs.reduce((s, d) => s + d.sessionCount, 0)
|
||||
const totalMessages = allDbs.reduce((s, d) => s + d.messageCount, 0)
|
||||
console.log()
|
||||
console.log(
|
||||
`Total across ${allDbs.filter((d) => d.sessionCount > 0).length} databases: ${totalSessions} sessions, ${totalMessages.toLocaleString()} messages`,
|
||||
)
|
||||
|
||||
const sessionMedians = computeSessionMedians(allDbs)
|
||||
const sessionP90s = computeSessionP90s(allDbs)
|
||||
const sessionMaxes = computeSessionMaxes(allDbs)
|
||||
const cacheRatio = computeOverallCacheRatio(allDbs)
|
||||
|
||||
const sortedMedians = [...sessionMedians].sort((a, b) => a - b)
|
||||
const sortedP90s = [...sessionP90s].sort((a, b) => a - b)
|
||||
const sortedMaxes = [...sessionMaxes].sort((a, b) => a - b)
|
||||
|
||||
console.log()
|
||||
console.log("## Aggregate Input Token Stats (per-session metrics)")
|
||||
console.log()
|
||||
console.log("| Metric | p50 | p90 | max |")
|
||||
console.log("| ------ | --- | --- | --- |")
|
||||
console.log(
|
||||
`| Per-session median input | ${percentile(sortedMedians, 50).toLocaleString()} | ${percentile(sortedMedians, 90).toLocaleString()} | ${percentile(sortedMedians, 100).toLocaleString()} |`,
|
||||
)
|
||||
console.log(
|
||||
`| Per-session p90 input | ${percentile(sortedP90s, 50).toLocaleString()} | ${percentile(sortedP90s, 90).toLocaleString()} | ${percentile(sortedP90s, 100).toLocaleString()} |`,
|
||||
)
|
||||
console.log(
|
||||
`| Per-session max input | ${percentile(sortedMaxes, 50).toLocaleString()} | ${percentile(sortedMaxes, 90).toLocaleString()} | ${percentile(sortedMaxes, 100).toLocaleString()} |`,
|
||||
)
|
||||
|
||||
console.log()
|
||||
const ratioLabel = cacheRatio >= 1 ? `${cacheRatio.toFixed(1)}:1` : `1:${(1 / cacheRatio).toFixed(1)}`
|
||||
const light = trafficLight(cacheRatio)
|
||||
console.log(`## Cache Read / Input Ratio: ${ratioLabel} ${light}`)
|
||||
console.log()
|
||||
|
||||
const desc =
|
||||
cacheRatio > 10
|
||||
? "Excellent — context reuse is very high, indicating effective caching"
|
||||
: cacheRatio >= 3
|
||||
? "Moderate — reasonable cache hits, room for improvement"
|
||||
: "Low — consider strategies to increase context cache reuse"
|
||||
console.log(` ${desc}`)
|
||||
|
||||
const mergedAgents: Map<string, { msgCount: number; totalTokens: number }> = new Map()
|
||||
for (const db of allDbs) {
|
||||
for (const [agent, stats] of db.agents) {
|
||||
const existing = mergedAgents.get(agent)
|
||||
if (existing) {
|
||||
existing.msgCount += stats.msgCount
|
||||
existing.totalTokens += stats.totalTokens
|
||||
} else {
|
||||
mergedAgents.set(agent, { ...stats })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let exploreTokens = 0
|
||||
let workerTokens = 0
|
||||
for (const [agent, stats] of mergedAgents) {
|
||||
const lower = agent.toLowerCase()
|
||||
if (lower.includes("explorer")) exploreTokens += stats.totalTokens
|
||||
else if (lower.includes("worker")) workerTokens += stats.totalTokens
|
||||
}
|
||||
|
||||
const m5Ratio = workerTokens > 0 ? exploreTokens / workerTokens : 0
|
||||
|
||||
function m5TrafficLight(ratio: number): string {
|
||||
if (ratio > 2) return "🟢"
|
||||
if (ratio >= 1) return "🟡"
|
||||
return "🔴"
|
||||
}
|
||||
|
||||
console.log()
|
||||
console.log("## Explore / Execute Ratio (M5)")
|
||||
console.log()
|
||||
console.log("| Agent | Messages | Total tokens |")
|
||||
console.log("| --------- | -------- | ------------ |")
|
||||
|
||||
const sortedAgents = [...mergedAgents.entries()].sort((a, b) => b[1].totalTokens - a[1].totalTokens)
|
||||
for (const [agent, stats] of sortedAgents) {
|
||||
console.log(
|
||||
`| ${agent.padEnd(9)} | ${stats.msgCount.toLocaleString().padStart(7)} | ${stats.totalTokens.toLocaleString().padStart(12)} |`,
|
||||
)
|
||||
}
|
||||
|
||||
console.log()
|
||||
if (filtersActive && mergedAgents.size === 0) {
|
||||
console.log("[NOTE: no data in window for M5 — no messages match the requested window/slug]")
|
||||
} else {
|
||||
const m5Light = m5TrafficLight(m5Ratio)
|
||||
console.log(`Explore/Execute: ${m5Ratio.toFixed(2)}:1 ${m5Light}`)
|
||||
console.log()
|
||||
|
||||
const m5Desc =
|
||||
m5Ratio > 2
|
||||
? "Explorer-heavy — exploration dominates execution, good for discovery but may need more synthesis"
|
||||
: m5Ratio >= 1
|
||||
? "Balanced — reasonable split between exploration and execution"
|
||||
: "Execution-heavy — workers are spending tokens on discovery work that explorers should handle"
|
||||
console.log(` ${m5Desc}`)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Compactor activity — compact-frequency proxy (#2601 pilot data).
|
||||
// The compactor agent runs once per agent-initiated compaction, so its
|
||||
// message count in the window approximates how often compaction fired.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
console.log()
|
||||
console.log("## Compactor Activity (#2601 compact-frequency proxy)")
|
||||
console.log()
|
||||
const compactor = mergedAgents.get("compactor")
|
||||
if (compactor) {
|
||||
console.log(
|
||||
`compactor: ${compactor.msgCount.toLocaleString()} messages, ${compactor.totalTokens.toLocaleString()} tokens in window`,
|
||||
)
|
||||
console.log(" (per-run distribution = the #2601 pilot metric; rising zero-compact")
|
||||
console.log(" share for short runs (bugfix / DAG task) with no late-stage degradation retires the pilot gate)")
|
||||
} else {
|
||||
console.log("[NOTE: no compactor messages in window — zero agent-initiated compactions recorded]")
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// M2 — Stage token distribution (ledger-gated).
|
||||
// Reads token-stage-ledger.jsonl (written by .octopus/plugin/token-stage-ledger.ts),
|
||||
// reconstructs a per-root-session stage timeline, and attributes every
|
||||
// assistant message's tokens to the stage that was active when the message
|
||||
// was created. Skips cleanly when no ledger exists.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type LedgerEntry = { sessionID: string; stage: string; t: number }
|
||||
|
||||
function loadLedger(): Map<string, { stage: string; t: number }[]> | null {
|
||||
if (!existsSync(LEDGER_PATH)) return null
|
||||
const bySession = new Map<string, { stage: string; t: number }[]>()
|
||||
let any = false
|
||||
for (const line of readFileSync(LEDGER_PATH, "utf8").split("\n")) {
|
||||
const trimmed = line.trim()
|
||||
if (!trimmed) continue
|
||||
try {
|
||||
const e = JSON.parse(trimmed) as LedgerEntry
|
||||
const arr = bySession.get(e.sessionID) ?? []
|
||||
arr.push({ stage: e.stage, t: e.t })
|
||||
bySession.set(e.sessionID, arr)
|
||||
any = true
|
||||
} catch {
|
||||
// skip malformed lines
|
||||
}
|
||||
}
|
||||
if (!any) return null
|
||||
for (const arr of bySession.values()) arr.sort((a, b) => a.t - b.t)
|
||||
return bySession
|
||||
}
|
||||
|
||||
function stageTokensForDb(
|
||||
dbPath: string,
|
||||
ledger: Map<string, { stage: string; t: number }[]>,
|
||||
since = 0,
|
||||
slug?: string,
|
||||
): Map<string, number> | null {
|
||||
let db: Database | null = null
|
||||
try {
|
||||
db = new Database(dbPath, { readonly: true })
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
try {
|
||||
const sessions = loadSessionRows(db)
|
||||
if (!sessions) return null
|
||||
|
||||
// timelines keyed by root sessions present in this db
|
||||
const timelines = new Map<string, { stage: string; t: number }[]>()
|
||||
for (const [sid, entries] of ledger) {
|
||||
if (sessions.has(sid)) timelines.set(sid, entries)
|
||||
}
|
||||
if (timelines.size === 0) return null
|
||||
|
||||
let include: Set<string> | undefined
|
||||
if (slug !== undefined) {
|
||||
include = slugSessionIncludeSet(sessions, slug)
|
||||
if (include.size === 0) return null // no session matches the slug
|
||||
}
|
||||
|
||||
const rootOf = (id: string): string => {
|
||||
let cur = id
|
||||
let guard = 0
|
||||
while (guard++ < 100) {
|
||||
const parent = sessions.get(cur)?.parent_id
|
||||
if (!parent) break
|
||||
cur = parent
|
||||
}
|
||||
return cur
|
||||
}
|
||||
const stageAt = (rootId: string, time: number): string | null => {
|
||||
const tl = timelines.get(rootId)
|
||||
if (!tl) return null
|
||||
let stage: string | null = null
|
||||
for (const e of tl) {
|
||||
if (e.t <= time) stage = e.stage
|
||||
else break
|
||||
}
|
||||
return stage
|
||||
}
|
||||
|
||||
const byStage = new Map<string, number>()
|
||||
const rows = db
|
||||
.prepare(
|
||||
`
|
||||
SELECT session_id, time_created,
|
||||
CAST(json_extract(data, '$.tokens.input') AS REAL) as inp,
|
||||
CAST(json_extract(data, '$.tokens.output') AS REAL) as outp,
|
||||
CAST(json_extract(data, '$.tokens.reasoning') AS REAL) as rea
|
||||
FROM message
|
||||
WHERE json_extract(data, '$.role') = 'assistant'
|
||||
AND json_extract(data, '$.tokens.input') IS NOT NULL
|
||||
${since > 0 ? "AND time_created >= ?" : ""}
|
||||
`,
|
||||
)
|
||||
.all(...(since > 0 ? [since] : [])) as {
|
||||
session_id: string
|
||||
time_created: number
|
||||
inp: number | null
|
||||
outp: number | null
|
||||
rea: number | null
|
||||
}[]
|
||||
|
||||
for (const r of rows) {
|
||||
if (include !== undefined && !include.has(r.session_id)) continue
|
||||
const stage = stageAt(rootOf(r.session_id), r.time_created)
|
||||
if (!stage) continue
|
||||
const tokens = Number(r.inp ?? 0) + Number(r.outp ?? 0) + Number(r.rea ?? 0)
|
||||
byStage.set(stage, (byStage.get(stage) ?? 0) + tokens)
|
||||
}
|
||||
return byStage
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// M3 — Review rework fraction (#2591).
|
||||
// Two sources, merged with dedup by review identity `{slug}/reviews/{stage}`
|
||||
// (the ACTIVE status.json wins when both exist — it is canonical):
|
||||
// 1. ACTIVE runs — status.json under the Tier 1 location
|
||||
// .octopus/runs/{slug}/reviews/{stage}/status.json (reads history[],
|
||||
// legacy alias rounds[], current_round) and the legacy
|
||||
// .artifacts/**/reviews/*/status.json tree. In-flight runs only: the
|
||||
// active workspace is deleted at archive-at-close, so this source alone
|
||||
// structurally empties as runs close.
|
||||
// 2. ARCHIVED runs — the committed archive bundle
|
||||
// .octopus/runs/archive/{slug}.json. Bundles store digests, not
|
||||
// status.json content, so the per-review round count is reconstructed
|
||||
// from the documented Tier 1 layout `reviews/{stage}/round{N}/…`
|
||||
// (templates/runs-layout.md) by counting distinct roundN path segments
|
||||
// per stage across index.artifacts[].path. This archive source is what
|
||||
// makes M3 durable.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type ReviewRounds = Map<string, number> // `${slug}/reviews/${stage}` -> rounds
|
||||
|
||||
function collectActiveReviewRounds(roots: string[]): { rounds: ReviewRounds; startedMs: Map<string, number> } {
|
||||
const out: ReviewRounds = new Map()
|
||||
const startedMs = new Map<string, number>()
|
||||
const walk = (dir: string, top: string) => {
|
||||
let entries: ReturnType<typeof readdirSync>
|
||||
try {
|
||||
entries = readdirSync(dir, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const e of entries) {
|
||||
if (e.name === "_archive" || e.name === "archive") continue
|
||||
const full = join(dir, e.name)
|
||||
if (e.isDirectory()) walk(full, top)
|
||||
else if (e.name === "status.json" && dir.includes("/reviews/")) {
|
||||
try {
|
||||
const data = JSON.parse(readFileSync(full, "utf8")) as {
|
||||
rounds?: unknown[]
|
||||
history?: unknown[]
|
||||
current_round?: number
|
||||
started_at?: string
|
||||
}
|
||||
// Canonical field is `history[]` (per review-status.schema.json);
|
||||
// `rounds[]` is a legacy alias that maps to it. Prefer the array
|
||||
// forms; fall back to current_round.
|
||||
let rounds = 0
|
||||
if (Array.isArray(data.rounds) && data.rounds.length > 0) rounds = data.rounds.length
|
||||
else if (Array.isArray(data.history) && data.history.length > 0) rounds = data.history.length
|
||||
else if (typeof data.current_round === "number" && data.current_round > 0) rounds = data.current_round
|
||||
if (rounds > 0) {
|
||||
const key = relative(top, dir).split("\\").join("/")
|
||||
out.set(key, rounds)
|
||||
if (typeof data.started_at === "string") {
|
||||
const t = Date.parse(data.started_at)
|
||||
if (!Number.isNaN(t)) startedMs.set(key, t)
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// skip unreadable / malformed status files
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const root of roots) walk(root, root)
|
||||
return { rounds: out, startedMs }
|
||||
}
|
||||
|
||||
function collectBundleReviewRounds(archiveDir: string): { rounds: ReviewRounds; closedMs: Map<string, number> } {
|
||||
let files: string[] = []
|
||||
try {
|
||||
files = readdirSync(archiveDir).filter((f) => f.endsWith(".json"))
|
||||
} catch {
|
||||
return { rounds: new Map(), closedMs: new Map() } // no archive dir (e.g. a fresh checkout) — fine
|
||||
}
|
||||
const out: ReviewRounds = new Map()
|
||||
const closedMs = new Map<string, number>()
|
||||
for (const f of files) {
|
||||
try {
|
||||
const bundle = JSON.parse(readFileSync(join(archiveDir, f), "utf8")) as {
|
||||
index?: { artifacts?: { path?: unknown }[] }
|
||||
meta?: { closed_at?: string; updated_at?: string; created_at?: string }
|
||||
}
|
||||
// Identity = the bundle filename stem (= the archived run's workspace
|
||||
// dir name). meta.slug is NOT unique — epic task-node bundles carry the
|
||||
// parent epic slug while filenames stay per-node.
|
||||
const slug = f.replace(/\.json$/, "")
|
||||
const closedRaw = bundle.meta?.closed_at ?? bundle.meta?.updated_at ?? bundle.meta?.created_at
|
||||
// Distinct roundN segments per review stage across artifact paths
|
||||
// (paths may or may not carry the slug prefix — match the segment).
|
||||
const byStage = new Map<string, Set<string>>()
|
||||
for (const a of bundle.index?.artifacts ?? []) {
|
||||
if (typeof a?.path !== "string") continue
|
||||
const hit = a.path.match(/reviews\/([^/]+)\/(round\d+)\//)
|
||||
if (!hit?.[1] || !hit[2]) continue
|
||||
const set = byStage.get(hit[1]) ?? new Set<string>()
|
||||
set.add(hit[2])
|
||||
byStage.set(hit[1], set)
|
||||
}
|
||||
for (const [stage, rounds] of byStage) {
|
||||
if (rounds.size > 0) out.set(`${slug}/reviews/${stage}`, rounds.size)
|
||||
if (closedRaw !== undefined) {
|
||||
const t = Date.parse(closedRaw)
|
||||
if (!Number.isNaN(t)) closedMs.set(`${slug}/reviews/${stage}`, t)
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// skip unreadable / malformed bundles
|
||||
}
|
||||
}
|
||||
return { rounds: out, closedMs }
|
||||
}
|
||||
|
||||
// --- M2 output ---
|
||||
const ledger = loadLedger()
|
||||
console.log()
|
||||
console.log("## Stage Token Distribution (M2)")
|
||||
console.log()
|
||||
if (!ledger) {
|
||||
console.log("[NOTE: token-stage-ledger.jsonl absent — M2 skipped]")
|
||||
console.log(" (enable the .octopus/plugin/token-stage-ledger plugin to populate)")
|
||||
} else {
|
||||
const mergedStages = new Map<string, number>()
|
||||
for (const dbPath of dbFiles) {
|
||||
const byStage = stageTokensForDb(dbPath, ledger, windowOrSince, slugArg)
|
||||
if (!byStage) continue
|
||||
for (const [stage, tokens] of byStage) mergedStages.set(stage, (mergedStages.get(stage) ?? 0) + tokens)
|
||||
}
|
||||
const grandTotal = [...mergedStages.values()].reduce((a, b) => a + b, 0)
|
||||
if (grandTotal === 0) {
|
||||
if (filtersActive) {
|
||||
console.log("[NOTE: no data in window for M2 — no attributed tokens match the requested window/slug]")
|
||||
} else {
|
||||
console.log("[NOTE: ledger present but no sessions matched — M2 has no attributed data yet]")
|
||||
}
|
||||
} else {
|
||||
const sortedStages = [...mergedStages.entries()].sort((a, b) => b[1] - a[1])
|
||||
console.log("| Stage | Tokens | Share |")
|
||||
console.log("| ------------- | ------ | ----- |")
|
||||
for (const [stage, tokens] of sortedStages) {
|
||||
const pct = ((tokens / grandTotal) * 100).toFixed(1)
|
||||
console.log(`| ${stage.padEnd(13)} | ${tokens.toLocaleString().padStart(13)} | ${pct.padStart(5)}% |`)
|
||||
}
|
||||
const reviewTokens = mergedStages.get("review") ?? 0
|
||||
const reviewShare = (reviewTokens / grandTotal) * 100
|
||||
const m2Light = reviewShare > 60 ? "🔴" : reviewShare >= 35 ? "🟡" : "🟢"
|
||||
console.log()
|
||||
console.log(`Review-stage share: ${reviewShare.toFixed(1)}% ${m2Light}`)
|
||||
console.log(
|
||||
` ${
|
||||
reviewShare > 60
|
||||
? "Review dominates token spend — possible over-reviewing"
|
||||
: reviewShare >= 35
|
||||
? "Moderate review spend"
|
||||
: "Review spend is proportionate"
|
||||
}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// --- M3 output ---
|
||||
const runsDir = join(process.cwd(), ".octopus", "runs")
|
||||
const artifactsDir = join(process.cwd(), ".artifacts")
|
||||
const archiveDir = join(runsDir, "archive")
|
||||
const activeRoots = [runsDir, artifactsDir].filter((d) => existsSync(d))
|
||||
console.log()
|
||||
console.log("## Review Rework (M3)")
|
||||
console.log()
|
||||
const active = collectActiveReviewRounds(activeRoots)
|
||||
const archived = collectBundleReviewRounds(archiveDir)
|
||||
const activeRounds = active.rounds
|
||||
const archivedRounds = archived.rounds
|
||||
const mergedRounds: ReviewRounds = new Map(activeRounds)
|
||||
let archivedOnly = 0
|
||||
for (const [key, rounds] of archivedRounds) {
|
||||
if (mergedRounds.has(key)) continue // active status.json is canonical
|
||||
mergedRounds.set(key, rounds)
|
||||
archivedOnly++
|
||||
}
|
||||
// Cycle-window filters: constrain to reviews whose run identity contains the
|
||||
// slug and whose start (active) / close (archived) time falls in the window.
|
||||
// Reviews without a parseable timestamp are excluded when a window is set —
|
||||
// strict, so filtered numbers never silently fall back to all-time totals.
|
||||
const windowRounds: ReviewRounds = new Map()
|
||||
for (const [key, rounds] of mergedRounds) {
|
||||
const identity = key.split("/reviews/")[0] ?? key
|
||||
if (slugArg !== undefined && !identity.includes(slugArg)) continue
|
||||
if (windowSince !== undefined) {
|
||||
const t = active.startedMs.get(key) ?? archived.closedMs.get(key)
|
||||
if (t === undefined || t < windowSince) continue
|
||||
}
|
||||
windowRounds.set(key, rounds)
|
||||
}
|
||||
if (mergedRounds.size === 0) {
|
||||
console.log(
|
||||
activeRoots.length === 0 && archivedRounds.size === 0
|
||||
? "[NOTE: no .octopus/runs or .artifacts directory in cwd — M3 skipped]"
|
||||
: "[NOTE: no review rounds found (active status.json or archive bundles) — M3 skipped]",
|
||||
)
|
||||
} else if (filtersActive && windowRounds.size === 0) {
|
||||
console.log("[NOTE: no data in window for M3 — no reviews match the requested window/slug]")
|
||||
} else {
|
||||
const roundsMap = filtersActive ? windowRounds : mergedRounds
|
||||
const reviews = roundsMap.size
|
||||
const totalRounds = [...roundsMap.values()].reduce((a, b) => a + b, 0)
|
||||
const reworkRounds = [...roundsMap.values()].reduce((a, b) => a + (b - 1), 0)
|
||||
const nonFirstPass = [...roundsMap.values()].filter((r) => r > 1).length
|
||||
const fraction = totalRounds > 0 ? reworkRounds / totalRounds : 0
|
||||
const nonFirstPct = (nonFirstPass / reviews) * 100
|
||||
const m3Light = fraction > 0.3 ? "🔴" : fraction >= 0.15 ? "🟡" : "🟢"
|
||||
if (filtersActive)
|
||||
console.log(`Window filter: ${windowRounds.size}/${mergedRounds.size} reviews match (--since/--slug)`)
|
||||
console.log(`Reviews: ${reviews} | total rounds: ${totalRounds} | rework rounds: ${reworkRounds}`)
|
||||
console.log(
|
||||
`Sources: ${activeRounds.size} active status.json + ${archivedOnly} archive bundles (dedup by slug+stage)`,
|
||||
)
|
||||
console.log(`Non-first-pass reviews: ${nonFirstPass}/${reviews} (${nonFirstPct.toFixed(0)}%)`)
|
||||
console.log()
|
||||
console.log(`Rework fraction: ${(fraction * 100).toFixed(1)}% ${m3Light}`)
|
||||
console.log(
|
||||
` ${
|
||||
fraction > 0.3
|
||||
? "High rework — review findings not actionable or design unclear"
|
||||
: fraction >= 0.15
|
||||
? "Moderate rework — some review churn"
|
||||
: "Low rework — reviews converge efficiently"
|
||||
}`,
|
||||
)
|
||||
}
|
||||
Reference in New Issue
Block a user