Files
octopus-workflow/core/rules/testing.md
T

8.5 KiB
Raw Blame History

Core 中立版(Increment 2 改写)。术语对照见 core/adapters/TERMINOLOGY.md;本组织实例表述见 dogfood 对应文件。

Testing

  • Avoid mocks as much as possible
  • Test actual implementation, do not duplicate logic into tests
  • Tests cannot run from repo root (guard: do-not-run-tests-from-root); run from package dirs.
  • Local pre-check: run <test-cmd:changed> from the owning package after editing source. It selects only tests affected by your uncommitted + branch changes (path-mapping CodeGraph direct dependents) and runs them in a single process — seconds, machine stays responsive. Use --print to see the selection without running, --all to scan beyond the home package.
  • Before a delivery push: <test-cmd:changed> green + <typecheck-cmd> clean — same selector as CI's PR side (local parity, [org-internal #4425]). The full suite (<test-cmd:parallel>) stays with nightly / run-full-tests / manual pre-merge runs. 实例命令绑定见 core/adapters/TERMINOLOGY.md。
  • Never use the bare single-process test runner for the full suite; <test-cmd:parallel> shards it and process-isolates known-corrupting files. Parallelism is controlled by TEST_SHARDS (default 3; raise it on beefy CI runners, e.g. TEST_SHARDS=7).

Pre-existing test failures must be filed, not silently logged

When the full suite surfaces a failure that is NOT caused by the current change (it reproduces on the base branch / last-green commit too), it is a pre-existing / baseline failure. Do NOT silently drop it as "someone else's problem" and do NOT bury it in a report body.

  • Classify it: run the failing test on the base commit (see verify Phase 2.6). Fails there too → pre-existing (BF); passes there → regression you must fix.
  • De-duplicate BEFORE filing (retro [org-internal #2696]): search existing baseline-failure tickets — open AND closed — by test/file name to surface candidates, then match by failure signature (error signature + affected surface — one root-cause family may span several tests; see the ticket-lifecycle rule §"BF / FT family umbrellas"). On a signature hit, attach run evidence to the family umbrella; do NOT create a new ticket.
  • File it: the Verifier registers every un-tracked pre-existing failure as a ticket-backend issue labeled baseline-failure with a BF-NNN id (verify Phase 5.55). The number is repo-wide sequential: highest existing NNN across ALL baseline-failure tickets (open AND closed) + 1 — duplicate ids forbidden, including against closed tickets.
  • A pre-existing failure does not block the iteration that discovered it, but the iteration is NOT Done until it is filed — an un-filed baseline failure is a process gap.

Flaky tests must be filed, not silently tolerated

When a test passes and fails intermittently across runs (non-deterministic), it is a flaky test. Do NOT silently re-run until it passes and move on.

  • Detect: re-run the suite (or use the project's flaky detector). Record every test that shows intermittent behavior — name, failure rate, and run count.
  • De-duplicate BEFORE filing (retro [org-internal #2696]): same signature-based procedure as baseline failures above, against existing flaky-test tickets.
  • File it: the Verifier registers every un-tracked flaky test as a ticket-backend issue labeled flaky-test with an FT-NNN id (verify Phase 5.56). FT-NNN allocation follows the same repo-wide sequential rule as BF-NNN, across ALL flaky-test tickets (open AND closed).
  • A flaky test does not block the iteration verdict, but the iteration is NOT Done until every unfixed flaky test is filed — an un-filed flaky test is a process gap.

Filing is the whole job. If the failure was NOT caused by your change, do NOT fix it in place — classify, file, and return to your task per the session-scope-guard rule. A repository-wide flaky test is a shared magnet — fixing it in place drifts off-task and collides with its owner ([org-internal #1803]).

Testing Kobalte components with happydom

Kobalte's portal/context-based components (Collapsible, DropdownMenu, ContextMenu, FormControlContext) do not work reliably under happydom's DOM environment. For these components, use source-verification tests — assert that the correct JSX elements, props, and Kobalte primitives are present in the component's source rather than attempting full DOM rendering.

When implementing a component that uses these Kobalte primitives, prefer source-verification over render tests. The same applies to SolidJS SVG <filter>/<mask>/<linearGradient> elements, which happydom also cannot render reliably.

Test file naming convention (one file per component)

A single component MUST NOT carry both a .test.ts and a .test.tsx test file — the two-file split duplicated ~3300 lines of overlapping assertions across one UI package [org-internal #1712]. Use exactly one file per component, chosen by test style:

Test style Extension When to use
Source-verification .test.ts Kobalte portal/context components, SVG <filter>/<mask>/<linearGradient> components, and any component happydom cannot render reliably. Asserts against the component source.
DOM render / interaction .test.tsx Pure Solid components that render cleanly under happydom. Uses a testing-library harness.

Rules:

  • One file per component. If a component needs both source-verification and DOM-render assertions, keep them in a single file matching the dominant style; do not split into a .test.ts/.test.tsx pair.
  • When merging an existing dual pair, carry over any assertion unique to the deleted file into the surviving file before deletion.
  • The app package's .sv.test.* suffix is a separate convention for storybook/storyshots source-verification and is unaffected.

Source-verification assertion boundaries

Source-verification tests (per the Kobalte/happydom carve-out above) must assert structural contracts, not implementation literals.

  • Assert: props wiring, event handler attachment, i18n key presence, ARIA role/attributes present in JSX, namespace correctness, export shape.
  • Do NOT assert: exact source-string literals (event.preventDefault(), 'block: "nearest"', event.movementX === 0), branch operators counted via regex, or any expression that changes when a constant is renamed.
  • Do NOT write tautologiessource.includes("false") || source.includes("!") is true for nearly all TSX.
  • One source-verification file per component (see naming convention above). When two test files read the same source, merge into one.
  • Prefer behavioral tests — when source-verification and a behavioral/ interaction test cover the same contract, prefer the behavioral test; collapse the SV duplicate.

Namespace gate and test assertions

The namespace gate (pre-commit hook + the check-namespace script) scans ALL string literals in .ts/.tsx/.json files under the product packages for a reserved vendor namespace. Do NOT write test assertions containing that namespace literal — the gate cannot distinguish a test's namespace-absence check from an actual namespace violation, and will fail the commit.

Instead, use patterns that avoid the literal:

const NAMESPACE = "vendor-prefix"
expect(source).not.toContain(`@${NAMESPACE}`)

Single test root (per package)

Each package MUST have exactly ONE test root — no parallel tracks (src/**/__tests__/, a tests/ dir alongside colocated src/, or a test/ alongside tests/); a second root drifts and breaks test discovery ([org-internal #1711], extended to all packages by [org-internal #1821]).

Rules:

  • New tests: create them at the package's single root from the start.
  • Do NOT create src/**/__tests__/ directories in any package — parallel tracks drift (residuals eliminated by [org-internal #1821]).
  • For test/-root packages: mirror the src/ path (test/<src-relative>.test.ts); type-level tests live alongside the behavioral suites in the config area.
  • For colocated-root packages: keep tests next to the source file; one test file per component [org-internal #1712].

Per-package root tables live in the instance layer (they name concrete package paths); the invariant itself — ONE root per package — is universal.