SkillPedia Skills Encyclopedia
中文 EN
Today's updates 2026-11-18 UTC

No new commits yet today

Open the update feed

Skills 2026-10-10 23:18 UTC

skillsmith

Committed by wrsmith108. Update Category: Bug Fixes & Stability Improvements. Commit details: fix(scripts): SMI-7067 -- the gate-test guard counts every gate spawn, and a failed A/B test fails the experiment (#3054). This update resolves known runtime errors and enhances system reliability.

wrsmith108 f8c90b9 smith-horn/skillsmith

CommitRaw commit message

fix(scripts): SMI-7067 -- the gate-test guard counts every gate spawn, and a failed A/B test fails the experiment (#3054)

* fix(scripts): SMI-7067 -- the gate-test guard counts every gate spawn, and a failed A/B test fails the experiment

The post-merge governance retro on 9608a21fa (PR #3053) found no defect in
the transform() fix and four problems in that PR's new code, all confirmed
against main.

1. MEDIUM. The afterEach timeout guard was vacuous for two tests. Only
   runGate() counted runs, but "a compiler that hangs is still INCONCLUSIVE
   when the parent ignores or blocks SIGALRM" spawns the gate in a loop with
   spawnSync, and "backstop: an INCONCLUSIVE arm that falls through..." runs
   it through its own run() helper. The guard saw 0 runs for both and
   always passed. Now one countGateRun() is called by all three spawn sites.
   Red-tested, expectation declared first: with both tests set to
   gateBudget(1), exactly those two failed in the guard, each counted at 2
   runs; the other 29 passed. Both now declare gateBudget(2).
2. MEDIUM. The experiment counted a failed A/B test as completed. runABTest
   keeps only successful Claude invocations but returns a result even when a
   side is empty, so a run with `claude` missing printed "N A/B test(s)
   completed" and exited 0. abTestFailed() flags a result with no samples on
   either side; main() counts it in tally.abFailed, logs it, keeps it out of
   the report, and experimentExitCode() fails the run on it.
3. LOW. A comment claimed "a REAL compile takes 10-30s" under CI load; no
   such range was measured. The rule is now stated without the number.
4. LOW. MULTI_RUN_TIMEOUT_MS (360s) was stale and below gateBudget(3); its
   two users make 1 and 2 runs. They now declare gateBudget(1) and
   gateBudget(2), which the guard confirms, and the constant is deleted.

Red-tested in the worktree container, expectations declared first, each
mutant confirmed delivered and restored by hash:
- A1 (exit code ignores abFailed): only the A/B exit test fails.
- A2 (|| to && in abTestFailed): the two one-sided cases fail; both-empty
  passes.
- A3 survived at first -- its marker comment "(continue removed)" contained
  the word the wiring test looked for. The test now requires a `continue`
  statement on its own line; A3b (statement removed, neutral marker) fails it.
- A4 (abFailed increment removed): both wiring tests fail.
Helpers suite 23/23; gate suite 31 passed, 1 skipped (root-only), 89.6s;
typecheck:scripts PASS (268 checked, 0 errors); eslint, prettier clean.

Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: Claude <noreply@anthropic.com>

* test(scripts): SMI-7067 -- a gate spawn cannot go uncounted, and the A/B wiring test checks order

PR #3054's cross-family gate (on a13671747) found no defect in shipped
behaviour and two test-strength gaps in a13671747's own tests.

1. Counting was a manual countGateRun() call at each direct-spawn site.
   Deleting one left a count of 1 against gateBudget(2), which the guard
   accepts. Now spawnGate() is the only way the file runs the gate: it
   counts, then spawns, so a counted run cannot be skipped or moved after
   the spawn. runGate() and both direct-spawn tests use it, and those two
   tests assert their EXACT run count (wrappers.length; 2), not merely a
   count within budget.
2. The A/B wiring test checked that the increment and `continue` were
   PRESENT in the failed-A/B block, not their order; an increment moved
   after `continue` is unreachable yet present. It now asserts guard <
   increment < continue < push.

Red-tested in the worktree container, expectations declared first, each
mutant confirmed delivered and restored by hash:
- R1 (hang test spawns with spawnSync, bypassing spawnGate): "every
  wrapper ran the gate through spawnGate: expected +0 to be 2".
- R2 (backstop run() bypasses spawnGate): "control and subject both ran
  through spawnGate: expected +0 to be 2".
- R3 as first written DELETED the increment instead of moving it, and
  carried no marker; it failed the presence check, which proves nothing
  about order. R3b (increment moved after `continue`, marked) passed the
  presence checks and failed "increment before continue", as declared.
Helpers 23/23; gate suite 31 passed, 1 skipped (root-only), 93.5s;
prettier clean.

Co-Authored-By: claude-flow <ruv@ruv.net>
Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Ryan Smith <wrsmith108@users.noreply.github.com>
Co-authored-by: claude-flow <ruv@ruv.net>
Co-authored-by: Claude <noreply@anthropic.com>