Add route simulation quality summary

This commit is contained in:
2026-06-27 20:08:11 +02:00
parent eb1bdbf305
commit de6615c024
4 changed files with 192 additions and 0 deletions
@@ -39,6 +39,10 @@ The map audit currently sees:
- Route simulation now has an opt-in multi-seed sweep, and the smoke suite runs
a three-seed sweep so representative route/noise checks are not proven by one
lucky seed only.
- Route simulation now emits a `quality` summary that groups route health by
target, action family, and position family, separates route issues from
coverage/seed-check issues, buckets issue types, and reports weakest cases so
future prompt-logic passes can target the worst path first.
- Map audit now fails when a registered ComfyUI node display name is missing
from the route map or README, so utility nodes cannot silently drift out of
user-facing documentation.
+7
View File
@@ -1028,6 +1028,13 @@ issues for:
- multi-seed route sweeps that repeat the same route/noise/seed checks across
spaced seeds to catch random-pool drift hidden by a single clean seed.
The report also includes a `quality` section. This is the high-level progress
view for path cleanup: it groups route cases by target, action family, and
position family; counts route issues separately from coverage/seed-check
issues; buckets issue types such as label leaks, softcore noise, trace
mismatches, trigger drift, or reroll drift; and lists the weakest cases first
when a sweep finds failures.
Use `--json --include-prompts` when you need the exact raw and formatted text
for debugging a route. Use `--sweep-count 5 --seed-step 101` when changing pool
selection, route terms, or formatter noise rules and you need more than one