feat(pipeline): A/B mode for strength sweep (old-vs-new depth maps)

--alt-work renders each test frame from a second depth dir alongside the
new full-range maps, so the GPU run proves the rev-3 normalization fix
(018909b) beats the washed-out rev-2 maps rather than just looking
better. Persisted the rev-2 maps under tools/out/compare/rev2_depths so
the A/B is reproducible. Doc updated with the one-shot command.

Workstation note: host 192.168.1.26 is up but ComfyUI (:8188) wasn't
running this iteration — needs a manual start before the sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Parley Hatch 2026-06-15 21:18:13 -06:00
parent 53cd1023e0
commit e0cbd3d66b
2 changed files with 27 additions and 17 deletions

View file

@ -42,15 +42,20 @@ With a weak signal you crank strength to force adherence; a strong signal wants
*less*. `bake_atlas.py` defaults `--strength 0.85`.
1. Re-probe the workstation: `python3 tools/comfy-spike/comfy.py probe http://192.168.1.26:8188`
2. **Strength sweep** (tool already written, validated):
(2026-06-15 iter 2: host pings but **port 8188 closed — ComfyUI not running**; needs a
manual start on the workstation before any of the below.)
2. **A/B + strength sweep in one run** (tool written & validated; old rev-2 maps persisted
to `tools/out/compare/rev2_depths/` so the fix is provable, not just "looks better"):
```
python3 tools/comfy-spike/sweep_strength.py http://192.168.1.26:8188 \
--seed-dir tools/out/atlas/7 --open r8_N --closed r0_N \
--strengths 0.55,0.7,0.85,1.0
--seed-dir tools/out/atlas/7 --alt-work tools/out/compare/rev2_depths \
--open r8_N --closed r0_N --strengths 0.55,0.7,0.85,1.0
```
`tools/out/compare/sweep_<run>.png`. Read adherence: does the rendered passage
land on the black recess? does the closed wall stay a wall (no phantom door)?
Pick the lowest strength that still honors geometry, update `bake_atlas.py` default.
`tools/out/compare/sweep_<run>.png`: each frame gets a NEW row (full-range maps) and
an OLD row (washed-out rev-2) across every strength. Read adherence: does the rendered
passage land on the black recess? does the closed wall stay a wall (no phantom door)?
Expect NEW ≫ OLD; pick the lowest strength that still honors geometry, update
`bake_atlas.py --strength 0.85` default. (Drop `--alt-work` once the win is confirmed.)
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the
old `acc8137`/`a12bcbb` bakes.

View file

@ -73,6 +73,8 @@ def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("url")
ap.add_argument("--seed-dir", default=os.path.join(REPO, "tools", "out", "atlas", "7"))
ap.add_argument("--alt-work", default="", help="optional 2nd depth dir to A/B (e.g. tools/out/compare/rev2_depths "
"= the old washed-out maps); adds an OLD row per frame so old-vs-new adherence is conclusive")
ap.add_argument("--open", default="r8_N", help="frame id for the open-passage test")
ap.add_argument("--closed", default="r0_N", help="frame id for the closed-wall test")
ap.add_argument("--strengths", default="0.55,0.7,0.85,1.0")
@ -84,21 +86,24 @@ def main():
strengths = [float(s) for s in a.strengths.split(",") if s.strip()]
run = os.getpid()
cases = [(a.open, OPEN_BODY), (a.closed, WALL_BODY)]
# (source-tag, depth-dir): NEW always; OLD too when --alt-work is given.
sources = [("NEW", work)] + ([("OLD", a.alt_work)] if a.alt_work else [])
rows = []
for frame, body in cases:
depth = os.path.join(work, f"{frame}_depth.png")
if not os.path.exists(depth):
sys.exit(f"!! missing depth map {depth} — bake the atlas first")
prompt = STYLE + body
imgs = {}
for s in strengths:
prefix = f"sweep_{frame}_{int(s*100)}_{run}"
print(f"[{frame} @ strength {s}] rendering ...", flush=True)
out = render(a.url, depth, prompt, s, a.seed, prefix, a.wait)
if out:
imgs[s] = out
rows.append((frame, depth, imgs))
for tag, srcdir in sources:
depth = os.path.join(srcdir, f"{frame}_depth.png")
if not os.path.exists(depth):
sys.exit(f"!! missing depth map {depth} — bake the atlas first")
imgs = {}
for s in strengths:
prefix = f"sweep_{frame}_{tag}_{int(s*100)}_{run}"
print(f"[{frame}·{tag} @ strength {s}] rendering ...", flush=True)
out = render(a.url, depth, prompt, s, a.seed, prefix, a.wait)
if out:
imgs[s] = out
rows.append((f"{frame}·{tag}", depth, imgs))
out_dir = os.path.join(REPO, "tools", "out", "compare")
os.makedirs(out_dir, exist_ok=True)