reikhelm/.dev/2026-06-15-diffusion-pipeline-reliability.md

Diffusion pipeline reliability — making the depth ControlNet actually followed

Started: 2026-06-15 · ongoing (/loop) Problem (user): "controlnet maps generated are barely followed; images of the dungeons are very hit and miss."

Root cause found (iteration 1)

The FPV depth maps fed to the Z-Image Fun depth ControlNet used only a sliver of the tonal range. Histogram over all 68 baked seed-7 frames:

• median dynamic range 193 / 255, worst frame only 54
• 0/68 frames reached true black (<32); the deepest recesses topped out ~mid-gray
• only 35/68 reached near-white at the near surfaces

Cause in reikhelm-web/fpv.js: a fixed t = dpix / maxView with maxView = 12. The nearest floor sits ~1 cell away (never hit white) and a wall 3 cells ahead landed at mid-gray (never black), so small rooms collapsed into the 160–214 band. Depth ControlNets are trained on full-range MiDaS / depth-anything maps, so a washed-out map gives almost no structural signal → the model freelances.

Fix shipped — commit 018909b

Two-pass raycast in computeFPVDepth:

• Pass 1 records each column's wall distance + the frame's true near/far.
• Pass 2 maps depth through a per-frame window: near → white, deepest visible recess → black, every frame.
• Far plane tracks the deepest real surface but is clamped to minSpan (5 cells) so a shallow dead-end nook is NOT stretched into a fake tunnel — its faced wall stays an honest mid-gray slab (no black recess → no phantom-door bait).
• gamma 2.0 → 1.5 (normalization now owns the range). nearPlane/minSpan are opts.

Verified offline (headless bakeAtlasDepths(7), 68 frames): median range 193 → 255; near-white frames 35/68 → 68/68; worst range 54 → 106. Visual before/after (/tmp/ba.png during the session): open frames now frame a black passage; closed frames a clean mid-gray wall.

Next — needs the render workstation (192.168.1.26:8188, was OFFLINE this iteration)

The depth signal is now strong, so the old strength is almost certainly wrong. With a weak signal you crank strength to force adherence; a strong signal wants less. bake_atlas.py defaults --strength 0.85.

1. Re-probe the workstation: python3 tools/comfy-spike/comfy.py probe http://192.168.1.26:8188 (2026-06-15 iter 2: host pings but port 8188 closed — ComfyUI not running; needs a manual start on the workstation before any of the below.)
2. A/B + strength sweep in one run (tool written & validated; old rev-2 maps persisted to tools/out/compare/rev2_depths/ so the fix is provable, not just "looks better"):

   python3 tools/comfy-spike/sweep_strength.py http://192.168.1.26:8188 \
       --seed-dir tools/out/atlas/7 --alt-work tools/out/compare/rev2_depths \
       --open r8_N --closed r0_N --strengths 0.55,0.7,0.85,1.0
   

   → tools/out/compare/sweep_<run>.png: each frame gets a NEW row (full-range maps) and an OLD row (washed-out rev-2) across every strength. Read adherence: does the rendered passage land on the black recess? does the closed wall stay a wall (no phantom door)? Expect NEW ≫ OLD; pick the lowest strength that still honors geometry, update bake_atlas.py --strength 0.85 default. (Drop --alt-work once the win is confirmed.)
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the old acc8137/a12bcbb bakes.

Iteration 3 — first GPU pass (2026-06-15/16, workstation back up)

Ran the A/B + strength sweep (16 renders) and follow-ups. Findings:

1. Open frames: the rev-3 normalization works. r8_N renders a clean pillared hall with the passage landing exactly on the depth's black recess. NEW ≈ OLD here only because r8_N already had structure; the win shows on small-room recesses.
2. Blown-highlight regression (introduced by rev-3, now FIXED). Mapping the nearest pixel to pure white (nearPlane = nearRaw) made the ControlNet render overexposed near floors/ceilings at strength 0.85. Fix: nearPlane = 0 (anchor white at the camera, nearest real surface ≈ gray 214) + minSpan = 8. Re-baked + re-rendered: blown ceiling gone, geometry adherence intact, recesses still black.
3. Phantom doors on closed walls = a PROMPT problem, not depth. Present in OLD maps too. Root cause: opening-NOUNS in the positive prompt. threshold's body ("dungeon gatehouse, iron portcullis, entrance") painted a gate onto a solid wall; at cfg 1.0 the negative can't cancel it. A generic opening-free body ("rough-hewn stone chamber, bare granite walls") renders a clean dead-end. "Bricked-up archway" BACKFIRES — "archway" alone re-summons the arch. → Fix in bake_atlas.py: THEME_BODY_CLOSED (theme flavor, zero opening nouns) for closed facings + opening-nouns added to the negative on closed facings only.

Tooling fixes: sweep_strength.py re-exec-under-venv guard (the venv's python is a symlink to the base interpreter, so the realpath check wrongly skipped the hop — now an env-sentinel guard); montage now succeeds in one invocation.

Iteration 3 end-to-end result — VALIDATED ✅

Full seed-7 re-bake (67/68; one transient ComfyUI miss, re-rendered) through rev-3 maps + closed-body prompts. Old-vs-new montage (tools/out/compare/):

• r0_N (threshold, closed): OLD painted a grand gateway + portcullis on a solid wall; NEW renders a clean ashlar wall with torches. Phantom door gone.
• r16_E (throne, closed): clean wall with banners/columns — theme flavor survives with zero opening nouns. Generalizes past threshold.
• r16_N (throne, open) / r8_N (hall, open): rich themed scenes, passage exactly on the depth recess. Forge lava / crypt sarcophagi / library shelves / den all good.
• Strength 0.85 kept — honors geometry without stiffness on the high-contrast maps.

Campaign upgrade (in progress)

DESCENT loads these atlases directly (game.js setFrame /out/atlas/<seed>/r<room>_<dir>.png). Campaign = seeds 7, 69, 59 (game-web/game.js CAMPAIGN), all previously baked at the broken rev-2 (a12bcbb). Seed 7 done. Re-baking 69 + 59 through rev-3 (depths re-exported headless, old frames backed up to compare/old_bake_<seed>/).

Iteration 4 — QA sweep + strength tune (2026-06-16)

Systematic QA of all 208 baked campaign frames (not just spot-checks):

• Closed walls (83): zero phantom doors. Ranked every closed frame by center-vs-flanking-wall darkness; even the worst (r6_W forge, r11_E crypt, r0_W den) are genuine solid walls — the dark center is lighting falloff / a real stone mass, not a recess. The closed-body prompt fix holds campaign-wide.
• Open frames (125): median 0.18%% blown. Low-contrast flags were false positives (atmospheric dark throne halls). One real pattern: deep-corridor frames — mostly crypt — blew out a white ceiling band at the top edge (r9_N 27.7%%, r13_N 21%%, r11_N 18%%).
• Root cause + fix: the depth's near-ceiling top edge is gray ~209 (bright); at strength 0.85 the ControlNet forced it to overexposed white, amplified by crypt's "pale light". GPU A/B on r9_N: strength 0.70 → 0.0%% blown (vs 27.7%% at 0.85) with geometry intact; de-paling the prompt did nothing (cfg-1 negatives inert). Verified r0_N closed wall stays solid at 0.70 → one global strength is safe. Set bake_atlas.py --strength 0.85 → 0.70 (commit b661989). Campaign re-baked at 0.70; 0.85 frames archived under tools/out/compare/str85_<seed>/.

Optional later

• The single-frame retry-on-empty in bake_atlas would avoid the occasional ComfyUI miss.

Open levers if still hit-and-miss after strength tuning

• nearPlane/minSpan/gamma are now tunable per-export — A/B if near surfaces read too blown-out (the AFTER maps are ~47% near-white; may want a slightly deeper nearPlane to retain near-field texture).
• Subtle masonry-block relief on closed-wall slices so flat walls render as detailed stone (offline-buildable, but verify on GPU it doesn't read as structure).
• Strength start_percent/end_percent scheduling (currently 0→1 full) — easing control off in late steps frees texture while locking composition.