reikhelm/.dev/2026-06-15-diffusion-pipeline-reliability.md

Diffusion pipeline reliability — making the depth ControlNet actually followed

Started: 2026-06-15 · ongoing (/loop) Problem (user): "controlnet maps generated are barely followed; images of the dungeons are very hit and miss."

Root cause found (iteration 1)

The FPV depth maps fed to the Z-Image Fun depth ControlNet used only a sliver of the tonal range. Histogram over all 68 baked seed-7 frames:

• median dynamic range 193 / 255, worst frame only 54
• 0/68 frames reached true black (<32); the deepest recesses topped out ~mid-gray
• only 35/68 reached near-white at the near surfaces

Cause in reikhelm-web/fpv.js: a fixed t = dpix / maxView with maxView = 12. The nearest floor sits ~1 cell away (never hit white) and a wall 3 cells ahead landed at mid-gray (never black), so small rooms collapsed into the 160–214 band. Depth ControlNets are trained on full-range MiDaS / depth-anything maps, so a washed-out map gives almost no structural signal → the model freelances.

Fix shipped — commit 018909b

Two-pass raycast in computeFPVDepth:

• Pass 1 records each column's wall distance + the frame's true near/far.
• Pass 2 maps depth through a per-frame window: near → white, deepest visible recess → black, every frame.
• Far plane tracks the deepest real surface but is clamped to minSpan (5 cells) so a shallow dead-end nook is NOT stretched into a fake tunnel — its faced wall stays an honest mid-gray slab (no black recess → no phantom-door bait).
• gamma 2.0 → 1.5 (normalization now owns the range). nearPlane/minSpan are opts.

Verified offline (headless bakeAtlasDepths(7), 68 frames): median range 193 → 255; near-white frames 35/68 → 68/68; worst range 54 → 106. Visual before/after (/tmp/ba.png during the session): open frames now frame a black passage; closed frames a clean mid-gray wall.

Next — needs the render workstation (192.168.1.26:8188, was OFFLINE this iteration)

The depth signal is now strong, so the old strength is almost certainly wrong. With a weak signal you crank strength to force adherence; a strong signal wants less. bake_atlas.py defaults --strength 0.85.

1. Re-probe the workstation: python3 tools/comfy-spike/comfy.py probe http://192.168.1.26:8188 (2026-06-15 iter 2: host pings but port 8188 closed — ComfyUI not running; needs a manual start on the workstation before any of the below.)
2. A/B + strength sweep in one run (tool written & validated; old rev-2 maps persisted to tools/out/compare/rev2_depths/ so the fix is provable, not just "looks better"):

   python3 tools/comfy-spike/sweep_strength.py http://192.168.1.26:8188 \
       --seed-dir tools/out/atlas/7 --alt-work tools/out/compare/rev2_depths \
       --open r8_N --closed r0_N --strengths 0.55,0.7,0.85,1.0
   

   → tools/out/compare/sweep_<run>.png: each frame gets a NEW row (full-range maps) and an OLD row (washed-out rev-2) across every strength. Read adherence: does the rendered passage land on the black recess? does the closed wall stay a wall (no phantom door)? Expect NEW ≫ OLD; pick the lowest strength that still honors geometry, update bake_atlas.py --strength 0.85 default. (Drop --alt-work once the win is confirmed.)
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the old acc8137/a12bcbb bakes.

Iteration 3 — first GPU pass (2026-06-15/16, workstation back up)

Ran the A/B + strength sweep (16 renders) and follow-ups. Findings:

1. Open frames: the rev-3 normalization works. r8_N renders a clean pillared hall with the passage landing exactly on the depth's black recess. NEW ≈ OLD here only because r8_N already had structure; the win shows on small-room recesses.
2. Blown-highlight regression (introduced by rev-3, now FIXED). Mapping the nearest pixel to pure white (nearPlane = nearRaw) made the ControlNet render overexposed near floors/ceilings at strength 0.85. Fix: nearPlane = 0 (anchor white at the camera, nearest real surface ≈ gray 214) + minSpan = 8. Re-baked + re-rendered: blown ceiling gone, geometry adherence intact, recesses still black.
3. Phantom doors on closed walls = a PROMPT problem, not depth. Present in OLD maps too. Root cause: opening-NOUNS in the positive prompt. threshold's body ("dungeon gatehouse, iron portcullis, entrance") painted a gate onto a solid wall; at cfg 1.0 the negative can't cancel it. A generic opening-free body ("rough-hewn stone chamber, bare granite walls") renders a clean dead-end. "Bricked-up archway" BACKFIRES — "archway" alone re-summons the arch. → Fix in bake_atlas.py: THEME_BODY_CLOSED (theme flavor, zero opening nouns) for closed facings + opening-nouns added to the negative on closed facings only.

Tooling fixes: sweep_strength.py re-exec-under-venv guard (the venv's python is a symlink to the base interpreter, so the realpath check wrongly skipped the hop — now an env-sentinel guard); montage now succeeds in one invocation.

Still open after iteration 3

• Pick the production strength. Sweep rendered 0.55/0.7/0.85/1.0; 0.85 looks good on open frames now. Eyeball the full montage for the lowest strength that still honors geometry (lower = more natural texture). Then set bake_atlas.py --strength.
• Re-bake a full atlas (seed 7) with rev-3 maps + closed-body prompts and montage vs the old a12bcbb bake to confirm the end-to-end win across all themes.

Open levers if still hit-and-miss after strength tuning

• nearPlane/minSpan/gamma are now tunable per-export — A/B if near surfaces read too blown-out (the AFTER maps are ~47% near-white; may want a slightly deeper nearPlane to retain near-field texture).
• Subtle masonry-block relief on closed-wall slices so flat walls render as detailed stone (offline-buildable, but verify on GPU it doesn't read as structure).
• Strength start_percent/end_percent scheduling (currently 0→1 full) — easing control off in late steps frees texture while locking composition.