fix(render): kill blown highlights + phantom doors (GPU-verified)

First GPU pass on the rev-3 depth pipeline surfaced two issues; both fixed
and re-rendered clean:

1. Blown near-field highlights (regression from 018909b). Mapping the
   nearest pixel to pure white (nearPlane=nearRaw) made the depth
   ControlNet render overexposed near floors/ceilings at strength 0.85.
   fpv.js: nearPlane=0 (anchor white at the camera; nearest real surface
   ≈gray 214, headroom intact) + minSpan 5→8. Re-bake confirms the blown
   ceiling is gone while recesses still reach black and geometry holds.

2. Phantom doors on closed walls — a PROMPT problem, not the depth map
   (present in old maps too). Opening NOUNS in the positive prompt summon
   a passage onto solid stone: threshold's "gatehouse, portcullis,
   entrance" painted a gate on a blank wall, and at cfg 1.0 the negative
   can't cancel it. bake_atlas.py: THEME_BODY_CLOSED (full theme flavor,
   zero opening nouns) for closed facings + opening-nouns negated on
   closed facings only. r0_N (threshold) now renders clean ashlar wall.
   ("Bricked-up archway" backfires — "archway" alone re-summons the arch.)

Also: sweep_strength.py re-exec-under-venv guard (the venv python is a
symlink to the base interpreter, so the realpath check wrongly skipped
the hop and the montage died after every render); now env-sentinel
guarded. Findings in .dev/2026-06-15-diffusion-pipeline-reliability.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Parley Hatch 2026-06-15 21:57:49 -06:00
parent e0cbd3d66b
commit 5506c38a51
4 changed files with 100 additions and 16 deletions

View file

@ -59,6 +59,38 @@ With a weak signal you crank strength to force adherence; a strong signal wants
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the 3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the
old `acc8137`/`a12bcbb` bakes. old `acc8137`/`a12bcbb` bakes.
## Iteration 3 — first GPU pass (2026-06-15/16, workstation back up)
Ran the A/B + strength sweep (16 renders) and follow-ups. Findings:
1. **Open frames: the rev-3 normalization works.** `r8_N` renders a clean pillared
hall with the passage landing exactly on the depth's black recess. NEW ≈ OLD here
only because `r8_N` already had structure; the win shows on small-room recesses.
2. **Blown-highlight regression (introduced by rev-3, now FIXED).** Mapping the
nearest pixel to pure white (`nearPlane = nearRaw`) made the ControlNet render
overexposed near floors/ceilings at strength 0.85. Fix: `nearPlane = 0` (anchor
white at the camera, nearest real surface ≈ gray 214) + `minSpan = 8`. Re-baked +
re-rendered: blown ceiling gone, geometry adherence intact, recesses still black.
3. **Phantom doors on closed walls = a PROMPT problem, not depth.** Present in OLD
maps too. Root cause: opening-NOUNS in the positive prompt. `threshold`'s body
("dungeon gatehouse, iron portcullis, entrance") painted a gate onto a solid
wall; at cfg 1.0 the negative can't cancel it. A generic opening-free body
("rough-hewn stone chamber, bare granite walls") renders a clean dead-end.
"Bricked-up archway" BACKFIRES — "archway" alone re-summons the arch.
→ Fix in `bake_atlas.py`: `THEME_BODY_CLOSED` (theme flavor, zero opening nouns)
for closed facings + opening-nouns added to the negative on closed facings only.
**Tooling fixes:** `sweep_strength.py` re-exec-under-venv guard (the venv's python is
a symlink to the base interpreter, so the realpath check wrongly skipped the hop —
now an env-sentinel guard); montage now succeeds in one invocation.
### Still open after iteration 3
- **Pick the production strength.** Sweep rendered 0.55/0.7/0.85/1.0; 0.85 looks good
on open frames now. Eyeball the full montage for the lowest strength that still
honors geometry (lower = more natural texture). Then set `bake_atlas.py --strength`.
- **Re-bake a full atlas** (seed 7) with rev-3 maps + closed-body prompts and montage
vs the old `a12bcbb` bake to confirm the end-to-end win across all themes.
## Open levers if still hit-and-miss after strength tuning ## Open levers if still hit-and-miss after strength tuning
- `nearPlane`/`minSpan`/`gamma` are now tunable per-export — A/B if near surfaces - `nearPlane`/`minSpan`/`gamma` are now tunable per-export — A/B if near surfaces
read too blown-out (the AFTER maps are ~47% near-white; may want a slightly read too blown-out (the AFTER maps are ~47% near-white; may want a slightly

View file

@ -186,14 +186,20 @@ export function computeFPVDepth(env, opts = {}) {
if (x === centerX) ahead = dist; if (x === centerX) ahead = dist;
} }
// Per-frame normalization window. The far plane tracks the deepest surface // Per-frame normalization window.
// actually visible (so a real passage reads black), but is clamped to a floor // nearPlane = 0 (the camera), NOT the nearest surface: a GPU sweep showed
// (minSpan) so a shallow dead-end nook isn't stretched into a dramatic fake // that mapping the closest pixel to pure white (255) made the depth
// tunnel — its faced wall stays an honest mid-gray slab, not a black recess // ControlNet render blown-out, overexposed near floors/ceilings. Anchoring
// the model would paint a phantom door into. `gamma` then shapes the falloff // white at the camera leaves the nearest real surface (~1 cell) at ~gray
// within that window. // 214 — bright, but with headroom — which renders as lit stone, not glare.
const nearPlane = opts.nearPlane ?? nearRaw; // farPlane tracks the deepest surface actually visible so a real passage
const minSpan = opts.minSpan ?? 5; // reads black, but is floored (minSpan) so a shallow dead-end isn't
// stretched into a fake tunnel — its faced wall stays a brighter mid-gray
// slab. (Phantom doors on fully-closed walls are a separate camera-vantage
// problem, not a normalization one — the faced wall is unavoidably darker
// than the near floor; see the .dev reliability note.)
const nearPlane = opts.nearPlane ?? 0;
const minSpan = opts.minSpan ?? 8;
const farPlane = clamp(farRaw, nearPlane + minSpan, maxView); const farPlane = clamp(farRaw, nearPlane + minSpan, maxView);
const span = Math.max(farPlane - nearPlane, 1e-3); const span = Math.max(farPlane - nearPlane, 1e-3);

View file

@ -53,21 +53,48 @@ THEME_BODY = {
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient", "stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
} }
DEFAULT_BODY = THEME_BODY["stone"] DEFAULT_BODY = THEME_BODY["stone"]
# Closed-facing bodies: SAME theme flavor (material, contents, mood) but every
# architectural OPENING noun stripped — gatehouse/portcullis/entrance, wall
# niches, the throne's recessed dais, etc. A GPU sweep (2026-06-15) proved the
# phantom doors on closed walls were summoned by these words, not the depth map:
# `threshold`'s "gatehouse, portcullis, entrance" painted a gate onto a solid
# wall, and at cfg 1.0 the negative branch is too weak to cancel it. Working WITH
# a generic solid-wall description (no opening nouns) renders a clean dead-end;
# even "bricked-up archway" backfires because "archway" alone re-summons the arch.
THEME_BODY_CLOSED = {
"forge": "forge chamber, soot-blackened granite walls, iron anvils and hanging chains, dim fiery glow",
"cistern": "flooded cistern, dark still water, dripping wet stone walls, cold teal light",
"crypt": "ancient crypt, carved stone sarcophagi along blank stone walls, cobwebs, cold pale light",
"library": "ruined library, tall stone shelves of rotting books against the wall, drifting dust, warm dim light",
"throne": "throne hall, towering carved stone columns and a blank stone wall, tattered banners, cold grandeur",
"vault": "treasure vault, iron-bound chests and stone strongboxes against heavy locked stone walls, scattered gold",
"hall": "grand pillared hall, carved stone columns and a high blank stone wall, vaulted ceiling",
"den": "foul beast den, gnawed bones and filth, claw-scratched blank stone walls, dim red light",
"threshold": "heavy sealed gateroom of dungeon stone, worn flagstones, blank ashlar walls",
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
}
# These reinforce the depth (a deep recess vs a near wall straight ahead) so the # These reinforce the depth (a deep recess vs a near wall straight ahead) so the
# model paints a passage only where one actually exists — killing phantom doors. # model paints a passage only where one actually exists — killing phantom doors.
AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead" AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead"
# cfg 1.0 makes negatives inert, so the closed case is carried by emphatic # cfg 1.0 makes negatives inert, so the closed case is carried by emphatic
# positive description — themes like the vault say "locked", which otherwise # positive description (opening-free body above) + this sealed-wall clause.
# summons phantom doors on blank walls. AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of solid carved stone blocks, "
AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of carved stone blocks, " "sealed flat masonry, a dead end")
"sealed solid masonry with no door and no opening, a dead end")
NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, " NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, "
"holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern") "holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern")
# Closed facings additionally negate every opening noun (cheap insurance; the
# opening-free positive body does the real work). NOT applied to open facings —
# they WANT a passage.
NEG_OPENINGS = "doorway, door, archway, arch, opening, passage, tunnel, gate, portcullis, gateway, entrance, corridor"
def prompt_for(theme, is_open): def prompt_for(theme, is_open):
body = THEME_BODY.get(theme or "stone", DEFAULT_BODY) """Return (positive, negative) for this (theme, facing)."""
return f"{STYLE}{body}, {AHEAD_OPEN if is_open else AHEAD_WALL}" if is_open:
body = THEME_BODY.get(theme or "stone", DEFAULT_BODY)
return f"{STYLE}{body}, {AHEAD_OPEN}", NEGATIVE
body = THEME_BODY_CLOSED.get(theme or "stone", THEME_BODY_CLOSED["stone"])
return f"{STYLE}{body}, {AHEAD_WALL}", f"{NEGATIVE}, {NEG_OPENINGS}"
NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$") NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$")
@ -127,7 +154,7 @@ def main():
room = rooms.get(int(rid), {}) room = rooms.get(int(rid), {})
theme = room.get("theme") theme = room.get("theme")
is_open = bool(room.get("open", {}).get(d, False)) is_open = bool(room.get("open", {}).get(d, False))
prompt = prompt_for(theme, is_open) prompt, negative = prompt_for(theme, is_open)
strength = a.strength strength = a.strength
# Walls render under their own FIXED seed: the open-frame seed's # Walls render under their own FIXED seed: the open-frame seed's
# favorite composition is a centered doorway — exactly what a closed # favorite composition is a centered doorway — exactly what a closed
@ -140,7 +167,7 @@ def main():
if not run(["python3", COMFY, "zrender", a.url, "--depth", depth, if not run(["python3", COMFY, "zrender", a.url, "--depth", depth,
"--width", str(a.width), "--height", str(a.height), "--width", str(a.width), "--height", str(a.height),
"--strength", str(strength), "--seed", str(render_seed), "--strength", str(strength), "--seed", str(render_seed),
"--prompt", prompt, "--negative", NEGATIVE, "--prompt", prompt, "--negative", negative,
"--prefix", prefix, "--wait", "300"]): "--prefix", prefix, "--wait", "300"]):
print(f" {tag} RENDER FAILED"); continue print(f" {tag} RENDER FAILED"); continue

View file

@ -25,6 +25,25 @@ REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)
COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py") COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py")
VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python") VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python")
# The final montage needs Pillow, which only lives in the pixelart venv; the
# renders themselves shell out to `python3 comfy.py` (stdlib only). If we were
# launched under a Python without PIL, re-exec under the venv so one invocation
# both renders and montages. (Without this the whole sweep runs, then dies at the
# last step — wasting every render.)
if __name__ == "__main__":
try:
import PIL # noqa: F401
except ImportError:
# The venv's bin/python is a symlink back to the base interpreter, so a
# realpath() comparison wrongly reports "already in the venv" and skips
# the hop — what matters is sys.path (venv site-packages), not the binary.
# Guard the re-exec with an env sentinel so a still-missing PIL raises
# instead of looping forever.
if os.path.exists(VENV_PY) and not os.environ.get("_SWEEP_REEXEC"):
os.environ["_SWEEP_REEXEC"] = "1"
os.execv(VENV_PY, [VENV_PY, os.path.abspath(__file__), *sys.argv[1:]])
raise
# Mirror bake_atlas.py's proven prompt skeleton so the sweep reflects production. # Mirror bake_atlas.py's proven prompt skeleton so the sweep reflects production.
STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, " STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, "
"wet stone, volumetric haze, architectural photography of an empty ruin, " "wet stone, volumetric haze, architectural photography of an empty ruin, "