fix(render): kill blown highlights + phantom doors (GPU-verified)
First GPU pass on the rev-3 depth pipeline surfaced two issues; both fixed
and re-rendered clean:
1. Blown near-field highlights (regression from 018909b). Mapping the
nearest pixel to pure white (nearPlane=nearRaw) made the depth
ControlNet render overexposed near floors/ceilings at strength 0.85.
fpv.js: nearPlane=0 (anchor white at the camera; nearest real surface
≈gray 214, headroom intact) + minSpan 5→8. Re-bake confirms the blown
ceiling is gone while recesses still reach black and geometry holds.
2. Phantom doors on closed walls — a PROMPT problem, not the depth map
(present in old maps too). Opening NOUNS in the positive prompt summon
a passage onto solid stone: threshold's "gatehouse, portcullis,
entrance" painted a gate on a blank wall, and at cfg 1.0 the negative
can't cancel it. bake_atlas.py: THEME_BODY_CLOSED (full theme flavor,
zero opening nouns) for closed facings + opening-nouns negated on
closed facings only. r0_N (threshold) now renders clean ashlar wall.
("Bricked-up archway" backfires — "archway" alone re-summons the arch.)
Also: sweep_strength.py re-exec-under-venv guard (the venv python is a
symlink to the base interpreter, so the realpath check wrongly skipped
the hop and the montage died after every render); now env-sentinel
guarded. Findings in .dev/2026-06-15-diffusion-pipeline-reliability.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
e0cbd3d66b
commit
5506c38a51
4 changed files with 100 additions and 16 deletions
|
|
@ -59,6 +59,38 @@ With a weak signal you crank strength to force adherence; a strong signal wants
|
||||||
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the
|
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the
|
||||||
old `acc8137`/`a12bcbb` bakes.
|
old `acc8137`/`a12bcbb` bakes.
|
||||||
|
|
||||||
|
## Iteration 3 — first GPU pass (2026-06-15/16, workstation back up)
|
||||||
|
|
||||||
|
Ran the A/B + strength sweep (16 renders) and follow-ups. Findings:
|
||||||
|
|
||||||
|
1. **Open frames: the rev-3 normalization works.** `r8_N` renders a clean pillared
|
||||||
|
hall with the passage landing exactly on the depth's black recess. NEW ≈ OLD here
|
||||||
|
only because `r8_N` already had structure; the win shows on small-room recesses.
|
||||||
|
2. **Blown-highlight regression (introduced by rev-3, now FIXED).** Mapping the
|
||||||
|
nearest pixel to pure white (`nearPlane = nearRaw`) made the ControlNet render
|
||||||
|
overexposed near floors/ceilings at strength 0.85. Fix: `nearPlane = 0` (anchor
|
||||||
|
white at the camera, nearest real surface ≈ gray 214) + `minSpan = 8`. Re-baked +
|
||||||
|
re-rendered: blown ceiling gone, geometry adherence intact, recesses still black.
|
||||||
|
3. **Phantom doors on closed walls = a PROMPT problem, not depth.** Present in OLD
|
||||||
|
maps too. Root cause: opening-NOUNS in the positive prompt. `threshold`'s body
|
||||||
|
("dungeon gatehouse, iron portcullis, entrance") painted a gate onto a solid
|
||||||
|
wall; at cfg 1.0 the negative can't cancel it. A generic opening-free body
|
||||||
|
("rough-hewn stone chamber, bare granite walls") renders a clean dead-end.
|
||||||
|
"Bricked-up archway" BACKFIRES — "archway" alone re-summons the arch.
|
||||||
|
→ Fix in `bake_atlas.py`: `THEME_BODY_CLOSED` (theme flavor, zero opening nouns)
|
||||||
|
for closed facings + opening-nouns added to the negative on closed facings only.
|
||||||
|
|
||||||
|
**Tooling fixes:** `sweep_strength.py` re-exec-under-venv guard (the venv's python is
|
||||||
|
a symlink to the base interpreter, so the realpath check wrongly skipped the hop —
|
||||||
|
now an env-sentinel guard); montage now succeeds in one invocation.
|
||||||
|
|
||||||
|
### Still open after iteration 3
|
||||||
|
- **Pick the production strength.** Sweep rendered 0.55/0.7/0.85/1.0; 0.85 looks good
|
||||||
|
on open frames now. Eyeball the full montage for the lowest strength that still
|
||||||
|
honors geometry (lower = more natural texture). Then set `bake_atlas.py --strength`.
|
||||||
|
- **Re-bake a full atlas** (seed 7) with rev-3 maps + closed-body prompts and montage
|
||||||
|
vs the old `a12bcbb` bake to confirm the end-to-end win across all themes.
|
||||||
|
|
||||||
## Open levers if still hit-and-miss after strength tuning
|
## Open levers if still hit-and-miss after strength tuning
|
||||||
- `nearPlane`/`minSpan`/`gamma` are now tunable per-export — A/B if near surfaces
|
- `nearPlane`/`minSpan`/`gamma` are now tunable per-export — A/B if near surfaces
|
||||||
read too blown-out (the AFTER maps are ~47% near-white; may want a slightly
|
read too blown-out (the AFTER maps are ~47% near-white; may want a slightly
|
||||||
|
|
|
||||||
|
|
@ -186,14 +186,20 @@ export function computeFPVDepth(env, opts = {}) {
|
||||||
if (x === centerX) ahead = dist;
|
if (x === centerX) ahead = dist;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Per-frame normalization window. The far plane tracks the deepest surface
|
// Per-frame normalization window.
|
||||||
// actually visible (so a real passage reads black), but is clamped to a floor
|
// nearPlane = 0 (the camera), NOT the nearest surface: a GPU sweep showed
|
||||||
// (minSpan) so a shallow dead-end nook isn't stretched into a dramatic fake
|
// that mapping the closest pixel to pure white (255) made the depth
|
||||||
// tunnel — its faced wall stays an honest mid-gray slab, not a black recess
|
// ControlNet render blown-out, overexposed near floors/ceilings. Anchoring
|
||||||
// the model would paint a phantom door into. `gamma` then shapes the falloff
|
// white at the camera leaves the nearest real surface (~1 cell) at ~gray
|
||||||
// within that window.
|
// 214 — bright, but with headroom — which renders as lit stone, not glare.
|
||||||
const nearPlane = opts.nearPlane ?? nearRaw;
|
// farPlane tracks the deepest surface actually visible so a real passage
|
||||||
const minSpan = opts.minSpan ?? 5;
|
// reads black, but is floored (minSpan) so a shallow dead-end isn't
|
||||||
|
// stretched into a fake tunnel — its faced wall stays a brighter mid-gray
|
||||||
|
// slab. (Phantom doors on fully-closed walls are a separate camera-vantage
|
||||||
|
// problem, not a normalization one — the faced wall is unavoidably darker
|
||||||
|
// than the near floor; see the .dev reliability note.)
|
||||||
|
const nearPlane = opts.nearPlane ?? 0;
|
||||||
|
const minSpan = opts.minSpan ?? 8;
|
||||||
const farPlane = clamp(farRaw, nearPlane + minSpan, maxView);
|
const farPlane = clamp(farRaw, nearPlane + minSpan, maxView);
|
||||||
const span = Math.max(farPlane - nearPlane, 1e-3);
|
const span = Math.max(farPlane - nearPlane, 1e-3);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -53,21 +53,48 @@ THEME_BODY = {
|
||||||
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
|
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
|
||||||
}
|
}
|
||||||
DEFAULT_BODY = THEME_BODY["stone"]
|
DEFAULT_BODY = THEME_BODY["stone"]
|
||||||
|
# Closed-facing bodies: SAME theme flavor (material, contents, mood) but every
|
||||||
|
# architectural OPENING noun stripped — gatehouse/portcullis/entrance, wall
|
||||||
|
# niches, the throne's recessed dais, etc. A GPU sweep (2026-06-15) proved the
|
||||||
|
# phantom doors on closed walls were summoned by these words, not the depth map:
|
||||||
|
# `threshold`'s "gatehouse, portcullis, entrance" painted a gate onto a solid
|
||||||
|
# wall, and at cfg 1.0 the negative branch is too weak to cancel it. Working WITH
|
||||||
|
# a generic solid-wall description (no opening nouns) renders a clean dead-end;
|
||||||
|
# even "bricked-up archway" backfires because "archway" alone re-summons the arch.
|
||||||
|
THEME_BODY_CLOSED = {
|
||||||
|
"forge": "forge chamber, soot-blackened granite walls, iron anvils and hanging chains, dim fiery glow",
|
||||||
|
"cistern": "flooded cistern, dark still water, dripping wet stone walls, cold teal light",
|
||||||
|
"crypt": "ancient crypt, carved stone sarcophagi along blank stone walls, cobwebs, cold pale light",
|
||||||
|
"library": "ruined library, tall stone shelves of rotting books against the wall, drifting dust, warm dim light",
|
||||||
|
"throne": "throne hall, towering carved stone columns and a blank stone wall, tattered banners, cold grandeur",
|
||||||
|
"vault": "treasure vault, iron-bound chests and stone strongboxes against heavy locked stone walls, scattered gold",
|
||||||
|
"hall": "grand pillared hall, carved stone columns and a high blank stone wall, vaulted ceiling",
|
||||||
|
"den": "foul beast den, gnawed bones and filth, claw-scratched blank stone walls, dim red light",
|
||||||
|
"threshold": "heavy sealed gateroom of dungeon stone, worn flagstones, blank ashlar walls",
|
||||||
|
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
|
||||||
|
}
|
||||||
# These reinforce the depth (a deep recess vs a near wall straight ahead) so the
|
# These reinforce the depth (a deep recess vs a near wall straight ahead) so the
|
||||||
# model paints a passage only where one actually exists — killing phantom doors.
|
# model paints a passage only where one actually exists — killing phantom doors.
|
||||||
AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead"
|
AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead"
|
||||||
# cfg 1.0 makes negatives inert, so the closed case is carried by emphatic
|
# cfg 1.0 makes negatives inert, so the closed case is carried by emphatic
|
||||||
# positive description — themes like the vault say "locked", which otherwise
|
# positive description (opening-free body above) + this sealed-wall clause.
|
||||||
# summons phantom doors on blank walls.
|
AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of solid carved stone blocks, "
|
||||||
AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of carved stone blocks, "
|
"sealed flat masonry, a dead end")
|
||||||
"sealed solid masonry with no door and no opening, a dead end")
|
|
||||||
NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, "
|
NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, "
|
||||||
"holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern")
|
"holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern")
|
||||||
|
# Closed facings additionally negate every opening noun (cheap insurance; the
|
||||||
|
# opening-free positive body does the real work). NOT applied to open facings —
|
||||||
|
# they WANT a passage.
|
||||||
|
NEG_OPENINGS = "doorway, door, archway, arch, opening, passage, tunnel, gate, portcullis, gateway, entrance, corridor"
|
||||||
|
|
||||||
|
|
||||||
def prompt_for(theme, is_open):
|
def prompt_for(theme, is_open):
|
||||||
|
"""Return (positive, negative) for this (theme, facing)."""
|
||||||
|
if is_open:
|
||||||
body = THEME_BODY.get(theme or "stone", DEFAULT_BODY)
|
body = THEME_BODY.get(theme or "stone", DEFAULT_BODY)
|
||||||
return f"{STYLE}{body}, {AHEAD_OPEN if is_open else AHEAD_WALL}"
|
return f"{STYLE}{body}, {AHEAD_OPEN}", NEGATIVE
|
||||||
|
body = THEME_BODY_CLOSED.get(theme or "stone", THEME_BODY_CLOSED["stone"])
|
||||||
|
return f"{STYLE}{body}, {AHEAD_WALL}", f"{NEGATIVE}, {NEG_OPENINGS}"
|
||||||
|
|
||||||
NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$")
|
NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$")
|
||||||
|
|
||||||
|
|
@ -127,7 +154,7 @@ def main():
|
||||||
room = rooms.get(int(rid), {})
|
room = rooms.get(int(rid), {})
|
||||||
theme = room.get("theme")
|
theme = room.get("theme")
|
||||||
is_open = bool(room.get("open", {}).get(d, False))
|
is_open = bool(room.get("open", {}).get(d, False))
|
||||||
prompt = prompt_for(theme, is_open)
|
prompt, negative = prompt_for(theme, is_open)
|
||||||
strength = a.strength
|
strength = a.strength
|
||||||
# Walls render under their own FIXED seed: the open-frame seed's
|
# Walls render under their own FIXED seed: the open-frame seed's
|
||||||
# favorite composition is a centered doorway — exactly what a closed
|
# favorite composition is a centered doorway — exactly what a closed
|
||||||
|
|
@ -140,7 +167,7 @@ def main():
|
||||||
if not run(["python3", COMFY, "zrender", a.url, "--depth", depth,
|
if not run(["python3", COMFY, "zrender", a.url, "--depth", depth,
|
||||||
"--width", str(a.width), "--height", str(a.height),
|
"--width", str(a.width), "--height", str(a.height),
|
||||||
"--strength", str(strength), "--seed", str(render_seed),
|
"--strength", str(strength), "--seed", str(render_seed),
|
||||||
"--prompt", prompt, "--negative", NEGATIVE,
|
"--prompt", prompt, "--negative", negative,
|
||||||
"--prefix", prefix, "--wait", "300"]):
|
"--prefix", prefix, "--wait", "300"]):
|
||||||
print(f" {tag} RENDER FAILED"); continue
|
print(f" {tag} RENDER FAILED"); continue
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -25,6 +25,25 @@ REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)
|
||||||
COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py")
|
COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py")
|
||||||
VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python")
|
VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python")
|
||||||
|
|
||||||
|
# The final montage needs Pillow, which only lives in the pixelart venv; the
|
||||||
|
# renders themselves shell out to `python3 comfy.py` (stdlib only). If we were
|
||||||
|
# launched under a Python without PIL, re-exec under the venv so one invocation
|
||||||
|
# both renders and montages. (Without this the whole sweep runs, then dies at the
|
||||||
|
# last step — wasting every render.)
|
||||||
|
if __name__ == "__main__":
|
||||||
|
try:
|
||||||
|
import PIL # noqa: F401
|
||||||
|
except ImportError:
|
||||||
|
# The venv's bin/python is a symlink back to the base interpreter, so a
|
||||||
|
# realpath() comparison wrongly reports "already in the venv" and skips
|
||||||
|
# the hop — what matters is sys.path (venv site-packages), not the binary.
|
||||||
|
# Guard the re-exec with an env sentinel so a still-missing PIL raises
|
||||||
|
# instead of looping forever.
|
||||||
|
if os.path.exists(VENV_PY) and not os.environ.get("_SWEEP_REEXEC"):
|
||||||
|
os.environ["_SWEEP_REEXEC"] = "1"
|
||||||
|
os.execv(VENV_PY, [VENV_PY, os.path.abspath(__file__), *sys.argv[1:]])
|
||||||
|
raise
|
||||||
|
|
||||||
# Mirror bake_atlas.py's proven prompt skeleton so the sweep reflects production.
|
# Mirror bake_atlas.py's proven prompt skeleton so the sweep reflects production.
|
||||||
STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, "
|
STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, "
|
||||||
"wet stone, volumetric haze, architectural photography of an empty ruin, "
|
"wet stone, volumetric haze, architectural photography of an empty ruin, "
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue