fix(render): kill blown highlights + phantom doors (GPU-verified)
First GPU pass on the rev-3 depth pipeline surfaced two issues; both fixed
and re-rendered clean:
1. Blown near-field highlights (regression from 018909b). Mapping the
nearest pixel to pure white (nearPlane=nearRaw) made the depth
ControlNet render overexposed near floors/ceilings at strength 0.85.
fpv.js: nearPlane=0 (anchor white at the camera; nearest real surface
≈gray 214, headroom intact) + minSpan 5→8. Re-bake confirms the blown
ceiling is gone while recesses still reach black and geometry holds.
2. Phantom doors on closed walls — a PROMPT problem, not the depth map
(present in old maps too). Opening NOUNS in the positive prompt summon
a passage onto solid stone: threshold's "gatehouse, portcullis,
entrance" painted a gate on a blank wall, and at cfg 1.0 the negative
can't cancel it. bake_atlas.py: THEME_BODY_CLOSED (full theme flavor,
zero opening nouns) for closed facings + opening-nouns negated on
closed facings only. r0_N (threshold) now renders clean ashlar wall.
("Bricked-up archway" backfires — "archway" alone re-summons the arch.)
Also: sweep_strength.py re-exec-under-venv guard (the venv python is a
symlink to the base interpreter, so the realpath check wrongly skipped
the hop and the montage died after every render); now env-sentinel
guarded. Findings in .dev/2026-06-15-diffusion-pipeline-reliability.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
e0cbd3d66b
commit
5506c38a51
4 changed files with 100 additions and 16 deletions
|
|
@ -59,6 +59,38 @@ With a weak signal you crank strength to force adherence; a strong signal wants
|
|||
3. Then re-bake a full atlas at the chosen strength and spot-check a montage vs the
|
||||
old `acc8137`/`a12bcbb` bakes.
|
||||
|
||||
## Iteration 3 — first GPU pass (2026-06-15/16, workstation back up)
|
||||
|
||||
Ran the A/B + strength sweep (16 renders) and follow-ups. Findings:
|
||||
|
||||
1. **Open frames: the rev-3 normalization works.** `r8_N` renders a clean pillared
|
||||
hall with the passage landing exactly on the depth's black recess. NEW ≈ OLD here
|
||||
only because `r8_N` already had structure; the win shows on small-room recesses.
|
||||
2. **Blown-highlight regression (introduced by rev-3, now FIXED).** Mapping the
|
||||
nearest pixel to pure white (`nearPlane = nearRaw`) made the ControlNet render
|
||||
overexposed near floors/ceilings at strength 0.85. Fix: `nearPlane = 0` (anchor
|
||||
white at the camera, nearest real surface ≈ gray 214) + `minSpan = 8`. Re-baked +
|
||||
re-rendered: blown ceiling gone, geometry adherence intact, recesses still black.
|
||||
3. **Phantom doors on closed walls = a PROMPT problem, not depth.** Present in OLD
|
||||
maps too. Root cause: opening-NOUNS in the positive prompt. `threshold`'s body
|
||||
("dungeon gatehouse, iron portcullis, entrance") painted a gate onto a solid
|
||||
wall; at cfg 1.0 the negative can't cancel it. A generic opening-free body
|
||||
("rough-hewn stone chamber, bare granite walls") renders a clean dead-end.
|
||||
"Bricked-up archway" BACKFIRES — "archway" alone re-summons the arch.
|
||||
→ Fix in `bake_atlas.py`: `THEME_BODY_CLOSED` (theme flavor, zero opening nouns)
|
||||
for closed facings + opening-nouns added to the negative on closed facings only.
|
||||
|
||||
**Tooling fixes:** `sweep_strength.py` re-exec-under-venv guard (the venv's python is
|
||||
a symlink to the base interpreter, so the realpath check wrongly skipped the hop —
|
||||
now an env-sentinel guard); montage now succeeds in one invocation.
|
||||
|
||||
### Still open after iteration 3
|
||||
- **Pick the production strength.** Sweep rendered 0.55/0.7/0.85/1.0; 0.85 looks good
|
||||
on open frames now. Eyeball the full montage for the lowest strength that still
|
||||
honors geometry (lower = more natural texture). Then set `bake_atlas.py --strength`.
|
||||
- **Re-bake a full atlas** (seed 7) with rev-3 maps + closed-body prompts and montage
|
||||
vs the old `a12bcbb` bake to confirm the end-to-end win across all themes.
|
||||
|
||||
## Open levers if still hit-and-miss after strength tuning
|
||||
- `nearPlane`/`minSpan`/`gamma` are now tunable per-export — A/B if near surfaces
|
||||
read too blown-out (the AFTER maps are ~47% near-white; may want a slightly
|
||||
|
|
|
|||
|
|
@ -186,14 +186,20 @@ export function computeFPVDepth(env, opts = {}) {
|
|||
if (x === centerX) ahead = dist;
|
||||
}
|
||||
|
||||
// Per-frame normalization window. The far plane tracks the deepest surface
|
||||
// actually visible (so a real passage reads black), but is clamped to a floor
|
||||
// (minSpan) so a shallow dead-end nook isn't stretched into a dramatic fake
|
||||
// tunnel — its faced wall stays an honest mid-gray slab, not a black recess
|
||||
// the model would paint a phantom door into. `gamma` then shapes the falloff
|
||||
// within that window.
|
||||
const nearPlane = opts.nearPlane ?? nearRaw;
|
||||
const minSpan = opts.minSpan ?? 5;
|
||||
// Per-frame normalization window.
|
||||
// nearPlane = 0 (the camera), NOT the nearest surface: a GPU sweep showed
|
||||
// that mapping the closest pixel to pure white (255) made the depth
|
||||
// ControlNet render blown-out, overexposed near floors/ceilings. Anchoring
|
||||
// white at the camera leaves the nearest real surface (~1 cell) at ~gray
|
||||
// 214 — bright, but with headroom — which renders as lit stone, not glare.
|
||||
// farPlane tracks the deepest surface actually visible so a real passage
|
||||
// reads black, but is floored (minSpan) so a shallow dead-end isn't
|
||||
// stretched into a fake tunnel — its faced wall stays a brighter mid-gray
|
||||
// slab. (Phantom doors on fully-closed walls are a separate camera-vantage
|
||||
// problem, not a normalization one — the faced wall is unavoidably darker
|
||||
// than the near floor; see the .dev reliability note.)
|
||||
const nearPlane = opts.nearPlane ?? 0;
|
||||
const minSpan = opts.minSpan ?? 8;
|
||||
const farPlane = clamp(farRaw, nearPlane + minSpan, maxView);
|
||||
const span = Math.max(farPlane - nearPlane, 1e-3);
|
||||
|
||||
|
|
|
|||
|
|
@ -53,21 +53,48 @@ THEME_BODY = {
|
|||
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
|
||||
}
|
||||
DEFAULT_BODY = THEME_BODY["stone"]
|
||||
# Closed-facing bodies: SAME theme flavor (material, contents, mood) but every
|
||||
# architectural OPENING noun stripped — gatehouse/portcullis/entrance, wall
|
||||
# niches, the throne's recessed dais, etc. A GPU sweep (2026-06-15) proved the
|
||||
# phantom doors on closed walls were summoned by these words, not the depth map:
|
||||
# `threshold`'s "gatehouse, portcullis, entrance" painted a gate onto a solid
|
||||
# wall, and at cfg 1.0 the negative branch is too weak to cancel it. Working WITH
|
||||
# a generic solid-wall description (no opening nouns) renders a clean dead-end;
|
||||
# even "bricked-up archway" backfires because "archway" alone re-summons the arch.
|
||||
THEME_BODY_CLOSED = {
|
||||
"forge": "forge chamber, soot-blackened granite walls, iron anvils and hanging chains, dim fiery glow",
|
||||
"cistern": "flooded cistern, dark still water, dripping wet stone walls, cold teal light",
|
||||
"crypt": "ancient crypt, carved stone sarcophagi along blank stone walls, cobwebs, cold pale light",
|
||||
"library": "ruined library, tall stone shelves of rotting books against the wall, drifting dust, warm dim light",
|
||||
"throne": "throne hall, towering carved stone columns and a blank stone wall, tattered banners, cold grandeur",
|
||||
"vault": "treasure vault, iron-bound chests and stone strongboxes against heavy locked stone walls, scattered gold",
|
||||
"hall": "grand pillared hall, carved stone columns and a high blank stone wall, vaulted ceiling",
|
||||
"den": "foul beast den, gnawed bones and filth, claw-scratched blank stone walls, dim red light",
|
||||
"threshold": "heavy sealed gateroom of dungeon stone, worn flagstones, blank ashlar walls",
|
||||
"stone": "rough-hewn stone chamber, bare granite walls, plain and ancient",
|
||||
}
|
||||
# These reinforce the depth (a deep recess vs a near wall straight ahead) so the
|
||||
# model paints a passage only where one actually exists — killing phantom doors.
|
||||
AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead"
|
||||
# cfg 1.0 makes negatives inert, so the closed case is carried by emphatic
|
||||
# positive description — themes like the vault say "locked", which otherwise
|
||||
# summons phantom doors on blank walls.
|
||||
AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of carved stone blocks, "
|
||||
"sealed solid masonry with no door and no opening, a dead end")
|
||||
# positive description (opening-free body above) + this sealed-wall clause.
|
||||
AHEAD_WALL = ("the way ahead ends at a blank unbroken wall of solid carved stone blocks, "
|
||||
"sealed flat masonry, a dead end")
|
||||
NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, "
|
||||
"holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern")
|
||||
# Closed facings additionally negate every opening noun (cheap insurance; the
|
||||
# opening-free positive body does the real work). NOT applied to open facings —
|
||||
# they WANT a passage.
|
||||
NEG_OPENINGS = "doorway, door, archway, arch, opening, passage, tunnel, gate, portcullis, gateway, entrance, corridor"
|
||||
|
||||
|
||||
def prompt_for(theme, is_open):
|
||||
"""Return (positive, negative) for this (theme, facing)."""
|
||||
if is_open:
|
||||
body = THEME_BODY.get(theme or "stone", DEFAULT_BODY)
|
||||
return f"{STYLE}{body}, {AHEAD_OPEN if is_open else AHEAD_WALL}"
|
||||
return f"{STYLE}{body}, {AHEAD_OPEN}", NEGATIVE
|
||||
body = THEME_BODY_CLOSED.get(theme or "stone", THEME_BODY_CLOSED["stone"])
|
||||
return f"{STYLE}{body}, {AHEAD_WALL}", f"{NEGATIVE}, {NEG_OPENINGS}"
|
||||
|
||||
NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$")
|
||||
|
||||
|
|
@ -127,7 +154,7 @@ def main():
|
|||
room = rooms.get(int(rid), {})
|
||||
theme = room.get("theme")
|
||||
is_open = bool(room.get("open", {}).get(d, False))
|
||||
prompt = prompt_for(theme, is_open)
|
||||
prompt, negative = prompt_for(theme, is_open)
|
||||
strength = a.strength
|
||||
# Walls render under their own FIXED seed: the open-frame seed's
|
||||
# favorite composition is a centered doorway — exactly what a closed
|
||||
|
|
@ -140,7 +167,7 @@ def main():
|
|||
if not run(["python3", COMFY, "zrender", a.url, "--depth", depth,
|
||||
"--width", str(a.width), "--height", str(a.height),
|
||||
"--strength", str(strength), "--seed", str(render_seed),
|
||||
"--prompt", prompt, "--negative", NEGATIVE,
|
||||
"--prompt", prompt, "--negative", negative,
|
||||
"--prefix", prefix, "--wait", "300"]):
|
||||
print(f" {tag} RENDER FAILED"); continue
|
||||
|
||||
|
|
|
|||
|
|
@ -25,6 +25,25 @@ REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)
|
|||
COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py")
|
||||
VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python")
|
||||
|
||||
# The final montage needs Pillow, which only lives in the pixelart venv; the
|
||||
# renders themselves shell out to `python3 comfy.py` (stdlib only). If we were
|
||||
# launched under a Python without PIL, re-exec under the venv so one invocation
|
||||
# both renders and montages. (Without this the whole sweep runs, then dies at the
|
||||
# last step — wasting every render.)
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
import PIL # noqa: F401
|
||||
except ImportError:
|
||||
# The venv's bin/python is a symlink back to the base interpreter, so a
|
||||
# realpath() comparison wrongly reports "already in the venv" and skips
|
||||
# the hop — what matters is sys.path (venv site-packages), not the binary.
|
||||
# Guard the re-exec with an env sentinel so a still-missing PIL raises
|
||||
# instead of looping forever.
|
||||
if os.path.exists(VENV_PY) and not os.environ.get("_SWEEP_REEXEC"):
|
||||
os.environ["_SWEEP_REEXEC"] = "1"
|
||||
os.execv(VENV_PY, [VENV_PY, os.path.abspath(__file__), *sys.argv[1:]])
|
||||
raise
|
||||
|
||||
# Mirror bake_atlas.py's proven prompt skeleton so the sweep reflects production.
|
||||
STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, "
|
||||
"wet stone, volumetric haze, architectural photography of an empty ruin, "
|
||||
|
|
|
|||
Loading…
Reference in a new issue