reikhelm/.dev/2026-06-01-fpv-prompts.md
Parley Hatch ed00e31009 feat(render): real-geometry depth export + first-person explorable atlas
Render actual generated dungeons through the AI pipeline — both top-down and
first-person (Eye-of-the-Beholder style) — instead of synthetic test depth.
All renderer-side: derived from the envelope's tiles+regions+theme, the core
stores none of it.

- depth.js: top-down height-field depth exporter (walls raised, pools recessed,
  pillars as bumps, subtle per-theme relief), calibrated to the proven room_depth
  levels so one tall room can't crush every floor dark.
- fpv.js: first-person depth via a Wolfenstein-style grid raycaster (16:9),
  reports straight-ahead distance for opening detection.
- explore.js: room->room nav graph derived from REAL tile openings (scan
  boundary gaps, bin by cardinal, flood the corridor to the destination room) —
  nav + open share one source of truth with the rendered passages.
- bake_atlas.py: bakes per-room x4-facing pixel-art frames; per-theme + per-facing
  (wall vs passage) prompts, fixed diffusion seed for cross-frame style coherence,
  per-run unique prefixes so re-bakes don't silently skip.
- explore.html + explore-viewer.js: WASD first-person dungeon explorer
  (room-to-room movement, turning, minimap), integer-pixel fullscreen.
- serve.py: stdlib dev server — static web root + /out tree + POST /save, no-store.
- main.js/render.js/style.css: export hooks, the depth button, distanceToWall export.
- tools/README.md: Stage 0/1/2/3 docs; .dev/2026-06-01-fpv-prompts.md: prompt
  research (cfg-1 negatives are inert; trigger words summon hands; empty-ruin
  reframe; fixed seed = consistency).

Proven end-to-end on seed 7 (17 rooms). Outputs organized under git-ignored
tools/out/. Control strength: 0.80-0.85 top-down, 0.85 first-person.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 00:02:14 -06:00

9.7 KiB
Raw Permalink Blame History

First-Person Dungeon FPV Prompts (Z-Image-Turbo + Depth Fun-ControlNet)

Researched: 2026-06-01 Pipeline: Z-Image-Turbo (Qwen3-4B encoder), 8 steps, cfg 1.0, Fun-ControlNet Union depth patch, strength ~0.65, res_multistep/simple. Output downsampled to 256-color pixel art.

TL;DR

  • At cfg 1.0 the negative prompt is mechanically dead (classifier-free guidance is off). Hands/figures must be killed by the POSITIVE prompt + avoiding trigger words, NOT by negatives.
  • Top triggers that summon a player character: "first person", "POV", "dungeon crawler", "Eye of the Beholder", "adventurer", "hero", "you". Drop them all. Frame as architectural photography of an empty/abandoned place instead.
  • Let the depth map own the geometry; the prompt owns material + light. Don't fight the controlnet by over-describing structure.

1. Positive-prompt templates (copy-paste)

Frame every shot as a deserted architectural photograph. The model never adds a person to "an empty abandoned stone hall, no people."

Corridor

empty abandoned stone dungeon corridor, deserted, no people, uninhabited,
narrow ancient stonework passage receding into darkness, wet mossy flagstone
floor, rough granite block walls, iron sconces with flickering torchlight,
warm orange firelight and deep black shadows, volumetric haze, damp cold air,
dark fantasy, architectural photography of an empty ruin, atmospheric, detailed stone

Chamber / room

empty abandoned stone dungeon chamber, deserted, no people, uninhabited,
vast vaulted hall of ancient masonry, weathered granite pillars, cracked
flagstone floor, iron braziers with flickering torchlight, warm firelight
pooling in deep shadow, volumetric haze, dripping damp walls, dark fantasy,
architectural photography of an empty ruin, atmospheric, detailed stone

Tighter variant (if long prompts drift)

empty stone dungeon corridor, no people, abandoned ancient ruin, mossy
flagstone, granite walls, torchlight, deep shadows, volumetric fog, dark
fantasy, architectural photo

Style anchor block (PREPEND identically to EVERY frame in an atlas)

dark fantasy dungeon, ancient torchlit stone, warm orange torchlight, deep
black shadows, wet mossy granite masonry, volumetric haze,

Then append the per-room body (corridor vs chamber, any theme word like "crypt"/"flooded"/"library").

Words to AVOID (they summon a character / break the empty frame)

Avoid Why Use instead
first person, POV, first-person view model fills the bottom edge with hands/weapon (nothing — say "corridor receding into darkness")
dungeon crawler, Eye of the Beholder, Daggerfall strong "RPG with a player" prior → arms/HUD "dark fantasy", "old-school RPG art style"
adventurer, hero, explorer, party, you, your literal person "abandoned", "deserted"
holding, wielding, torch in hand summons a hand+weapon "iron sconce", "wall torch", "brazier"
game screenshot, HUD, interface, UI summons fake UI overlay "architectural photograph"

Note "torch" alone is fine (wall torches); only "holding/carrying a torch" triggers a hand. Anchor light to fixtures: sconce, brazier, wall torch.


2. Negative prompt — and why it barely matters here

Caveat first: Z-Image-Turbo is a distilled model running at cfg/guidance 1.0, where classifier-free guidance is effectively off, so the negative branch is ignored or near-inert. Two independent sources confirm the Qwen-family text encoder + distillation means "the model does not use negative prompts at all." Do NOT rely on it to remove hands. The positive prompt and trigger-word avoidance do ~95% of the work.

Still set one (cheap insurance; matters only if you ever nudge cfg to ~1.21.5), ordered by importance:

person, people, human, figure, character, hands, fingers, hand, arm, arms,
holding weapon, sword, feet, legs, body, silhouette, UI, HUD, health bar,
text, watermark, signature, modern, photorealistic skin

If hands still appear, the real fixes (in order):

  1. Remove every trigger word above from the positive prompt.
  2. Add stronger empty-scene tokens up front: empty, deserted, no people, uninhabited, abandoned.
  3. Floor-bias the depth so the lower frame is clearly continuous floor (your corridor_depth --floor-bias) — fewer ambiguous near-camera regions for the model to "fill" with a body. A bright featureless blob at the bottom of the depth map invites hands.
  4. Only as a last resort raise cfg to ~1.31.5 (re-enables negatives) and accept slower, slightly stiffer renders.

3. Cross-frame style consistency (atlas of one dungeon)

Goal: many views, one coherent place. Levers, strongest first:

  1. Identical style-anchor block on every frame (see §1). This is the #1 consistency lever since the prompt owns all style/lighting/material and the depth map varies per view. Keep the lighting + material tokens byte-identical across the whole atlas; only swap the geometry-neutral body ("corridor" vs "chamber") and optional theme word.
  2. Fix the seed across the whole batch. Same seed + same model + same style block → consistent palette and lighting character; the per-view depth map supplies the differing geometry. (Note: Z-Image is reported to "hardwire" composition to the seed even across prompt changes — that's a feature here, it locks the look.)
  3. Lock everything else: sampler res_multistep, scheduler simple, steps 8, cfg 1.0, shift, resolution. Any change to these shifts the rendering character.
  4. Keep control strength constant across frames (one value, e.g. 0.65) so geometry adherence — and thus how much "model style" bleeds in — is uniform.
  5. Bake a fixed <style block> constant in code (sibling of the prompt strings in tools/comfy-spike/comfy.py) and concatenate per-room, so it can never drift between frames.
  6. Optional later: a small style LoRA is the documented route for hard brand/style lock if prompt+seed isn't tight enough. The 256-color pixel-art downsample after the fact also strongly homogenizes the palette, hiding minor per-frame drift — lean on it.

4. Depth-ControlNet-specific tips

  • Don't re-describe geometry that's already in the depth map. Documented guidance: "Fix the reference, not the prompt" — prompt+model handle style, controlnet handles structure/composition. Saying "archway ahead" when the depth already encodes it is redundant at best; if it conflicts with the depth it fights the controlnet and can warp the scene. Describe materials and light, let depth dictate walls/floor/vanishing point.
  • A geometry-neutral noun ("corridor"/"chamber") is fine and helps — it tells the model what kind of surface to paint, not where. Avoid spatial directives ("door on the left", "stairs ahead").
  • Control strength: 0.6 is the documented sweet spot; your 0.65 is good. 0.4 = more style freedom / loose geometry; 0.8 = near-locked geometry; 1.0 = stiff/over-constrained; 0.2 = barely controls. For a clean architectural atlas you want geometry honored, so 0.60.7 is right. If renders look flat/over-rigid, drop toward 0.55.
  • Depth map hygiene = anatomy control. Near=white/far=black is correct. The bottom-center near region is exactly where hands spontaneously appear; keep that area as unambiguous floor (use --floor-bias to push the vanishing point up so the lower frame reads as continuous ground, not a vague near object). Smooth falloff (your default bands=0) avoids hard ring seams that the model misreads as objects.
  • Prompt-vs-control balance: with strength ~0.65 and cfg 1.0, prompt adherence is gentle — which is good for empty scenes (less chance of hallucinating a subject), but means you must front-load the most important tokens (empty/no-people/material) early in the prompt.

5. Open tension to test (flagged, not resolved)

Your brief assumes "turbo → short concise prompts beat long flowery ones." The official Tongyi-MAI prompting guide says the opposite for Z-Image: it favors long, detailed, structured prompts (80250 words) and "responds very strongly to lighting keywords." These can both be true — concrete beats flowery regardless of length; Z-Image likes detail but not purple prose. The templates above are concrete-detailed (not flowery, not terse). Recommendation: A/B the long corridor template vs the tighter variant at fixed seed and keep whichever yields cleaner empty frames in your atlas. Detail seems to help this model; just keep every token concrete and avoid character words.


Sources