Parley Hatch
ed00e31009
Render actual generated dungeons through the AI pipeline — both top-down and first-person (Eye-of-the-Beholder style) — instead of synthetic test depth. All renderer-side: derived from the envelope's tiles+regions+theme, the core stores none of it. - depth.js: top-down height-field depth exporter (walls raised, pools recessed, pillars as bumps, subtle per-theme relief), calibrated to the proven room_depth levels so one tall room can't crush every floor dark. - fpv.js: first-person depth via a Wolfenstein-style grid raycaster (16:9), reports straight-ahead distance for opening detection. - explore.js: room->room nav graph derived from REAL tile openings (scan boundary gaps, bin by cardinal, flood the corridor to the destination room) — nav + open share one source of truth with the rendered passages. - bake_atlas.py: bakes per-room x4-facing pixel-art frames; per-theme + per-facing (wall vs passage) prompts, fixed diffusion seed for cross-frame style coherence, per-run unique prefixes so re-bakes don't silently skip. - explore.html + explore-viewer.js: WASD first-person dungeon explorer (room-to-room movement, turning, minimap), integer-pixel fullscreen. - serve.py: stdlib dev server — static web root + /out tree + POST /save, no-store. - main.js/render.js/style.css: export hooks, the depth button, distanceToWall export. - tools/README.md: Stage 0/1/2/3 docs; .dev/2026-06-01-fpv-prompts.md: prompt research (cfg-1 negatives are inert; trigger words summon hands; empty-ruin reframe; fixed seed = consistency). Proven end-to-end on seed 7 (17 rooms). Outputs organized under git-ignored tools/out/. Control strength: 0.80-0.85 top-down, 0.85 first-person. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
120 lines
7.6 KiB
Markdown
120 lines
7.6 KiB
Markdown
# reikhelm tools — AI-render → pixel-art pipeline
|
||
|
||
Post-processing tools that turn a depth map into atmospheric, palette-locked pixel
|
||
art via a local diffusion render. Three stages, glued by PNG files.
|
||
|
||
```
|
||
real geometry ──► depth map ──► Stage 1: ComfyUI/Z-Image ──► Stage 2: dither ──► pixel art
|
||
(reikhelm-web/ (top-down (comfy-spike/comfy.py) (pixelart/pixelate.py)
|
||
depth.js export) heightfield)
|
||
```
|
||
|
||
**Now rendering real generated dungeons** (top-down). The depth exporter lives
|
||
renderer-side in `reikhelm-web/depth.js` (derives a height field from the same
|
||
envelope render.js paints — walls raised, floors mid, pools recessed, pillars as
|
||
bumps, plus subtle per-theme relief; near=white/far=black). It honors the "map
|
||
holds no rendering info" contract: height, like color, is derived from
|
||
tiles+regions+theme, never stored in the core.
|
||
|
||
## Setup
|
||
The pixelart tools share a venv (Pillow + numpy + opencv-headless):
|
||
```
|
||
python3 -m venv tools/pixelart/.venv
|
||
tools/pixelart/.venv/bin/pip install pillow numpy opencv-python-headless
|
||
```
|
||
`comfy-spike/comfy.py` and `serve.py` are pure stdlib (no venv). LAN calls to ComfyUI need the Bash sandbox disabled.
|
||
|
||
## Stage 0 — export a depth map from a real dungeon (`serve.py` + `reikhelm-web/{depth,fpv}.js`)
|
||
Serve the playground and let it POST depth PNGs back to disk:
|
||
```
|
||
python3 tools/serve.py # http://127.0.0.1:8000, saves → tools/comfy-spike/
|
||
```
|
||
Two depth modes, both derived from the same envelope (renderer-side, core stays clean):
|
||
|
||
**Top-down** (`depth.js`) — a height field, walls raised / pools recessed. Hit the
|
||
playground's **⬇ depth** button, or headless:
|
||
```js
|
||
// in the page (e.g. via Playwright browser_evaluate):
|
||
window.reikhelm.setSeed(7);
|
||
await window.reikhelm.exportDepthToServer('depth_seed7.png'); // → tools/comfy-spike/depth_seed7.png
|
||
// window.reikhelm.exportDepth({stage,scale,bevel,ao,normalize}) → {url,width,height} for in-context capture
|
||
```
|
||
≈1024px on the long edge (16px/cell at the default 64×40 map), aspect-matched to the render.
|
||
|
||
**First-person** (`fpv.js`) — an Eye-of-the-Beholder / Daggerfall view via a Wolfenstein
|
||
DDA raycaster over the grid (wall slice + floor/ceiling ramp, near=white). Camera at a
|
||
room's most-open cell, one of 4 cardinal facings:
|
||
```js
|
||
window.reikhelm.setSeed(7);
|
||
const at = window.reikhelm.vantage(); // most-open cell of the largest room
|
||
for (const dir of ['N','E','S','W'])
|
||
await window.reikhelm.exportFPVToServer(`fpv_s7_${dir}.png`, { dir });
|
||
// exportFPV({at:[x,y],dir,width,height,fov,maxView,gamma}) → {url,width,height,at,dir}
|
||
```
|
||
1280×720 (16:9 — integer-scales to 1080p/1440p). Doorways are non-blocking, so they read as dark recesses leading deeper.
|
||
|
||
## Stage 1 — depth → diffusion render (`comfy-spike/comfy.py`)
|
||
Talks to ComfyUI over HTTP (workstation `http://192.168.1.26:8188`).
|
||
```
|
||
# real dungeon (Stage 0 export), top-down prompt, geometry-honoring strength:
|
||
python3 comfy-spike/comfy.py zrender http://192.168.1.26:8188 --depth comfy-spike/depth_seed7.png \
|
||
--width 1024 --height 640 --strength 0.85 --prefix rk_s7 \
|
||
--prompt "overhead top-down view of an ancient stone dungeon, rock-cut chambers linked by corridors, flagstone floors, molten lava pool, torchlight, dark fantasy, highly detailed, cinematic"
|
||
# synthetic test depth (no real geometry): comfy.py depth --kind room --width 1024 --height 640 --out d.png
|
||
# also: probe <url> | nodeinfo <url> <Node...> for introspection
|
||
```
|
||
Proven Z-Image depth recipe: `ModelPatchLoader` + `ZImageFunControlnet`, CLIP type `qwen_image`,
|
||
`EmptySD3LatentImage`, `res_multistep`/`simple`, 8–12 steps, cfg 1.0, `ModelSamplingAuraFlow` shift 3.0.
|
||
Depth strength ~0.6 for freeform; **0.80–0.85 is the sweet spot for real geometry** (confirmed on
|
||
seeds 7/33: <0.6 drifts off the layout, ≥1.0 goes rigid and flattens the pools). Set `--width/--height`
|
||
to the depth PNG's dimensions and use a **top-down/overhead** prompt to match the geometry.
|
||
|
||
## Stage 2 — render → pixel art (`pixelart/pixelate.py`)
|
||
LOCKED "Primordyn" recipe = the defaults:
|
||
```
|
||
tools/pixelart/.venv/bin/python pixelart/pixelate.py render.png --palette pixelart/primordyn_v2.json
|
||
# → 640×360 · Floyd–Steinberg · OKLab match · linear-light · serpentine · fill
|
||
# --size 320x180 = chunky cut; --dither atkinson|jjn|bayer4|none to experiment
|
||
```
|
||
Outputs a true indexed PNG (+ optional `--preview-scale` nearest-neighbor preview). Palette loader
|
||
auto-detects .json / .gpl / JASC .pal / hex / Paint.NET / .png / `adaptive:N`.
|
||
|
||
## Stage 3 — bake & explore a first-person dungeon (`bake_atlas.py` + `reikhelm-web/explore.html`)
|
||
Pre-render every room's 4 cardinal views into an atlas, then walk it Eye-of-the-Beholder style.
|
||
```
|
||
# 1) export the depth maps + room-graph manifest for a seed (browser, via Playwright):
|
||
# window.reikhelm.bakeAtlasDepths(7) → tools/out/atlas/7/_work/r<id>_<dir>_depth.png + manifest.json
|
||
# 2) render every view through comfy → pixelate (FIXED diffusion seed = style lock across frames):
|
||
python3 tools/bake_atlas.py --seed 7 # → tools/out/atlas/7/r<id>_<dir>.png (640x360 pixel art)
|
||
# 3) walk it: http://127.0.0.1:8000/explore.html?seed=7
|
||
```
|
||
The explorer (`explore.html` + `explore-viewer.js`) loads the manifest + frames: you occupy a room and a
|
||
cardinal facing; **W/S** step forward/back to the room in that direction, **A/D** turn. `explore.js` builds
|
||
the room graph (corridors contracted; each neighbour binned to a cardinal) and per-room vantage points.
|
||
Prompt is tank's "empty abandoned ruin" recipe — **no** *first person / POV / dungeon-crawler* trigger
|
||
words (they summon player hands; at cfg 1.0 the negative prompt is inert, so the positive must stay clean).
|
||
See `.dev/2026-06-01-fpv-prompts.md`. The baker then varies the body **per room theme** (forge→lava,
|
||
throne→throne+banners, library→shelves, crypt→niches…) and adds a **wall-vs-passage clause per facing**
|
||
(from `open[dir]` in the manifest) so rooms read distinctly and solid walls don't grow phantom doors.
|
||
Strength **0.85** honors the footprint. `--rooms 6,7,9` re-bakes a subset.
|
||
|
||
## Helpers & references
|
||
- `pixelart/montage.py` — tile labeled images into a contact sheet.
|
||
- `pixelart/samples/` — curated reference: before/after pair + the kernel/size & palette-stretch contact sheets.
|
||
- `out/gallery/` — local keepers (top-down + first-person hero shots). Everything under `out/` is git-ignored.
|
||
|
||
## Next
|
||
- **Top-down real-geometry export — done** (`reikhelm-web/depth.js`, Stage 0). Per-tile heights + subtle
|
||
per-theme relief; the BFS distance field is reused only for crevice AO (it's an openness map, not a
|
||
depth map — feeding it raw would dome the floors).
|
||
- **First-person depth — working** (`reikhelm-web/fpv.js`). A grid raycaster, NOT the heavy 3D path once
|
||
assumed. 16:9 dungeon-crawler views from a room centre; strength **0.55–0.70** gives the richest results.
|
||
- **Playable EoB explorer — shipped** (Stage 3). Per-room ×4-facing atlas bake + a WASD viewer. Movement is
|
||
room-to-room along the contracted corridor graph.
|
||
- **Next for the explorer:** per-*cell* baking (smooth step-by-step movement, not just room-to-room);
|
||
entity sprites (monsters/treasure) composited on frames; door cells rendered as doors vs open archways;
|
||
a "bake this seed" button in the playground. Walk-forward render sequence lives in `out/walk/`.
|
||
- **Richer relief (top-down Stage B+):** push per-theme height harder (throne dais, deeper cisterns) if
|
||
top-down renders feel uniform.
|
||
- **Rust port:** move depth export (both modes) + the locked Stage-2 dither into a crate so the engine
|
||
owns the whole chain. See project memory.
|