reikhelm/tools/README.md

reikhelm tools — AI-render → pixel-art pipeline

Post-processing tools that turn a depth map into atmospheric, palette-locked pixel art via a local diffusion render. Three stages, glued by PNG files.

real geometry  ──►  depth map  ──►  Stage 1: ComfyUI/Z-Image  ──►  Stage 2: dither  ──►  pixel art
(reikhelm-web/      (top-down       (comfy-spike/comfy.py)          (pixelart/pixelate.py)
 depth.js export)    heightfield)

Now rendering real generated dungeons (top-down). The depth exporter lives renderer-side in reikhelm-web/depth.js (derives a height field from the same envelope render.js paints — walls raised, floors mid, pools recessed, pillars as bumps, plus subtle per-theme relief; near=white/far=black). It honors the "map holds no rendering info" contract: height, like color, is derived from tiles+regions+theme, never stored in the core.

Setup

The pixelart tools share a venv (Pillow + numpy + opencv-headless):

python3 -m venv tools/pixelart/.venv
tools/pixelart/.venv/bin/pip install pillow numpy opencv-python-headless

comfy-spike/comfy.py and serve.py are pure stdlib (no venv). LAN calls to ComfyUI need the Bash sandbox disabled.

Stage 0 — export a depth map from a real dungeon (serve.py + reikhelm-web/{depth,fpv}.js)

Serve the playground and let it POST depth PNGs back to disk:

python3 tools/serve.py                       # http://127.0.0.1:8000, saves → tools/comfy-spike/

Two depth modes, both derived from the same envelope (renderer-side, core stays clean):

Top-down (depth.js) — a height field, walls raised / pools recessed. Hit the playground's ⬇ depth button, or headless:

// in the page (e.g. via Playwright browser_evaluate):
window.reikhelm.setSeed(7);
await window.reikhelm.exportDepthToServer('depth_seed7.png');   // → tools/comfy-spike/depth_seed7.png
// window.reikhelm.exportDepth({stage,scale,bevel,ao,normalize}) → {url,width,height} for in-context capture

≈1024px on the long edge (16px/cell at the default 64×40 map), aspect-matched to the render.

First-person (fpv.js) — an Eye-of-the-Beholder / Daggerfall view via a Wolfenstein DDA raycaster over the grid (wall slice + floor/ceiling ramp, near=white). Camera at a room's most-open cell, one of 4 cardinal facings:

window.reikhelm.setSeed(7);
const at = window.reikhelm.vantage();                          // most-open cell of the largest room
for (const dir of ['N','E','S','W'])
  await window.reikhelm.exportFPVToServer(`fpv_s7_${dir}.png`, { dir });
// exportFPV({at:[x,y],dir,width,height,fov,maxView,gamma}) → {url,width,height,at,dir}

1280×720 (16:9 — integer-scales to 1080p/1440p). Doorways are non-blocking, so they read as dark recesses leading deeper.

Stage 1 — depth → diffusion render (comfy-spike/comfy.py)

Talks to ComfyUI over HTTP (workstation http://192.168.1.26:8188).

# real dungeon (Stage 0 export), top-down prompt, geometry-honoring strength:
python3 comfy-spike/comfy.py zrender http://192.168.1.26:8188 --depth comfy-spike/depth_seed7.png \
  --width 1024 --height 640 --strength 0.85 --prefix rk_s7 \
  --prompt "overhead top-down view of an ancient stone dungeon, rock-cut chambers linked by corridors, flagstone floors, molten lava pool, torchlight, dark fantasy, highly detailed, cinematic"
# synthetic test depth (no real geometry): comfy.py depth --kind room --width 1024 --height 640 --out d.png
# also: probe <url> | nodeinfo <url> <Node...> for introspection

Proven Z-Image depth recipe: ModelPatchLoader + ZImageFunControlnet, CLIP type qwen_image, EmptySD3LatentImage, res_multistep/simple, 8–12 steps, cfg 1.0, ModelSamplingAuraFlow shift 3.0. Depth strength ~0.6 for freeform; 0.80–0.85 is the sweet spot for real geometry (confirmed on seeds 7/33: <0.6 drifts off the layout, ≥1.0 goes rigid and flattens the pools). Set --width/--height to the depth PNG's dimensions and use a top-down/overhead prompt to match the geometry.

Stage 2 — render → pixel art (pixelart/pixelate.py)

LOCKED "Primordyn" recipe = the defaults:

tools/pixelart/.venv/bin/python pixelart/pixelate.py render.png --palette pixelart/primordyn_v2.json
# → 640×360 · Floyd–Steinberg · OKLab match · linear-light · serpentine · fill
# --size 320x180 = chunky cut;  --dither atkinson|jjn|bayer4|none to experiment

Outputs a true indexed PNG (+ optional --preview-scale nearest-neighbor preview). Palette loader auto-detects .json / .gpl / JASC .pal / hex / Paint.NET / .png / adaptive:N.

Stage 3 — bake & explore a first-person dungeon (bake_atlas.py + reikhelm-web/explore.html)

Pre-render every room's 4 cardinal views into an atlas, then walk it Eye-of-the-Beholder style.

# 1) export the depth maps + room-graph manifest for a seed (browser, via Playwright):
#    window.reikhelm.bakeAtlasDepths(7)   → tools/out/atlas/7/_work/r<id>_<dir>_depth.png + manifest.json
# 2) render every view through comfy → pixelate (FIXED diffusion seed = style lock across frames):
python3 tools/bake_atlas.py --seed 7            # → tools/out/atlas/7/r<id>_<dir>.png (640x360 pixel art)
# 3) walk it:  http://127.0.0.1:8000/explore.html?seed=7

The explorer (explore.html + explore-viewer.js) loads the manifest + frames: you occupy a room and a cardinal facing; W/S step forward/back to the room in that direction, A/D turn. explore.js builds the room graph (corridors contracted; each neighbour binned to a cardinal) and per-room vantage points. Prompt is tank's "empty abandoned ruin" recipe — no first person / POV / dungeon-crawler trigger words (they summon player hands; at cfg 1.0 the negative prompt is inert, so the positive must stay clean). See .dev/2026-06-01-fpv-prompts.md. The baker then varies the body per room theme (forge→lava, throne→throne+banners, library→shelves, crypt→niches…) and adds a wall-vs-passage clause per facing (from open[dir] in the manifest) so rooms read distinctly and solid walls don't grow phantom doors. Strength 0.85 honors the footprint. --rooms 6,7,9 re-bakes a subset.

Helpers & references

• pixelart/montage.py — tile labeled images into a contact sheet.
• pixelart/samples/ — curated reference: before/after pair + the kernel/size & palette-stretch contact sheets.
• out/gallery/ — local keepers (top-down + first-person hero shots). Everything under out/ is git-ignored.

Next

• Top-down real-geometry export — done (reikhelm-web/depth.js, Stage 0). Per-tile heights + subtle per-theme relief; the BFS distance field is reused only for crevice AO (it's an openness map, not a depth map — feeding it raw would dome the floors).
• First-person depth — working (reikhelm-web/fpv.js). A grid raycaster, NOT the heavy 3D path once assumed. 16:9 dungeon-crawler views from a room centre; strength 0.55–0.70 gives the richest results.
• Playable EoB explorer — shipped (Stage 3). Per-room ×4-facing atlas bake + a WASD viewer. Movement is room-to-room along the contracted corridor graph.
• Next for the explorer: per-cell baking (smooth step-by-step movement, not just room-to-room); entity sprites (monsters/treasure) composited on frames; door cells rendered as doors vs open archways; a "bake this seed" button in the playground. Walk-forward render sequence lives in out/walk/.
• Richer relief (top-down Stage B+): push per-theme height harder (throne dais, deeper cisterns) if top-down renders feel uniform.
• Rust port: move depth export (both modes) + the locked Stage-2 dither into a crate so the engine owns the whole chain. See project memory.