diff --git a/.dev/2026-06-01-fpv-prompts.md b/.dev/2026-06-01-fpv-prompts.md new file mode 100644 index 0000000..818e031 --- /dev/null +++ b/.dev/2026-06-01-fpv-prompts.md @@ -0,0 +1,118 @@ +# First-Person Dungeon FPV Prompts (Z-Image-Turbo + Depth Fun-ControlNet) + +**Researched:** 2026-06-01 +**Pipeline:** Z-Image-Turbo (Qwen3-4B encoder), 8 steps, cfg 1.0, Fun-ControlNet Union depth patch, strength ~0.65, res_multistep/simple. Output downsampled to 256-color pixel art. + +## TL;DR +- At **cfg 1.0 the negative prompt is mechanically dead** (classifier-free guidance is off). Hands/figures must be killed by the POSITIVE prompt + avoiding trigger words, NOT by negatives. +- Top triggers that summon a player character: **"first person", "POV", "dungeon crawler", "Eye of the Beholder", "adventurer", "hero", "you"**. Drop them all. Frame as **architectural photography of an empty/abandoned place** instead. +- Let the depth map own the geometry; the prompt owns material + light. Don't fight the controlnet by over-describing structure. + +--- + +## 1. Positive-prompt templates (copy-paste) + +Frame every shot as a deserted architectural photograph. The model never adds a person to "an empty abandoned stone hall, no people." + +### Corridor +``` +empty abandoned stone dungeon corridor, deserted, no people, uninhabited, +narrow ancient stonework passage receding into darkness, wet mossy flagstone +floor, rough granite block walls, iron sconces with flickering torchlight, +warm orange firelight and deep black shadows, volumetric haze, damp cold air, +dark fantasy, architectural photography of an empty ruin, atmospheric, detailed stone +``` + +### Chamber / room +``` +empty abandoned stone dungeon chamber, deserted, no people, uninhabited, +vast vaulted hall of ancient masonry, weathered granite pillars, cracked +flagstone floor, iron braziers with flickering torchlight, warm firelight +pooling in deep shadow, volumetric haze, dripping damp walls, dark fantasy, +architectural photography of an empty ruin, atmospheric, detailed stone +``` + +### Tighter variant (if long prompts drift) +``` +empty stone dungeon corridor, no people, abandoned ancient ruin, mossy +flagstone, granite walls, torchlight, deep shadows, volumetric fog, dark +fantasy, architectural photo +``` + +### Style anchor block (PREPEND identically to EVERY frame in an atlas) +``` +dark fantasy dungeon, ancient torchlit stone, warm orange torchlight, deep +black shadows, wet mossy granite masonry, volumetric haze, +``` +Then append the per-room body (corridor vs chamber, any theme word like "crypt"/"flooded"/"library"). + +### Words to AVOID (they summon a character / break the empty frame) +| Avoid | Why | Use instead | +|---|---|---| +| first person, POV, first-person view | model fills the bottom edge with hands/weapon | (nothing — say "corridor receding into darkness") | +| dungeon crawler, Eye of the Beholder, Daggerfall | strong "RPG with a player" prior → arms/HUD | "dark fantasy", "old-school RPG art style" | +| adventurer, hero, explorer, party, you, your | literal person | "abandoned", "deserted" | +| holding, wielding, torch in hand | summons a hand+weapon | "iron sconce", "wall torch", "brazier" | +| game screenshot, HUD, interface, UI | summons fake UI overlay | "architectural photograph" | + +Note "torch" alone is fine (wall torches); only **"holding/carrying a torch"** triggers a hand. Anchor light to fixtures: **sconce, brazier, wall torch.** + +--- + +## 2. Negative prompt — and why it barely matters here + +**Caveat first:** Z-Image-Turbo is a distilled model running at **cfg/guidance 1.0**, where classifier-free guidance is effectively off, so the negative branch is **ignored or near-inert.** Two independent sources confirm the Qwen-family text encoder + distillation means "the model does not use negative prompts at all." Do NOT rely on it to remove hands. The positive prompt and trigger-word avoidance do ~95% of the work. + +Still set one (cheap insurance; matters only if you ever nudge cfg to ~1.2–1.5), ordered by importance: +``` +person, people, human, figure, character, hands, fingers, hand, arm, arms, +holding weapon, sword, feet, legs, body, silhouette, UI, HUD, health bar, +text, watermark, signature, modern, photorealistic skin +``` + +**If hands still appear, the real fixes (in order):** +1. Remove every trigger word above from the positive prompt. +2. Add stronger empty-scene tokens up front: `empty, deserted, no people, uninhabited, abandoned`. +3. Floor-bias the depth so the lower frame is clearly continuous floor (your `corridor_depth --floor-bias`) — fewer ambiguous near-camera regions for the model to "fill" with a body. A bright featureless blob at the bottom of the depth map invites hands. +4. Only as a last resort raise cfg to ~1.3–1.5 (re-enables negatives) and accept slower, slightly stiffer renders. + +--- + +## 3. Cross-frame style consistency (atlas of one dungeon) + +Goal: many views, one coherent place. Levers, strongest first: + +1. **Identical style-anchor block on every frame** (see §1). This is the #1 consistency lever since the prompt owns all style/lighting/material and the depth map varies per view. Keep the lighting + material tokens byte-identical across the whole atlas; only swap the geometry-neutral body ("corridor" vs "chamber") and optional theme word. +2. **Fix the seed across the whole batch.** Same seed + same model + same style block → consistent palette and lighting character; the per-view depth map supplies the differing geometry. (Note: Z-Image is reported to "hardwire" composition to the seed even across prompt changes — that's a *feature* here, it locks the look.) +3. **Lock everything else:** sampler `res_multistep`, scheduler `simple`, steps 8, cfg 1.0, shift, resolution. Any change to these shifts the rendering character. +4. **Keep control strength constant** across frames (one value, e.g. 0.65) so geometry adherence — and thus how much "model style" bleeds in — is uniform. +5. Bake a fixed ` + + + dungeon view +
+
loading atlas…
+
⛏ this view isn't baked yet
+ +

W/S forward·back · A/D turn

+ + + diff --git a/reikhelm-web/explore.js b/reikhelm-web/explore.js new file mode 100644 index 0000000..2adce96 --- /dev/null +++ b/reikhelm-web/explore.js @@ -0,0 +1,127 @@ +// explore.js — turn a dungeon envelope into a navigable room graph for the +// pre-baked first-person explorer. +// +// The player occupies a ROOM and a cardinal FACING. Moving forward (W) steps to +// the neighbouring room in the faced direction; A/D rotate the facing. To support +// that we need, per room: a vantage cell (where the camera stands), and which +// room lies to its N/E/S/W. Rooms connect through corridors, so we contract the +// region graph (rooms = nodes, corridors = edges to traverse through) and bin +// each room-neighbour into the cardinal of its bearing. +// +// Renderer-side, like depth.js/fpv.js: derived from regions + edges, nothing +// stored in the core. Room ids here are the region ARRAY INDEX (== the wire id; +// edges already reference regions by that index), used as the atlas frame key. + +import { getStage, distanceToWall } from './render.js'; +import { mostOpenCell, centroidOf } from './fpv.js'; + +const WALL = 0; +const DIRV = [[0, -1, 'N'], [1, 0, 'E'], [0, 1, 'S'], [-1, 0, 'W']]; // y grows downward = south + +// Build the room navigation graph from the ACTUAL tile openings (not centroid +// bearings, which disagree with what the views render). For each room we scan its +// boundary for real gaps, bin each gap by the cardinal it sits on, and flood the +// corridor behind it to find the room it truly leads to. nav + open then share a +// single source of truth with the rendered passages. +// +// Returns { seed, width, height, startRoom, rooms }; each room = +// { id, theme, vantage:[x,y], centroid:[x,y], bounds, nav:{N,E,S,W}, open:{N,E,S,W} } +// (nav = neighbour room id or null; open = a real gap exists that way). +export function dungeonGraph(env, opts = {}) { + const { tiles, regions } = getStage(env, opts.stage); + const w = env.width, h = env.height; + const dist = distanceToWall(tiles, w, h); + const isRoom = (i) => regions[i] && regions[i].kind === 'Room' && regions[i].cells && regions[i].cells.length > 0; + const inBounds = (x, y) => x >= 0 && y >= 0 && x < w && y < h; + const open = (x, y) => inBounds(x, y) && tiles[y][x] !== WALL; // floor/door/pillar/water/lava all pass + + // cell → owning room id (region index), -1 for corridors/walls. + const cellRoom = new Int32Array(w * h).fill(-1); + for (let i = 0; i < regions.length; i++) { + if (!isRoom(i)) continue; + for (const [x, y] of regions[i].cells) if (inBounds(x, y)) cellRoom[y * w + x] = i; + } + + // From a gap cell just outside `home`, flood through non-room corridor cells + // until we reach a different room; return its id (or null for a dead end/loop). + const destFrom = (sx, sy, home) => { + const seen = new Set([sx + ',' + sy]); + const q = [[sx, sy]]; + while (q.length) { + const [x, y] = q.shift(); + const rid = cellRoom[y * w + x]; + if (rid !== -1 && rid !== home) return rid; // arrived at another room + for (const [dx, dy] of DIRV) { + const nx = x + dx, ny = y + dy, k = nx + ',' + ny; + if (seen.has(k) || !open(nx, ny)) continue; + if (cellRoom[ny * w + nx] === home) continue; // never wander back into home + seen.add(k); q.push([nx, ny]); + } + } + return null; + }; + + const rooms = []; + for (let i = 0; i < regions.length; i++) { + if (!isRoom(i)) continue; + const r = regions[i]; + const centroid = centroidOf(r.cells); + const van = mostOpenCell(dist, w, r.cells); + const cellSet = new Set(r.cells.map(([x, y]) => x + ',' + y)); + + // Collect boundary gaps per cardinal: a room cell whose neighbour that way is + // outside the room and not a wall. + const gaps = { N: [], E: [], S: [], W: [] }; + for (const [x, y] of r.cells) { + for (const [dx, dy, dir] of DIRV) { + const nx = x + dx, ny = y + dy; + if (!inBounds(nx, ny) || cellSet.has(nx + ',' + ny) || tiles[ny][nx] === WALL) continue; + gaps[dir].push([nx, ny]); + } + } + + const nav = { N: null, E: null, S: null, W: null }; + const openDir = { N: false, E: false, S: false, W: false }; + for (const dir of ['N', 'E', 'S', 'W']) { + if (gaps[dir].length === 0) continue; + openDir[dir] = true; + for (const [gx, gy] of gaps[dir]) { // first gap that reaches a room wins + const d = destFrom(gx, gy, i); + if (d != null) { nav[dir] = d; break; } + } + } + + rooms.push({ + id: i, + theme: r.theme || null, + vantage: van, + centroid: [Math.round(centroid[0]), Math.round(centroid[1])], + bounds: r.bounds, + nav, + open: openDir, + }); + } + + // Start room: the one holding the Entrance entity, else the largest. + let startRoom = rooms.length ? rooms[0].id : null; + const ent = (env.entities || []).find((e) => e.kind === 'Entrance'); + if (ent) { + const hit = rooms.find((rm) => regions[rm.id].cells.some(([x, y]) => x === ent.at[0] && y === ent.at[1])); + if (hit) startRoom = hit.id; + } else { + let bestArea = 0; + for (const rm of rooms) { + const a = regions[rm.id].cells.length; + if (a > bestArea) { bestArea = a; startRoom = rm.id; } + } + } + + return { seed: env.seed, width: w, height: h, startRoom, rooms }; +} + +// Full manifest = the nav graph + the tile grid (for the viewer's minimap). +export function dungeonManifest(env, opts = {}) { + const g = dungeonGraph(env, opts); + const { tiles } = getStage(env, opts.stage); + return { ...g, tiles }; +} diff --git a/reikhelm-web/fpv.js b/reikhelm-web/fpv.js new file mode 100644 index 0000000..f8c42f1 --- /dev/null +++ b/reikhelm-web/fpv.js @@ -0,0 +1,157 @@ +// fpv.js — first-person (Eye-of-the-Beholder / Daggerfall style) depth exporter. +// +// A renderer concern, like depth.js: it reads the same envelope and produces an +// 8-bit depth map for the AI-render pipeline — but viewed from *inside* the +// dungeon rather than top-down. A grid dungeon is a 2D map extruded: walls are +// full-height blocks, floor and ceiling are flat planes, the eye sits at mid +// height, and you face one of four cardinal directions. So we don't need any 3D +// data from the core — a classic Wolfenstein-style raycaster turns the grid into +// a perspective depth map directly. +// +// Per screen column we cast one ray, march the grid (DDA) to the first blocking +// cell, and fill the column: a wall slice at that distance, floor ramping toward +// the camera below it, ceiling above. Distance → gray with NEAR=white/FAR=black +// (the same convention comfy.py expects). Doorways are non-blocking, so they +// read as dark recesses leading deeper — which is exactly what we want. + +import { getStage, distanceToWall } from './render.js'; +import { depthToCanvas } from './depth.js'; + +const WALL = 0, PILLAR = 3; + +// Cardinal facings as (dx, dy) on the grid (y grows downward / "south"). +export const DIRS = { N: [0, -1], E: [1, 0], S: [0, 1], W: [-1, 0] }; + +const clamp = (v, lo, hi) => v < lo ? lo : v > hi ? hi : v; +const blocks = (t) => t === WALL || t === PILLAR; // line-of-sight blockers + +// Centroid of a cell list as [cx, cy]. +export function centroidOf(cells) { + let cx = 0, cy = 0; + for (const [x, y] of cells) { cx += x; cy += y; } + return [cx / cells.length, cy / cells.length]; +} + +// The *most open* cell (max distance-to-wall) among `cells`, so a camera stands +// in open space rather than against a wall (a room centroid can land near a wall +// when the room is non-convex). Ties break toward the cell nearest the centroid, +// keeping it visually centred. `dist` is a precomputed distanceToWall field. +export function mostOpenCell(dist, w, cells) { + const [cx, cy] = centroidOf(cells); + let pick = cells[0], pmax = -1, pcen = Infinity; + for (const [x, y] of cells) { + const d = dist[y * w + x]; + const cen = (x - cx) ** 2 + (y - cy) ** 2; + if (d > pmax || (d === pmax && cen < pcen)) { pmax = d; pcen = cen; pick = [x, y]; } + } + return pick; +} + +// Vantage cell for one room region (its most-open cell). +export function roomVantage(env, region, opts = {}) { + const { tiles } = getStage(env, opts.stage); + return mostOpenCell(distanceToWall(tiles, env.width, env.height), env.width, region.cells); +} + +// Default demo vantage: the most-open cell of the largest room (or, failing that, +// the most open floor cell anywhere). +export function vantage(env, opts = {}) { + const { tiles, regions } = getStage(env, opts.stage); + const w = env.width, h = env.height; + const dist = distanceToWall(tiles, w, h); + let best = null, bestArea = 0; + for (const r of regions) { + if (r.kind !== 'Room' || !r.cells || r.cells.length === 0) continue; + if (r.cells.length > bestArea) { bestArea = r.cells.length; best = r; } + } + if (best) return mostOpenCell(dist, w, best.cells); + const all = []; + for (let y = 0; y < h; y++) for (let x = 0; x < w; x++) if (!blocks(tiles[y][x])) all.push([x, y]); + return all.length ? mostOpenCell(dist, w, all) : [w >> 1, h >> 1]; +} + +// Build a first-person depth field. Returns { width, height, gray, at, dir }. +// opts: { stage?, at?[x,y], dir?'N'|'E'|'S'|'W', width?, height?, fov?deg, maxView?, gamma? } +export function computeFPVDepth(env, opts = {}) { + const { tiles } = getStage(env, opts.stage); + const w = env.width, h = env.height; + const W = opts.width ?? 1280, H = opts.height ?? 720; // 16:9 — integer-scales to 1080p/1440p + const fov = (opts.fov ?? 80) * Math.PI / 180; // wider horizontal FOV to fill the widescreen frame + const planeLen = Math.tan(fov / 2); + const maxView = opts.maxView ?? 12; // how far the eye sees (cells) before it's pure dark + const gamma = opts.gamma ?? 2.0; // >1 deepens the falloff to black (matches the proven corridor look) + + const at = opts.at ?? vantage(env, opts); + const dirName = opts.dir ?? 'N'; + const [dirX, dirY] = DIRS[dirName]; + const posX = at[0] + 0.5, posY = at[1] + 0.5; + // Camera plane ⟂ to the view direction (rotate dir 90°), scaled to the FOV. + const planeX = -dirY * planeLen, planeY = dirX * planeLen; + + const gray = new Uint8ClampedArray(W * H); + const half = H / 2; + const centerX = W >> 1; + let ahead = maxView; // straight-ahead wall distance (cells) — big = an opening leads onward + + for (let x = 0; x < W; x++) { + const camX = 2 * x / W - 1; // −1 … 1 across the screen + const rayX = dirX + planeX * camX; + const rayY = dirY + planeY * camX; + + // DDA grid march. + let mapX = posX | 0, mapY = posY | 0; + const deltaX = Math.abs(1 / rayX), deltaY = Math.abs(1 / rayY); // Infinity when axis-aligned — fine + let stepX, stepY, sideX, sideY; + if (rayX < 0) { stepX = -1; sideX = (posX - mapX) * deltaX; } else { stepX = 1; sideX = (mapX + 1 - posX) * deltaX; } + if (rayY < 0) { stepY = -1; sideY = (posY - mapY) * deltaY; } else { stepY = 1; sideY = (mapY + 1 - posY) * deltaY; } + + let hit = false, side = 0; + const maxSteps = maxView * 2 + 4; + for (let s = 0; s < maxSteps; s++) { + if (sideX < sideY) { sideX += deltaX; mapX += stepX; side = 0; } + else { sideY += deltaY; mapY += stepY; side = 1; } + if (mapX < 0 || mapY < 0 || mapX >= w || mapY >= h) break; // off-map → open/far + if (blocks(tiles[mapY][mapX])) { hit = true; break; } + } + const perp = side === 0 ? sideX - deltaX : sideY - deltaY; + const dist = hit ? clamp(perp, 0.02, maxView) : maxView; + if (x === centerX) ahead = dist; + + // Project the 1-cell-tall wall: lineH = H/dist (fills the screen at dist 1). + const lineH = H / dist; + const ds = clamp(Math.floor(half - lineH / 2), 0, H); + const de = clamp(Math.floor(half + lineH / 2), 0, H); + + for (let y = 0; y < H; y++) { + let dpix; + if (y >= ds && y < de) { + dpix = dist; // wall slice + } else { + // Floor (below) / ceiling (above). (0.5·H)/p is continuous with the + // wall distance exactly at the slice edge, so the column has no seam. + const p = y < half ? (half - y) : (y - half + 1); + dpix = Math.min((0.5 * H) / p, maxView); + } + const t = clamp(dpix / maxView, 0, 1); + gray[y * W + x] = Math.round(255 * Math.pow(1 - t, gamma)); + } + } + + return { width: W, height: H, gray, at, dir: dirName, ahead }; +} + +// --- export glue (mirrors depth.js) -------------------------------------- + +export function exportFPVDataURL(env, opts = {}) { + const field = computeFPVDepth(env, opts); + return { url: depthToCanvas(field).toDataURL('image/png'), width: field.width, height: field.height, at: field.at, dir: field.dir, ahead: field.ahead }; +} + +export async function exportFPVToServer(env, name, opts = {}) { + const field = computeFPVDepth(env, opts); + const blob = await new Promise((res) => depthToCanvas(field).toBlob(res, 'image/png')); + const resp = await fetch('/save/' + encodeURIComponent(name), { method: 'POST', body: blob }); + if (!resp.ok) throw new Error('save failed: ' + resp.status); + const info = await resp.json(); + return { ...info, width: field.width, height: field.height, at: field.at, dir: field.dir, ahead: field.ahead }; +} diff --git a/reikhelm-web/main.js b/reikhelm-web/main.js index ac22d9b..1dafabd 100644 --- a/reikhelm-web/main.js +++ b/reikhelm-web/main.js @@ -4,6 +4,9 @@ import init, { generate, default_config_json } from './pkg/reikhelm_wasm.js'; import { renderMap, getStage, THEMES } from './render.js'; +import { downloadDepth, exportDepthDataURL, exportDepthToServer } from './depth.js'; +import { exportFPVDataURL, exportFPVToServer, vantage } from './fpv.js'; +import { dungeonManifest } from './explore.js'; const canvas = document.getElementById('view'); const ctx = canvas.getContext('2d'); @@ -131,7 +134,8 @@ function buildControls() { seedRow.innerHTML = ` - `; + + `; panel.appendChild(seedRow); seedRow.querySelector('#seed').addEventListener('input', (e) => { state.seed = Math.max(0, parseInt(e.target.value || '0', 10)); @@ -142,6 +146,9 @@ function buildControls() { document.getElementById('seed').value = String(state.seed); regenerate(); }); + seedRow.querySelector('#depth').addEventListener('click', () => { + if (state.env) downloadDepth(state.env, { stage: state.stage }); + }); // Sliders. for (const [path, label, min, max, step] of SLIDERS) { @@ -211,6 +218,35 @@ window.reikhelm = { setStage(i) { stageSlider.value = String(i); stageSlider.dispatchEvent(new Event('input')); }, setOpt(k, v) { state.opts[k] = v; const cb = document.querySelector(`[data-opt="${k}"]`); if (cb) cb.checked = v; draw(); }, setConfig(path, v) { setPath(state.config, path, v); regenerate(); }, + // Depth-map export (drives the AI-render pipeline). exportDepth → data URL + + // dims (for headless capture); exportDepthToServer → POST the PNG to serve.py. + exportDepth(opts) { return state.env ? exportDepthDataURL(state.env, opts) : null; }, + downloadDepth(opts) { return state.env ? downloadDepth(state.env, opts) : null; }, + exportDepthToServer(name, opts) { return state.env ? exportDepthToServer(state.env, name, opts) : null; }, + // First-person (Eye-of-the-Beholder style) depth — raycast from a room centre. + exportFPV(opts) { return state.env ? exportFPVDataURL(state.env, opts) : null; }, + exportFPVToServer(name, opts) { return state.env ? exportFPVToServer(state.env, name, opts) : null; }, + vantage(opts) { return state.env ? vantage(state.env, opts) : null; }, + dungeonManifest() { return state.env ? dungeonManifest(state.env) : null; }, + // Save a JSON blob to the dev server (used to drop an atlas manifest to disk). + async saveJSON(name, obj) { + const r = await fetch('/save/' + name, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(obj) }); + if (!r.ok) throw new Error('saveJSON failed: ' + r.status); + return r.json(); + }, + // Export every room × 4 cardinal FPV depth maps + the manifest for a baked atlas. + async bakeAtlasDepths(seed) { + if (seed != null) this.setSeed(seed); + const m = dungeonManifest(state.env); // nav + open already derived from real tile openings + const sd = state.env.seed; + for (const r of m.rooms) { + for (const dir of ['N', 'E', 'S', 'W']) { + await exportFPVToServer(state.env, `atlas/${sd}/_work/r${r.id}_${dir}_depth.png`, { at: r.vantage, dir }); + } + } + await this.saveJSON(`atlas/${sd}/manifest.json`, m); + return { seed: sd, rooms: m.rooms.length, frames: m.rooms.length * 4, themes: m.rooms.map((r) => `${r.id}:${r.theme}`) }; + }, state: () => ({ seed: state.seed, stage: state.stage, genMs: state.genMs, config: state.config, rooms: state.env?.regions.filter((r) => r.kind === 'Room' && r.cells.length > 0).length, pillarTiles: state.env ? countTiles(state.env, 3) : 0, diff --git a/reikhelm-web/render.js b/reikhelm-web/render.js index b9e5acc..d562363 100644 --- a/reikhelm-web/render.js +++ b/reikhelm-web/render.js @@ -94,8 +94,9 @@ export function getStage(env, stage) { // --- precomputed fields -------------------------------------------------- // Multi-source BFS distance (in cells) from every floor cell to the nearest -// wall. Wall cells are 0; open cells grow outward. Drives ambient occlusion. -function distanceToWall(tiles, w, h) { +// wall. Wall cells are 0; open cells grow outward. Drives ambient occlusion +// here, and crevice AO in the depth-map exporter (depth.js) — hence exported. +export function distanceToWall(tiles, w, h) { const dist = new Float32Array(w * h).fill(Infinity); const q = new Int32Array(w * h); let tail = 0; diff --git a/reikhelm-web/style.css b/reikhelm-web/style.css index 9da440a..30237bf 100644 --- a/reikhelm-web/style.css +++ b/reikhelm-web/style.css @@ -84,7 +84,8 @@ input[type="range"] { height: 4px; } -.seed-row { grid-template-columns: 40px 1fr auto; } +.seed-row { grid-template-columns: 40px 1fr auto auto; } +.seed-row button { padding: 5px 7px; white-space: nowrap; } .seed-row input[type="number"] { background: #0c0c11; border: 1px solid var(--panel-edge); diff --git a/tools/.gitignore b/tools/.gitignore index c6d2d49..04784c3 100644 --- a/tools/.gitignore +++ b/tools/.gitignore @@ -6,8 +6,14 @@ __pycache__/ # macOS .DS_Store -# Ad-hoc generated render / dither outputs (regenerable). Curated keepers live in pixelart/samples/. +# All pipeline outputs live here — depth maps, renders, dither, baked atlases, +# and the local gallery of keepers. Regenerable; organized; never committed. +out/ + +# Legacy ad-hoc outputs (pre-tools/out/ layout). Curated keepers live in pixelart/samples/. comfy-spike/out_*.png +comfy-spike/depth_*.png +comfy-spike/fpv_*.png pixelart/p_*.png pixelart/crypt_*.png pixelart/godray_*.png diff --git a/tools/README.md b/tools/README.md index b8c972c..c5c12ef 100644 --- a/tools/README.md +++ b/tools/README.md @@ -1,33 +1,73 @@ # reikhelm tools — AI-render → pixel-art pipeline Post-processing tools that turn a depth map into atmospheric, palette-locked pixel -art via a local diffusion render. Two stages, glued by PNG files. +art via a local diffusion render. Three stages, glued by PNG files. ``` -depth map ──► Stage 1: ComfyUI/Z-Image render ──► Stage 2: dither ──► pixel art -(synthetic now; (comfy-spike/comfy.py) (pixelart/pixelate.py) - real reikhelm - geometry next) +real geometry ──► depth map ──► Stage 1: ComfyUI/Z-Image ──► Stage 2: dither ──► pixel art +(reikhelm-web/ (top-down (comfy-spike/comfy.py) (pixelart/pixelate.py) + depth.js export) heightfield) ``` +**Now rendering real generated dungeons** (top-down). The depth exporter lives +renderer-side in `reikhelm-web/depth.js` (derives a height field from the same +envelope render.js paints — walls raised, floors mid, pools recessed, pillars as +bumps, plus subtle per-theme relief; near=white/far=black). It honors the "map +holds no rendering info" contract: height, like color, is derived from +tiles+regions+theme, never stored in the core. + ## Setup The pixelart tools share a venv (Pillow + numpy + opencv-headless): ``` python3 -m venv tools/pixelart/.venv tools/pixelart/.venv/bin/pip install pillow numpy opencv-python-headless ``` -`comfy-spike/comfy.py` is pure stdlib (no venv). LAN calls to ComfyUI need the Bash sandbox disabled. +`comfy-spike/comfy.py` and `serve.py` are pure stdlib (no venv). LAN calls to ComfyUI need the Bash sandbox disabled. + +## Stage 0 — export a depth map from a real dungeon (`serve.py` + `reikhelm-web/{depth,fpv}.js`) +Serve the playground and let it POST depth PNGs back to disk: +``` +python3 tools/serve.py # http://127.0.0.1:8000, saves → tools/comfy-spike/ +``` +Two depth modes, both derived from the same envelope (renderer-side, core stays clean): + +**Top-down** (`depth.js`) — a height field, walls raised / pools recessed. Hit the +playground's **⬇ depth** button, or headless: +```js +// in the page (e.g. via Playwright browser_evaluate): +window.reikhelm.setSeed(7); +await window.reikhelm.exportDepthToServer('depth_seed7.png'); // → tools/comfy-spike/depth_seed7.png +// window.reikhelm.exportDepth({stage,scale,bevel,ao,normalize}) → {url,width,height} for in-context capture +``` +≈1024px on the long edge (16px/cell at the default 64×40 map), aspect-matched to the render. + +**First-person** (`fpv.js`) — an Eye-of-the-Beholder / Daggerfall view via a Wolfenstein +DDA raycaster over the grid (wall slice + floor/ceiling ramp, near=white). Camera at a +room's most-open cell, one of 4 cardinal facings: +```js +window.reikhelm.setSeed(7); +const at = window.reikhelm.vantage(); // most-open cell of the largest room +for (const dir of ['N','E','S','W']) + await window.reikhelm.exportFPVToServer(`fpv_s7_${dir}.png`, { dir }); +// exportFPV({at:[x,y],dir,width,height,fov,maxView,gamma}) → {url,width,height,at,dir} +``` +1280×720 (16:9 — integer-scales to 1080p/1440p). Doorways are non-blocking, so they read as dark recesses leading deeper. ## Stage 1 — depth → diffusion render (`comfy-spike/comfy.py`) Talks to ComfyUI over HTTP (workstation `http://192.168.1.26:8188`). ``` -python3 comfy-spike/comfy.py depth --kind corridor --width 1536 --height 864 --out d.png # synthetic test depth -python3 comfy-spike/comfy.py zrender http://192.168.1.26:8188 --depth d.png --prompt "..." --prefix scene +# real dungeon (Stage 0 export), top-down prompt, geometry-honoring strength: +python3 comfy-spike/comfy.py zrender http://192.168.1.26:8188 --depth comfy-spike/depth_seed7.png \ + --width 1024 --height 640 --strength 0.85 --prefix rk_s7 \ + --prompt "overhead top-down view of an ancient stone dungeon, rock-cut chambers linked by corridors, flagstone floors, molten lava pool, torchlight, dark fantasy, highly detailed, cinematic" +# synthetic test depth (no real geometry): comfy.py depth --kind room --width 1024 --height 640 --out d.png # also: probe | nodeinfo for introspection ``` Proven Z-Image depth recipe: `ModelPatchLoader` + `ZImageFunControlnet`, CLIP type `qwen_image`, `EmptySD3LatentImage`, `res_multistep`/`simple`, 8–12 steps, cfg 1.0, `ModelSamplingAuraFlow` shift 3.0. -Depth strength ~0.6 for freeform; **~0.65–0.85 to honor real geometry**. +Depth strength ~0.6 for freeform; **0.80–0.85 is the sweet spot for real geometry** (confirmed on +seeds 7/33: <0.6 drifts off the layout, ≥1.0 goes rigid and flattens the pools). Set `--width/--height` +to the depth PNG's dimensions and use a **top-down/overhead** prompt to match the geometry. ## Stage 2 — render → pixel art (`pixelart/pixelate.py`) LOCKED "Primordyn" recipe = the defaults: @@ -39,11 +79,42 @@ tools/pixelart/.venv/bin/python pixelart/pixelate.py render.png --palette pixela Outputs a true indexed PNG (+ optional `--preview-scale` nearest-neighbor preview). Palette loader auto-detects .json / .gpl / JASC .pal / hex / Paint.NET / .png / `adaptive:N`. +## Stage 3 — bake & explore a first-person dungeon (`bake_atlas.py` + `reikhelm-web/explore.html`) +Pre-render every room's 4 cardinal views into an atlas, then walk it Eye-of-the-Beholder style. +``` +# 1) export the depth maps + room-graph manifest for a seed (browser, via Playwright): +# window.reikhelm.bakeAtlasDepths(7) → tools/out/atlas/7/_work/r__depth.png + manifest.json +# 2) render every view through comfy → pixelate (FIXED diffusion seed = style lock across frames): +python3 tools/bake_atlas.py --seed 7 # → tools/out/atlas/7/r_.png (640x360 pixel art) +# 3) walk it: http://127.0.0.1:8000/explore.html?seed=7 +``` +The explorer (`explore.html` + `explore-viewer.js`) loads the manifest + frames: you occupy a room and a +cardinal facing; **W/S** step forward/back to the room in that direction, **A/D** turn. `explore.js` builds +the room graph (corridors contracted; each neighbour binned to a cardinal) and per-room vantage points. +Prompt is tank's "empty abandoned ruin" recipe — **no** *first person / POV / dungeon-crawler* trigger +words (they summon player hands; at cfg 1.0 the negative prompt is inert, so the positive must stay clean). +See `.dev/2026-06-01-fpv-prompts.md`. The baker then varies the body **per room theme** (forge→lava, +throne→throne+banners, library→shelves, crypt→niches…) and adds a **wall-vs-passage clause per facing** +(from `open[dir]` in the manifest) so rooms read distinctly and solid walls don't grow phantom doors. +Strength **0.85** honors the footprint. `--rooms 6,7,9` re-bakes a subset. + ## Helpers & references - `pixelart/montage.py` — tile labeled images into a contact sheet. - `pixelart/samples/` — curated reference: before/after pair + the kernel/size & palette-stretch contact sheets. +- `out/gallery/` — local keepers (top-down + first-person hero shots). Everything under `out/` is git-ignored. ## Next -Wire **real reikhelm geometry** → depth-map export (top-down from the renderer's distance field, then a -height layer) into Stage 1, so we render actual generated dungeons. Later: port the locked Stage-2 dither -into a Rust crate so the engine owns the whole chain. See project memory. +- **Top-down real-geometry export — done** (`reikhelm-web/depth.js`, Stage 0). Per-tile heights + subtle + per-theme relief; the BFS distance field is reused only for crevice AO (it's an openness map, not a + depth map — feeding it raw would dome the floors). +- **First-person depth — working** (`reikhelm-web/fpv.js`). A grid raycaster, NOT the heavy 3D path once + assumed. 16:9 dungeon-crawler views from a room centre; strength **0.55–0.70** gives the richest results. +- **Playable EoB explorer — shipped** (Stage 3). Per-room ×4-facing atlas bake + a WASD viewer. Movement is + room-to-room along the contracted corridor graph. +- **Next for the explorer:** per-*cell* baking (smooth step-by-step movement, not just room-to-room); + entity sprites (monsters/treasure) composited on frames; door cells rendered as doors vs open archways; + a "bake this seed" button in the playground. Walk-forward render sequence lives in `out/walk/`. +- **Richer relief (top-down Stage B+):** push per-theme height harder (throne dais, deeper cisterns) if + top-down renders feel uniform. +- **Rust port:** move depth export (both modes) + the locked Stage-2 dither into a crate so the engine + owns the whole chain. See project memory. diff --git a/tools/bake_atlas.py b/tools/bake_atlas.py new file mode 100644 index 0000000..e220cfd --- /dev/null +++ b/tools/bake_atlas.py @@ -0,0 +1,155 @@ +#!/usr/bin/env python3 +"""Bake a first-person dungeon atlas: every room's 4 cardinal views → pixel art. + +Input (written by the browser, see explore-bake hook): + tools/out/atlas//_work/r__depth.png FPV depth maps (1280x720) + tools/out/atlas//manifest.json room graph (nav + vantages + tiles) + +For each depth map this runs the proven chain — comfy.py zrender (Z-Image depth +ControlNet) → pixelate.py (locked Primordyn dither) — and writes the explorer frame: + tools/out/atlas//r_.png 640x360 indexed pixel art + +The prompt is tank's "empty abandoned ruin" recipe: NO first-person/POV/crawler +trigger words (they summon player hands), a fixed style-anchor block, and a FIXED +seed across every frame so the palette + lighting stay coherent across the atlas. + + python3 tools/bake_atlas.py --seed 7 [--url http://192.168.1.26:8188] + [--strength 0.65] [--size 640x360] [--limit N] +""" +import argparse +import glob +import json +import os +import re +import subprocess +import sys + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +COMFY = os.path.join(REPO, "tools", "comfy-spike", "comfy.py") +PIXELATE = os.path.join(REPO, "tools", "pixelart", "pixelate.py") +VENV_PY = os.path.join(REPO, "tools", "pixelart", ".venv", "bin", "python") +PALETTE = os.path.join(REPO, "tools", "pixelart", "primordyn_v2.json") +RUN = os.getpid() # unique per invocation → ComfyUI never reuses a SaveImage prefix (re-bakes stay clean) + +# Tank's anti-hands framing ("empty abandoned ruin, no people, architectural +# photography" — never "first person / POV / dungeon crawler"), but the room's +# CHARACTER now comes from its procgen theme and the view's actual openings, so +# the atlas stops being a mashup of one generic hall. The style anchor stays +# byte-identical across frames (consistency lever); the theme body + opening +# clause vary per (room, facing). +STYLE = ("dark fantasy, ancient torchlit stone, warm orange torchlight, deep black shadows, " + "wet stone, volumetric haze, architectural photography of an empty ruin, " + "empty, deserted, no people, uninhabited, atmospheric, highly detailed, ") +THEME_BODY = { + "forge": "molten forge chamber, channels of glowing orange lava, iron anvils and hanging chains, soot-blackened granite, fiery glow", + "cistern": "flooded stone cistern, still dark water with mirror reflections, dripping wet walls, cold teal light", + "crypt": "ancient crypt, carved stone sarcophagi and bone-filled wall niches, cobwebs, cold pale light", + "library": "ruined library, tall stone shelves of rotting books and scrolls, drifting dust, warm dim light", + "throne": "vast throne hall, a raised dais with a great stone throne, tattered banners, towering carved columns", + "vault": "treasure vault, iron-bound chests and stone strongboxes, scattered gold, heavy locked stone", + "hall": "grand pillared hall, rows of carved stone columns, high vaulted ceiling", + "den": "foul beast den, gnawed bones and filth, claw-scratched walls, a feral lair, dim red light", + "threshold": "dungeon gatehouse, a great iron portcullis and worn stone steps, torchlit entrance", + "stone": "rough-hewn stone chamber, bare granite walls, plain and ancient", +} +DEFAULT_BODY = THEME_BODY["stone"] +# These reinforce the depth (a deep recess vs a near wall straight ahead) so the +# model paints a passage only where one actually exists — killing phantom doors. +AHEAD_OPEN = "a dark arched passage leads onward into shadow ahead" +AHEAD_WALL = "a solid carved stone wall closes the way ahead" +NEGATIVE = ("person, people, human, figure, character, hands, fingers, hand, arm, arms, " + "holding weapon, sword, feet, legs, body, silhouette, UI, HUD, text, watermark, modern") + + +def prompt_for(theme, is_open): + body = THEME_BODY.get(theme or "stone", DEFAULT_BODY) + return f"{STYLE}{body}, {AHEAD_OPEN if is_open else AHEAD_WALL}" + +NAME_RE = re.compile(r"^r(\d+)_([NESW])_depth\.png$") + + +def run(cmd): + p = subprocess.run(cmd, capture_output=True, text=True) + if p.returncode != 0: + sys.stderr.write(p.stdout + p.stderr + "\n") + return p.returncode == 0 + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--seed", type=int, required=True, help="dungeon seed (selects the atlas dir)") + ap.add_argument("--url", default="http://192.168.1.26:8188") + ap.add_argument("--strength", type=float, default=0.85, help="depth control strength (0.85 honors the room footprint)") + ap.add_argument("--render-seed", type=int, default=7, help="FIXED diffusion seed for every frame (style lock)") + ap.add_argument("--size", default="640x360", help="pixel-art frame size (16:9; 320x180 for chunky)") + ap.add_argument("--width", type=int, default=1280) + ap.add_argument("--height", type=int, default=720) + ap.add_argument("--rooms", default="", help="comma-separated room ids to bake (default all)") + ap.add_argument("--limit", type=int, default=0, help="bake only the first N frames (smoke test)") + a = ap.parse_args() + + atlas = os.path.join(REPO, "tools", "out", "atlas", str(a.seed)) + work = os.path.join(atlas, "_work") + + # Per-room theme + per-facing openness come from the manifest the browser wrote. + rooms = {} + mpath = os.path.join(atlas, "manifest.json") + if os.path.exists(mpath): + with open(mpath) as f: + for r in json.load(f).get("rooms", []): + rooms[int(r["id"])] = r + + only = {int(x) for x in a.rooms.split(",") if x.strip()} if a.rooms else None + + def order(p): + m = NAME_RE.match(os.path.basename(p)) + return (int(m.group(1)), "NESW".index(m.group(2))) if m else (1 << 30, 0) + depths = sorted(glob.glob(os.path.join(work, "r*_*_depth.png")), key=order) + if only is not None: + depths = [p for p in depths if NAME_RE.match(os.path.basename(p)) and int(NAME_RE.match(os.path.basename(p)).group(1)) in only] + if not depths: + sys.exit(f"!! no depth maps to bake in {work} — run the browser export first") + if a.limit: + depths = depths[:a.limit] + + print(f"baking {len(depths)} frames → {os.path.relpath(atlas, REPO)} " + f"(strength {a.strength}, render-seed {a.render_seed}, {a.size}, per-theme prompts)") + ok = 0 + for i, depth in enumerate(depths, 1): + m = NAME_RE.match(os.path.basename(depth)) + if not m: + continue + rid, d = m.group(1), m.group(2) + room = rooms.get(int(rid), {}) + theme = room.get("theme") + is_open = bool(room.get("open", {}).get(d, False)) + prompt = prompt_for(theme, is_open) + prefix = f"r{rid}_{d}_{RUN}" # per-run unique; avoids ComfyUI prefix-counter collisions on re-bake + frame = os.path.join(atlas, f"r{rid}_{d}.png") + tag = f"[{i}/{len(depths)}] r{rid} {d} {theme or '?'}{'·open' if is_open else '·wall'}" + + if not run(["python3", COMFY, "zrender", a.url, "--depth", depth, + "--width", str(a.width), "--height", str(a.height), + "--strength", str(a.strength), "--seed", str(a.render_seed), + "--prompt", prompt, "--negative", NEGATIVE, + "--prefix", prefix, "--wait", "300"]): + print(f" {tag} RENDER FAILED"); continue + + renders = glob.glob(os.path.join(work, f"out_{prefix}_*.png")) + if not renders: + print(f" {tag} no render output"); continue + render = renders[0] + + if not run([VENV_PY, PIXELATE, render, "--palette", PALETTE, + "--size", a.size, "--preview-scale", "0", "--out", frame]): + print(f" {tag} DITHER FAILED"); continue + + os.remove(render) # keep the depth, drop the big intermediate render + ok += 1 + print(f" {tag} → {os.path.relpath(frame, REPO)}") + + print(f"done: {ok}/{len(depths)} frames baked into {os.path.relpath(atlas, REPO)}") + + +if __name__ == "__main__": + main() diff --git a/tools/serve.py b/tools/serve.py new file mode 100644 index 0000000..5eda393 --- /dev/null +++ b/tools/serve.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python3 +"""Dev server for the reikhelm playground, headless depth/FPV export, and the +pre-baked first-person explorer. Pure stdlib — no deps. + + python3 tools/serve.py # serve reikhelm-web on :8000 + +Three things, one server: + GET / static file from the web root (reikhelm-web/) + GET /out/ static file from the output tree (tools/out/) — lets the + explorer load baked atlas frames + manifests + POST /save/.png write the raw request body (a PNG) under tools/out/; + sub-dirs allowed and created (e.g. atlas/7/r3_N.png) + → {"ok": true, "path": "...", "bytes": N} + +All generated images live under tools/out/ (git-ignored, organized): + out/depth/ top-down + one-off depth maps out/fpv/ first-person stills + out/atlas// a baked explorable dungeon (frames + manifest.json) +""" +import argparse +import json +import os +import sys +from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import unquote + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + + +def safe_join(base, rel): + """Join rel under base, collapsing any '..' so it can't escape base.""" + rel = os.path.normpath("/" + rel).lstrip("/") + return os.path.join(base, rel) + + +class Handler(SimpleHTTPRequestHandler): + out_dir = "." + + extensions_map = { + **SimpleHTTPRequestHandler.extensions_map, + ".js": "text/javascript", + ".mjs": "text/javascript", + ".wasm": "application/wasm", + ".json": "application/json", + } + + # Route /out/ to the output tree; everything else from the web root (cwd). + def translate_path(self, path): + p = unquote(path.split("?", 1)[0].split("#", 1)[0]) + if p == "/out" or p.startswith("/out/"): + return safe_join(self.out_dir, p[len("/out/"):]) if p != "/out" else self.out_dir + return super().translate_path(path) + + def end_headers(self): + self.send_header("Cache-Control", "no-store") # dev server: always serve fresh modules + super().end_headers() + + def do_POST(self): + if not self.path.startswith("/save/"): + self.send_error(404, "only /save/.png is supported") + return + rel = unquote(self.path[len("/save/"):]) + if not (rel.endswith(".png") or rel.endswith(".json")): + self.send_error(400, "path must end in .png or .json") + return + dest = safe_join(self.out_dir, rel) + length = int(self.headers.get("Content-Length", 0)) + body = self.rfile.read(length) + os.makedirs(os.path.dirname(dest), exist_ok=True) + with open(dest, "wb") as f: + f.write(body) + payload = json.dumps({"ok": True, "path": dest, "bytes": len(body)}).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(payload))) + self.end_headers() + self.wfile.write(payload) + sys.stderr.write(f"saved {os.path.relpath(dest, REPO)} ({len(body)} bytes)\n") + + def log_message(self, fmt, *args): # quiet GETs; POSTs/errors print via do_POST/send_error + pass + + +def main(): + ap = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + ap.add_argument("--port", type=int, default=8000) + ap.add_argument("--root", default=os.path.join(REPO, "reikhelm-web"), help="static web root") + ap.add_argument("--out", default=os.path.join(REPO, "tools", "out"), help="output tree (served at /out, target of /save)") + args = ap.parse_args() + + Handler.out_dir = os.path.abspath(args.out) + os.makedirs(Handler.out_dir, exist_ok=True) + root = os.path.abspath(args.root) + os.chdir(root) + httpd = ThreadingHTTPServer(("127.0.0.1", args.port), Handler) + print(f"serving {os.path.relpath(root, REPO)} at http://127.0.0.1:{args.port}") + print(f" /out/ → {os.path.relpath(Handler.out_dir, REPO)} POST /save/.png writes there") + try: + httpd.serve_forever() + except KeyboardInterrupt: + httpd.shutdown() + + +if __name__ == "__main__": + main()