There is no 3D in that picture. The map is a flat grid of integers. For every vertical column of pixels on screen, the code fires one ray across that grid, measures how far it travelled before it hit something, and draws a vertical stripe whose height is 1 / distance. Near wall, tall stripe. Far wall, short stripe. Do that 580 times and your eye assembles a hallway.
That's it. That's the entire technique. Everything else in this tutorial is detail work stacked on top of that one sentence — and the detail work is where it gets interesting, because each layer has a specific way it goes wrong, and those wrong versions are worth seeing before you fix them.
Why this project
Two things you might want are usually in tension. Learning how rendering actually works normally means writing a software rasterizer that renders a spinning cube and nothing else. Learning how to make a game normally means picking up Unreal or Godot, where the rendering is a black box you are specifically prevented from touching.
A raycaster collapses both. You write every line of the renderer — projection, texture mapping, depth sorting, all of it — and the finished artifact is a playable first-person shooter that loads in a browser tab from a static host. No download, no cloud GPU, no install.
It's also the right size. Roughly 600 lines for a version with textured walls, sprites, enemies and a weapon. That's a week of evenings, not a season.
And then the part that isn't 1992
Stages 0 through 9 rebuild a 1992 renderer. Stages 10 through 12 do something that renderer's authors had no reason to think of, and it comes from one observation:
A raycaster isn't a renderer. It's a perception system that happens to be pointed at the screen.
In 1992 you cast rays to draw a world the machine already knew. Today the identical algorithm runs on robots and delivery drones to learn a world the machine doesn't know — a 2D LiDAR scan is a ray cast in a loop returning distances, which is exactly what cast() returns. Same math, inverted purpose.
So once the game works, we point the sensor the other way. Stage 10 gives enemies their own beliefs about a map they have to discover, which produces stealth that nobody has to fake. Stage 11 hands that sensor feed to a language model and lets it drive. Stage 12 ports the cast loop to a GPU compute shader and measures whether that was worth doing.
Four phases
The thirteen stages group into four phases. Each phase ends with something you'd show someone — not a checkpoint in a build log, an actual thing. You can stop at the end of any phase and have a finished artifact rather than a half-finished one.
Stage numbering runs straight through 00 to 12, so a stage number always means the same thing regardless of which phase you're in.
Contents
One · The sensor
ends with: a ray that measuresFour · The inversion
ends with: a perception systemHow to read a stage
Each stage ends with something on screen that works. You are never more than one evening from a checkpoint. Each one also has a characteristic bug — the specific mistake nearly every first implementation makes. Those are called out on purpose. Hitting them is not a detour; it's the part you'll remember.
Every stage carries a spec written the way you'd hand it to a Claude Code agent team: goal, requirements, non-goals, acceptance criteria. Paste it verbatim or write your own from the same shape. If you're building this by hand, read the spec as a checklist and ignore the framing.
Two decisions made at the top that save real pain later. Render at low internal resolution — 320×200 in a canvas that's CSS-scaled to full size. It's period-correct, it's fast enough that you'll never think about performance, and the chunky pixels look better than a crisp high-res version. And settle the cast function's return shape at stage 2, because texturing, sprites, hitscan and the minimap all consume that one interface.
The sensor
No 3D in this phase at all. You end with a top-down view of a grid, a player who can walk around it, and one function — cast() — that fires a ray and reports exactly what it hit and how far away it was. That function is the entire project. Everything after this consumes it.
Map, player, and a top-down debug view
no 3D yetMost tutorials skip straight to casting rays. That's a mistake. Every bug from here to the end is a math bug, and math bugs are invisible in a first-person view — a wall that's subtly in the wrong place looks exactly like a wall that's in the right place. A top-down view turns invisible math into a picture you can check at a glance.
Build it first, bind it to a key, and keep it for the entire project. It becomes your minimap in stage 9 anyway.
GOAL
A top-down debug view of a grid map with a movable player.
CONTEXT
Stage 0 of a browser raycaster (Wolfenstein-style FPS).
Vanilla JS, single HTML file, one <canvas>. No libraries,
no build step, no framework.
REQUIREMENTS
- MAP: flat Int array, 24x24. 0 = empty, 1..N = wall type.
The border must be solid wall on all four sides.
- player: { x, y, angle } where 1 world unit = 1 grid cell.
x and y are floats. The player is NOT snapped to the grid.
angle is radians, 0 = +x axis.
- Draw top-down at a fixed scale (12px per cell): walls filled,
empty cells outlined, player as a dot with a short line
showing facing direction.
- Input: W/S move along facing, A/D strafe,
ArrowLeft/ArrowRight turn.
- Movement must be frame-rate independent. Use a delta-time
game loop with requestAnimationFrame, not a fixed per-frame
step. Speed in units/second, turn in radians/second.
- Bind the debug view to the M key. It stays in the project.
NON-GOALS
No raycasting. No 3D. No collision detection.
ACCEPTANCE
- Holding W moves the dot in the direction the line points.
- Turning changes the line, not the position.
- The player walks through walls. This is expected here.
- Movement speed is identical at 60fps and 30fps.
A dot with a whisker, sliding around a grid of squares, passing through them.
Cast one ray, badly
~10 linesMarch along the ray in small fixed steps — 0.01 units at a time — checking the grid cell at each step until you land in a wall. Draw the ray on the top-down view.
This is the wrong algorithm and you should write it anyway. It's ten lines, it works, and it gives you a correct picture to compare against when you replace it. Building the naive version first is how you end up understanding why the real one exists instead of just copying it.
GOAL
Cast a single ray from the player along their facing angle
and draw it on the top-down view.
REQUIREMENTS
- castNaive(x, y, angle) marches in fixed 0.01-unit steps
along the ray, sampling MAP at each step, and stops when it
enters a non-zero cell or exceeds a max distance of 32 units.
- Returns { distance, tile }.
- Draw the ray from the player to the hit point on the
top-down view, and mark the hit point.
NON-GOALS
Do not optimise this. Do not write DDA. Do not render 3D.
ACCEPTANCE
- The ray line always terminates at a wall face, never inside
open space and never past a wall.
- Turning sweeps the ray around the player smoothly.
NOTE FOR REVIEW
This implementation is deliberately naive and will be replaced
in stage 2. Do not flag the fixed step size as a defect.
A champagne line from your dot to the nearest wall, sweeping as you turn.
Replace it with DDA
the load-bearing stageThe naive march has two problems. It's slow — hundreds of samples per ray, times 320 rays, times 60 frames. And it's imprecise: it reports the distance to somewhere inside the wall, not to the wall's face, which will show up as wobbling textures later.
The fix is a digital differential analyzer. Instead of stepping a fixed distance, you step from one grid line to the next — always jumping exactly to the next place the ray crosses a cell boundary, choosing whichever of the two candidate crossings (vertical or horizontal) is nearer. Typically five to fifteen steps instead of several hundred, and the distance it returns is exact.
This is also where you lock the return shape. Everything downstream reads it.
GOAL
Replace castNaive with a DDA grid traversal. Identical visible
output, exact distances, far fewer iterations.
REQUIREMENTS
- cast(px, py, dirX, dirY) takes a normalised direction vector,
not an angle. Callers do the trig.
- Standard DDA: compute deltaDist per axis, seed sideDist from
the player's fractional position within the cell, then loop
advancing whichever sideDist is smaller.
- Return exactly this shape. Every later stage depends on it:
{
distance, // perpendicular distance, fisheye-free
tile, // the non-zero MAP value that was hit
side, // 0 = hit a N/S face, 1 = hit an E/W face
hitOffset, // 0..1 along the wall face at the hit point
mapX, mapY // the cell that was hit
}
- distance comes from the sideDist arithmetic directly
(sideDist - deltaDist for the axis that stepped last).
Do NOT compute Euclidean distance from a hit point.
- Bail out at 64 iterations and return distance = Infinity so a
malformed map cannot hang the frame.
NON-GOALS
No rendering changes. The top-down view should look the same.
ACCEPTANCE
- Ray endpoints are visually identical to stage 1's, within a
pixel, from at least six different positions and angles.
- Iteration count logged per ray is under 20 in a 24x24 map.
- A ray fired exactly along an axis (angle 0, PI/2, PI) does not
produce NaN or Infinity in the delta calculations.
CONSTRAINT
This return shape is a public interface. Changing it later means
touching texturing, sprites, hitscan and the minimap. Get it
reviewed before moving on.
Off-by-one on which cell you test. You step mapX and then read the map, or read and then step — one order is right and the other stops your rays one square early or late, consistently, in a way that looks almost fine. The top-down view is how you catch it.
A ray pointing exactly along an axis makes one component of the direction vector zero, and 1/0 gives you Infinity. That's actually the correct behaviour here — an infinite delta means "never crosses that axis" and the comparison handles it. But 0/0 gives NaN, which poisons everything silently. Guard it.
Same picture as stage 1, roughly thirty times faster, with exact distances.
The world
Point the sensor at the screen. You end with a solid 3D hallway you can walk down — untextured, flat-shaded, but genuinely three-dimensional and genuinely solid. This is the phase where it stops being a diagram and starts being a place.
One ray per column — the 3D appears
the payoffCast one ray per screen column across your field of view. Wall height is a constant divided by distance. Draw a vertical stripe of that height, centred on the horizon. Ceiling above it, floor below it, both flat colours.
Twenty lines. It goes from top-down squares to a hallway you can walk down.
Two things to do here that look pointless now and matter enormously later. Shade walls hit on one axis darker than the other — you already have side from the DDA, so it's one boolean, and it reads as free directional lighting. And store every column's distance in an array. You will not use it until stage 7, at which point it becomes your depth buffer and you'll be glad it's there.
GOAL
Render a first-person view: one ray per screen column,
wall stripes sized by distance.
REQUIREMENTS
- Internal render resolution 320x200. The canvas element is
CSS-scaled to fill its container. Set
ctx.imageSmoothingEnabled = false.
- Use the camera-plane formulation, not an angle sweep:
dir = (cos(angle), sin(angle))
plane = perpendicular to dir, length = tan(fov/2)
for column x: cameraX = 2*x/W - 1
rayDir = dir + plane * cameraX
FOV 60 degrees.
- lineHeight = floor(H / distance).
Stripe is centred vertically on H/2. Clip to screen bounds.
- Flat ceiling colour above the stripe, flat floor below.
- Wall colour is chosen by `tile`. Multiply RGB by 0.72 when
side === 1 for directional shading.
- Write each column's distance into a preallocated
Float32Array(320) named zBuffer. Nothing reads it yet.
- Keep the top-down view on M. It renders the ray fan.
NON-GOALS
No textures. No fisheye correction (stage 4). No collision.
ACCEPTANCE
- Walking forward makes walls grow smoothly, not in jumps.
- Corners read as corners because of the side shading.
- zBuffer[x] equals the distance used to size column x.
- Frame time under 4ms for the cast+draw loop on a 24x24 map.
The obvious approach is to sweep the ray angle evenly from angle - fov/2 to angle + fov/2. It's intuitive and it's wrong — evenly spaced angles do not land on evenly spaced columns of a flat screen, so the image stretches toward the edges. The plane formulation spaces the rays across a flat projection plane, which is what your monitor actually is.
A hallway. Genuinely three-dimensional, and you wrote every line of it.
Meet the fisheye, then kill it
do not skip aheadIf you followed stage 3 exactly you already have this right, because perpWallDist from the DDA is fisheye-free. So do it wrong on purpose first: temporarily size your stripes with the raw Euclidean distance to the hit point instead. Walk around.
The walls bow outward. Flat surfaces bulge toward you in the centre of the screen, and the whole world breathes as you turn. That's the fisheye, and it happens because a ray toward the edge of your view travels further to reach the same flat wall — so it reports a bigger number, so its stripe is shorter, so a straight wall renders as a curve.
The correction is to project the ray distance onto the view direction: multiply by the cosine of the angle between the ray and the player's facing. The DDA's perpendicular distance does this for free, which is why it's the right thing to have returned.
Seeing the distortion before you correct it is the whole lesson. It's also the best screenshot pair in the writeup.
GOAL Demonstrate and then correct fisheye distortion. REQUIREMENTS - Add a debug toggle (F key) that switches column height between Euclidean ray distance (distorted) and perpendicular distance (correct). Default is correct. - The toggle is a teaching device and stays in the build. - Add an on-screen indicator when distortion mode is active so a reader can tell which screenshot is which. ACCEPTANCE - With F on, a long flat wall visibly bows outward and warps when turning. - With F off, the same wall renders as a straight horizontal edge from every angle and position. - Standing square-on to a wall, the top edge of the wall is a perfectly level line across the screen.
Correcting twice. If you already used perpWallDist and then also multiply by cos(rayAngle - playerAngle), you get an inverse fisheye — walls that bow inward. Subtler than the original and much easier to stare past.
Stop walking through walls
short stageCheck the destination cell before committing the move. The important detail: test the x and y components separately. If you test the combined move and reject it wholesale, the player sticks to walls the moment they approach at any angle. Testing per-axis means a rejected x still allows the y, which is what "sliding along a wall" is.
Give the player a small radius too — roughly 0.2 units — or you'll clip diagonally through corners where two walls meet.
GOAL Collision with wall sliding. REQUIREMENTS - Player collision radius: 0.2 world units. - Resolve x and y independently. Compute the candidate new x; if the cell at (newX +/- radius, y) is solid, reject only x and keep the current x. Repeat for y. - Check the cell offset by the radius in the direction of travel, not just the centre point. - Applies to all movement: forward, back and strafe. ACCEPTANCE - Walking into a wall head-on stops the player dead. - Walking into a wall at 45 degrees slides the player along it rather than sticking. - Running diagonally into an inside corner does not squeeze the player through the seam between two wall cells.
The game
Textures, sprites, a weapon, and the small timing details that separate a tech demo from something people play twice. You end with a first-person shooter you could put on the showcase page — the finished 1992 artifact.
Textures
first parallel stageThis is where hitOffset earns its place in the return shape. It tells you how far along the wall face the ray landed, 0 to 1, which picks the texture column. The stripe's vertical position picks the texture row.
For each screen pixel in the stripe, work out where it falls in the stripe's full (possibly off-screen) height, map that to a row of the texture, and read the pixel. Doing this per-pixel with drawImage is too slow — write to an ImageData buffer directly.
Textures can be generated in code rather than loaded. A brick pattern is a nested loop and forty lines, it keeps the project to a single file with no assets, and it means nobody has to hunt for licensed art.
GOAL
Textured walls, drawn per-pixel into an ImageData buffer.
REQUIREMENTS
- 64x64 textures, generated procedurally at startup into
Uint8ClampedArrays. At least four: brick, panel, stone, door.
No external image files.
- texX = floor(hitOffset * 64).
- Mirror texX when the ray hits the back face of a wall, so
adjacent walls don't read as mirror images of each other.
Condition depends on `side` and the sign of the ray direction.
- Per stripe pixel y:
d = (y - H/2 + lineHeight/2) * 64 / lineHeight
texY = floor(d) & 63
Compute the step incrementally; do not divide per pixel.
- Apply the side === 1 shading by bit-shifting the channels, not
by a float multiply per pixel.
- Write into one ImageData and putImageData once per frame.
NON-GOALS
No floor or ceiling texturing. Those stay flat colours.
ACCEPTANCE
- Texture stays locked to the wall as the player moves. It must
not swim, slide or shear.
- Two walls meeting at a corner show no seam and no repeated
or reversed column at the join.
- Standing right against a wall does not produce a stretched
single column or an out-of-bounds read.
- Frame time under 8ms at 320x200.
Textures mirrored on half your walls. The sign error on hitOffset for one of the two facings is nearly invisible on a symmetric brick pattern and screamingly obvious the moment you put lettering or a door on a wall. Test with an asymmetric texture.
Walking close to a wall makes lineHeight enormous, so texY runs off the end of the texture. Masking with & 63 instead of clamping handles it and is faster, but only because 64 is a power of two — which is the actual reason texture sizes are powers of two.
Sprites
the hard oneWalls are easy because there's exactly one per column and it's always the nearest thing. Sprites are hard because there can be several, they overlap each other, and they can be in front of or behind walls depending on which column you're looking at.
The procedure: transform each sprite's world position into camera space using the inverse of your direction/plane matrix. That gives you a depth and a horizontal offset. Project those to a screen x and a size. Then draw the sprite column by column — and for each column, only draw if the sprite's depth is less than zBuffer[x].
That's the array you filled in stage 3 and haven't touched since. It's your depth buffer, and it's the entire reason a sprite standing behind a pillar is correctly hidden by it while the half of it sticking out is still visible.
Sort sprites far-to-near before drawing, or two overlapping enemies will stack in whatever order the array happens to be in.
GOAL
Billboard sprites with correct occlusion against walls
and against each other.
REQUIREMENTS
- sprites: [{ x, y, texture, alive }] in world coordinates.
- Each frame: compute squared distance from player, sort
descending (far to near), then draw in that order.
- Camera transform using the inverse of the [plane | dir] matrix:
invDet = 1 / (planeX*dirY - dirX*planeY)
transX = invDet * (dirY*relX - dirX*relY)
transY = invDet * (-planeY*relX + planeX*relY)
transY is the depth. Skip the sprite entirely if transY <= 0.
- screenX = (W/2) * (1 + transX / transY)
spriteSize = abs(floor(H / transY))
- Per sprite column, skip unless:
column is on screen AND transY < zBuffer[column]
- Sprite textures need transparency. Reserve one colour as the
key and skip those pixels, or carry an alpha channel.
NON-GOALS
No animation frames. No AI. No collision with sprites.
ACCEPTANCE
- A sprite behind a wall is fully hidden.
- A sprite half behind a pillar shows exactly the visible half,
with a clean vertical cut at the pillar edge.
- Two overlapping sprites: the nearer one draws on top,
from every viewing angle including circling around them.
- A sprite directly beside or behind the player does not render
and does not throw.
Sprites drawing on top of walls they're standing behind. That's always a missing or inverted depth test. The tell is that it looks correct until you back away from the sprite and it starts floating through geometry.
Sprites behind the player appearing mirrored in front of them. Negative transY flips the projection instead of rejecting it. The transY <= 0 guard is not optional.
Weapon and combat
now it's a gameYou already built the hard part. Hitscan shooting is casting one ray straight down the centre of the screen and asking what it hits first — which is a function you wrote in stage 2, plus a check against the sprite list using the same camera transform from stage 7.
The rest is game feel, and game feel is almost entirely timing and feedback rather than logic. A muzzle flash for two frames. A hit marker. Screen shake for 80 milliseconds. These are cheap and they're the difference between a tech demo and something people play twice.
GOAL
Shooting, damage, and a HUD.
REQUIREMENTS
- Weapon sprite anchored to the bottom centre of the viewport,
scaled with the render resolution.
- Fire on click or Space. Cooldown between shots.
- Hitscan resolution:
1. cast() down the centre ray -> wall distance
2. for each alive sprite, camera-transform it; a hit requires
transY > 0, |transX| within the sprite's half-width at
that depth, and transY < wall distance
3. nearest qualifying sprite takes the damage
- Fire animation: at least 3 frames, driven by elapsed time,
never by frame count.
- Feedback on hit: 2-frame muzzle flash, a hit marker, and a
screen shake of ~80ms. Feedback fires before any state update
so it never feels delayed.
- HUD: health, ammo, score. Mono type, bottom of the screen.
- Enemy death removes it from the draw list after its death
frames finish, not immediately.
ACCEPTANCE
- Shooting an enemy standing behind a wall does no damage.
- Shooting an enemy standing in a doorway with a wall beside it
registers a hit.
- Two enemies in a line: only the near one takes damage.
- Holding the fire button respects the cooldown and does not
drain ammo faster at higher frame rates.
Feel
cheapest wins in the projectEverything here is optional and every item is worth more than it costs.
Distance fog is three lines — darken each wall column toward the ceiling colour as a function of depth — and it does more for the atmosphere than anything else on this list. It also hides your draw distance for free.
Head bob is a sine wave on the horizon offset while moving. Mouse look via the Pointer Lock API. Enemy AI can be genuinely dumb — move toward the player if there's a clear ray between you, which is again the cast function you already have.
GOAL Atmosphere and polish. Each item independently toggleable. REQUIREMENTS - Distance fog: blend wall colour toward the ceiling colour by min(distance / fogEnd, 1). Applies to sprites too, or they will float out of the fog and look pasted on. - Head bob: sine on the horizon offset, amplitude scaled by current movement speed, zero when stationary. - Mouse look via Pointer Lock. Keyboard turning still works. Sensitivity is a named constant. - Minimap: reuse the stage 0 top-down view, scaled down, corner-anchored, toggled with M. - Enemy AI: if cast() from enemy to player returns a distance greater than the straight-line distance between them, line of sight is clear -> advance. Otherwise idle. - Audio: fire, hit, enemy alert, footsteps. WebAudio oscillators are acceptable; no asset files required. ACCESSIBILITY - Respect prefers-reduced-motion: disable head bob and screen shake, keep everything else. - The game must be fully playable on keyboard alone. ACCEPTANCE - Every item above can be switched off individually without breaking any other item. - Frame time stays under 12ms at 320x200 with 12 sprites active.
The inversion
Turn the sensor around. The same cast() that draws the world can be used to discover one — so enemies get their own beliefs about a map they've never seen, a language model gets handed the raw ray feed and told to drive, and the column loop gets ported to a GPU to find out whether that was worth doing. You end with something nobody published in 1992, because nobody had a reason to.
Enemies that don't know the map
the novel oneEvery FPS since Wolfenstein cheats. The enemy holds a reference to the same map array you do, always knows exactly where you are, and "AI difficulty" is a knob controlling how convincingly it pretends otherwise. Stealth in that world is a meter, because there is nothing real to hide from.
Invert it. Give each enemy its own grid — a Float32Array of log-odds, one per cell, initialised to zero meaning unknown. Every tick it casts around thirty rays across its own field of view. Cells a ray passes through get nudged toward free; the cell it stops at gets nudged toward occupied. Clamp so no cell becomes unshakeably certain, and repeat.
That's a Bayesian occupancy grid — the standard technique in mobile robotics — and it's roughly forty lines, because you already built the sensor in stage 2.
Then navigate on the belief instead of the truth. Treat unknown cells as passable but expensive and you get frontier exploration for free: the enemy walks toward the boundary between what it knows and what it doesn't, because that's where the cheap unexplored space is.
Watch the right panel fill in. Nothing seeds it — the enemy starts blind in a corner and builds that picture entirely out of ray returns. The champagne cells are walls it has personally observed. The dark cells are places it has never looked.
What this actually buys you
- Enemies can be wrong. Slip through a side passage while one is watching a corridor and it still believes you're in the corridor, because that is genuinely the last thing it observed. It acts on a stale map and looks in the wrong place.
- Hiding becomes a real mechanic rather than an abstraction. Breaking line of sight breaks the sensor, and the sensor is the only thing the enemy has.
- Enemies that meet can merge maps. One spots you, walks to another, and now both know — an explicit, visible moment of information transfer between agents.
That last one is worth dwelling on. It's the same structure as the pipeline on the front page: independent agents with partial views, and the interesting behaviour emerging at the handoff.
GOAL
Give each enemy a private occupancy grid built only from its own
ray casts, and navigate on that belief rather than on MAP.
CONTEXT
Stage 10 of a browser raycaster. cast() from stage 2 already
returns { distance, tile, side, hitOffset, mapX, mapY }. This
stage adds a variant that reports every cell traversed, not just
the cell hit.
REQUIREMENTS
- castCells(px, py, dirX, dirY, maxDist) runs the same DDA but
accumulates each empty cell it passes through, and returns
{ cells, hit, distance } where hit is the cell index or null
if the ray reached maxDist without stopping.
- Each enemy owns belief: Float32Array(MAP.length), all zeros.
Zero means unknown, not free. This distinction is the stage.
- Sensor model, per tick, per enemy:
28 rays across a 90 degree cone centred on facing
range 6.5 world units
each traversed cell: L[i] = max(-4, L[i] - 0.42)
the hit cell: L[i] = min( 4, L[i] + 0.85)
Clamping at +/-4 is required. Without it a cell observed for
long enough becomes unrevisable and the enemy cannot learn
that a door opened.
- A cell is: unknown if |L| < 0.2, free if L <= -0.2,
occupied if L >= 0.2.
- Frontier selection: the nearest believed-free cell that has at
least one 4-neighbour still unknown. Recompute when the target
is reached or becomes unreachable.
- Steering: turn toward the target at a capped rate (do not snap
the angle), advance at walking speed.
- Physical collision still resolves against MAP. The enemy's
belief governs where it decides to go; the world governs where
it can actually be. Never let belief override physics.
- Player detection uses the same rays. Seeing the player writes
a lastSeen { x, y, t } on the enemy. It pursues lastSeen, NOT
the player's live position, and gives up after a timeout.
NON-GOALS
No pathfinding around known obstacles yet (steer-and-bump is
fine). No map merging between enemies (stage 10b). No change to
rendering.
ACCEPTANCE
- An enemy released in a corner with an all-zero belief explores
and reaches over 95% of reachable cells classified.
- Zero misclassified cells: every cell it has decided about
matches MAP. Assert this in a headless test loop.
- Breaking line of sight and moving makes the enemy travel to
where the player WAS. Observable, repeatable, not stochastic.
- Two enemies in the same room build measurably different grids,
because they have looked at different things.
- Sensor cost stays under 2ms per enemy per tick.
DEBUG REQUIREMENT
A side-by-side view: true map left, selected enemy's belief
right, with the frontier target marked. This is not optional —
every bug in this stage is invisible without it.
Treating unknown as free. It's the natural reading of a zero-initialised array, and it silently deletes the entire feature — an enemy that assumes unexplored space is walkable behaves identically to one that has the map. The three-state distinction has to survive every function that touches the grid.
Forgetting to clamp the log-odds. Everything looks perfect for two minutes, then the enemy stops responding to change, because a cell it stared at for a thousand ticks now has a confidence no amount of contradicting evidence can move. Bayesian updating with unbounded certainty is just stubbornness.
An enemy that gets lost, searches the wrong room, and eventually finds you — and none of that behaviour is scripted.
Let a model drive
2026 properStage 10 established that cast() is a sensor. Follow that one more step: the sensor's output is a short array of floats, and a short array of floats is something you can hand to a language model as a sentence.
So don't give the model the screen, and don't give it the map. Give it twelve rays and what each one hit. That's a genuine egocentric depth reading — precisely what a robot gets — and nothing more.
The design decision that makes this work rather than embarrassing itself is that the model issues plans, not keystrokes. A round trip takes a second or two; a shooter runs at sixty frames. Asking a model for a keypress per frame is a category error. Asking for follow the left wall until an opening and executing that locally for the next two seconds sidesteps the latency entirely instead of pretending it isn't there. That's also how you'd build an embodied agent for real.
Record it once. Ship the recording.
The obvious way to publish this is a live agent anyone can poke, and that's the wrong way. Every decision is a paid API call, so an idle browser tab left open overnight is somebody playing a game on your card. It's also fragile — network hiccups, rate limits, malformed output — and all of that fails in public.
So make the default record and replay. Run the session yourself, dump every observation, plan and reason string to a JSON file, and have the page play it back at whatever pace reads well. Costs nothing at runtime, works offline, deterministic, and unbreakable.
That's the better artifact regardless of cost. What's interesting here was never that a stranger gets to drive — it's the agent's reasoning as it works out a floor plan from twelve distance readings. A recording shows that more clearly than a live run with latency gaps in it, and you can scrub back to the moment it got something wrong. The transcript is the content, the same way the rejection counts are.
My honest prediction: it's a poor shooter and a surprisingly good navigator. Publish that either way.
GOAL
An agent-controlled player driven by ray data alone, issuing
short plans that the game loop executes locally. Ships as a
recorded session, not a live one.
THREE MODES — build them in this order
1. local left-wall-follower, no network. The baseline, and
the fallback. Must complete a level on its own.
2. replay plays a recorded session from JSON. THE DEFAULT
for anything published. No network, no cost.
3. live calls the model. Opt-in only, never the default.
REQUIREMENTS — observation
- 12 rays evenly across the FOV, each reported as distance (2dp)
and what it hit. Plus health, ammo, and the last plan's
outcome. Serialise compactly:
rays: [1.20 wall, 3.40 wall, 8.10 open, 2.05 enemy, ...]
health: 80 ammo: 14 last: "advance 4" -> "blocked after 2"
- NEVER include player x/y, the map, or enemy positions. The
agent gets what the sensor sees. If it can infer position from
ray patterns over time, that is the interesting part.
REQUIREMENTS — plans
- The model returns one plan as JSON, nothing else:
{ "action": "advance"|"turn"|"strafe"|"fire"|"wait",
"amount": number, "reason": string }
Strip markdown fences before parsing. Reject and retry once on
malformed output, then fall back to local.
- The game loop executes the plan over up to 2 seconds, then
reports back what actually happened, including early
termination ("blocked after 2 of 4").
- Plans are interruptible: taking damage cancels the current
plan and forces a new observation immediately.
- Show the agent's reason string on screen while it executes.
The reasoning is the content here, more than the play.
REQUIREMENTS — recording
- Live mode writes a session log as it plays:
{ seed, level, startedAt, steps: [
{ t, observation, plan, outcome, pos, angle } ] }
pos and angle are recorded for playback only. They are NOT in
the observation the agent saw. Keep that boundary clean or the
replay stops being an honest record of what it knew.
- Replay mode reconstructs the run from the log alone: no
simulation, no re-deriving state. If the log cannot drive the
playback by itself, it is not a complete record.
- Playback controls: play/pause, scrub, step one decision at a
time, and adjustable speed. Scrubbing back to a bad decision
is the main thing a reader will want to do.
- Export/import the log as a .json file. Sessions are artifacts.
REQUIREMENTS — live mode guard rails
- Off by default. Requires an explicit user action to enter.
- Hard caps: max calls per session, max session duration, and a
token budget. Exceeding any one drops to local with a visible
notice, permanently for that session.
- Pause on tab blur. An unattended tab must not spend anything.
- If the deployment has no server-side key, live mode is
bring-your-own-key: the visitor supplies their own, held in
memory only, never persisted and never sent anywhere but the
API. If no key is present, hide the mode entirely.
NON-GOALS
No fine-tuning. No vision input. No agent-vs-agent play.
ACCEPTANCE
- The published page never makes a network call on load.
- Replay of a recorded session is frame-identical every time.
- Killing the network mid-live-session degrades to the
wall-follower without a stall or an exception.
- Malformed JSON never crashes the loop.
- A session log opened in a text editor is readable on its own:
a reader can follow what the agent saw and why it moved
without running the game.
- Instrumented per session: calls made, tokens used, wall clock,
and completion vs the local policy on the same level.
THE MEASUREMENT
Run the same level three ways: human, local wall-follower, agent.
Record time, deaths, and cost. The wall-follower will likely beat
the agent on the maze and lose on anything requiring a decision.
Publish the table either way.
The session log records the player's position and angle so playback can reconstruct the run — but those values were never in the observation the agent received. Keep that separation absolutely clean. The moment position leaks from the log into the observation builder, the recording stops being an honest record of what the agent knew, and the whole demonstration is worthless. It's the one bug in this stage that wouldn't look like a bug.
A session you can scrub through, watching a model reason its way around a floor plan it has never seen — costing nothing to serve and impossible to break.
Write the cast loop twice
closing stageThe column loop is embarrassingly parallel — 320 independent rays, no shared state, no ordering. That's the textbook shape for a GPU compute shader, and porting it means writing the same twenty lines in WGSL and running the two side by side.
Here's why this closes the tutorial rather than opening it. At 320×200 the GPU version will probably be slower. Dispatch overhead and the buffer round-trip swamp the actual work, which is trivial. It only starts winning around 1920 columns, or once you add secondary rays for reflections, or take multiple samples per column.
Which is exactly the finding from the agent-workflow section further down: parallelism has a fixed entry cost, and below some threshold the serial version wins. Same lesson, two unrelated domains, one project. Measure it and put the crossover point in the article — that number is more useful than any claim about which approach is better.
GOAL
Port the per-column cast loop to a WebGPU compute shader and
measure the crossover point against the JS implementation.
REQUIREMENTS
- The WGSL kernel is a direct translation of the JS DDA. One
invocation per column. Do not restructure the algorithm; the
point is that it is the same code in two languages.
- Buffers: MAP as a storage buffer uploaded once, camera state
as a small uniform updated per frame, results written to a
storage buffer of { distance, tile, side, hitOffset }.
- Runtime switch between the JS and GPU paths. Both must produce
identical output.
- Feature-detect navigator.gpu. No WebGPU means the JS path,
silently, with no console noise.
ACCEPTANCE
- Column distances from both paths agree to within 1e-4 across
a full 360 degree sweep. Assert this, do not eyeball it.
- A benchmark harness that sweeps column counts from 160 to 3840
and reports median frame time for both paths.
- The report names the crossover column count, or states clearly
that JS won at every tested width.
WRITE-UP REQUIREMENT
Report the measured result, including if the GPU path lost. A
negative result with a crossover number is the finding.
Two implementations, one benchmark table, and a defensible number where the tradeoff flips.
Running this with an agent team
This project is unusually good for testing a multi-agent setup, because the phases split cleanly into ones that cannot parallelise and ones that genuinely can. Most toy projects don't, which is why most multi-agent demos prove nothing.
Phases one and two are one serial chain
Each stage's output is the next stage's input, and the interfaces are still moving. Splitting these across agents buys you nothing but handoff cost — every handoff is a fresh context load, and every rejection is a full extra round trip. Run these yourself, or single-agent. Hand them to four agents and three of them wait.
Phases three and four genuinely parallelise
Once cast() is frozen at the end of phase one and zBuffer is populated at the end of stage 3, texturing, sprites and combat touch almost nothing in common — three builders can work simultaneously against a stable interface. Phase four is looser still: the occupancy grid, the agent driver and the GPU port share only the sensor, and none of them share files.
That's the actual experiment. Same project, same team, a serial half and a parallel half, so the comparison controls for everything except the thing you're measuring.
The measurement worth publishing
Time each phase, wall clock, start to working code, single-agent and team. Record the rejection count too. Everybody selling agent workflows claims a speedup and shows the happy path. "I timed it, and here's the phase where it was slower" is a more useful finding and a more credible one.
Notice what the stage 2 spec does that the others don't: it names the return shape as a public interface and requires review before anyone moves on. That single constraint is what makes stages 6 through 8 parallelisable at all. Interface-first isn't ceremony here — it's the thing that decides whether the team can work at once or has to queue.
What you end up with
A first-person shooter, in a browser tab, served off a static host, with no engine underneath it and no line of rendering code you didn't write. It loads instantly, costs nothing to host, and runs on a phone.
And the next time you open Unreal, you'll know what Nanite is doing instead of just knowing that it's on.