GitHub - mottopanikeiku/doomformer: Neural networks computed by DOOM's game mechanics: voodoo dolls, conveyor belts, lava and doors. Every value checked bit-exact.

Neural networks that run on DOOM's game mechanics: voodoo dolls, conveyor belts, lava and doors.

Watch the films · Try it in Colab: draw a digit and DOOM reads it, nothing to install A number is a crowd of voodoo dolls on a conveyor belt. An activation function is a staircase they have to climb. A matrix multiply is dolls running across lava lines, each line raising a room's light level. Attention is dolls squeezing under ceilings that another number has lowered. When the dolls stop, the light levels are read back out of the running game and compared bit for bit with an independent integer reference. One value off by one and the run fails. The marine's brain is compiled into the level he is fighting in. The dots on the right are 1,345 voodoo dolls of him. They run his transformer and choose his next move; the level senses the demons and pulls the trigger.

Watch the film.

Walk a digit, DOOM reads it The input is a 28×28 floor in the level. Every tile the marine walks on while holding FIRE inks itself, and the level writes that pixel into the brain.

He presses READ.

1,355 voodoo dolls run an MNIST network, compare all 45 pairs of digit scores and count the wins. Only the winner's doll gets through, and it lights that digit on the wall. Python advances the clock, then reads back the 784 pixels, the hidden layer, every comparison and the answer, and checks all 248 values against NumPy. doomformer draw # opens a window: arrow keys to walk, hold Ctrl to draw, Space on the READ switch doomformer draw --digit 7 # headless: the marine walks a 7 (Python presses his keys in place of a player) ▄▄▄▄▄▄▄▄▄▄▄▄▄▄ ▄████████████████ ████▀▀ ▀████▀ ▀▀▀ ▄███▀ ████ ███ ███ ███ ████ ███▀ ████ ▀███ DOOM reads: 7 (right) -- the winner's doll lit it on the 7-segment display EXACT: all 248 values DOOM computed (hidden layer, 45 score comparisons and 10 win counts, each with its twin, and the one-hot answer) equal the NumPy integer reference computed from the 784 pixel ports DOOM holds (tolerance 0); one display firing pixels: the 784 ports equal the level's canvas (the tiles the pen inked) READ pass: 5,705 game tics = 163 s of real-time DOOM at 35 tics/s, 79 s wall | Python sent 58 commands, 29 readout scripts each followed by the ack script it waits for (0 waited more than one advance), and nothing else (0 rejected) Every digit 0 to 9 in three walked variants: 30 read correctly, all 30 exact. In a window DOOM runs in real time, so a READ takes 2¾ minutes; headless it takes 76 to 99 s. How it works · film Choose your skill level pip install git+https://github.com/mottopanikeiku/doomformer # Python 3.11+, Linux or macOS The first run builds DOOM's script compiler, so you need git, cmake and a C compiler. Or skip all of it: the Colab notebook installs it, lets you draw a digit for DOOM to read, and runs the softmax demo. Skill Command What DOOM does Time I'm too young to die doomformer hello runs a tiny neural network 3 s Hey, not too rough doomformer mnist reads handwritten digits ~2 min Hurt me plenty doomformer demo invents a DOOM level name 20 s Ultra-Violence doomformer ask "what does iddqd do" answers your question 7-11 min Nightmare! doomformer demo --nightmare 106 maps, 56,390 voodoo dolls 2-4 min Times are from a 16-core Linux machine running 8 DOOM games at once.

I'm too young to die: what you'll see tiny integer MLP 4-6-5-3 compiled into one DOOM map: 60 voodoo dolls, 1222 sectors input 1/5 x=[-71, -82, 64, 37] -> y=[11, 4, -15] EXACT (3 outputs + 22 hidden readouts equal the integer reference; 488 game tics) ...

5/5 EXACT Hey, not too rough: DOOM reads handwritten digits 784 pixels → 64 → 45 comparisons → 10 win counts → the answer, all in one DOOM map: 1,355 voodoo dolls and 158,550 sectors. Each ink pixel releases a doll that runs across the hidden layer's weight lines. Dolls then compare every pair of digit scores, and only the digit that wins all nine of its comparisons gets a doll through. DOOM picks the digit and Python checks it. In DOOM the network read 99 of the first 100 MNIST test digits correctly and 966 of 1,000 sampled from all 10,000, every one of them exact.

On all 10,000 the same integer network gets 97.16 %.

Hurt me plenty: a transformer names a level A transformer trained on 136 official DOOM level names, split into 38 small maps that a pool of DOOM games runs.

Python copies the integers between games and picks each letter from the logits DOOM computed.

--one-map puts a whole transformer into a single map (embedding, attention, MLP and logits with no Python between layers), at about 90 s per letter. --softmax runs the same model trained with softmax attention. The dolls compute the exponentials (a staircase whose steps are spaced like a logarithm) and divide by their sum at run time (a count of comparisons).

How that works is in docs/HOW_IT_WORKS.md.

--model doomnames_modern adds RMSNorm and a gated (ReGLU) MLP to the softmax attention.

The dolls also compute the sums of squares (as sums of odd numbers), the integer square root (a staircase at d·k²) and the division by it. It wrote MILL in 170 tile runs, all 62,000 values exact. It writes on its own.

--one-map --loop builds one map that writes a whole name with no Python in the loop. At every position the dolls compare all 30 logits, and the winner's doll moves the next position's token input to its letter. The level then rewinds and runs the next pass by itself, five passes for a 6-token context. Python only advances the clock and reads: every value of every pass is checked, and the final text has to equal greedy generation.

It wrote DAN (5,875 dolls, about 3 minutes per pass), and DAGE with --softmax.

DOOM wrote: 'DEMON CANYON' 1,690 tile runs in 8 parallel DOOM games | 56,390 voodoo dolls in the tile maps | 120,666 values read back and compared with the NumPy integer reference (tolerance 0) | 135 s EXACT: every value DOOM exposed equals the integer reference, and the reference writes the same name Close the doors in front of one neuron (L1.m unit 33) and the same brain writes DEMON DEATH.

How it works A matrix multiply.

Each doll is one unit of an activation, and each lava line it crosses adds a weight to a room's light level.

Attention.

Dolls squeeze under ceiling gates that another live value has lowered; counting who gets through is a multiply.

Ablation is a door.

Shoot the switch, the door slams, and one neuron's dolls never arrive.

DEMON CANYON, a level built from the name the DOOM brain wrote, played by a marine whose brain also runs on DOOM.