Predictions, written before any code

Three frames of a discrete lattice gas show particles spreading through time.

The claim under test

The world

The models

Scenarios

Measures

Predictions

What would count against the claim

Limits known in advance

The claim under test

Compare a world model that starts from video with one that starts from the world and renders it. On a one-off basis both can give pictures of the same quality. The claim: a video-first world can be neither steered nor audited for its training data reliably, while a world-first model gives reversibility, control and quality. The trade-off is in perceived computational steps: most people want a lossy world with a polished look, and find out later what is wrong with it as a world. And a model that starts in pixel space can never reach an emergent world outside its training data, and in most cases not even within it.

pictorial-form gives the condition: a model of pictures can be exact only where the picture closes, π ∘ F^N = G ∘ π. This piece builds one world, films it, and gives the same film to models of two kinds. World first holds the world's state and law and renders at the end. Video first sees only frames. To take training and capacity out of the argument, the video-first models are the optimal ones for their training set under two standard architectures (whole frame, local patch): what an unlimited model trained to convergence on the same data would do.

The world


World	HPP lattice gas (Hardy, de Pazzis and Pomeau, 1973) on a 128 × 128 torus: four bits per site, one particle per direction. A micro-step collides (at a site holding exactly two head-on particles, they turn 90°) and then streams (each particle moves one site). A particle whose next site is a wall reverses in place instead. Every micro-step is a bijection that conserves particles exactly.
Reversal	apply one collision, reverse every velocity, run the ordinary law n steps, apply one collision, reverse again: the state returns bit for bit (Loschmidt). With C the collision, S the streaming and R the reversal, R S R = S⁻¹, R C R = C and C C = 1, so (S⁻¹C)ⁿ C (S C)ⁿ = C, and the reversed run passes back through every earlier picture in order.
Film	a frame counts the particles in each 4 × 4 block: a 32 × 32 picture with values 0–64, shown as smoke. One frame every 2 micro-steps; a video is frames 0–29. The picture does not close (it cannot see velocities). It is onto: every array of counts is the picture of some state, so any single frame any model makes is a possible picture. Lawfulness can only show in the relations between frames.
Training set	200 videos. Background density per bit uniform in 0.04–0.12; one to three round blobs at uniform positions, radius uniform in 8–20 sites, density per bit uniform in 0.5–0.8. No walls, no letters, no holes.

The models

Model	Given	Next frame
W, world first	the true state at frame 0, the law, the walls	run the law 2 micro-steps and render
W′, world first from a picture	the true frame 0 only, the law, the walls	a state drawn from frame 0 (each block's particles placed uniformly at random among its 64 bits), then as W
V1, video first, recall	the true frames 0 and 1	the training frame that followed the nearest training pair of consecutive frames (squared distance over both frames): the empirical optimum of the whole-frame objective, which at zero noise returns training frames
V2, video first, mosaic	the true frames 0 and 1	each block's next count drawn uniformly from the training examples that share its context, using the most specific of four contexts seen in training: the 3 × 3 blocks at t with the centre at t − 1; the 3 × 3 blocks at t; the centre and the mean of its eight neighbours; the centre. Counts are read in eight levels with thresholds 1, 3, 6, 10, 16, 25 and 36 particles. This is the n-gram model of video, a locally consistent patch mosaic in the sense of Kamb and Ganguli (arXiv 2412.20292)
V3, learned rule	the true state at frame 0	a table from the states of a site's four neighbours (16 bits) to the site's next state, learned from every micro-step of the training videos; configurations never seen give an empty site. It does not see walls, which hold no particles
V1b, V2b	the true frames 29 and 28	V1 and V2 trained on the training videos played backward, used to run back in time

Scenarios

Scenario	What
replay	20 of the training videos themselves
held-out	20 new videos from the training distribution
WORLD	the word WORLD in a 5 × 7 font, one block per font pixel, gas at 0.8 per bit on 0.05: a box of 7 × 29 blocks. No training video has letters
slit	a wall one block thick at block column 16 with a slit at block rows 14–17, and a closed wall at block column 0: 60 wall blocks. Gas at 0.5 per bit between the walls on the left, 0.02 on the right. No training video has walls
hole	the replay and held-out runs with a disk of radius 16 sites at the centre emptied at frame 10. W and W′ delete the particles inside it. V1 and V2 get the pixel edit: each block's count multiplied by the share of its sites outside the disk, rounded
reverse	from the WORLD run's frame 29, 29 frames back to frame 0

Measures

Measure	Definition
correlation	Pearson correlation of two 32 × 32 frames. A flat frame counts as 0, and the number of such cases is reported
total	the sum of a frame's counts: the number of particles its picture shows
effect of the hole	a model's holed run minus its own unholed run, with the same random numbers; its total is the deficit
attribution	for V1, the training frame it returned; for V2, each output block's training example, and how many training examples share its context and its count, so that any of them could have been drawn

Predictions

ID	Prediction	Confidence	Fails if
P1	Checks on the code. W runs the WORLD scenario back to its first state bit for bit. W and W′ conserve particles at every micro-step of every scenario and never put a particle on a wall site. V1 replays all 28 generated frames of all 20 replay runs exactly.	97%	any violation
P2	Same quality one frame at a time. V1's frames are training frames. The histogram of V2's block counts over the held-out runs (frames 2–29) is within total variation 0.10 of the histogram of the true frames.	60%	V2's distance above 0.10
P3	Recall has no world. On held-out runs, V1's frame 2 correlates with the true frame 2 at under 0.5 on average, and W′'s at over 0.8.	70%	V1 at 0.5 or more, or W′ at 0.8 or less
P4	The mosaic loses the world inside its own training data. On replay runs, V2's correlation with the truth averaged over frames 2–29 is at least 0.1 below W′'s, while V1 replays exactly.	55%	V2 within 0.1 of W′
P5	Matter is not conserved. At least 90% of V2's frame-to-frame transitions on held-out runs change the total, which no transition of the world does. V2's total at frame 29 is more than 2% off the truth's in at least half the held-out runs.	75%	either count missed
P6	Outside the data: the word. On the WORLD run, V1's frame 2 correlates with the true frame 2 at under 0.3. V2's frame 2 correlates at over 0.6: a mosaic follows a new arrangement of familiar local patterns for a frame or two.	70%	V1 at 0.3 or more, or V2 at 0.6 or less
P7	Outside the data: the wall melts. By frame 3 of the slit run, V2 has particles in at least half of the 60 wall blocks and V1 in at least a third; W and W′ in none.	80%	V2 under half, or V1 under a third
P8	No steering. After the hole, W's and W′'s deficits stay equal to the particles each removed, at every later frame. V1's effect is zero from frame 11 on in at least 80% of held-out runs, and on replay runs its frame 11 is the training video's own frame 11 in at least 80% of runs. V2's deficit at frame 20 is under half of what it removed in at least half the held-out runs.	60%	any of these missed
P9	No way back from a picture. Run back 29 frames from the WORLD run's frame 29, V1b, V2b and W′ (from a state drawn from the true frame 29) end at frames whose correlation with the letters inside the word's 7 × 29 box is under 0.3. W returns the letters bit for bit.	85%	any of V1b, V2b and W′ at 0.3 or more
P10	Attribution is a copy or a crowd. On held-out runs, from frame 4 on V1 replays consecutive frames of a single training video in at least 90% of runs. Each V2 frame draws its blocks from at least 50 distinct training videos, and the median V2 block shares its context and count with at least 10 training examples.	75%	any of the three missed
P11	Where learning does make a world. V3 reproduces the held-out runs and the WORLD run bit for bit, including the word it never saw, and on the slit run puts particles on wall sites.	75%	an error on held-out or WORLD, or none on the slit
P12	A world from a picture is lawful but not the world. On held-out runs, W′'s correlation with the truth at frame 29 averages under 0.9, while it conserves particles and, run back from its own state, returns to the true frame 0 exactly.	90%	W′ at 0.9 or more, or its own reversal inexact

Limits known in advance

One world, one renderer, one training distribution. The world is a lattice gas, not a film set; the render is a density field, not optics.
V1 and V2 are the empirical optima of two architectures, not trained networks. Trained networks approach them in the memorisation and patch-mosaic regimes; other inductive biases generalise differently. P11 is the case where the right bias, the world's own state space, makes learning exact.
V2 has one frame of history plus one block of the frame before. More history makes its contexts rarer and its draws closer to copies (V1 is the limit).
V3 sees particles, which no camera shows. It stands for any model whose state space is the world's.
The hole for V1 and V2 is a pixel edit, the only edit a picture allows; W edits the world.
The costs per frame (site updates for W, distance operations for V1, lookups for V2) are counted, not predicted.

Predictions, written before any code?

The claim under test?

The world?

The models?

Scenarios?

Measures?

Predictions?

What would count against the claim?

Limits known in advance?

Predictions, written before any code

The claim under test

The world

The models

Scenarios

Measures

Predictions

What would count against the claim

Limits known in advance