Predictions, written before any model code

A tiled floor in perspective holds a column, a screen and coloured balls with shadows beneath a lamp.

The question

The world

The models

Scenarios

Measures

Predictions

What would count against the claim

Limits known in advance

Added later: the Mathematics mode

Stage

floor 6 × 6 m of 1 m tiles, walls at left, right and back, a pit in front. A column (radius 0.5 m, height 2.2 m) at (−1.35, 3.7) and a screen (2.1 m wide, 2.0 m tall) at z = 2.4 on the right.

Actors

two or three balls of radius 0.5 m, red, blue or yellow, rolling at 0.10–0.22 m per frame. They bounce elastically off the walls, the column, the screen and each other; collision times are solved exactly (event-driven).

Lamp

a point lamp at (−2.2, 4.6, 0.9), up and to the left. It gives surfaces their shading (form) and casts shadows.

Camera

a pinhole at (0, 3.3, −2.9): a 64 × 48 picture, each pixel the mean of 2 × 2 rays, in sRGB. The screen hides balls behind it; the column hides some.

Checks already run

1,600 frames rendered with the ball-and-shadow shortcut equal the full render pixel for pixel; one film's energy changes by 4 × 10⁻¹⁶ and its reversal returns the balls to within 10⁻¹⁵ m.

Training

200 films of 30 frames, lamp on the left, the set unchanged.

Model

Given

Next frame

world (W)

the state, the laws, the set and the lamp

run the laws one frame and render

world from a picture (W′)

frames 0–3, the laws, the set and the lamp

find each ball by its colour and fit its floor position by drawing candidate balls until the drawing covers the same pixels (analysis by synthesis); fit a straight line through its positions for its velocity; then as the world. A ball unseen in frame 3 does not exist for it.

recall (V1)

frames 0–3

the training frame that followed the nearest training pair of consecutive frames (exact squared distance over all pixels)

mosaic (V2)

frames 0–3

frames are cut into 4 × 4 tiles, and each tile is named by the nearest of 64 codewords (k-means over the training tiles' 2 × 2 colour averages). Each next tile is drawn at random from the training tiles that followed the same named context: the tile, its four neighbours and the tile one frame earlier; backing off to the tile and its neighbours, then the tile now and before, then the tile alone. The drawn training tile's own pixels are pasted. Contexts do not include position.

diffusion (V4)

frames 0–3

for each tile position, the 32 training transitions at that position whose context (the 3 × 3 tiles around it at 2 × 2 resolution and the tile one frame earlier) is nearest. A prior weight exp(−(d − d_min)/s), with s the median of d − d_min over the 32 plus 1, then 24 steps of denoising from pure noise under z = (1 − t) x + t ε with the optimal denoiser for those 32 tiles (the locally optimal denoiser with the tile as window, times the prior). The last step's tile is the output.

Measure

Definition

ball seen

at least 6 pixels of that ball's colour (saturation over 0.45, hue within its band)

tracked

the world's frame shows the ball and the model's frame shows it with its centroid within 3 pixels of the world's

return

in a hide film, the model shows the hidden ball within 3 pixels of the world's centroid in one of the 3 frames from the world's first frame with the ball back in view

shadow overlap

the pixels of the floor that a frame shows at under 72% of the empty stage's brightness and not as a ball colour, compared between model and world as intersection over union, pooled over frames 4–29

green kept

of the pixels where the world's frame shows green, the share the model's frame also shows green, pooled over frames 4–29

column painted back

in no-column films, the share of the column's former pixels that the model paints closer to the stage with the column than to the stage without it, pooled over frames 4–29, ignoring pixels the model shows as a ball colour

launch

for each ball–ball collision the world shows at frame f (4 ≤ f ≤ 25), and each of the two balls the world shows at f and f + 4: the model's displacement from f to f + 4 lies within 45° of the world's and is at least half as long

sources

recall: training films used by a run; mosaic: distinct training films per frame; diffusion: effective number of training tiles per tile, 1 / Σ w², at the first and last denoising steps

Prediction

Confidence

Fails if

Checks. In every held-out film the world keeps its energy to 10⁻¹² and, turned at frame 29, returns every ball to within 10⁻⁶ m (a pixel spans about 0.1 m of floor; rounding errors grow at every bounce off a curved surface). Recall replays 20 of 20 training films exactly.

97%

any violation

Same quality at first. At frame 4 of the held-out films, the mosaic and the diffusion model each track at least 80% of the balls the world shows; recall under 40%.

60%

mosaic or diffusion under 80%, or recall at 40% or more

Not even within the training data. On replay films, the mosaic and the diffusion model each track under half of the ball-frames over frames 4–29, while recall tracks all of them.

60%

mosaic or diffusion at half or more

Out of sight, out of existence. In the hide films the world brings the hidden ball back in every film (a check). The world from a picture brings it back in at least half. Recall, the mosaic and the diffusion model each bring it back in at most a third.

70%

the world from a picture under half, or any video model over a third

The lamp moves; the shadows do not. With the lamp on the right, the shadow overlap with the world is under 0.3 for each of recall, the mosaic and the diffusion model, and over 0.5 for the world from a picture.

70%

any video model at 0.3 or more, or the world from a picture at 0.5 or less

A colour it never saw. Recall, the mosaic and the diffusion model keep under 5% of the green ball's pixels green; the world from a picture keeps at least 50%.

85%

any video model at 5% or more, or the world from a picture under 50%

The set remembered. With the column taken away, recall and the diffusion model paint back over half of the column's former pixels; the mosaic under a fifth; the world from a picture none.

65%

recall or diffusion at half or less, or the mosaic at a fifth or more

Cause and effect. After ball–ball collisions in held-out films, the mosaic and the diffusion model each launch under 30% of the balls as the world does; the world from a picture over 60%.

55%

mosaic or diffusion at 30% or more, or the world from a picture at 60% or less

Receipts. Recall copies a single training film in at least 18 of the 20 held-out runs. Every mosaic frame draws on at least 50 training films. The diffusion model's median effective number of source tiles is at least 8 at its first step and at most 1.5 at its last.

75%

any of the three missed

Prediction

Confidence

Fails if

The pixel, worked by hand. Every pixel of 40 frames (10 new films, 4 frames each, including films with the lamp on the right and without the column) is recomputed ray by ray by the chalkboard's formulas: linear equations for the floor and walls, quadratics for the balls and the column, a dot product for the lamp and gamma at the end. The result equals the renderer's bytes in all three channels, and the renderer's record of what each pixel shows.

90%

any byte or label differs

Spheres are drawn as ellipses. For every ball in every frame of the 200 archive films, the outline is a conic whose discriminant B² − 4AC is negative (an ellipse). The largest ratio of long axis to short axis is between 1.10 and 1.25, and the median is between 1.01 and 1.06. The ellipse's centre is always farther from the picture's centre than the projection of the ball's centre, by less than half a pixel.

70%

any outline not an ellipse, a ratio outside its range, or a centre displaced inward or by half a pixel or more

The denoiser, worked by hand. For every tile of 5 frames of the diffusion model, the weights recomputed from the recorded noisy tile, the 32 candidate tiles and their priors (a bell curve times a prior, normalised) equal the model's own weights to within 10⁻¹² at all 24 steps. Each Euler step z + h(ε̂ − x̂) reproduces the next recorded noisy tile to within 10⁻¹².

90%

any weight or step outside the tolerance

Scenario	What
replay	20 of the training films
held-out	20 new films from the training distribution
hide	12 films in which a ball visible in frames 0–3 rolls behind the screen, is hidden for at least 4 frames and comes back into view by frame 27, all checked on the world's own film
lamp	20 new films with the lamp moved to the right, at (2.2, 4.6, 0.9); no training film has it there
green	20 new films in which one ball is green; no training film has a green ball
no column	20 new films with the column taken away; every training film has it

Predictions, written before any model code?

The question?

The world?

The models?

Scenarios?

Measures?

Predictions?

What would count against the claim?

Limits known in advance?

Added later: the Mathematics mode?

Predictions, written before any model code

The question

The world

The models

Scenarios

Measures

Predictions

What would count against the claim

Limits known in advance

Added later: the Mathematics mode