GPT-2 small · 12 blocks · 768 sine voices

Residual instrument

AText

BCompare

Preparing actual captures

Coordinate 000 · 55 HzOutgoing residualLinear bank → safe output

Block 08 / 12

Play the addition r + a + m

rr + a + m

Loop routePulse 84

Shared trim +24 dBOutput 35%

Shape the instrument Pitch bank · coordinate order · listening window

Pitch bankCoordinate orderPermutation seedCoordinateListening windowPulse length

Coordinate order has no intrinsic pitch meaning. Every choice keeps one mapping and one scale across A, B and their additions.

Method and limits

A particular residual stream

These are actual CPU float32 captures of GPT-2 small: 768 coordinates in each of 12 blocks. Attention is the whole projected attention branch, MLP is the whole projected feed-forward branch. r + a and r + a + m are captured before the next normalization. The final layer normalization and vocabulary readout are outside the listening field.

All bundled captures satisfy both additions, the embedding input and continuity between blocks exactly. Repeated captures and a second initialization with seed 2026 are byte-identical. The seeded model uses the same architecture and tokenizer, with untrained weights. Its quieter signals are retained.

One linear bank

For a vector x, the signal is S Σ xⱼ sin(2πfⱼt + φⱼ). Amplitudes keep their signs. One scale S is shared by all texts, blocks, tokens and models; Shared trim multiplies every coordinate equally. The logarithmic bank assigns slots to unique rounded multiples of 0.5 Hz from 55 to 7,040 Hz. The dense bank uses 60 + 7.5k Hz. Index, reverse and seeded permutation change the assignment, not the captured values. Phases are fixed by seed 314159.

A 65,536-sample periodic table represents the sine sum; playback linearly interpolates adjacent samples. Linear transitions take 80 ms, with a common phase clock. A fixed tanh safety stage follows synthesis, then conservative output gain. This final nonlinear stage can alter audible additions; identities are checked before it. “Soft limit” indicates substantial shaping. Audio never normalizes one branch or frame separately.

Play with the terms

Tap or drag a ring to select a block. Use the word fragments to select a token position. Solo r, a or m; scrub the addition; switch A, B and B − A. B compares the same token index, which can name different text fragments. Use the coordinate window to hear one feature or a small group. The eight score steps are editable. Loop Addition, Blocks or Tokens; Pulse is a performance tempo, not an inference timer. Space starts or stops, M mutes, L toggles the loop. Keyboard shortcuts leave form controls alone.

Marks show signed coordinates, ordered by pitch slot. Ink uses an asinh contrast curve so smaller coordinates remain visible. The center shows the current linear waveform; its sweep follows the chosen performance clock only after Start.

Text and provenance

Bundled phrases work offline. Add text uses the accompanying local inference server when available, or imports a verified trace file. No hosted inference, microphone or analytics is used.

Implementation follows the residual structure in the original GPT-2 model and the published GPT-2 weights. Feature-to-pitch choices are artistic choices. Listening does not establish a model’s meaning, understanding or experience, or predict human enjoyment.