Multiprocessors on the Grid VM: four scenarios

Four concrete ways to put more than one control unit on the 1 µm byte grid — a message-passing multicore, a SIMT lane array, a systolic coprocessor, and a static SPMD replica machine — judged for physical realism, scorability and expressivity, with animated examples of a parallel reduction and a parallel matrix multiply. Companion to The Grid VM: competition design. September 7, 2026.

1. The brief, and how this note was produced

The single-core machine is settled: a 2-D grid of byte cells at 1 µm pitch, one sequential control unit at the origin, 1 fJ for every byte moved one micron, signals at c/160, and three serial tapes carrying input, output and instructions. The design report shows what that machine leaves on the table: its sequential-time leaderboard measures tier-3 MNIST training in days, and the I/O-order note already remarked that nothing in the model rewards parallelism. The question here is how to add more than one control unit realistically — anchored to what silicon actually does and free of free lunches — while keeping the grid model: cells, distances, tapes, and cost as a static property of the submitted schedule.

2. The settled single-core machine

Every scenario below must reduce to this machine at P = 1. The constants are the ones the competition design fixed; the multiprocessor extensions add to them and never change them.

Two consequences of these constants drive everything that follows. First, a working set of W bytes costs about √W per access, so the energy of a sweep is ≈ T·√W: distributing W over P cores cuts the mean distance by √P. Second, at 1 B/ns a tape delivers 2²⁰ bytes in 1.05 ms whatever the core count, while one core consumes them at 2 ns per byte (recv + add): a second core doubles throughput and a third does nothing. Both facts are physics, not artefacts, but the second one hides an unstated calibration (one byte per nanosecond is a single wire at 1 GHz, not a memory interface), and §4 says what to do about it.

3. What the sources say

Dally: a parallel model with a distance function

Dally's CACM point [1] is usually read as an argument about single-processor cost, but the model it proposes is explicitly parallel. His PECM (parallel explicit communication model) is PRAM with two changes: arithmetic stays unit cost, and “each processing element and each memory is assigned a location l ∈ L and a distance function D: L×L ⇒ R is defined to determine the cost of sending a message or making a memory access between two locations,” with 2-D Manhattan distance for on-chip communication and extra distance when a coordinate crosses a chip boundary. Multiple PEs at locations are native to it; the settled Grid VM is its P = 1 case with the distance function made literal.

His numbers, verbatim: a 32-bit add is 20 fJ and 150 ps; moving its two operands 1 mm is 1.9 pJ and 400 ps; 40 mm corner to corner on a 400 mm² chip is 77 pJ and 16 ns; going off chip is 320 pJ per 64 bits with 6 ns per metre. On-chip SRAM is built from 8 KB sub-arrays: 64 b from a sub-array costs 0.64 pJ and 300 ps; the same 64 b from a 256 MB memory (about 100 mm²) costs 58 pJ and 12.3 ns, of which 57.4 pJ and 12 ns are communication, 15 mm each way. And the worked multiprocessor example that §11 animates: summing a table held in the on-chip memory of a 4×4 array of 256-core chips, 16 mm on a side, each core at the centre of a 2 MB SRAM with a 1 mm local round trip (1.9 pJ, 400 ps per 64 b access). A random access costs 21.3 mm on chip for the 1/16 of accesses that stay local and an additional 640 pJ + 21.3 mm for the 15/16 that leave, 680 pJ on average; letting each core sum its own 256K words and forward the result up a tree is “354×” cheaper (230× elsewhere in the same article) — and PRAM cannot tell the two programs apart.

Roune: the AI CPU

Bjarke Roune's design document [2] argues from the other end — from what accelerators should look like — and lands on “traditional-ish CPUs with big caches and systolic arrays, where the caches and systolic arrays take up most of the chip area and power.” The facts that bear on a grid model: doubling a systolic array's vector width quadruples its work per cycle while less than doubling the cost of the core that drives it, so the controlling scalar and vector logic shrinks relative to the array (“4⁵ = 1024×” in scalar work from Volta's 4×4 to TPUv3's 128×128); after configuration a systolic array fetches no instructions, which is where its energy advantage over a vector processor comes from; the reduction dimension K is the hard utilisation limit (attention at 16–128, feed-forward at 8,192), so mono-sized arrays are unbalanced; data should move SRAM → network → SRAM without a detour through a central store; a torus with neighbour-only traffic is 100% link-efficient while any router caps at 50%; and all-reduce traffic per chip is 2X(1 − 1/P), so within-layer parallelism more than doubles per-chip bandwidth when P doubles. He is explicit that there are few numbers in the document, and that a 2-D plane is enough to understand most of it.

The calibration thread

A Codex thread on matmul energies [3] turned Dally's SRAM figures into a hierarchy the grid can represent: 256 MiB is 32,768 sub-arrays of 8 KiB; at 100 mm² each sub-array gets a 55.2 µm square-equivalent pitch and each 64-bit word site a 1.73 µm pitch (0.61 µm per byte); an 8 KiB sub-array laid out as 32 half-diamond shells has a mean one-way distance of 37.7 µm; Dally's 15 mm needs a routing factor of about 2.24 over the 6.7 mm mean of a flat 100 mm² layout; and at the area-derived byte pitch and the AHA keynote's 100 fJ per bit-mm, a byte-wide round trip over one grid step costs 0.98 fJ — the physical reading of “1 fJ per score step.” Effective speeds: c/120 for global wires, c/1200 for a complete sub-array access. One slip in the thread's descendants is corrected here: 320 pJ per 64 b is 5 pJ per bit, 40,000 fJ per byte, a board-level crossing; the 5,000 fJ per byte the competition uses for E_in is a chiplet- or HBM-class crossing (0.6 pJ/bit), and the two must not be cited for each other.

4. Three axes, and twenty rules applied to all four scenarios

Read together, the six designs and eighteen reviews say that multiprocessor extensions of the grid differ on exactly three axes, and that everything else is a constant to be calibrated once:

1. Instruction delivery. MIMD cores each fetch their own program (Dally's 80 pJ “add instruction” on a CPU is this overhead); SIMT lanes share one fetch across a broadcast tree whose wire length grows with P; a systolic array fetches nothing after a priced configuration. A grid model that broadcasts instructions for free erases the axis that separates GPUs from TPUs from multicores.
2. Memory ownership and geometry. One shared plane read from anywhere at the distance from the reader (PECM literal), or private regions around each head with explicit messages (Dally's 256-core chip, Roune's SRAM-to-SRAM DMAs), or a hierarchy of tiles, dies and packages with a crossing constant at each boundary. Where the head sits and how large its footprint is decides the minimum distance a byte ever travels.
3. Communication pricing. Byte-microns of wire (the settled rule) plus, once routers exist, a per-hop constant and a link rate; whether idle silicon costs anything; and whether makespan is a closed form or a simulation.

Twenty problems recurred across designs. Each is resolved once here and the resolution is applied to every scenario, so their numbers are comparable with each other and with the single-core leaderboard. The load-bearing ones:

5. Scenario 1: the SPMD tile mesh with a collective library

In one paragraph. Add one header line, replicate P, to a settled schedule. The judge stamps P copies of the settled machine — head, three port cells, private half-diamond — each with a 128×128-cell core block under its head, on a P_x×P_y lattice at a derived pitch. All tiles run the same text with static lane_x, lane_y; bytes cross a tile boundary only inside a fixed library of scoped collectives whose energy and time are closed forms; the three tapes attach to tile (0,0) at 8 B/ns and distribute by a static stripe library. It is Dally's 256-core chip programmed the way GSPMD programs a TPU pod, priced by the PECM lattice plus a real network's constants. Adopt it first.

   y ^      tile (0,1)                          tile (1,1)
     |   .  .  .  ◊  .  .  .                 .  .  .  ◊  .  .  .      diamond: k shells, lin(a)
     |   .  .  ◊  ◊  ◊  .  .                 .  .  ◊  ◊  ◊  .  .      reads cost ⌈√a⌉ fJ/byte
     |   .  ◊  ◊  ◊  ◊  ◊  .                 .  ◊  ◊  ◊  ◊  ◊  .
     |   I [H] O ═══════ 8 B/ns link, S µm ═══════ I [H] O            head row, ports at ±1
     |   +----------------+                  +----------------+
     |   | 128x128 block  |                  | 128x128 block  |       sequencer, 2 KiB I-buf,
     |   | seq ibuf ALU   |                  | seq ibuf ALU   |       ALU, router, 4x1 KiB NI
     |   | router  NI     |                  | router  NI     |       (no data cells; clock
     |   +------║---------+                  +------║---------+        tree 1,344 µm, 168 fJ/ns)
     |          ║ vertical link, S µm               ║
     |   .  .  .  ◊  .  .  .                 .  .  .  ◊  .  .  .
 in ═══► I [H] O ═════════════════════════════════════ I [H] O        tile (0,0): the settled
 out ◄══ tile (0,0)    (0,-1) ◄═══ instr tape           tile (1,0)     machine + block; 3 tapes
     |   +----------------+                  +----------------+
     +----|<----- S ----->|-------------------------------------------> x

Geometry and constants

Instruction delivery, tapes, memory

The text (S_text canonical bytes, up to 2 KiB before the block grows) crosses the instruction tape once and is multicast over the XY spanning tree: E_text = 4·L_static + S_text·(P − 1)·(S + 105) fJ, arriving after S_text/8 + (P_x + P_y − 2)(S/v + 1) ns. Each tile then issues from its own buffer at the settled 4 fJ per static line and 1 fJ per issue; the physical reading is a per-core loop buffer with a location, filled once from a shared instruction memory over the network. A reported, unscored column E_ctl = 1,000 fJ per issued instruction records what a microcontroller-class sequencer would actually spend.

Input is cut into segments by a declared stripe — cyclic (element kP+l to lane l), block n, block2d(bx,by), replicate (one E_in, then a priced multicast with charged landings) — and output is the mirror under credit pull: the pad's 1 KiB buffer is credit for the owners of the next segments in stream order, a send blocks until its tile holds credit, and the pad drains a segment only after it has arrived. Rule R9 makes this decidable: cyclic is safe by construction; block stripes are accepted only when no barrier separates two tape operations of the same block or the block fits the FIFO, and their phase time is a per-segment recurrence (push, arrival, receive-start, receive-end) that the judge evaluates in O(number of segments) with periodic extrapolation for long streams. Memory is private per tile at the settled lin(a) addresses; local writes stay free; the landing writes of collectives are charged, into a judge-allocated slot just above the program's own a_max.

Collectives

Every collective is a barrier among the lanes in scope, every combine order is fixed, and every round snapshots its sources before any landing write, so values form a Kahn network with one actor class: determinism is syntactic. Element-wise arithmetic inside a collective is issued (one issue per element on the combining lane) and movement runs at DMA rate; a collective is not a free vector unit.

Makespan, area, and the ISA delta

Makespan is MIMD-between-barriers: the prologue, plus for every barrier-to-barrier segment the maximum over lanes of that lane's settled time for the lines it executes, plus the collective rows, plus the tape phases of R9. Because lane coordinates are affine in addresses (rule R1′), the lane maximum sits at a corner of each guard class and is found by evaluation, not search. Modes are padded segment-wise: every arm is charged the mode's maximum issues and byte-microns between consecutive blocking operations, so the timeline is data-independent by construction. Nothing is simulated. Area is P·((2k−1)² + 16,384) µm² for P ≥ 2 and the settled (2k−1)² at P = 1.

Worked examples

(a1) The streamed 2²⁰-byte sum. stripe in cyclic, one recv.i32 and four adds per trip per lane, one reduce, one send. The body is 5.52 ns per trip, the tape 4P/8 ns, so the program is issue-bound to P = 8 and tape-bound from P = 16. Speed-ups are given against the settled 1 B/ns machine and against a same-tape single core (8 B/ns), because the first number mixes an eight-fold tape change with parallelism.

Energy rises 18% at P = 16 (12% distribution, 7% clock) for an 11× time gain, and then the tape is the wall: the streamed sum is where a parallel machine buys time only.

(a2) The resident sum, Dally's framing. Each lane holds N/P bytes, sums them, then one reduce. The load phase (a cyclic stripe) is reported separately and added for the end-to-end row, because Dally's example charges it.

Three readings. The √P locality dividend is real (mean read distance 683 µm at P = 1, 171 at P = 16) but the clock caps it near 3× at P = 256 and it falls again at 4,096, so the energy board's optimum is finite. Speed-ups above P are a v1 artefact the reader must know: the settled machine charges operand flight serially (386,177 of its 1,434,757 ns is unpipelined flight over a 683 µm mean path) and the √P shrinkage of that path is booked as speed-up; real SRAM pipelines it. And a once-swept table that had to be loaded over the tape is worse in energy at every P ≥ 4 — Dally's 354× needs a table produced in place, which is exactly what the MNIST tiers' resident weights are.

(b) The 64×64 int8 matmul. Tile (x,y) owns a block of C; stripe in block2d delivers its blocks of A and B; allgather.row and allgather.col give it its block-row of A and block-column of B; the settled 4×4 register-block kernel does the MACs; output leaves row-major under credit pull.

Time falls 15× and energy doubles. The clock is 42% of the 16-tile energy, and it is honest: P·T = 16 × 46 µs = 737 core-microseconds against 700 for the single core (95% parallel efficiency), and the single core pays no clock by rule. Like for like (the single core charged its 117.7 MfJ), the 16-tile machine costs 1.11× the energy for 15× the time. The energy board rewards P only where locality beats P·T × 168 fJ; the time board rewards it outright.

Which machine, scoring, exploits

Dally's 256-core chip with per-core SRAM on a mesh, programmed as GSPMD does with partition_id and all-reduce, all-gather and collective-permute; the SRAM-plus-network half of Roune's AI CPU without the array. One honest mismatch: Dally's 2 MB core does not fit 256 to a 16 mm die in this model (k = 1,449, S = 2,897 µm, five tiles per die side), because the byte pitch is 1 µm against his 0.61 and a half-diamond fills a quarter of its bounding square; 256 tiles per die are 243 KiB tiles. Scoring is the settled closed forms with two more affine dimensions plus a table row per collective and the R9 recurrence: about 2,600 new judge lines, 0.2–10 ms per submission, no simulator. Exploits closed: huge P for free work (the clock, the block, the text delivery); a replicate 2 shell with one idle lane, 1.45× faster than the settled streamed sum on the tape rate alone (it scores on the P ≥ 2 time board only); collectives as unissued vector units; FIFO deadlock through block stripes (R9); corner-rule under-estimation with mod addresses (R1′); modes as free MIMD (segment padding); port balance by mnemonic (bytes); shift as a free memset (rejected at P = 1); the table bomb; head placement (fixed lattice). It rewards small per-tile working sets, neighbour chains, large messages, 2-D decompositions (SUMMA's row and column scopes cost 0.4× the link energy of a 1-D allgather) and model parallelism for MLPs; it punishes streaming once-touched data, replication to all lanes, divergence, high P on small problems, and every rung-(a) trace (a 700k-line trace costs 323 mm² of instruction buffer at P = 16, so rung (a) stays P = 1 only).

Strongest surviving objection. Collectives-only is a closed world: no overlap of a lane's own communication with its compute, no point-to-point, no data-dependent destinations. Roune's “three-stage software pipeline you write every time” has no expression, and the makespan is pessimistic against hardware that barriers only at collectives. That is the price of a judge with no simulator; Scenario 3 is the answer when it is too high.

6. Scenario 2: the systolic MXU coprocessor with a vector pass

In one paragraph. The settled head, or a Scenario-1 tile's head, gets a K×K mesh of 16×16-µm processing elements in its otherwise empty lower half-plane, fed and drained through four edge buffers by K-lane vector loads and stores at min(4K, 16) B/ns. The PE program (up to eight four-byte micro-ops in up to four phases, including shr/shl/cmp/select) is loaded once at a priced configuration; afterwards there is no instruction traffic, only 1 fJ per clocked PE-beat, the clock trunk of the declared array per beat, and the bytes moved. Time is a Kung–Leiserson wavefront: mesh.run starts a skewed phase sequence, the head keeps issuing, mesh.wait joins. It is a TPU matrix unit and vector unit attached to a scalar core, Roune's “traditional CPU with extra vector compute and a systolic array.” K = 0 is the settled machine bit for bit. Adopt it second, inside a Scenario-1 tile.

              y ^      head memory: lin(a) half-diamond above the head (unchanged)
                |               . . . . . . .
   IN ─► (−1,0) [H] (0,0) ─► (1,0) OUT        (0,−1) instruction tape; core block, if a Scenario-1 tile,
                |                              sits beside the array to the west
  x = −w_buf−1 .. −1 │ x = 0 ................. K·s−1 │ K·s .. K·s+w_buf−1
  ┌──────────────────┼────────────────────────────────┼──────────────────┐ y = −2
  │                  │  NORTH buffer: K lanes, 2D B   │                  │  w_buf = 2D/s columns
  ├──────────────────┼──┬──┬──┬──┬────────────────┬──┼──────────────────┤ y = −2−w_buf
  │ WEST buffer      │PE│PE│PE│PE│ …              │  │ EAST buffer      │
  │ lane l: s rows   ├──┼──┼──┼──┼────────────────┼──┤ (pass-through    │  PE: 16 x 16 cells,
  │ slot (h,t) at    │PE│PE│PE│PE│                │  │  writes priced)  │  4x4 controller,
  │ x = −1−c,        ├──┼──┼──┼──┼────────────────┼──┤                  │  32 data B, 64 prog B
  │ y = y_top(l)−r   │ …                             │                  │  links: 4 B/beat,
  ├──────────────────┼────────────────────────────────┼──────────────────┤  16 fJ per byte-hop
  │                  │  SOUTH buffer (drain reads priced)                │
  └──────────────────┴────────────────────────────────┴──────────────────┘ y = −2−2w_buf−K·s

Constants and rules

Worked example: the 64×64 int8 matmul

Output-stationary OS_MAC (zero, then mac.i32 acc, @W.u8, @N.u8 with the operands forwarded east and south, then a northward drain), tiles of K×K outputs over a reduction length of 64, software-pipelined halves. A MAC-beat at s = 16 is 41 fJ (1 + 8 + 16 + 16), a drain beat 73. Compute-only energies; the common I/O is 124.45 nJ. The baseline is the blocked single core: 21,967,280 fJ compute-only in 674,920 ns.

The parenthetical components are the calculator's attribution and do not sum exactly to the totals (they leave out the one-time program load and head control at small K and overlap slightly at large K); the totals, E divided by 262,144 MACs, are the scored values.

The coprocessor is a time engine: 35–325× faster than the blocked single core. On energy it beats the single core only with the declared 8-µm PE (68.7–73.9 fJ/MAC at K = 8–16, 1.13–1.22× below); at 16 µm the in-mesh floor is 41 fJ of hops per MAC and the head's diamond feed adds 19–94, so K = 8 lands 1.20× above. Roune's forces appear correctly stated: issues per MAC fall 4× per doubling of K, but the head's bytes per MAC fall at most 2× on loads and not at all on stores (4K² i32 bytes per 64K² MACs at every K), the drain grows with K, and an array larger than the head port can feed idles — at K = 16 the mesh is 54% busy because a 16 B/ns port, not the array, sets the pace. The honest caveat: a TPU matrix unit is fed at K bytes per cycle per side from a banked vector memory, 16–32× wider than this inherited head port, so every K ≥ 16 row describes the port. The “TPU beats CPU on energy” story needs a banked head memory (deferrable), panel-resident buffers (D ≥ 8,192) and a reduction length far above K: the 8192³ GEMM and the tier-3 MLP, not 64×64. For the tier-2 SGD step the vector pass is load-bearing: without it the step is 73–89% head-bound and saturates at 13–16× over v1; with it about 90×.

On the 2²⁰-byte sum the mesh is a K-lane SIMD strip: 960 nJ (1.32× v1) in 67 µs at K = 16. It loses on energy at every K and cannot reach Dally's distributed-memory result at all, with 20–32 bytes per PE. Its scoring is entirely closed-form (static counts times per-beat costs, quasi-polynomial edge sums of period s, lattice sums for loads, the first-order recurrence for time); it cannot deadlock and determinism is a circuit property. Exploits closed: free head bandwidth (the port cap), the K = 1 relayout engine (4 B/ns), free idle silicon (trunk and PE beats regardless of gating), the straw-man baseline, zero-footprint PEs, area absorption, modes reaching into the mesh, declared half flags, mesh.prog as a cheap broadcast, link registers as free storage, untyped stores that scatter bytes.

Strongest surviving objection. Its expressivity is one dataflow shape. Only the 4K−4 edge PEs receive independent data, so anything that is not a rank-1-update wavefront runs at O(K) parallelism, and it answers one of the four target algorithms on its own. It is a component, ranked second as the first extension of a Scenario-1 tile, not as a multiprocessor; its energy case rests on the PE pitch and on a head port the model inherited rather than designed.

7. Scenario 3: the message-passing multicore with a chiplet level

In one paragraph. Scenario 1's lattice, tiles, tapes and collective library (kept as primitives, so every Scenario-1 program scores identically here), plus explicit two-sided messages (msend/mrecv, mxchg), a one-sided put into declared landing windows with an explicit release, credit-based end-to-end flow control per ordered pair of tiles, and a die/package level with priced crossings. Values stay a Kahn network (per-pair FIFOs, blocking receive, no probe); energy and area stay closed-form; makespan needs a deterministic, data-free discrete-event simulation over message and tape-segment events. It is Epiphany-, SCC- or Cerebras-class manycore, Dally's 4×4 array of 16 mm chips, and the SRAM-to-network-to-SRAM DMAs of Roune's AI CPU. Adopt it third, once the reference simulator exists and has been differentially tested against Scenario 1's closed forms.

   die (0,1)                            die (1,1)
   +-----------------------------+  gap  +-----------------------------+
   | [t][t][t][t] ... C x C      |  ~~~  | [t][t][t][t] ...            |
   | [t][t][t][t]  tiles, S µm   |═E_off═| [t][t][t][t]                |   ═ die-to-die link:
   | ...                         |  ~~~  | ...                         |     5,000 fJ/B, +20 ns,
   +-----------------------------+       +-----------------------------+     8 B/ns
        ║ E_off                                ║
   +-----------------------------+       +-----------------------------+
   | [t][t][t][t] ...            |═E_off═| [t][t][t][t] ...            |
 in═►[t]                         |       |                             |
 out◄[t]  tile (0,0) has the     |       |                             |
 instr►   three tapes            |       |                             |
   +-----------------------------+       +-----------------------------+
   die (0,0), 16,384 µm                  die (1,0)

Determinism and deadlock. Values are Kahn-determinate: per-pair FIFOs, blocking receives, no probe, single-signaller flags, single-writer disjoint windows, put snapshots its source at issue. Deadlock is decided by an untimed execution of the credit network (every tile run as a process with the declared credit depths and no clock; by the Kahn property the answer is schedule-independent), because FIFO-size matching alone is not enough: two tiles that each send twice before their first receive deadlock at depth one with perfectly matching sequences. Makespan is the discrete-event simulation of the rulebook: events are messages, tape segments, output-segment handovers and boot completions; between events each tile advances by the settled closed-form segment time; integer ticks of 1/(1,874×8) ns; oldest-head-flit arbitration with row-major ties; steady-state extrapolation of index-invariant outer loops. It is O(M log M) with M ≤ 10⁷ events, milliseconds for the worked examples and about 10⁴ events for the 8192³ GEMM and tier-3 MNIST after segment-level tape events and 64 KiB messages. A closed-form lower bound (per-tile sums, per-link bytes/8) is reported beside it.

What it adds, measured. Cannon on the 64×64 matmul, which Scenario 1 cannot write (point-to-point shifts with wrap on a mesh without a torus): per tile two operand blocks double-buffered, √P steps, a message of A left and of B up per step, C accumulated in place.

At 64×64 the model does not separate Cannon from SUMMA in energy (+1.9%) or time (+2.5%), because tapes and clock dominate; Cannon holds 4×256 B of operands instead of a 1 KiB block-row plus block-column and wins 14% of the area. At 8192³, where replication is 2 MB per tile, only Cannon can be written. On Dally's 4×4 dies the partial-sum tree costs at most 15 crossings × 4 bytes × 5,000 = 0.3 nJ, while loading the table across dies costs three quarters of 2²⁰ bytes at 5,000 or more, at least 3.9 µJ — which is his point about where the 640 pJ goes. Scoring: energy and area closed-form (piecewise-affine hop counts and die floors through the settled floor-sum engine); about 4,700 new judge lines including an 800-line simulator; 0.2–2 s per submission at the event cap. It rewards pipelines and stages with different text, overlap of one tile's DMA with another's compute, coarse messages and skew by tape schedule; it punishes fine-grained messaging (a 1-byte message is a 9-byte packet at ≥ 105 fJ per payload byte), die crossings for anything but results, and out-of-order output.

Strongest surviving objection. The makespan is a simulation, and every tie-break in it is a rule of the competition: the published simulator is the definition of time on the general path. The closed-form lower bound lets a suspicious makespan be compared, but the time axis rests on 800 lines of simulator rather than on a formula.

8. Scenario 4: the SIMT lane array

In one paragraph. One instruction tape, one decoder at the port strip, P identical lanes at fixed sites on an n_x×n_y array at pitch s, every instruction executed by every lane in lockstep. The word is driven over an H-tree whose entire wire length is charged per issue; data reaches lanes by striped recv over a second tree; lanes exchange through a distance-priced shuffle library with a per-link rate term or through a 32-bank shared plane; dp4a and dp2a do four int8 or two int16 MACs per issue. Everything is closed-form. It is one GPU streaming multiprocessor. Adopt it fourth, on its own P ≥ 2 board, because on the energy board it loses to the single core for every instruction-bound task by construction.

              SHARED PLANE  n_x·s wide × H_SH rows, 32 banks × 4 B (bank = ⌊q/4⌋ mod 32)
   y=n_y·s  ┌──────────────────────────────────────────────────────────────┐
            └──────────────────────────────────────────────────────────────┘
            ┌───────────────┬───────────────┬───────────────┬───────────────┐
            │  ◊◊◊ private  │  ◊◊◊          │  ◊◊◊          │  ◊◊◊          │  l_y = 1
            │   H  head     │   H           │   H           │   H           │  head at local row 16
            │ ┌────┐ 16x16  │ ┌────┐        │ ┌────┐        │ ┌────┐        │  head block rows 0..15
            │ │blk │ ══╗    │ │blk │ ╔══    │ │blk │ ══╗    │ │blk │ ╔══    │  (decoder leaf, dp4a,
            ├─┴────┴───╫────┼─┴────┴─╫──────┼─┴────┴───╫────┼─┴────┴─╫──────┤   predicate, ports)
            │  ◊◊◊     ╚════╪════ROOT╩══════╪══════════╝    │        ║      │  l_y = 0
            │   H           │   ║           │   H           │   H    ║      │
            │ ┌────┐        │ ┌─╫──┐        │ ┌────┐        │ ┌────┐ ║      │  ═ H-tree (I and D
            │ │blk │        │ │blk│ trunk   │ │blk │        │ │blk │═╝      │    trees, same shape)
   y=0      └───────────────┴───╫───────────┴───────────────┴───────────────┘
   y=−1      port strip      [in] [instr] [out]   decoder at the port; s ≥ 32

Worked examples

(a) The 2²⁰-byte sum, streamed, P = 16, s = 32: one striped recv.i32 (333,132 fJ, 9.09 ns) and one dp4a (6,204 fJ, 1.10 ns) per iteration, 16,384 iterations: 5.572 µJ in 166.95 µs, 12.6× the settled machine and 8.7× a same-tape single core, on 16,384 µm². Resident (the table in the lanes, s forced by capacity):

Instruction delivery is 83–89% of the parallel machine's energy at every P: fat lanes lose 2.9×, thin lanes win 2.7×, the crossover is near P = 100. This is the GPU lesson in one table.

(b) The 64×64 int8 matmul, P = 16, s = 32, a 128×68 plane holding B^T padded to bank-conflict-free rows, lane l owning columns j ≡ l (mod 16), a 4-row register tile and dp4a:

64× faster than the blocked single core (62× same-tape) for 1.32× the energy; on-chip energy triples because instruction delivery, 56–68% of it, is priced honestly, and dp4a is the only thing that keeps it there. The comparison with Scenario 1 is confounded by an ISA choice (decision P7): a Scenario-1 tile granted dp4a would cut its compute phase about eight-fold and land near 9,300 ns at P = 16, ahead of this array's 11,021 at 0.9× the energy — an estimate, not a calculator output.

Which machine: one streaming multiprocessor, with lane = CUDA core and register slice, decoder = warp scheduler, instruction tree = dispatch, data tree = operand collector, plane = shared memory with 32 banks, shfl = SHFL, predication = the SIMT mask, dp4a = IDP4A, brecv = the constant cache; 350–790 fJ of delivery per lane-instruction is of the order of published per-lane overheads. Scoring: closed-form at every rung (lane sums are lattice sums of piecewise-affine forms; about 2,500 judge lines; 1–20 ms per submission); deadlock impossible. Exploits closed: the P-ported plane and crossbar (32 banks, warp-scoped shuffles, a per-link rate), zero-footprint lanes (s ≥ 32 and the head block), data-dependent tape position across arms (bytes), data-dependent writer counts, the table bomb (immediates cost trunk plus tree per byte per issue), the unlabelled baseline. It rewards thin lanes with small private sets, coalesced striped I/O, register tiles, packed MACs, and ownership that makes lane order equal stream order; it punishes fat lanes, divergence, plane traffic without reuse, streaming, and scalar instruction-bound work (the sparse-parity search at P = 32 is 13× faster and 12.6× the energy).

Strongest surviving objection. On the energy board this scenario loses to P = 1 for every instruction-bound or streamed task, because the settled machine fetches a 14-byte instruction for 1 fJ and the thinnest legal lane array pays at least 110 fJ per issue; only resident, read-bound, packable problems flip. Either SIMT is scored as its own P ≥ 2 board (the recommendation) or P = 1 gets a priced instruction store, which breaks the P = 1 exactness that everything else here depends on.

9. Comparison

The rank is an adoption order under two constraints (the judge stays execution-free wherever possible; P = 1 stays exact) weighted by expressivity over the four target algorithms. No scenario is first on every criterion:

10. Recommendation and migration path

Adopt the SPMD tile mesh first. It is the smallest delta that makes the three invisible things visible — parallel makespan, cross-core locality, Dally's tree — while keeping the judge a formula: one header line, a stripe library with rule R9, seven scoped collectives, the 128×128 block, the clock rule and 8 B/ns tapes. Its scores are what the other three are compared against: every message-passing program that uses only collectives scores identically, the systolic array attaches to its tiles, SIMT is its own division. And it is the scenario in which the MNIST target reads correctly: the tier-3 MLP, whose weights are resident across epochs, is the “table produced in place” that makes the resident-sum dividend real rather than an artefact of dropping the load phase. Three things must be in its checker before it ships: R9 (stripe validity and the tape-phase recurrence), R1′ (affine lane addresses), and the P = 1 | P ≥ 2 split of the time board.

Then the systolic coprocessor inside a tile (“AI-CPU tile”: replicate P plus mesh K, D per tile), because it is the only scenario that prices Roune's thesis and its scoring is closed-form; the per-challenge flag is the PE pitch, and its first honest showcase is the 8192³ GEMM with panel-resident buffers. Message passing third, when the reference simulator exists and has been differentially tested against the closed forms on the collective primitives; its first user is the 8192³ Cannon. SIMT as a separate P ≥ 2 division, with decision P7 settling whether dp4a reaches the mesh's tiles.

Migration from the settled spec.

1. Pin two settled conventions the designs found unwritten, as decisions: the port term (a recv.T occupies sizeof(T) ns at 1 B/ns with no issue slot; program-text transit is not in T at P = 1) and the area function ((2k−1)² over the lin(a) half-diamond). Correct the “5 pJ per byte” slip to per bit.
2. The mesh: header, stripes with R9, collectives with charged landings, the block, the clock rule, 8 B/ns tapes, byte-based port balance and segment-wise arm padding (these two also tighten P = 1 modes: a settled program whose arms differ in recv width becomes invalid, a grammar change at no score change for programs that remain valid), and the reported columns E_static, E_ctl, E_clk-if-charged. Publish the resident-sum and matmul rows of §5 as reference scores; split the time board. Rung (a) stays P = 1 only.
3. mesh K, D inside a tile, with the three PE programs shipped as a verified library and the vector pass.
4. msend/mrecv/mxchg/put/release, windows, dies, with the simulator and the untimed credit run published as normative.
5. lanes P, s, H_SH on its own leaderboard.

Deferrable, with no worked number depending on them: scoring E_static; a banked head memory for the coprocessor; torus wrap links; asynchronous msend with a descriptor queue; heterogeneous tile radii and Roune-style mixed core kinds; per-die tape triples at E_pkg; a v2 critical-path time division.

11. Animations

The three animations below are driven by event-level schedules, not by drawings. Every dot is a byte transfer along its Manhattan path, every filled region is a local phase, and the counters under each canvas accumulate the same energy and time the scorer would charge; the totals were cross-checked against an independent Python calculator to the femtojoule and nanosecond. Press play, drag the speed slider (log scale), or step from event to event. Long local phases are fast-forwarded while nothing visibly moves, and the readout says so.

Animation 1: Dally's table sum on a 4×4 tile lattice (Scenario 1, resident framing)

Sixteen tiles at a 162 µm pitch each hold 1 KiB of the table in their private diamond (k = 33; the 128×128 blocks under the heads are the core footprints). The 38-byte program crosses the instruction tape once and is multicast over the spanning tree (152,206 fJ, 11.27 ns); every tile sums its 1,024 bytes (30,669 fJ and 1,039.82 ns each: one issue per byte plus the accumulator round trip plus the read at ⌈√a⌉); then 4-byte partial sums climb a recursive-halving tree in four rounds (1,349 fJ per one-hop message, 2,157 per two-hop, 24,275 fJ and 26.8 ns in all); the result leaves by the output tape.

Counters: dynamic energy 687,197 fJ; clock 168 fJ/ns × 16 tiles × 1,079.40 ns = 2,901,436 fJ; total 3,588,633 fJ; makespan 1,079.40 ns; area 16 × (65² + 16,384) = 329,744 µm². The same 16,384 bytes on the settled single core (k = 129, mean read distance 85.9 µm against 21.95 on the lattice): 1,557,877 fJ in 17,199.88 ns on 66,049 µm². The lattice is 15.9× faster; on dynamic energy it is 2.27× cheaper (the sixteen local sums alone 3.13×); with its clock it costs 2.30× more than the uncharged single core and 1.24× less like for like. At 1 KiB per core the clock is 4.2× the movement; the dividend grows with the table (§5, resident rows). Streamed instead of resident, the same job is 0.82× the single core's energy and 10.7× faster, because the 16 KiB has to cross the tape and be distributed either way.

Animation 2: 32×32 int8 SUMMA on a 2×2 tile lattice (Scenario 1)

Four tiles at a 176 µm pitch (k = 47). The 420-byte program is multicast (354,180 fJ, 54.7 ns). The input tape streams 128 sixteen-byte segments of A and B in stream order into the owning tiles under the block2d stripe (the tape at 8 B/ns, each tile receiving 16 B per 6 ns); each tile all-gathers its row of A and column of B in eight 256-byte messages (673,068 fJ, 141 ns); four register-blocked sweeps of 20,992 issues compute C (535,084 fJ and 21,266 ns each); then 64 sixty-four-byte rows of C leave under the credit rule, and the pad is visibly starved for its first half by the 24-ns injection cadence of the two tiles that own the first rows.

Counters: dynamic energy 35,687,672 fJ (text 354,180; input 10,784,768; allgather 673,068; compute 2,140,336; output 21,735,320); clock 168 × 4 × 22,411.90 = 15,060,797 fJ; total 50,748,469 fJ; makespan 22,411.90 ns; area 4 × (93² + 16,384) = 100,132 µm²; compute 65.3 fJ/MAC. The settled single core with the same blocked kernel: 33,206,376 fJ in 91,389.82 ns on 10,609 µm². The lattice is 4.08× faster (3.91× against a same-tape single core) for 1.53× the energy (1.05× like for like); the two tapes are 64% of its energy and the clock 30%. The picture shows why: eight block messages and four long sweeps, with the input and output tapes as the only serial resources.

Animation 3: a Kung–Leiserson wavefront on a 4×4 systolic mesh (Scenario 2)

A 4×4 mesh of 16-µm processing elements hangs below the head in the Scenario-2 geometry, with the west and north edge buffers beside it and the head's memory diamond above holding A, B and C at lin(a) addresses. The 64-byte OS_MAC program is loaded once over the array's H-tree (23,873 fJ, 16.0 ns); the A rows and B columns are loaded into the west and north lanes through the head port at 16 B/ns (5.0 ns each); then the wavefront runs. PE(i,j) starts at beat i+j, each active beat does one MAC for 41 fJ (8 local, 16 forwarding A east, 16 forwarding B south, 1 of clock) while the trunk toggles for 36 fJ, for 22 beats including six skew beats; the four-beat northward drain (73 fJ per PE-beat) delivers the i32 tile to the north lanes, and the store carries the 64-byte C tile back through the head port (2,337 fJ, 5.0 ns).

Counters: dynamic energy 50,729 fJ (configuration 23,873; loads 5,873; mesh 18,620, of which trunk 972 and PE clock 432; store 2,337); makespan 58.26 ns; area 5,913 µm² by the Scenario-2 rule (a 729 µm² diamond plus the 72² array block); 1,613 events; 198 fJ/MAC with the one-time configuration, 105 without it. The settled single core with the blocked kernel (two issues per MAC, since the head has no fused MAC): 15,454 fJ (60.4 fJ/MAC) in 679.89 ns on 1,089 µm². The wavefront is 11.7× faster and 3.28× the energy (1.74× with the configuration amortised; 0.47× like for like with the head's clock charged). At this toy size the array's fixed costs dominate; the 64×64 table in §6 is where the PE pitch decides the verdict. Two choices the Scenario-2 text left open are recorded in the schedule: the drain is drawn as a synchronous shift of one PE per beat, which is what its pricing implies, and the skew applies to the MAC phase only.

12. Open decisions

Summarized in one sentence: put the settled machine on a lattice and let tiles talk only through priced, statically scheduled collectives, and the judge stays a formula while parallel time, cross-core locality and the cost of idle silicon all become visible; everything more expressive than that is a simulator, and everything more specialised is a coprocessor.

13. References

1. W. J. Dally, “On the model of computation: Point. We must extend our model of computation to account for cost and location,” Communications of the ACM 65(9), September 2022, pp. 30–32, cacm.acm.org — the PECM, the SRAM numbers, the 256-core table-sum example.
2. B. H. Roune, “Designing AI Chip Software and Hardware,” 2026, Google Doc — the AI-CPU thesis, systolic-array size forces, torus and all-reduce arithmetic, SRAM-to-network DMAs.
3. “Compare matmul costs across chips,” Codex thread, September 2026 — the 1 fJ-per-step calibration against Dally's SRAM hierarchy; c/120 vs c/1200; the 8 KiB sub-array as a 32-shell half-diamond. Companion pages: the A100 grid energy report.
4. “The Grid VM: competition design and feasibility,” this repository, September 4, 2026 — the settled single-core machine, decisions D1–D6, the rung-(a)–(e) language.
5. “The expressivity–scorability ladder” and “Fixed or moving I/O order?”, this repository — closed-form scoring by rung; the tapes, determinacy, and the remark that nothing rewards parallelism.
6. “Proposal: a streamed instruction processor and a compact schedule language,” this repository, September 4, 2026 — the four-port geometry, the canonical instruction encoding, charged reads and writes.
7. H. T. Kung and C. E. Leiserson, “Systolic arrays (for VLSI),” 1978/79 — the wavefront schedule of Scenario 2 and Animation 3.
8. Y. Xu et al., “GSPMD: general and scalable parallelization for ML computation graphs,” 2021 — the programming model Scenario 1 prices (partition ids, all-reduce, all-gather, collective-permute on a mesh).
9. N. P. Jouppi et al., “Ten lessons from three generations shaped Google's TPUv4i,” ISCA 2021; “In-datacenter performance analysis of a tensor processing unit,” ISCA 2017 — the matrix unit fed from a banked vector memory that Scenario 2 approximates with a 16 B/ns port.
10. A. Olofsson, “Epiphany-V: a 1024-processor 64-bit RISC system-on-chip,” 2016; Intel SCC; Cerebras WSE — the manycore anchors of Scenario 3.
11. G. Kahn, “The semantics of a simple language for parallel programming,” IFIP 1974 — determinacy of the message-passing and collective networks.
12. C. Sun et al., “DSENT: a tool connecting emerging photonics with electronics for opto-electronic networks-on-chip modeling,” NOCS 2012 — the router energy band behind the 40 fJ transit constant.
13. lowRISC, the Ibex RISC-V core — the microcontroller-class core behind the 128×128 block and the 1,000 fJ per instruction control column.
14. A. Aggarwal, B. Alpern, A. Chandra, M. Snir, “A model for hierarchical memory,” STOC 1987; L. Yavits, A. Morad, R. Ginosar, “Cache hierarchy optimization,” 2014 — the √-priced memory and the bank-plus-network latency decomposition that the hierarchy numbers follow.
15. NVIDIA, PTX ISA (dp4a, shfl, predication, shared-memory banks) — the vocabulary of Scenario 4.