One femtojoule per scored grid step.

01 · Unit audit

The conversion is exact once the counted quantity is fixed.

The scorer charges each source-operand read and each final output read by ceil(sqrt(address)). The score is already the sum of those read charges. It should not be multiplied by a second “total distance” unless that second quantity has a documented, different definition.

Here, “grid-step” means the scorer’s one-way address-radius abstraction. It is not a trace of producer-to-consumer Manhattan routes, router hops, or physical millimetres of wire.

Reconciliation required: 50,000 fJ and a total traversed distance of 550,000 cannot both follow the same 1 fJ/unit rule. If 50,000 is a normalized objective, publish the normalization equation and call it “score units,” not raw distance.

02 · Compiler report

The optimizer bought locality with explicit copies.

The record performs 10.25% more paid reads, entirely through 1,653 copies, while cutting average distance by 82.35%. The weighted total falls by 80.54%. This is precisely the kind of scheduling trade that a distance-sensitive compiler objective should expose.

Record history under the 1 fJ rule

Reproducibility: inspect the record report, submitted IR, machine-readable audit, or comparison CSV.

Precision is not a footnote

The verifier uses unbounded Python integers. Current inputs reach 256 and outputs reach 161,368, requiring 18 unsigned bits. An INT8 chip comparison is therefore a scale reference, not semantic equivalence. A hardware experiment should first choose uint8 modular, int8 × int8 → int32, or exact wide-integer semantics; that choice changes both arithmetic and transfer energy.

03 · Hardware literature

Published “energy per matmul” is rarely one comparable thing.

The table below converts each source’s reported efficiency to an 8192-operation energy equivalent. It answers: “If the published large-workload or peak pJ/op held for 8,192 useful ops, what energy scale would result?” It does not predict the energy of an isolated 16×16 launch.

The especially relevant non-systolic case: Tesla

Repeated 16×16 measurements show the utilization cliff

A 2012 study reports energy and elapsed time for a window of 1,000 repeated 32-bit 16×16 multiplies. Dividing its table values by 1,000 gives 17.62 µJ on a Cyclone II systolic FPGA, 20.11 µJ on a Clarkdale Core i5, and 34.79 µJ on a Pineview Atom per multiply. These are old platforms and software stacks, not modern efficiency targets. The CPU figures subtract idle power, while the FPGA measurement excludes regulator loss. The experiment is still valuable because it shows why scaling large-GEMM TOPS/W down to 8,192 operations can miss small-call reality by orders of magnitude.

Modern fixed-function engines have the same qualitative issue: the Apple ANE study reports roughly a 0.23 ms host/firmware dispatch floor and a separate engine-rail reading near 0.9 W for tiny matmuls. Their rough cross-product is 207 µJ, not a measured complete-event energy and not evidence that an exact 16×16 shape is supported. Saturated pJ/op and isolated-call joules answer different questions.

04 · 8K INT8 extrapolation

A scalable Sutro schedule lands at 119 mJ. Blackwell B200 estimates 244 mJ.

For a full square C[8192×8192] = A[8192×8192] · B[8192×8192], the workload is 549,755,813,888 MACs, or 1,099,511,627,776 conventional operations. The numbers below assume INT8 inputs and INT32 accumulation/output for the hardware comparison.

“8K model” is ambiguous. This section computes a synthetic full 8192³ GEMM. One-token autoregressive decode usually multiplies a 1×8192 activation by an 8192×8192 weight matrix—8,192× less arithmetic—and is much more likely to be memory-bound, so its energy cannot be obtained by simply dividing these joules by 8,192.

How the 0.118953 J Sutro estimate was derived #

This is an exact score for one parameterized schedule family, followed by the chosen 1 fJ/unit conversion. It is not the 66,300-point 16×16 record multiplied by a scale factor.

The checked-in 66,300-point record is a final physical-address program specialized to 16×16; it has no generator that can be rerun at 8K. Cubically scaling 66,300 gives 8.90 mJ, but incorrectly freezes average access distance while the live memory footprint grows. Instead, the calculation generalizes the repository’s sa_cache loop nest to N = 8192 and searches every tile pair whose dimensions divide N.

The sweep is exhaustive within this literal two-parameter sa_cache family, not across all possible matmul algorithms. The next two shapes are 128×128 at 121,105,737,911,286 and 256×64 at 121,681,290,169,654 score units. The 8K result is evaluated in closed form; it does not materialize a trillion-scale IR.

Exact scorer equation

Let d(a)=ceil(sqrt(a)) be the cost of reading address a, and let D(R) be the sum of d(a) over every address in region R. Cells are packed by descending reads per cell—sA, TMP, sB, sC, A, B, C—which is optimal for this fixed schedule because d(a) never decreases with address.

Substituting N = 8192, T_i = 256, and T_j = 128 gives this exact ledger:

The last step is only the proposed unit conversion: 118,953,083,721,334 × 10⁻¹⁵ J = 0.118953083721334 J. No throughput or clock rate is used, so the model produces energy but does not produce time. Arithmetic, destination writes, finite SRAM/HBM capacity, data conversion, control, clocks, and leakage remain unpriced.

The naive placement grows as Θ(n⁴). A size-retuned locality schedule is approximately Θ(n^3.5) because the scorer’s ceil(sqrt(address)) distance grows with the footprint. The selected schedule averages 53.94 score units per paid read; its 0.118953 J is 108.2 fJ per conventional operation, or 9.24 TOPS/W-equivalent for movement alone.

Blackwell and Rubin: what the official INT8 tables imply #

The Rubin result is intentionally counterintuitive. The same preliminary product table emphasizes 50 PFLOPS NVFP4 and 17.5 PFLOPS FP8, but those formats are not interchangeable with exact INT8×INT8→INT32 work. A 72-GPU rack cross-check—18 dense INT8 POPS at 136 kW provisioned power—gives 8.31 J per GEMM when amortized across 72 simultaneous GEMMs. It is consistent with, but does not measure, the 7.92 J per-GPU estimate.

SKU names matter: NVIDIA’s current Blackwell Ultra B300 table lists only 153.5 dense INT8 TOPS and up to 1,100 W, which implies 7.88 J. That is radically different from B200’s 0.244 J and reflects the published legacy-INT8 specification—not a general claim that B300 or Rubin is less efficient on its target FP4/FP8 workloads.

Four bytes and four bits are different Blackwell paths #

Taking “four byte” literally means FP32: four bytes per matrix element. NVIDIA publishes 75 TFLOP/s FP32 for one HGX B200 GPU. The same FP32 storage can enter the much faster TF32 Tensor Core path, but TF32 reduces multiply precision. NVFP4 is the other common reading: four bits, or half a byte, plus block-scale metadata.

All four B200 rows divide the same 1.0995 trillion conventional operations by NVIDIA’s published dense rate and multiply by the 1,000 W maximum TDP. They are matched roofline-style estimates, not measured per-precision power. FP32/TF32 traffic assumes C=A·B without reading a prior C matrix. For B300’s 1,100 W maximum TDP, the corresponding strict-FP32 and TF32 estimates are 16.13 J and 1.10 J; its per-GPU dense-FP4 figure gives 0.086 J.

8K comparison with evidence boundaries #

Only the T4 rows use an exact 8192³ published run paired with measured card power—and even those joules are derived from a graph-read throughput. No primary source located reports integrated exact-8K INT8 kernel joules for B200, B300, Rubin, H100, MI300X, TPUv1, or Tesla FSD. Their rows are roofline-style estimates, not measurements.

Two INT8 inputs occupy 128 MiB and an INT32 output another 256 MiB, for at least 384 MiB of logical device traffic. Sutro’s widthless, unbounded 2-D memory charges none of the HBM transfers, writes, arithmetic, clocks, leakage, or dispatch. Also, the repository verifier currently uses unbounded Python integers, so this hardware table is a scale comparison rather than semantic equivalence.

Reproducibility: download the 8K comparison CSV, inspect the machine-readable derivations, or run the standalone scaling script.

05 · Calibration and extension

Distance optimization survives. Absolute energy needs more terms.

The 1 fJ coefficient is best treated as a transparent hypothesis. It answers “what if link movement costs 1 fJ per operand-grid-step?” It does not calibrate SRAM activation, arithmetic, or the physical length and width of a wire.

Sensitivity to the earlier endpoint-offset idea

The second model charges 400 fJ of fixed endpoint energy per paid read. Because the record intentionally performs more reads, that term nearly erases its distance advantage. This does not prove that either calibration is correct; it proves that read count and read distance must remain separately visible.

What a stronger compiler report should export

XLA’s “cost” is semantic FLOPs and bytes; TVM’s cost model generally predicts schedule latency; Timeloop’s counts are actions; Accelergy turns actions into modeled joules. Keeping those namespaces separate prevents an attractive unit conversion from acquiring more physical meaning than it has.

06 · Sources and method

Primary sources, explicit derivations.

All hardware conversions use one MAC = two conventional operations: 8,192 operations at n = 16 and 1,099,511,627,776 operations at n = 8,192. Reported sparse peaks were excluded. Values marked measured retain the source’s measurement boundary; peak/TDP values are explicitly labeled derived. Sources were reviewed 28 August 2026.

1. Sutro scorer implementation and 66,300 record audit.
2. Reproducible 8K closed-form scorer and the report’s derivation audit.
3. NVIDIA B200 Blackwell datasheet, B300 Blackwell Ultra datasheet, and the current HGX specification table.
4. NVIDIA Vera Rubin preliminary specifications, datasheet, and rack-power analysis.
5. Exact 8192³ INT8 T4 study (DOI).
6. AutoMM measured INT8 efficiency on A100 and VCK190.
7. Tesla FSD computer, IEEE Micro; Hot Chips 31 slides.
8. In-Datacenter Performance Analysis of a TPU.
9. NVIDIA H100 Tensor Core GPU datasheet.
10. A100 matmul power measurement study.
11. AMD Instinct MI300X datasheet.
12. Dissecting the Graphcore IPU Architecture.
13. Apple Neural Engine reverse-engineering and energy study.
14. Eyeriss row-stationary accelerator.
15. NM-Carus near-memory SIMD accelerator.
16. Intel 22FFL analog SRAM compute-in-memory macro.
17. Energy efficiency of matrix multiplication on CPUs and GPUs.
18. Direct 16×16 CPU/FPGA energy study.
19. Horowitz, computing’s energy problem.
20. CACTI memory and interconnect model.
21. Timeloop, Accelergy, and MAESTRO.

Limitations: no cited chip implements the exact Sutro abstract machine or verifier contract. Published energy varies with matrix shape, data values, sparsity, placement, voltage/frequency, batching, and measurement scope. The normalized table is a scale comparison, not a ranking of products.