sutro-problems

D5: cross-taught Muon ReLU-MLP ensemble with local tangent jitter (fast)

Independent reviewer reproduction — October 14, 2026

The first frozen reviewer group passed 3/3 fresh canonical sandboxed Modal A100-80GB runs, with valid above-idle energy measurements on three distinct GPUs. Median ranked time is 1,915.402496 ms/call and median energy is 343,151.622557 mJ/call; pooled MNIST error is 1.789091% (324,096/330,000 correct).

Reviewer run MNIST error Margin, labels Ranked ms Above-idle mJ
1 1.810000% +99 1,922.228505 331,163.170874
2 1.812727% +96 1,913.758972 343,151.622557
3 1.744545% +171 1,915.402496 346,421.252166

Source is unchanged at SHA-256 9adc570a02279b34d0766344ec72f5589ff10fae06e1b6b341b58dc351a6445a. The scorer and runner are unchanged. Verification reconstructs every complete canonical score and energy result from the raw records; source review covers per-call fitting, label isolation and cached-state reset. No reviewer outcome was replaced or retried.

The leaderboard uses the independent reviewer medians above. Author results below are retained separately. The published instructions specify three official Modal runs and their median; this independent reproduction is review evidence, not a newly imposed second-batch submission rule.

Using Python 3.13 as in the canonical runner, recheck saved records from this submission directory with python evidence/reviewer-20261014/verify_records.py evidence/reviewer-20261014 submission.py --output /tmp/pr120-verification.json. This reconstructs recorded verdicts without executing the learner.

Original author report (as submitted)

Present-tense record and pending-review claims below describe the submission date; the dated reviewer section above gives the accepted review status.

Entrant: @likelytobelaura. The joint train+query graph-propagation readout is @sjbaebae’s, from the D3 record (see Credits).

Three canonical Modal A100-80GB sandbox evaluations, the first and only set run for this source, all passed difficulty 5 (mean MNIST error at most 1.90%). Median ranked time is 1,898.373 ms per call; median energy above idle is 333,939.8 mJ per call. The 33 MNIST calls labelled 324,016 / 330,000 rows correctly (1.813% aggregate error). This is the same method as the current D5 record (tangent-crossteach-d5-20261012, 3,335.4 ms) with faster members and a faster readout: 43% less time per call at about half the energy.

Run GPU MNIST error Margin, labels Hold-out Ranked ms mJ above idle Record
1 A100 80GB PCIe 1.809% +100 Fashion 88.64% 1,916.868 333,940 json
2 A100-SXM4-80GB 1.810% +99 Fashion 88.78% 1,894.813 339,773 json
3 A100-SXM4-80GB 1.821% +87 KMNIST 97.22% 1,898.373 321,951 json

The three runs used three distinct GPU UUIDs. The worst single MNIST draw was 2.15% (the cap is band + 1.5 = 3.40%), and every call was within 1.5% of its dataset’s median time. verification.json recomputes every verdict, the ranked times and the per-call gates from the raw per-call data in the records.

Source: submission.py, 13,363 bytes, SHA-256 9adc570a02279b34d0766344ec72f5589ff10fae06e1b6b341b58dc351a6445a (identical to the source_sha256 in all three records). mnist.check_source passes and mnist.review_flags reports nothing. The file is source only: scalar hyperparameters, no arrays, no fitted data.

Method

Rules review

Checked against the rules in mnist.py’s docstring:

Development (A100, dev/quick.py, seeds 401-416, not official scores)

Step Mean error ms per call
GELU MLPs with gain and bias, 2,500 updates (current D5 record, PR #118) 1.803% 3,349
+ in-graph hash dropout, fused tangent jitter, block readout 1.787-1.804% 2,950-2,977
+ ReLU, no hidden gain or bias 1.842% 2,187
+ 2,200 updates (this file) 1.838-1.849% 1,893-1,949

Removing the gain and bias takes the norm block from 143 to 47 us per layer (their gradients are column sums over all rows, which Inductor cannot fuse with the row-wise norm backward). With these members, a re-tune of tangent strength, Muon learning rate, weight decay, cross-teaching coverage and cadence, dropout and member count found nothing beyond noise.

Credits

Reproduce

From mnist-a100:

python run_modal.py submissions/tangent-relu-d5-20261012/submission.py:classify --difficulty 5 --runs 3 --json out/d5-tangent-relu