sutro-problems

Research archive: D5 width-128 cross-taught Muon ensemble

Research inclusion; D5 qualification remains unresolved. Both existing author sets (2/3, then 3/3; later-set median 1,490.895 ms) remain separately preserved. The initial independent reviewer group passed 1/3: two complete runs missed D5 by 28 and 74 labels, with all 45 timed calls retained. There is no qualifying reviewer-group median and no leaderboard promotion. This inclusion preserves the source, original author evidence and attribution. See reviewer evidence.

Entrant: @likelytobelaura. The joint train+query graph-propagation readout is @sjbaebae’s, from the D3 record (see Credits).

Two official sets were run for this exact source; the first failed by one label and is disclosed below. The second set of three canonical Modal A100-80GB sandbox evaluations all passed difficulty 5 (mean MNIST error at most 1.90%): median ranked time 1,490.895 ms per call, median energy above idle 190,385.8 mJ per call, 323,965 / 330,000 MNIST labels correct (1.829% aggregate error).

Set 2 run MNIST error Margin, labels Hold-out Ranked ms mJ above idle Record
1 1.830% +77 Fashion 88.94% 1,490.895 186,415 json
2 1.833% +74 Fashion 89.01% 1,473.153 200,918 json
3 1.824% +84 KMNIST 96.97% 1,496.069 190,386 json

Three distinct GPU UUIDs; every call within 2% of its dataset’s median time. verification.json recomputes every verdict and the ranked times from the raw per-call data.

Disclosure: the first official set failed

The first set of three runs of the same source (identical SHA-256) passed 2 of 3. Run 3 labelled 107,909 of 110,000 correctly, one fewer than the 107,910 the band requires (1.901% error), and was disqualified. No run of either set was dropped or replaced; the second set was run afterwards as a separate, complete set of three.

Set 1 run MNIST error Margin, labels Ranked ms Record
1 1.837% +69 1,535.147 json
2 1.809% +100 1,533.598 json
3 1.901% -1 disqualified json

Across all six official runs the source labelled 647,863 / 660,000 correctly (1.839% aggregate error); five of six runs passed. Set 1 verification. The margin is thinner than the width-256 entries’, so an occasional failed run is to be expected.

Source: submission.py, 15,223 bytes, SHA-256 d3bc8a2bf543c8db7f90e57d9b974532eb4e15d3f5415c1f79015ac34a2075a9 (identical in all six records). mnist.check_source passes and mnist.review_flags reports nothing. Source only: scalar hyperparameters, no arrays, no fitted data.

Method

Rules review

Development (A100, dev/quick.py, not official scores)

Step (seeds 401-416 unless noted) Mean error ms per call
Width 256, 2,200 updates (#120) 1.838-1.849% ~1,900
Width 128, dropout 0.1 2.024% 1,327
+ dropout 0.05 + widened final readout 1.872% (two runs); 1.881% on fresh seeds 501-516 ~1,290
+ hidden bias (this file) 1.841% / 1.842% (two runs) ~1,440

With width 128, more updates (to 5,000), stronger tangent jitter, 10-16 members, and re-tuning Muon learning rate, weight decay, EMA decay, cross-teaching coverage and start, mixup strength and the tangent neighbourhood gave nothing beyond noise.

Credits

Reproduce

From mnist-a100:

python run_modal.py submissions/tangent-relu-w128-bias-d5-20261013/submission.py:classify --difficulty 5 --runs 3 --json out/d5-w128-bias