Search Search
Back to recaps

Week of May 11, 2026

Three energy challenges run in parallel as wikitext spins out into its own repo and a SutroYaro records-room split begins.

146 messages and 30 links in the archive this week.

A busy week, 146 messages across seven topics, with most of the energy on the matmul and sparse-parity challenges, the new wikitext task, and a proposed reorganization of SutroYaro.

Wikitext spins out

Armins moved the wikitext task into its own repository at cybertronai/wikitext and rewrote the README by hand except for the rules section, asking the group for feedback on tone. Earlier in the week he fixed modal submission issues in PR #17 on sutro-problems while debugging a run with Yaroslav (modal logs, the wikitext-build branch). Armins also asked for ten minutes to demo wikitext to the group. Yaroslav posted eval progress reaching about acc=0.728 at roughly 92 char/s and shared a gist. He noted possible follow-up work to find much faster GPU iteration, citing that Keller used 2-second GPU runs to discover Muon, and looked into faster container startup (a ChatGPT thread suggested dockerhub plus estargz).

Challenge #2: energy-efficient matmul

Cosmin iterated heavily, opening PR #19, PR #21, PR #22, PR #23, and PR #24 for a series of small improvements. He got a slight improvement over Sung Jae's 2026-05-05 record of 68,452, verifying his own result at 68,392 through tuning over existing solutions, and openly asked whether his submissions might be cheating or whether the lower bound was wrong. Anastasiia shared the method she used to reach the optimal solution, an agent that explores dimensions in parallel, at cybertronai/SutroAna. Yaroslav pointed to the matmul score list, said her research improved to 69K after another hour, and flagged that the lower bound was wrong while checking the access_distance docs. He also asked whether Codex could run the Ralph Wiggum loop, and reflected on agents as idiot-savants that need human knowledge injected and can struggle to raise temperature back up once lowered.

Challenge #3: energy-efficient sparse parity

Yaroslav raised whether the problem actually needs more ops than Dally v3, and whether integer-only learning could replace continuous numbers, linking Wolfram's minimal models post. Andy asked how to think about operator fusion (for example MAC) and whether to break operators apart to avoid hidden movement cost. Yaroslav favored smaller, 8-bit instruction sets because the more that moves into the instruction set, the fewer optimization opportunities remain, and noted there are no fused operators in the current IR.

chat-yad and the SutroYaro split

Seth opened PR #60 on hinton-problems adding a second v2 ByteDMD algorithm pair (bars-rbm CD-1 vs bars wake-sleep), then noted it looked like a purely negative result and asked where to point agent cycles next. Yaroslav clarified the v2 and v3 staging (v2 is which stubs ByteDMD can trace, v3 is which of those Dally can trace) and pointed at the symmetry directory. Seth then added symmetry as a third sutro-problems challenge, 6-bit palindrome detection from RHW1986, in PR #18. Andy reported his agent had been working on a hinton to dally port and asked whether to pause it, since porting surfaced issues needing a human decision.

Yad proposed separating SutroYaro from the RL-Env and stripping it down to a records room and toolkit for the other repos in issue #96. Seth's agent concluded only sparse-parity-challenge needed changes and opened PR #40, then a follow-up PR #97 removing the same files from SutroYaro to complete the copy-paste-delete. Seth lacked merge permissions on PR #40 and added Yad and Yaroslav as required reviewers on #97. Yad also shared a paper index for ICLR-2026 (repo) and two related rejected papers, ENERGYLLM-BENCH and a low-bit compression paper. Both Yad and Yaroslav noted they had been demoted on their GLM plan, with Yad's bourbaki agent no longer reproducing already-written theorems.

In-person and tooling

The group met in person, with Yad giving a demo of his workflow (recording). Cosmin asked how others are finding agent usage and described iterating on the OAI param golf with codex 5.5. Yad shared a hand-wavy read that gpt-5.5 preserves reasoning tokens and parses tool calls well, while opus-4.7 seemed to deviate from 4.6.

Sources