Search Search
Back to recaps

June 2026

The cross-entropy track lands, nakaprop gets its name, the caffeine benchmark forms, and the month's thesis becomes equation solving with multiplicative updates.

Where the Sutro Group stands in June 2026, current through June 10 (128 archive messages so far). The month opened on the Modal hackathon debrief and the v0.31.0 dispatcher kit carried in from May, then found its own shape fast: the wikitext cross-entropy question got settled in code, two new optimizer directions opened, and a minimal optimizer benchmark started to form. This recap is built from the four-challenge index, the auto-research-loop kit doc, the changelog, and the June archive.

Where the work sits

The center of gravity is auto-research loops. The Modal hackathon on May 30 was built around them; Yaroslav wanted "a candidate which uses auto-research itself as a deliverable, tooling to visualize and explore the evolution of ideas." The team did not place, and the June 1 meeting was the debrief. A new thread also opened on structure-specific optimizers, which sharpens the group's oldest question: how much better than gradient descent can you do by exploiting structure.

By the second week that question had its sharpest formulation yet. Yaroslav's learning-rotations repo shows a method he describes as too weird to derive from gradient descent, and his framing is to treat learning as an equation-solving problem rather than a minimization problem. The carry-forward observation: the best update there is multiplicative rather than additive, and matrix multiplication suits GPUs better than addition because of the memory wall, so a multiplicative transformer update would be promising. The June 8 session took up Tilde Research's two-matrix update as a related direction.

By challenge

Full detail is in the challenge index.

  • #1 Sparse parity: the settled benchmark. The submission pipeline is live (submissions as recent as May 12), scored under ByteDMD. KM-min and GF(2) lead, but have not been re-measured under the byte-level metric.
  • #2 Energy-efficient matmul: active hill-climbing on Bill Dally's 2D grid; the 16x16 record fell to 67,821 in mid-May. Lower bounds are an open hard problem.
  • #3 Sparse parity on the grid: solve it in about 9 grid instructions (launched May 8). Precomputing XORs plus bit-packing works; tiling does not. The live question is whether integer and 8-bit op sets beat floating point.
  • #4 wikitext: minimum-Joules language modeling on WikiText-103 (Modal, NVML). The baseline modded_nanogpt is 54,784 J at 0.7285 accuracy in 322.7 s; a forward-forward entry reaches about 0.39 accuracy at roughly 10x fewer Joules. The cross-entropy debate resolved in code on June 4: Gabriel's CE track landed, and under its threshold only nanoGPT passes while every count-based method fails by a wide margin, so CE now stands as a real performance bar.

Active threads

  • Structure-specific optimizer for self-attention. A thread that grew out of the wikitext work (Armins, May 27; Yaroslav, June 1): design an update specialized to self-attention instead of a generic optimizer like Adam. The framing question (Yaroslav, June 2): how much room for improvement there is using structure-specific solvers instead of gradient descent. The current bottleneck (Armins, June 2): finding a toy problem whose attention optimization tracks the full LLM training setting, kept small enough to check correctness.
  • Nakaprop. The fixed-backward-pass variant beat backprop at short horizons in unit tests (June 2 to 4) and converged once sweeps ran long enough; its updates are order-dependent, which the PyTorch optimizer interface cannot express. Gabriel is extending it to cross-entropy loss, and the name sticks if it works at larger scale. Details in the week of June 1.
  • Caffeine, a minimal optimizer benchmark in formation with its own Telegram topic. The open design question is the data: Yaroslav argues random data isolates architecture structure from data structure, Armins that incompressible data gives a learner nothing to learn and real text is the honest test. The proposed harness shape is a solver that takes (x, y) pairs and returns weights.
  • Auto-research loops. Yad's dispatcher kit (v0.31.0) packages the wave-build method as a driver brief, copy-paste TeamCreate and worker templates, and a worked LeCun SPEC, so a second operator can run it without his tacit knowledge. The loop has run end to end twice (hinton, schmidhuber), both driven by Yad. The kit is set up as a transferability test: whether a fresh driver reproduces the result, and at what autonomy ceiling (about 25.7 lead turns per load-bearing prompt, 8 of 40 prompts load-bearing on the Schmidhuber run), is the open measurement.

Reusable this period

  • Yad's auto-research-loop kit, for reproducing a researcher's body of work as runnable stubs. See Insights.
  • The hinton (53) and schmidhuber (58) baseline catalogs, both pure numpy.

What is next

  • Instrument the baseline catalogs with ByteDMD (the v2 pass), turning them from correctness baselines into energy data.
  • Settle caffeine's task design, the random-versus-real-data question.
  • Take nakaprop to cross-entropy at scale, and chase the multiplicative transformer update.
  • The wikitext CPU-energy (RAPL) question remains open.

Sources