Search Search
Back to recaps

April 2026

ByteDMD becomes the primary metric, Yad and Seth measure how far methods sit from the floor, and Yaroslav pivots to modeling energy directly.

April took the byte-level idea from the end of March and made it the lab's metric, then spent the rest of the month stress-testing it. This recap draws on the changelog (v0.27.0 to v0.29.0), the April 30 catch-up, and the Telegram archive.

ByteDMD becomes the metric

ByteDMD was adopted as the primary metric (v0.27.0, April 14; wired into the repos via PR #80, April 15). It measures data movement at byte granularity in pure Python, which closes the escape hatch that let agents look efficient on paper. The case for it was made by an exploit: Seth found that the top sparse-parity entry escaped the element-level TrackedArray via np.asarray() to do bit-packed GF(2) on raw Python ints, work the metric never saw. Counting those operations honestly moved the entry's cost from about 45K to 81K. Byte-granularity tracking removes the numpy escape hatch and rewards smaller dtypes. The metric got its own repos (ByteDMD, ByteDMD-definition, ByteDMD-examples), partly to keep its definition out of reach of agents working in the main repo, and was validated against Karpathy's microGPT.

Yad and Seth measure the floor

With an honest metric, the question was how far the methods sit from the best possible. Yad and Seth designed the floor-gap survey and ran it April 23 to 25 (PR #87 Seth, PR #88 Yad):

MethodByteDMDTimes the read-floor (~70)
KM-min2683.8x
GF(2)101,5011,450x

A geometric lower bound (0.3849 times the measured ByteDMD), proven with a Tarjan-style argument and reviewed externally, put a floor on the real VLSI allocation cost. KM-min sits a mere 3.8x above the read-floor, while algebraic methods span three orders of magnitude.

Negative results, kept honest

A habit showed up this month: reporting what did not work without dressing it up. Seth's NoProp experiment (PR #78), diffusion-style local learning with no inter-layer backprop, solved parity but did not beat SGD plus Curriculum, and was merged as a clean negative finding rather than retrofitted to the new metric.

The pivot: from bounding energy to modeling it

The strategic move came late. After weeks trying to prove ByteDMD lower-bounds real energy without success, Yaroslav concluded it is a heuristic, not a bound, and shifted the research front to implementing algorithms directly in a 2D Manhattan-distance grid (Bill Dally's cost model). The active work moved to experiments/grid, where agents had implemented 9 energy metrics across 45 algorithms, and to a new idea: force agents to submit an intermediate representation (load from position N, store to position M) and score energy on the grid directly. On April 30 Yaroslav opened Challenge #2, energy-efficient matmul (sutro-problems/matmul).

Repo hygiene at agent scale

Yad kept the workspace clean with the same agent loops doing the science. His parallel-agent sprint (PR #82) closed seven issues in one pass; branch protection was enabled on main; about two dozen stale issues were triaged and closed against evidence. Agent compatibility broadened (a GEMINI.md alongside CLAUDE.md and CODEX.md via PR #72), and a Nix devShell put sqlite3 one command away for telegram.db queries.

StackUnderflow ships

Yad shipped StackUnderflow on April 21, a one-line-install tool that indexes past AI coding sessions so prior decisions and failure modes can be queried before redoing work. It is now used across the Sutro repos. Andy Zhang became a regular contributor (three PRs in the window) and shared his full parallel-agent setup; Sung Jae Bae contributed ByteDMD fixes; Silen Naihin shared an experiment-organization pattern from his own autoresearch work.

Reusable this period

  • ByteDMD: a pure-Python, byte-granularity metric that agents cannot game with representation tricks.
  • The floor-gap method: measuring methods against a read-floor and a geometric lower bound rather than only against each other.
  • The norm of publishing negative results as first-class findings.

What's next

  • Pursue the direct grid model (Manhattan distance and IR submission) as the successor to ByteDMD-as-bound.
  • Stand up Challenge #2 (matmul) with real submissions.
  • Re-measure the legacy best-methods tables under ByteDMD.

Sources