Search Search
Back to recaps

Week of May 25, 2026

Yad packages the auto-research loop into a reusable dispatcher kit, and the wikitext thread debates predicting characters instead of probability distributions.

181 messages and 44 links in the archive this week.

The main thread this week is Yad's auto-research loop. The idea is to point a team of agents at a researcher name and have them reproduce that researcher's work in the style of the existing hinton-problems and schmidhuber-problems builds. Yad walked through the build internals, orchestration, worker template, and FAQ pages (https://cybertronai.github.io/schmidhuber-problems/build-internals/), and pointed at the two driver docs (https://github.com/cybertronai/SutroYaro/tree/main/docs/auto-research-loop, with a worked LeCun spec at https://github.com/cybertronai/SutroYaro/blob/main/docs/auto-research-loop/lecun-spec-draft.md). This shipped as SutroYaro v0.31.0 on 2026-05-29: an auto-research-loop dispatcher kit that distills the two prior builds into reusable TeamCreate and worker-prompt templates, records the measured human-in-the-loop cost, and adds the LeCun spec seed (not yet wired into the nav). Yaroslav asked whether it makes sense to rerun the harness on all of LeCun's papers and what a good starting point would be. Gabriel tried LeCun on a couple of problems with his own agent and reported it mostly one-shot the result (https://td7xv5.pub.runspx.com/). Yad also shared the StackUnderflow writeup, repo, and docs that back these projects (https://yad.codes/posts/building-stackunderflow/, https://github.com/0bserver07/StackUnderflow, https://0bserver07.github.io/StackUnderflow/) and the Chimera examples agents can use to build a loop (https://github.com/0bserver07/chimera/tree/master/examples).

The wikitext thread moved on direction. Armins and Yaroslav discussed the next milestone after Gabriel left Monday, and Armins proposed a task to find a better optimizer for a transformer block. The concrete change: Armins fixed the model interface to require outputting character predictions directly rather than probability distributions (https://github.com/cybertronai/wikitext/pull/7), prompted by a model he was benchmarking that does not predict probabilities (https://arxiv.org/abs/2506.14202). This ties back to Yaroslav's argument, via Ben Recht, that there is no data-generating distribution and that probability theory may have been the wrong frame for learning algorithms (https://www.argmin.net/p/there-is-no-data-generating-distribution). Gabriel pushed on whether autoregressive distributions are still a fair way to model the data, and floated using an LLM to annotate which words are most important so count-based methods are forced to fail. Armins suggested infinigram for understanding the ceiling of n-gram capabilities (https://arxiv.org/pdf/2401.17377).

Challenge #2, energy-efficient matmul, saw Sung Jae get a leaderboard result. He noted it took a fair bit of hand-holding, and that having the agent look directly at the IR files to optimize worked better than just adjusting the algorithm. Yaroslav shared a grid of graphs across algorithms where the FFT looks fractal-like (https://github.com/cybertronai/ByteDMD/tree/dev/experiments/grid).

There was a hackathon this week. Armins seeded a braindump doc (https://docs.google.com/document/d/1k8jYDPawmq4gnzSkObm0sHX_x48-l-LFBl6lWfTWReY), and the group looked at a VizDoom duel MVP on Modal (https://ab-10--vizdoom-duel-mvp-web.modal.run/docs, https://github.com/sjbaebae/modalauto), plus EvolutionGym and the VDAIC2017 VizDoom competition repo (https://evolutiongym.github.io/, https://github.com/mihahauke/VDAIC2017). Yaroslav reported back that they did not win.

Smaller threads: in an in-person meeting the group discussed chain-of-thought monitoring and reward hacking (https://openai.com/index/chain-of-thought-monitoring/, https://x.com/marksaroufim/status/2057252914905932253). Yaroslav decided on The Unlicense for the wikitext repo over MIT (https://github.com/cybertronai/SutroYaro/blob/main/LICENSE). Yad looked at an auto-research hackathon offering Modal and ChatGPT credits and asked whether someone could attend on his behalf (https://luma.com/fvz1h1dq), and tried Google's Antigravity CLI with Gemini 3.5 Flash and GLM 5 Turbo as model options (https://antigravity.google/product/antigravity-cli, https://docs.z.ai/guides/llm/glm-5-turbo).

Open question into next week: whether to commit to rerunning the full auto-research loop on LeCun, and how much babysitting that takes. Cosmin also flagged a Codex credits issue where added credits did not unblock him.

Sources