Search Search
Back to recaps

Week of February 9, 2026

A theory-heavy week on compression and energy efficiency metrics. Plenty of debate on how Sutro should be framed.

126 messages and 13 links in the archive this week.

A busy week of theory and framing, 126 messages, most of it in the chat-yaroslav thread and General.

The compression thread ran the deepest. Yaroslav argued that if the dataset is treated as effectively infinite, the length of the compressed internet is set by Shannon's coding theorem, and he flagged that this is Shannon compression rather than Kolmogorov compression, pointing to the Hutter Prize. He also noted that computational cost cannot be ignored: a perfect next-token predictor is useless if obtaining it costs more than reading all the data. Andy pushed on the implications, asking whether decompression time can be a proxy for how much interesting structure a representation contains, something he called depth, and whether the deepest representation would therefore maximize computation time. Yaroslav shared a paper he liked at arXiv:2601.03220.

On metrics, Daria raised wanting an energy analogue to MFU, an MEU, plus a sense of what counts as SOTA for LLM training and inference, noting that 25 percent or more MFU is generally considered a target for training. Yaroslav's framing: with an algorithm in hand you count operations against the theoretical maximum to get MFU, but here the learning task is examined while the algorithm is still being decided, to get a theoretical minimum energy. The thread also touched physical limits, with Gabriel working through Landauer's principle and the thermodynamic floor for a GPT-4 scale training run, and a separate note on finding longitude in the 19th century.

Gabriel ran a small experiment to test two assumptions, that someone with no ML background could use LLMs to run rigorous experiments, and an initial hypothesis on optimizing tokens per joule, sharing a Colab notebook and the makemore names list. Yaroslav posted a concrete makemore task: train on 1000 random names, predict the last 3 characters of another 1000, build a baseline for accuracy and total operations, then improve it. He also linked the MegaKernels post on SM independence, a LeCun video on energy usage, and a Google Doc.

On branding and framing, Seth weighed in on naming, liking gliding but flagging that albatross carries the wrong connotation, and described Sutro as a tower looming over SF. Later in the week Seth asked for an elevator pitch on how Sutro's gradient dissent improves on the more aesthetic optimization Karpathy has been doing with microgpt, linking Karpathy's post. Yaroslav's own pitch was half-joking, a tweet he called a joke only until someone hands him a billion dollars, and he mused about getting Henry to invite him to a reading group in Emeryville.

On process, Anish shared the how to software engineer Feb 2026 piece as related to commit the prompts, not the code. Seth said it was reasonably close to his own practice, while noting he has not settled on which Claude artifacts belong in a repo. Daria revisited Yaroslav's preference for a large group with varied backgrounds attacking the energy problem rather than a small focused one, and asked for examples. There was also a brief open thread on funding shape, with Anish floating crypto, a VC, and a nonprofit, and the question of whether it is just a VC left hanging. Other shared links: a Google Doc from Gabriel, a favorite slide from Anish, and a video Yaroslav was watching.

Open questions left on the table: whether decompression time is a sound proxy for depth, what counts as SOTA MEU, and how to frame the pitch and the funding vehicle.

Sources