Schmidhuber Problems

A reproducible-baseline catalog of the synthetic learning problems that appear in Jürgen Schmidhuber’s experimental papers from 1989 through 2025 — implemented in pure numpy, runnable on a laptop CPU, with paper-comparison metrics per stub.

• GitHub: https://github.com/cybertronai/schmidhuber-problems
• Site: https://cybertronai.github.io/schmidhuber-problems/
• Catalog: RESULTS.md
• Visual tour: VISUAL_TOUR.md
• Build notes: BUILD_NOTES.md
• Status: 58 of 58 stubs implemented (PRs #4–#16, all merged 2026-05-08)

Introduction

The field has standardized on backprop by the end of the ’80s, and Hinton gives a sample of problems that were used at the time. In the last 20 years, we have transitioned to GPUs, and the math has changed considerably. Instead of being bottlenecked by arithmetic, the shrinking of transistors means that arithmetic is essentially free, and all of the work comes from data movement. Backprop is inefficient in terms of “commute to compute ratio” because it requires fetching all of the activations for each gradient add.

So a natural experiment would be to redo key experiments of this time with a focus on data movement. The first step is to get a baseline — to establish the list of problems which are famous, reasonable to implement, and easy to run/reproduce.

— Yaroslav, hinton-problems issue #1 (Sutro Group)

This repository is the algorithmic-lineage companion to hinton-problems.

• Hinton’s catalog emphasizes representational toy tasks: small benchmarks where hidden-unit inspection is the experimental payoff (4-2-4 encoder, family trees, shifter, Forward-Forward MNIST).
• Schmidhuber’s lineage emphasizes algorithmic capability. Four threads run through this catalog:
  • Long-time-lag indexing: 1990 flip-flop → 1992 chunker → 1996 adding-problem → 1997 temporal-order
  • Key-value binding: 1992 fast-weights → 2021 linear Transformers (the same outer-product math, 29 years apart)
  • Kolmogorov-complexity search: 1995 Levin search → 2003 OOPS (program enumeration, no gradients)
  • Controller + model + curiosity loops in tiny stochastic environments: 1990 pole-balance → 2018 World Models

v1 + v1.5 ship 58 implementations covering this lineage from the 1989 NBB through the 2022 Neural Data Router. Each stub is a self-contained folder with model + train + eval + visualization + animated GIF, all in numpy, all runnable in <5 min per seed on an M-series laptop.

What’s here

Pure numpy + matplotlib throughout. Every stub runs on a laptop CPU. Each problem lives in its own folder with <slug>.py (model + train + eval), README.md (8 sections: Header / Problem / Files / Running / Results / Visualizations / Deviations / Open questions), make_<slug>_gif.py, visualize_<slug>.py, an animated <slug>.gif, and a viz/ folder of training curves and weight visualizations.

Per the SPEC’s RL-stub rule, RL/env-heavy stubs (pole-balance-*, pomdp-flag-maze, world-models-*, torcs-vision-evolution, upside-down-rl, double-pole-no-velocity) use numpy mini-environments that capture the algorithmic claim of the original paper, not the original simulator. The substitution is documented in each stub’s §Deviations. Original-simulator reruns are tracked as v2 follow-ups.

Development

This repository includes a minimal Nix development shell with Python and NumPy:

nix develop
python3 nbb-xor/nbb_xor.py --seed 0

Or run one stub directly without Nix (assumes python3 -m pip install numpy matplotlib):

cd flip-flop
python3 flip_flop.py --seed 0
python3 visualize_flip_flop.py
python3 make_flip_flop_gif.py

Visual tour

nbb-xor — Schmidhuber 1989 NBB local rule on XOR. The wave-0 sanity validator: WTA + bucket-brigade dissipation, no backprop.	flip-flop — Schmidhuber 1990 controller + differentiable world-model on the canonical LSTM-precursor latch.
linear-transformers-fwp — Schlag/Irie/Schmidhuber 2021. Linear-attention V^T(Kq) ≡ 1992-FWP (V^T K)q to 2.22e-16 (float64 ulp).	world-models-carracing — Ha & Schmidhuber 2018 V+M+C on a numpy 2D track. Returns +103.8 mean across 5 seeds (random +4.84).

For the long-form picture-first walk through all 58 stubs — every GIF, organized by era, with notes on what each visualization is meant to show — see VISUAL_TOUR.md.

Catalog

Each table shows the v1 result per stub. Full per-stub metrics (run wallclock, headline numbers, implementation budget) are in RESULTS.md.

Reproduces? legend: yes = matches paper qualitatively or quantitatively; partial / qualitative = method works, paper-config gap documented in stub README; no = paper claim does not replicate (gap analysis documented).

1980s — Local rules and the Neural Bucket Brigade

Schmidhuber (1989) — A local learning algorithm for dynamic feedforward and recurrent networks (FKI-124-90 / Connection Science)

1990 — Controller + world-model + flip-flop

Schmidhuber (1990) — Making the world differentiable (FKI-126-90 / IJCNN-90)

Schmidhuber (1990) — Recurrent networks adjusted by adaptive critics (NIPS-3)

Schmidhuber & Huber (1990) — Learning to generate focus trajectories (FKI-128-90)

1991 — Curiosity, subgoals, the chunker

Schmidhuber (1991) — Adaptive confidence and adaptive curiosity (FKI-149-91)

Schmidhuber (1991) — Learning to generate sub-goals for action sequences (ICANN-91)

Schmidhuber (1991) — Reinforcement learning in Markovian and non-Markovian environments (NIPS-3)

Schmidhuber (1991/1992) — Neural sequence chunkers / Learning complex extended sequences using the principle of history compression

1992 — Neural Computation triple

Schmidhuber (1992) — Learning to control fast-weight memories (NC 4(1))

Schmidhuber (1992) — Learning factorial codes by predictability minimization (NC 4(6))

1993 — Predictable classifications, self-reference, very deep chunking

Schmidhuber & Prelinger (1993) — Discovering predictable classifications (NC 5(4))

Schmidhuber (1993) — A self-referential weight matrix (ICANN-93)

Schmidhuber (1993) — Habilitationsschrift, Netzwerkarchitekturen, Zielfunktionen und Kettenregel

1995–1997 — Levin search and the LSTM benchmark suite

Schmidhuber (1995/1997) — Discovering solutions with low Kolmogorov complexity (ICML / NN 10)

Hochreiter & Schmidhuber (1996) — LSTM can solve hard long time lag problems (NIPS 9)

Hochreiter & Schmidhuber (1997) — Long Short-Term Memory (NC 9(8)) — canonical 6-experiment battery

Mid-90s — Evolutionary, RL, and feature detection

Salustowicz & Schmidhuber (1997) — Probabilistic Incremental Program Evolution

Schmidhuber, Zhao, Wiering (1997) — Shifting inductive bias with SSA (ML 28)

Wiering & Schmidhuber (1997) — HQ-learning (Adaptive Behavior 6(2))

Schmidhuber, Eldracher, Foltin (1996) — Semilinear PM produces well-known feature detectors (NC 8(4))

Hochreiter & Schmidhuber (1999) — Feature extraction through LOCOCODE (NC 11)

2000–2002 — LSTM follow-ups

Gers, Schmidhuber, Cummins (2000) — Learning to forget (NC 12(10))

Gers & Schmidhuber (2001) — Context-free and context-sensitive languages (IEEE TNN 12(6))

Gers, Schraudolph, Schmidhuber (2002) — Learning precise timing (JMLR 3)

Eck & Schmidhuber (2002) — Blues improvisation with LSTM (NNSP)

2002–2010 — Evolutionary RL, OOPS, BLSTM+CTC

Schmidhuber, Wierstra, Gomez (2005/2007) — Evolino

Gomez & Schmidhuber (2005) — Co-evolving recurrent neurons (GECCO)

Graves et al. (2005/2006) — BLSTM and Connectionist Temporal Classification

Graves, Liwicki, Fernández, Bertolami, Bunke, Schmidhuber (2009) — Unconstrained handwriting (TPAMI)

Schmidhuber (2002–2004) — Optimal Ordered Problem Solver (ML 54)

2010–2017 — Deep learning at scale

Cireşan, Meier, Gambardella, Schmidhuber (2010) — Deep, big, simple nets (NC 22(12))

Cireşan, Meier, Schmidhuber (2012) — Multi-column deep neural networks (CVPR)

Cireşan, Giusti, Gambardella, Schmidhuber (2012) — EM segmentation (NIPS)

Srivastava, Masci, Kazerounian, Gomez, Schmidhuber (2013) — Compete to compute (NIPS)

Srivastava, Greff, Schmidhuber (2015) — Training very deep networks (NIPS)

Greff, Srivastava, Koutník, Steunebrink, Schmidhuber (2017) — LSTM: a search space odyssey (TNNLS)

Koutník, Greff, Gomez, Schmidhuber (2014) — A clockwork RNN (ICML)

Koutník, Cuccu, Schmidhuber, Gomez (2013) — Vision-based RL via evolution (GECCO)

Greff, van Steenkiste, Schmidhuber (2017) — Neural Expectation Maximization (NIPS)

van Steenkiste, Chang, Greff, Schmidhuber (2018) — Relational Neural EM (ICLR)

2018–2025 — World models, fast-weight Transformers, systematic generalization

Ha & Schmidhuber (2018) — Recurrent World Models Facilitate Policy Evolution (NeurIPS)

Schmidhuber et al. (2019) — Reinforcement Learning Upside Down (arXiv)

Schlag, Irie, Schmidhuber (2021) — Linear Transformers are secretly fast weight programmers (ICML)

Csordás, Irie, Schmidhuber (2022) — The Neural Data Router (ICLR)

Structure

problem-folder/
├── README.md                  source paper, problem, results, deviations
├── <slug>.py                  dataset + model + train + eval
├── visualize_<slug>.py        training curves + weight viz (writes to viz/)
├── make_<slug>_gif.py         animated GIF (writes <slug>.gif)
├── <slug>.gif                 committed animation
└── viz/                       committed PNGs

Methodological caveat

Many of the early TUM technical-report PDFs (FKI-124-90, FKI-126-90, FKI-128-90, FKI-149-91, the 1993 Habilitationsschrift, Hochreiter’s 1991 diploma thesis) are difficult to retrieve in original form. Stub READMEs reconstruct the experiments from corroborated secondary sources — Schmidhuber’s Deep Learning: Our Miraculous Year 1990–1991 (2020), the 1997 LSTM paper’s literature review, the 2001 Hochreiter/Bengio/Frasconi/Schmidhuber chapter Gradient Flow in Recurrent Nets, the 2015 Deep Learning in Neural Networks survey, and IDSIA HTML transcriptions where available — and flag claims that rest on secondary citation rather than verbatim quotation.

Schmidhuber vs Hinton: what’s different

The companion catalog hinton-problems emphasizes representational toy tasks: small benchmarks (4-2-4 encoder, family trees, shifter) designed to expose what kind of internal representation a network develops. Hidden-unit inspection is the experimental payoff.

Schmidhuber’s lineage emphasizes algorithmic capability: long-time-lag indexing (flip-flop, chunker, adding, temporal-order, a^n b^n c^n), key-value binding (1992 fast-weights → 2021 linear Transformers), Kolmogorov-complexity search (Levin → OOPS), and controller+model+curiosity loops in tiny stochastic environments (1990 pole-balance → 2018 World Models). The signature methodological move is the controlled difficulty sweep — (q=50, p=50) → (q=1000, p=1000) in the 1997 LSTM paper, the 5,400-experiment grid in the 2017 Search Space Odyssey.

Roadmap

• v2: ByteDMD instrumentation — measure data-movement cost per stub on these baselines (the actual research goal). The 58 implementations here are the substrate the data-movement cost tracer will run against.
• Original-simulator reruns — RL/env-heavy stubs in v1+v1.5 use numpy mini-environments per the SPEC’s RL-stub rule. v2 follow-ups will close the loop on the original simulators (gym CarRacing-v0, VizDoom DoomTakeCover, TORCS, TIMIT, IAM, ISBI).
• See Open questions / next experiments section in each stub README for stub-specific follow-ups.

Contributing

Implementations follow the v1 spec:

• Each stub fills in <slug>.py (model + train + eval), an 8-section README.md, make_<slug>_gif.py, visualize_<slug>.py, an animated <slug>.gif, and viz/ PNGs.
• Acceptance: reproduces in <5 min on a laptop; final accuracy with seed in Results table; GIF illustrates problem AND learning dynamics; “Deviations from the original” section honest; at least one open question.
• v1 metrics in PR body: "Paper reports X; we got Y. Reproduces: yes/no." + run wallclock + implementation budget.
• Algorithmic faithfulness: implement the actual algorithm the paper introduces (NBB local rule, RS over weight space, Levin search, BPTT through LSTM, peephole LSTM, PIPE on PPT, ESP co-evolution, FWP outer-product writes, etc.) — not a backprop shortcut.
• Pure numpy + matplotlib only. torchvision allowed for MNIST/CIFAR loaders; gymnasium / gym not allowed (use numpy mini-envs per the RL-stub rule).

License

Released into the public domain under the Unlicense.