In Epoch Idle, papers publish automatically when your run hits the historical condition that forced the field to invent them.
The first time a model's loss reaches the noise floor, gradient descent has done its job: the parameters sit on the hidden truth. Every training run since 1958 has been a variation on this moment. In-game, converging banks the ×2.5 epoch bonus and serves a harder dataset.
UNLOCK CONDITION
Publishes when you converge your first dataset.
Minsky and Papert (1969) showed a single line cannot even compute XOR - networks need non-linear activations to bend. The game's E3+ datasets are shaped for exactly this reason: only the matching σ family can converge them.
UNLOCK CONDITION
Publishes when you unlock your first activation.
Curriculum learning trains on progressively harder examples - easy tasks first, structure later - and converges faster than random ordering (Bengio et al., 2009). The epoch ladder is exactly that: a seeded curriculum of ever-harder datasets.
UNLOCK CONDITION
Publishes at epoch 6.
GELU gates each input by its own Gaussian CDF - a smooth, probabilistic ReLU that became the default inside transformers. The game's soft-hinge datasets fit it and nothing earlier: smoothness is a capability, not a detail.
UNLOCK CONDITION
Publishes when you unlock GELU (from epoch 8).
Compute per dollar doubled roughly every two years for half a century - the tide that lifted all of AI before the algorithms caught up. Each hardware tier here is ×10 compute for ×12 cost: the same diminishing-return curve, playable.
UNLOCK CONDITION
Publishes at hardware tier 4.
In 2012 AlexNet's convolutional network halved ImageNet's error and the field pivoted to learned spatial features overnight. Unlocking Conv2D here is that pivot: local experts replace the single global line.
UNLOCK CONDITION
Publishes when you unlock Conv2D (from epoch 11).
Convolution works because structure is local: edges, textures, and motifs repeat across the input, so small receptive windows find them anywhere. Two kernels are the proof of concept - each owns its slice of the domain.
UNLOCK CONDITION
Publishes when your stack grows to 2 kernels.
Depth buys composition: each layer's features become the next layer's inputs, building hierarchies no single layer can express. A full kernel stack tiles the whole domain with experts - the capacity that makes segmented structure fittable.
UNLOCK CONDITION
Publishes with a maxed kernel stack.
An oversized model stops learning the pattern and starts memorizing the noise: training loss falls while held-out loss climbs. The game charges the exact OLS price - 2σ² of sampling error per estimated parameter - so an oversized stack stalls at OVERFIT // VAL GAP until data catches up.
UNLOCK CONDITION
Publishes when your training fit first diverges from validation.
Jitter, flip, crop: augmenting data multiplies effective sample size without collecting anything new, shrinking estimation error directly. It is the honest cure for overfitting - more data, not more parameters.
UNLOCK CONDITION
Publishes on your first AUGMENT purchase.
Rich Sutton's observation: methods that lean on raw compute consistently beat clever hand-engineered ones. Stride here is exactly that trade - buy throughput instead of subtlety, and let the machine out-train the tricks.
UNLOCK CONDITION
Publishes at STRIDE level 5.
ImageNet (14M labeled images) mattered less for its labels than for its scale: big data is what lets big models generalize instead of memorize. The dataset cap here is 64 points - the same tension in miniature.
UNLOCK CONDITION
Publishes when datasets reach 64 samples.
Vaswani et al. (2017) dropped recurrence entirely: stacked self-attention, scaled dot-product similarity, parallel over the whole sequence. Unlocking Multi-Head Attention is the same bet - global, content-based routing instead of fixed local windows.
UNLOCK CONDITION
Publishes when you unlock Multi-Head Attention (from epoch 15).
Attention scores every query against every key, then scales by √d so the softmax stays sharp instead of saturating. Multiple heads each learn their own routing table - coverage no single attention map provides.
UNLOCK CONDITION
Publishes at 2 heads.
Kaplan et al. (2020) found loss falls as a smooth power law in model size, data, and compute - predictably, across nine orders of magnitude. The context lane's quadratic bonus is the game's N² scaling law, compounding on schedule.
UNLOCK CONDITION
Publishes at CONTEXT level 3.
Hoffmann et al. (2022) rebalanced the scaling law: most large models were starved of data - for a fixed compute budget, train a smaller model on more tokens. The paper fires exactly when you balance model size (context) against data (augmentation): the era's lesson, materialized.
UNLOCK CONDITION
Publishes when augmentation and context both grow together.
Some capabilities appear only past a scale threshold - few-shot learning, chain-of-thought - absent one order of magnitude earlier (Wei et al., 2022). Compound the era's lanes far enough and the multiplier simply takes off.
UNLOCK CONDITION
Publishes when the era multiplier passes ×10,000.
Rumelhart, Hinton & Williams (1986) made training deep networks tractable: one backward sweep assigns every parameter its share of the blame. The Backprop Step mirrors it - convert a finished run into gradients, then start the next one knowing everything the last taught you.
UNLOCK CONDITION
Publishes on your first Backprop Step.
Polyak's heavy-ball method adds velocity to gradient descent: steps accumulate direction, gliding through ravines and damping oscillation. In-game it automates parameter training - descent that no longer needs a finger on it.
UNLOCK CONDITION
Publishes when Momentum comes online in the optimizer tree.
Kingma & Ba (2014) combined momentum with per-parameter step-size adaptation and got the field's default optimizer. The apex node does the same for you: full training automation plus the tree's largest multiplier.
UNLOCK CONDITION
Publishes when the Adam apex compiles (both branches maxed).
Checkpoints make training resumable, and loading a frozen model into a new run - transfer learning - became how modern AI is actually built. FREEZE banks each mastered epoch as Model Weights ω: permanent value no reset can touch.
UNLOCK CONDITION
Publishes when you freeze your first checkpoint.
Progress is not smooth - it pivots. When the old architecture plateaus, the field switches paradigms and the curve bends again: perceptrons → backprop → convolution → attention. The Transformer shift is the game's version of 2017: the plateau detected, the multiplier rerouted, the next era opened.
UNLOCK CONDITION
Publishes when you initiate the paradigm shift at the token plateau.
Modality stopped being a boundary when vision transformers began eating images as patch sequences: pixels became tokens, and attention routed across them like any other text (GPT-4V, 2023). The game's own stack is the argument - convolution learned to see, attention learned to read, and the paradigm runs both in one model.
UNLOCK CONDITION
Publishes when convolution and attention have both come online.
A base model predicts text; it does not behave. InstructGPT (2022) showed a small model tuned against human preferences outperforms a giant one on usefulness: character is a fine-tune, not a parameter count. The game's version - the paradigm multiplies power only once the data discipline that keeps it honest has been paid for.
UNLOCK CONDITION
Publishes after the paradigm shift, with augmentation well grown.
o1 (2024) broke a thirty-year assumption: quality is not only a training-budget function. Drafting intermediate steps - chain of thought - turns runtime compute into accuracy, and DeepSeek-R1 (2025) published the recipe. The context lane is your answer-time dial: every extra level is more room to think before answering.
UNLOCK CONDITION
Publishes at CONTEXT level 6.
The 2025 shift was not a bigger model but a busier one: plan, act, check, repeat - the loop that turns a language model into a lab assistant. Your lab runs the same loop on itself: the auto-prune agent realigns the expert stack while you are away, no finger required.
UNLOCK CONDITION
Publishes when the Auto-Prune agent runs after the paradigm shift.
Sparse routing changed the scaling math: Mixtral 8x7B (2023) ran each token through 2 of 8 sub-networks, and DeepSeek-V3 (2024) trained 671B parameters while activating only 37B per token. Capacity stopped costing per-token compute - your stable adds output without joining the estimation burden a bigger stack pays.
UNLOCK CONDITION
Publishes when the first sparse expert joins the routing stable.
The 2026 frontier is widely believed to be a sparse mixture (est.): GPT-6-class systems route every token through specialists instead of running one dense model. A full stable tiles the routing space the way the kernel stack tiles the domain - the paradigm's lesson, completed.
UNLOCK CONDITION
Publishes when the expert stable is full.
Mixtral kept the top-2 router honest; the Switch Transformer (2021) went sparser still - one expert per token - and scaled past a trillion parameters on the budget of a dense tenth. Routing, it turns out, is a curriculum: half a stable already teaches the load balancer most of what a full one knows.
UNLOCK CONDITION
Publishes when the expert stable reaches four specialists.
Attention's N² memory wall broke when the sequence itself was sharded: Ring Attention (2023) passes key-value blocks around a ring of devices and overlaps the transfer with compute, pushing context windows toward the million-token scale. Your context lane is the same dial - every level is more rope for the window to grow into.
UNLOCK CONDITION
Publishes at CONTEXT level 8.
When the public run of high-quality text was exhausted, the labs turned the models loose on the bottleneck: teacher models wrote the textbooks, the exercises, and the grading keys (phi-1, 2023; Self-Instruct, 2022). A frontier-rated model (Elo 1650+) is itself a curriculum engine - the arena is the syllabus.
UNLOCK CONDITION
Publishes when the arena rating reaches 1650.
o1 (2024) did not just think longer - it searched: candidate chains proposed, verified, and pruned until one survived, trading wall-clock for accuracy the weights alone could not reach (AlphaGeometry, 2024, showed the same at olympiad geometry). Past Elo 1800 the rating stops measuring what the model knows and starts measuring what it can find.
UNLOCK CONDITION
Publishes when the arena rating reaches 1800.