For nearly a decade, the scaling hypothesis was the bible of AI research: larger models, trained on more data with more compute, predictably produce better models. The Kaplan Scaling Laws of 2020 formalized this mathematically. The Chinchilla Scaling Laws of 2022 refined the optimal ratios between parameters and tokens. And empirical results confirmed it: from GPT-3 to GPT-4, each generation was substantially better.

In 2026, this narrative met increasing resistance. Not because models stopped improving — but because the cost for each incremental improvement grew in a way that calls the paradigm's sustainability into question.

The Exponential Cost Problem

The numbers are concrete. GPT-4 cost an estimated US$ 100 million to train. Next-generation models, following the pure scaling paradigm, would require US$ 1 billion or more. Epoch AI projects this cost will continue growing exponentially — and that most organizations training frontier models today will fail to fund the next generation.

More concerning: Lilian Weng, in an analysis published on her influential Lil'Log, detailed that the power-law fits sustaining scaling laws are more sensitive to implementation choices than practitioners acknowledge. Results are replicable in controlled conditions — but actual training runs involve failed experiments, hyperparameter sweeps, and debugging sessions that can double or triple the reported final cost. The US$ 5.6 million for DeepSeek-V3 excluded infrastructure, experimentation, and failed runs.

The Data Wall

Parallel to the cost problem is the data wall. Epoch AI projects that the stock of high-quality public text available for pre-training will be fully exhausted between 2026 and 2032 at current training rates. Frontier labs are already facing constraints on the budget of unique tokens.

The Chinchilla framework was built assuming infinite unique data. When unique tokens run out, labs must train on repeated data — and research by Muennighoff et al. (2023) showed that repeated tokens do not dilute training efficiency linearly: their value decays exponentially with each repetition, with a constant determining how rapidly each additional pass through the same data loses its return.

DeepSeek as a Proof of Concept for Efficiency

The "DeepSeek Moment" of January 2025 was the event that crystallized the transition. DeepSeek-V3 — 671 billion total parameters, but a Mixture of Experts with only 37 billion active per token — was trained on 14.8 trillion tokens for a reported cost of US$ 5.6 million. It offered competitive performance with GPT-4o. It was MIT-licensed. It was downloadable.

The market impact was immediate: NVIDIA lost approximately US$ 589 billion in market capitalization on January 27, 2025 — the largest single-day value loss in stock market history. In one news cycle, the event invalidated the dominant premise that frontier AI required massive infrastructure controlled by the US.

In 2026, DeepSeek-V3.2 matches GPT-5 across several key benchmarks, runs at roughly a 70% lower cost, and is MIT-licensed. The lesson: architectural innovation can replace brute scale.

The Shift to Inference

The field's strategy has migrated. Instead of "who has more GPUs," the question in 2026 is "who has the most efficient architecture." Fine-tuning pre-trained models with LoRA can cost between US$ 500 and US$ 5,000 — a fraction of the millions needed to train from scratch. Distillation of DeepSeek-R1 to 7B retains 85% of the performance. Meta's Llama 4 and the latest Mistral releases adopted latent attention and MoE strategies similar to DeepSeek.

Scaling laws have not died — but they have been relativized. What scales is no longer just parameters and data: it is architectural efficiency, data quality over quantity, and inference compute over training compute. The 2026 paradigm is efficiency-first, not scale-first.