Researchers announced a new reinforcement-learning framework called Ring-Zero that lets a trillion-parameter language model learn to reason while staying within token-budget limits, and the model scored 84.2 % on the 2026 AIME math exam. The result shows that, at this scale, the model can acquire the step-by-step problem-solving habits engineers have traditionally hard-coded into prompts.

Why it matters

AIME-level performance has long been a benchmark for “human-level” quantitative reasoning. Hitting the 84.2 % range without any human-written examples suggests the model’s reasoning abilities emerge from the training dynamics themselves, not from hand-crafted scaffolds. For companies building AI agents, the implication is clear: writing and maintaining elaborate prompt recipes could be replaced by a training loop that teaches the model when to think harder and when a cheap shortcut will do.

From prompt engineering to training dynamics

For years, developers relied on prompt templates such as “break the problem into parts” or “verify your answer” to coax large language models into multi-step reasoning. Those templates act as external scaffolding; the model follows the instructions but does not internalize the process. Ring-Zero flips that paradigm. The model receives a task description together with a verifiable answer—a ground-truth that can be automatically checked. It then generates a series of attempts, receives a reward only when the attempt matches the verifiable answer, and updates its policy through reinforcement learning.

Because the reward ties to an objective that is hard to cheat (the answer must be correct, not just plausible), the model discovers reasoning patterns on its own. The paper argues that once a reliable reward signal is in place, scaling the model to a trillion parameters automatically surfaces the kinds of prompt-engineering tricks that engineers have manually inserted for smaller models.

The four-step training pipeline

Ring-Zero’s training proceeds through four distinct stages:

  1. First-stage RL – The model explores possible reasoning traces, learning which token sequences tend to lead to correct answers.
  2. Self-distillation – High-quality traces flow back into the model, stabilizing the emerging reasoning patterns.
  3. Second-stage RL – A finer-grained loop refines the policy while preserving earlier improvements.
  4. Tiered training – The model learns to allocate token budgets intelligently, choosing between short (≈2 k tokens) and long (≈20 k tokens) reasoning paths depending on problem difficulty.

Tiered training is the most production-relevant piece. In real deployments, each additional token adds compute cost. By teaching the model to recognize when a problem is simple enough for a short answer and when it warrants a deep, multi-step analysis, Ring-Zero directly ties reasoning quality to economic efficiency.

Two phases of learning: discovery and sharpening

The authors describe the learning curve as a two-phase process:

  • Discovery – Early reinforcement-learning steps are noisy; the model tries a wide variety of strategies, many of which fail. This variance is essential for uncovering novel reasoning pathways.
  • Sharpening – Once a promising pattern surfaces, later RL steps tighten the policy, reducing variance and solidifying the behavior.

The progression mirrors how humans improve: initial brainstorming followed by focused practice. The key insight is that the “messy” early stage should be embraced rather than suppressed, because it supplies the raw material for later refinement.

What could go wrong?

The paper’s optimism hinges on a few assumptions that merit scrutiny:

  • Reward design – The framework needs a reward that is both verifiable and resistant to shortcutting. For many real-world tasks, defining such a reward is non-trivial and may require costly annotation pipelines.
  • Environmental bottlenecks – The authors point out that the model’s performance is limited not by its size but by the surrounding evaluation infrastructure (test suites, logs, clear metrics). Building and maintaining that infrastructure can be a substantial engineering effort.
  • Compute cost of training – Training a trillion-parameter model with multiple RL stages is expensive. While tiered training cuts inference costs, the upfront training budget remains high, potentially restricting the approach to well-funded labs.

Що подивитися далі

Практичні поради для фахівців зі штучного інтелекту

  • Не сприймайте промпти як єдиний важіль впливу – Запитайте себе, чи не можна замість кодування поведінки в промптах вивчити її через цикл навчання на основі винагороди.
  • Зробіть використання токенів пріоритетною ціллю – Додавайте сигнали з урахуванням бюджету на ранніх етапах; модель навчиться балансувати між точністю та витратами на обчислення.
  • Замикайте цикл зворотного зв'язку – Впроваджуйте систему, яка автоматично надає завдання, порівнює результати з перевіреними відповідями та передає ці результати назад у навчання. Саме в цьому циклі модель відкриває власний робочий процес.

Коли винагорода чітка, а середовище може надійно вимірювати успіх, масштабування моделі робить більше, ніж просто збільшує обчислювальну потужність — воно вчить модель обирати спосіб мислення. Цей перехід від вручну створених структур до самокерованого мислення може змінити те, як ми створюємо та оцінюємо вартість ШІ-агентів.