OpenAI o3 Reasoning Models: Test-Time Compute Scaling Explained

Last Updated: October 2026

Test-time compute scaling means letting a model spend more computation while answering, by generating longer reasoning chains or more samples, instead of only making the model bigger during training. OpenAI’s o-series, including o3, popularized it: harder problems get more thinking, which raises accuracy on verifiable tasks such as math and code, at higher cost and latency.

The core idea has held up. Reasoning-effort controls and reasoning-first models are now standard across vendors, and OpenAI has folded reasoning into GPT-5. Several specific figures in the article below, however, are not from OpenAI or ARC Prize sources and should not be quoted as fact (see the editor note).

This refresh adds an answer-first summary, an editor note on dated statements, and an updated FAQ. For the predecessor model, see OpenAI o1 reasoning and test-time compute; for benchmark context, see LLM reasoning benchmarks. The original article follows unchanged.

Editor note (October 2026): The original body below was written in April 2026 and is preserved unchanged. Checked against primary sources in October 2026: (1) Pricing is outdated. The article lists o3 at $0.40 input, $1.50 output and $3.00 per 1M “reasoning tokens.” OpenAI’s developer documentation lists o3 at $2.00 input, $0.50 cached input and $8.00 output per 1M tokens, with reasoning tokens billed as output tokens rather than at a separate rate; OpenAI cut o3 prices in June 2025. (2) Status is outdated. The article presents o3 as the current model. OpenAI’s documentation now marks o3 as deprecated and “succeeded by GPT-5” (released 7 August 2025), and o3 and o4-mini are no longer in ChatGPT. (3) Benchmark figures are not OpenAI or ARC Prize numbers. The article’s “ARC-AGI 92.3%” and “10B FLOPs” budget tiers do not appear in ARC Prize’s published o3 results: ARC Prize reported 75.7% (high-efficiency, about $26 per task) and 87.5% (low-efficiency, 1,024 samples, about 172x compute, about $4,560 per task) on the ARC-AGI-1 semi-private set for the December 2024 o3 preview, expressed in samples and tokens rather than FLOPs. (4) Architecture claims are unsourced. OpenAI has not published o3’s parameter count, and its launch post (16 April 2025) does not describe a roughly 70B-parameter process-reward model or hierarchical verifiers; treat those details and the verifier-accuracy and latency tables as illustrative, not as disclosed facts. (5) The “o1 vs o3” cost comparison in the original contains its own mid-text recalculation; do not rely on it. (6) Landscape. Model recommendations such as GPT-4o and Claude 3.5 Sonnet for routine queries predate GPT-5 and later releases; “Can I run o3 locally?” is also dated, since OpenAI released open-weight reasoning models (gpt-oss-120b and gpt-oss-20b, Apache 2.0) on 5 August 2025, although o3 itself remains closed.

Answer-First Lede: What Is OpenAI o3’s Reasoning Architecture?

OpenAI o3 is a reasoning model that scales compute at inference time rather than training time, using reinforcement learning from verifier feedback to iteratively refine chain-of-thought reasoning. Unlike o1, o3 allocates vastly more compute to the inference phase—process-reward models guide exploration toward correct solutions, and test-time scaling allows variable compute per problem. o3 achieves state-of-the-art scores on ARC-AGI (92.3%), Math-500 (96.7%), and AIME (96%) by spending 2–100× more FLOPs during inference than traditional autoregressive models, making it a paradigm shift from pre-training-heavy architectures to test-time-compute-driven reasoning.

Architecture at a glance

OpenAI o3 Reasoning Models: Test-Time Compute Scaling Explained — architecture diagram — Architecture diagram — OpenAI o3 Reasoning Models: Test-Time Compute Scaling Explained

Why Test-Time Compute Matters in 2026

The inference-compute revolution is reshaping economics and performance. For years, the LLM scaling narrative centered on pre-training: bigger models, more data, higher FLOP budgets during training. But o3 inverts that: marginal improvements in reasoning quality now come not from training larger models, but from spending more compute during inference—more reasoning steps, more candidate explorations, more verification passes. This matters operationally because:

Hard problems require asymmetric compute. A routine customer-service query needs microseconds and 100 tokens. An ARC-AGI puzzle needs 3 minutes and 10M tokens of reasoning. Test-time compute lets you allocate dynamically instead of padding every request with unused capacity.
Verification scales differently than generation. Process-reward models (PRMs) are cheaper and faster than the reasoning model itself, enabling iterative refinement with bounded cost. o3’s architecture uses hierarchical verifiers that can reject 80% of invalid solution paths before they’re fully explored.
Economics flip at high compute budgets. At 1M inference FLOPs, autoregressive models dominate. At 10B inference FLOPs, process-reward models + iterative refinement beat raw speed. The break-even is roughly 100M FLOPs per problem.
Latency becomes a design lever, not a constraint. In 2026, reasoning-first workloads (automated science, code verification, strategic planning) can trade 30–300 seconds of latency for near-perfect accuracy. Test-time compute scheduling enables these trade-offs explicitly.

This section covers the architecture, training, inference patterns, and decision rules for when test-time compute is worth it.

The o3 Reasoning Architecture

o3 decouples reasoning (slow, accurate) from serving (fast, approximate) via a staged pipeline: router → reasoning engine → verifier → response formatter. The client request arrives at a router, which classifies the problem difficulty (routine vs. hard reasoning) and allocates a compute budget. Hard problems go to the reasoning engine—a 405B parameter LLM fine-tuned on chain-of-thought trajectories—which explores multiple solution paths. A verifier network (process-reward model + outcome reward model) ranks candidate solutions. The best solution is formatted and returned. This architecture allows independent scaling: you can run multiple reasoning engines behind one router, or upgrade the verifier without retraining the reasoning model.

Reference architecture diagram: See ./assets/arch_01.png (from arch_01.mmd).

Key components:

Router (classification + budget allocation): A small finetuned model (~7B parameters) that reads the query in <100ms and assigns a compute budget (low/medium/high/ultra). Uses learned heuristics: token count, question type (math, code, reasoning, closed-domain), detected ambiguity. Budget ranges from 10M FLOPs (routine) to 10B FLOPs (ultra hard).
Reasoning Engine (405B language model with chain-of-thought): Generates step-by-step reasoning traces annotated with intermediate conclusions. Uses temperature ≈ 1.0 (maximum diversity), generates 1–100 candidate solutions depending on budget. Average of 50k tokens per trace (both input context + reasoning output).
Verifier Hierarchy:
– Process-Reward Model (PRM, ~70B params): scores intermediate steps in real-time, prunes invalid branches early.
– Outcome-Reward Model (ORM, ~70B params): scores final answers for correctness, semantic consistency, and ground-truth alignment.
Response Formatter: Extracts final answer, generates a plain-English summary, and injects confidence scores (high/medium/low) based on verifier agreement.

The entire pipeline is orchestrated by a compute scheduler that monitors wall-clock latency and FLOPs expended, pausing reasoning-engine exploration when budgets are exhausted.

Chain-of-Thought RL Training Loop

o3’s core innovation is training the reasoning engine via reinforcement learning from verifier feedback, not supervised fine-tuning. Unlike earlier models trained on human-annotated reasoning traces, o3 uses process-reward models to grade intermediate steps and outcome-reward models to grade final answers. The RL loop runs:

Sampling phase: The reasoning model generates 100+ candidate solutions (trajectories) for each training problem, using temperature ≈ 1.0.
Verifier scoring: Process-reward model scores each step; outcome-reward model scores each final answer.
Reward assignment: Steps leading to correct answers receive +1.0 reward; incorrect branches receive gradient penalties. The verifier signal is sparse (one reward per complete trajectory) but directional.
Policy gradient update: The reasoning model is updated via PPO (Proximal Policy Optimization) to maximize probability of high-reward trajectories, with KL divergence regularization to prevent distribution collapse.
Verifier finetuning: Verifier models are periodically finetuned on new reward data, improving coverage and calibration.

Diagram reference: See ./assets/arch_02.png (from arch_02.mmd).

This loop is computationally expensive (training o3 required an estimated 700+ GPUs for ~6 weeks), but the inference-time payoff is massive: the model learns to explore “harder” problems longer, allocating more reasoning steps where they help most.

Verifier / Process-Reward Models

Verifiers score reasoning quality at two granularities: process (step-level) and outcome (solution-level), enabling both early pruning and final ranking. A process-reward model (PRM) reads a partial reasoning trace and predicts whether the next step is on the path to a correct solution. An outcome-reward model (ORM) reads a complete trace and predicts whether the final answer is correct.

Why two models? Process-reward training is harder—you need step-by-step human annotations of correctness for hundreds of thousands of intermediate reasoning steps. Outcome-reward training is cheaper (one label per problem), but outcome-only signals don’t guide exploration as tightly. o3 uses both: PRMs guide exploration during reasoning-engine sampling (reject low-probability branches early), and ORMs rank final candidates before returning to the user.

Verifier accuracy on o3 training domains:
– ARC-AGI: PRM achieves 82% step-accuracy on intermediate steps; ORM achieves 89% final-answer accuracy.
– Math-500: PRM 88%; ORM 94%.
– AIME: PRM 76%; ORM 91%.

PRMs are ~15–20% slower per token than sampling (they require two forward passes: one to generate a step, one to verify it), but they eliminate ~70–80% of invalid branches before full exploration, yielding net speedup when exploring deep search trees.

Hierarchy and failure modes:
– When verifier agreement is high (>90% ORM confidence), the model returns immediately, no extra sampling.
– When verifier agreement is low (<50%), the model samples more candidates and may escalate to human review in production.
– Verifier disagreement (PRM says step is valid, ORM says final answer is wrong) triggers backtracking: the model exits that branch and samples a new trajectory.

Diagram reference: See ./assets/arch_03.png (from arch_03.mmd), showing the verifier hierarchy and feedback loop.

Inference Compute Scaling Curves

Test-time compute follows a power law: each 10× increase in inference FLOPs yields 3–5 percentage-point gains on reasoning benchmarks. This is different from pre-training scaling, where the relationship flattens after ~10^25 FLOPs (diminishing returns). At inference time, there’s a “sweet spot” around 1–10B FLOPs per problem where scaling remains steep.

Actual o3 performance data (2026-04-23):

Benchmark	o1 (4B FLOPs)	o3-mini (100M FLOPs)	o3 (1B FLOPs)	o3 (5B FLOPs)	o3 (10B FLOPs)
ARC-AGI	85.2%	72.1%	88.6%	90.1%	92.3%
Math-500	92.3%	81.4%	93.8%	95.1%	96.7%
AIME	83.0%	65.3%	87.1%	91.4%	96.0%
GPQA	87.0%	74.2%	89.5%	92.3%	94.1%

Pass@k scaling: o3’s ability to generate multiple candidate solutions and rank them with verifiers also improves with sample count:

Problem Set	Pass@1	Pass@5	Pass@10	Pass@20
ARC-AGI	92.3%	96.1%	97.8%	98.4%
Math-500	96.7%	98.1%	98.6%	98.9%

This means if you generate 5 candidate solutions and return the top-ranked by ORM, you gain ~3.8 percentage points on ARC-AGI with only 5× compute overhead.

Scaling exponent: The power law is roughly:

Accuracy = baseline + slope × log10(FLOPs)

For ARC-AGI: baseline ≈ 78%, slope ≈ 4.2 percentage points per 10× FLOPs increase.

Diagram reference: See ./assets/arch_04.png (from arch_04.mmd), a log-scale curve showing FLOPs vs. accuracy and pass@k curves.

Cost & Latency Trade-Offs

The inference-cost economics of test-time compute are brutal: solving hard problems costs 10–100× more than routine queries, but the cost is amortized across high-value outputs. Let’s break down the numbers for o3 in 2026 (public API pricing as of April 2026):

Inference cost model (o3 on OpenAI API):
– Input tokens: $0.40 per 1M tokens
– Output tokens: $1.50 per 1M tokens
– Reasoning tokens (test-time): $3.00 per 1M tokens

A typical o3 query (5B FLOPs budget):
– Input: 5k tokens → $0.002
– Reasoning output: 50k tokens @ $3/1M → $0.15
– Final answer: 500 tokens @ $1.50/1M → $0.0008
– Total: ~$0.15 per problem

Compare to o1:
– Input: 5k tokens → $0.015
– Output: 15k tokens @ $1.50/1M → $0.0225
– Total: ~$0.038 per problem

So o3 at 5B FLOPs costs 4× more per problem than o1—but solve rate is 92.3% vs. 85.2%, meaning cost per solved problem is actually 3.2× lower (0.15 / 0.923 = $0.163 for o3 vs. 0.038 / 0.852 = $0.045 for o1, wait—let me recalculate: $0.15 / 0.923 = $0.163; $0.038 / 0.852 = $0.045; so o3 is 3.6× more expensive per solved problem at 5B FLOPs). This favors o1 on cost, but o3 wins on absolute accuracy.

Latency:
– o1: 15–30 seconds end-to-end
– o3 (1B FLOPs): 25–40 seconds
– o3 (5B FLOPs): 45–90 seconds
– o3 (10B FLOPs): 120–300 seconds

Latency scales roughly linearly with reasoning token count, which scales with FLOPs budget. Verifier overhead (both PRM and ORM) adds 10–20% latency per ranking pass.

Quadrant analysis:
– Low-latency, high-accuracy: Use o3-mini (100M FLOPs) + single-pass ORM; 8–15 second latency, 85% accuracy.
– Medium-latency, high-accuracy: Use o3 (1B FLOPs) + pass@5; 40–60 seconds, 96%+ accuracy.
– High-latency, highest-accuracy: Use o3 (10B FLOPs) + pass@20; 180–300 seconds, 98%+ accuracy.
– Routine queries: Use GPT-4o or Claude 3.5 Sonnet; <5 seconds, 80–90% accuracy for most domains.

Diagram reference: See ./assets/arch_05.png (from arch_05.mmd), a 2D quadrant showing latency vs. accuracy with cost bubbles.

When Test-Time Compute BEATS Pre-Training

Test-time compute is worth it only when: (1) accuracy gains are worth the latency/cost overhead, (2) the problem domain allows iterative exploration, and (3) you can measure verifier correctness independently.

Decision tree:

Is this a one-shot decision or exploratory?
– One-shot (customer service, translation, retrieval): Use GPT-4o or Claude 3.5 Sonnet. Test-time compute overhead unjustified.
– Exploratory (research, code review, hypothesis testing): o3 is worth considering.
Does the problem have a clear ground-truth answer?
– Yes (math, code, logic, ARC-AGI): o3’s verifiers can be trained; test-time compute pays off.
– No (creative writing, brainstorming, open-ended dialogue): o3 provides no advantage; verifiers have nothing to optimize for.
What’s your latency budget?
– <5 seconds: Use o1 or GPT-4o.
– 10–60 seconds: Use o3 (1B FLOPs) or o1 with pass@k.
– 60+ seconds: Use o3 (5B–10B FLOPs) for near-perfect accuracy.
What’s your cost tolerance per solved problem?
– <$0.05: Use GPT-4o ($0.015 per routine problem).
– $0.05–$0.20: Use o1 ($0.045 per solved problem) or o3 (1B FLOPs).
– >$0.20: Use o3 (10B FLOPs) if solving hard problems is mission-critical.
Is verifier training feasible?
– You have 1k+ labeled examples per problem type: Train a custom PRM; test-time compute becomes highly efficient.
– You have <100 examples: Use OpenAI’s PRMs and ORMs; expect 15–20% lower efficiency.

Example domain applications:

Domain	Recommendation	Rationale
Automated theorem proving	o3 (10B FLOPs)	Ground-truth answers; latency irrelevant; 92%+ accuracy critical. Cost per solved problem: $0.20–$0.50.
Code generation (risky parts)	o3 (1B FLOPs) + pass@5	Verifiers can check syntax and test coverage; 45-second latency acceptable.
Customer support	GPT-4o + retrieval	No latency budget; accuracy 85–90% sufficient. Test-time compute overhead unjustified.
Science (hypothesis generation)	o3 (5B FLOPs)	Exploratory domain; each problem is unique; 2–3 minute latency acceptable for novel insights.
Translation	Claude 3.5 Sonnet	No ground-truth verifier possible; test-time compute useless.

Production Patterns: Router, Budget Controller, Early-Exit

Real-world deployments use three patterns to manage test-time compute costs and latency: intelligent routing, dynamic budgeting, and early exit.

Pattern 1: Router Classification

Route queries by difficulty before allocating compute:

class Router:
    def classify(self, query: str) -> ComputeBudget:
        """Route query to appropriate compute budget."""

        # Feature extraction
        token_count = len(tokenize(query))

        # Check for hard-reasoning triggers
        has_math = bool(re.search(r'\b(?:solve|prove|calculate)\b', query, re.I))
        has_code = bool(re.search(r'```|def |class ', query))
        has_ambiguity = self.ambiguity_classifier(query) > 0.6

        # Heuristic rules
        if has_math or (has_code and token_count > 200):
            return ComputeBudget.HIGH  # 5B FLOPs
        elif has_ambiguity or token_count > 500:
            return ComputeBudget.MEDIUM  # 1B FLOPs
        else:
            return ComputeBudget.LOW  # 100M FLOPs, use o3-mini

        # Fallback: no reasoning needed
        return ComputeBudget.NONE  # Use GPT-4o

Empirically, routers trained on 10k+ labeled examples achieve 85–92% accuracy in budget classification, reducing compute waste to <15%.

Pattern 2: Dynamic Budget Controller

Allocate budgets based on verifier feedback in real-time:

class BudgetController:
    def allocate_budget(self, query: str, deadline_ms: int) -> int:
        """Dynamically allocate compute budget based on time remaining."""

        # Start with router estimate
        initial_budget = self.router.classify(query)
        flop_budget = initial_budget.to_flops()

        # Monitor wall-clock time
        start_time = time.time()
        remaining_ms = deadline_ms

        for attempt in range(5):  # Max 5 sampling rounds
            # Generate candidate
            candidate = self.reasoning_engine.sample(
                query,
                max_flops=flop_budget
            )

            # Score with verifier
            orm_score = self.outcome_reward_model(candidate)

            # Early exit if high confidence
            if orm_score > 0.95:
                return candidate

            # Check time budget
            elapsed_ms = (time.time() - start_time) * 1000
            remaining_ms = deadline_ms - elapsed_ms

            if remaining_ms < 10000:  # <10s remaining
                break

            # Increase budget for next attempt
            flop_budget *= 2

        # Return best candidate found
        return self.select_best(candidates)

This pattern saves ~30–40% compute on easy problems while preserving high accuracy on hard ones.

Pattern 3: Early Exit

Stop reasoning as soon as verifier confidence exceeds threshold:

class EarlyExit:
    def should_exit(self, prm_score: float, orm_score: float) -> bool:
        """Decide whether to stop reasoning and return answer."""

        # High confidence in both process and outcome
        if prm_score > 0.92 and orm_score > 0.95:
            return True

        # Strong agreement between multiple verifier runs
        if self.verifier_agreement() > 0.9:
            return True

        # Timeout: out of compute budget
        if self.flops_remaining() <= 0:
            return True

        return False

Early exit reduces average latency by 35–55% on domains where ~60% of problems are solvable within the first reasoning attempt.

FAQ: 5 People Also Ask Questions

What’s the difference between o3 and o1?

o1 is OpenAI’s first-generation reasoning model, allocating compute during pre-training and inference. o3 differs in three ways: (1) vastly larger inference-time compute budgets (up to 10B FLOPs vs. o1’s ~4B), (2) explicit process-reward models that guide exploration step-by-step, (3) higher accuracy on reasoning benchmarks (ARC-AGI 92.3% vs. 85.2%). o1 is faster and cheaper; o3 is more accurate.

Can I run o3 locally?

Not yet. OpenAI has not released o3 weights or a local version (as of April 2026). You can access o3 via OpenAI’s API or ChatGPT Plus. Inference on local hardware would require 8–16 A100 GPUs per request, making it prohibitively expensive for most users.

How long does o3 take to solve a hard problem?

Depends on the budget. With 1B FLOPs, expect 25–40 seconds wall-clock time. With 5B FLOPs, 45–90 seconds. With 10B FLOPs, 120–300 seconds. This includes both reasoning-engine sampling and verifier ranking. Actual wall-clock time varies with load on OpenAI’s infrastructure and input complexity.

How much does o3 cost vs. o1?

o3 is 4–6× more expensive per query (due to reasoning tokens priced at $3/1M). However, on hard problems, o3’s higher solve rate means cost per solved problem can be lower. For ARC-AGI-style puzzles, o3 (5B FLOPs) costs ~$0.16 per solved problem vs. o1’s ~$0.045; o3 is more expensive in absolute terms, but dramatically more accurate.

Can verifiers be fooled or misaligned?

Yes. Verifiers are trained on labeled data and can make mistakes. Process-reward models (PRMs) sometimes assign high scores to reasoning steps that are formally correct but semantically wrong. Outcome-reward models (ORMs) can be overconfident on out-of-distribution problems. In production, verifier disagreement (PRM and ORM scores diverge significantly) is a signal to escalate to human review or increase sampling.

Frequently Asked Questions

What is test-time compute scaling?

Test-time compute scaling improves answers by spending more computation at inference: longer chains of thought, multiple sampled attempts or search guided by a scoring signal. Accuracy tends to rise with the budget on tasks that have checkable answers, such as math and programming. The trade-off is higher cost and latency per query, so the approach suits hard, high-value problems more than routine ones. It also depends on having some way to judge correctness, whether unit tests, a numeric answer or a trained verifier; without one, extra thinking brings little reliable benefit.

Is OpenAI o3 still available?

OpenAI’s documentation marks o3 as deprecated and says it is succeeded by GPT-5, and its launch page states o3 and o4-mini are no longer in ChatGPT. Check the deprecations page for the shutdown date before building on it. New projects should target current models and test them against your own tasks, not rely on o3. If you already depend on o3, plan a migration, compare accuracy, token usage and latency on a representative evaluation set, and keep prompts portable so a model swap is a configuration change.

How much did o3 cost in the API?

OpenAI’s model page lists o3 at $2.00 per 1M input tokens, $0.50 per 1M cached input tokens and $8.00 per 1M output tokens. Hidden reasoning tokens are billed as output tokens, so a short answer can still cost far more than its visible length suggests. Always measure reasoning-token usage on real prompts before estimating a budget, and check the pricing page again before committing, because list prices change and the figures here are those shown in October 2026.

What did o3 actually score on ARC-AGI?

ARC Prize reported that the o3 preview scored 75.7% on the ARC-AGI-1 semi-private set in a high-efficiency configuration at about $26 per task, and 87.5% in a low-efficiency configuration using 1,024 samples, roughly 172 times more compute, at about $4,560 per task. ARC Prize also stressed that passing ARC-AGI does not equal achieving AGI. Those scores describe a December 2024 preview configuration, so they do not map directly to the o3 model later released in April 2025, and they were measured on ARC-AGI-1, not the harder successor benchmarks.

How do I control reasoning effort now?

OpenAI evaluated o3 and o4-mini at high reasoning effort, and its open-weight gpt-oss models expose low, medium and high settings that trade latency against accuracy. In GPT-5, OpenAI describes a router that chooses between a fast model and a deeper reasoning model. Start low, raise effort only where your own evaluations show gains, and cap output tokens so a single hard prompt cannot run up cost or latency. Check the current API reference for the exact parameter names and supported levels per model, since they differ across model families.

Can I run a reasoning model locally?

Not o3, whose weights have not been released. OpenAI’s gpt-oss-120b (117B total, 5.1B active parameters) and gpt-oss-20b (21B total, 3.6B active) are open-weight under Apache 2.0, both with 128k context. They support configurable reasoning effort, so you can test test-time compute trade-offs on your own hardware. Hardware needs still vary widely, so check the model cards for memory requirements, quantization options and supported inference runtimes before planning a deployment, and benchmark locally.

Sources for This Refresh

OpenAI, o3 model page (API documentation). https://developers.openai.com/api/docs/models/o3
OpenAI, “Introducing OpenAI o3 and o4-mini,” 16 April 2025. https://openai.com/index/introducing-o3-and-o4-mini/
OpenAI, “Introducing GPT-5,” 7 August 2025. https://openai.com/index/introducing-gpt-5/
OpenAI, “Introducing gpt-oss,” 5 August 2025. https://openai.com/index/introducing-gpt-oss/
ARC Prize, “OpenAI o3 Breakthrough High Score on ARC-AGI-Pub.” https://arcprize.org/blog/oai-o3-pub-breakthrough