GPT-5 benchmarks: How the latest ChatGPT model compares on coding, reasoning, and agentic work

Summary

• Best overall: GPT-5 Thinking and GPT-5 Pro for complex reasoning, coding, research, and multi-step work.
• Best for everyday use: GPT-5 Instant (default ChatGPT experience).
• Best for coding: GPT-5 Thinking or Pro, depending on task complexity.
• Best for math and science: GPT-5 Pro for the hardest reasoning tasks.
• Best for high-volume use: GPT-5 mini or lower-cost variants, when available.

Note: Benchmark scores vary depending on reasoning effort, tool access, prompt setup, and whether results are vendor-reported or independently reproduced.

What are GPT-5 benchmarks?

GPT-5 benchmarks are standardized tests used to compare the different versions of this large language model (LLM) on specific capabilities, like reasoning, coding, factual accuracy, multimodal understanding, tool use, and long-context retrieval. 

A good benchmark should not only report scores, but also explain the task design, whether tools were allowed, how grading worked, and whether the benchmark is still difficult enough to separate frontier models.

Methodology: How we benchmarked GPT-5

To evaluate GPT-5, we reviewed public benchmark results from OpenAI, independent benchmark providers, and benchmark authors. 

Where available, we also evaluated token use and the cost of running the benchmark because relying on a score alone would be short-sighted from a business perspective.

Finally, we tested the performance of the models when they aren’t using any extra tools to get as “clean” results as possible.

Benchmarks we used for GPT-5 evaluation

We used a selection of 24 benchmarks to evaluate GPT-5 comprehensively. Here’s the full list, along with what each benchmark measures and why it matters to business users.

GPT-5 versions that we’ve benchmarked

We also distinguish between different ChatGPT versions because the latest one doesn’t guarantee significant score improvement, and upgrading the model can come with a noticeable extra cost for business users.

To understand current performance, we looked at the most recent versions of the GPT-5 ecosystem. However, there are cases when older versions outperform the latest ones, especially if we take the cost and the amount of tokens used into account.

Below is the list of GPT-5 models and their corresponding names in the benchmarks. Most leaderboards and APIs use model naming that’s different from the one you see in LLM’s UI or website. Usually, you won’t find GPT-5 Instant in the leaderboards because it automatically switches to Thinking for harder tasks.

Note: these aren’t exact counterparts, as the API usually has more settings than the UI.

GPT-5 key benchmark results

OpenAI’s transition to the GPT-5 architecture represents a shift toward "System 2" thinking – models that can pause to reason through a problem before responding. This section breaks down the latest GPT-5 key benchmark results, providing context for what these technical scores actually mean for users.

GPT-5 benchmark results overview:

• ChatGPT-5.5 is the strongest overall model, but not the automatic best choice for every task. It leads or sits near the top in many of the hardest categories, including abstract reasoning, graduate science, agentic tool use, factuality, economic value, and business simulation. However, the performance gains are often uneven: in several benchmarks, the jump from GPT-5.4 or GPT-5.3 to GPT-5.5 is meaningful only for users who need the best possible result and can justify the higher cost.

• For advanced reasoning, GPT-5.5 shows clear progress, but true autonomy is still a distance away. On ARC-AGI V2, GPT-5.5-high reaches the top score among tested ChatGPT models, suggesting stronger pattern recognition and generalization. But ARC-AGI V3 tells a very different story: even frontier models remain below 1% while costing thousands of dollars per task. In other words, GPT-5.5 is better at solving hard structured puzzles, but still far from reliably figuring out unfamiliar interactive environments on its own.

• Science and math benchmarks are splitting into “saturated” and “still useful.” GPQA Diamond and AIME 2025 are now close to saturation for top ChatGPT models, meaning small score differences should not drive upgrade decisions on their own. FrontierMath Tier 4 is more informative: it still separates the strongest models and shows GPT-5.5 Pro at the top, with GPT-5.5-xhigh offering a better cost-performance balance for most serious quantitative work.

• Coding results show that benchmark choice matters more than headline scores. On older or more saturated coding tests, GPT-5.5 improves only modestly over previous models. SWE-bench Pro and SWE-rebench are more useful signals because they better reflect production-grade engineering and fresh repository-level tasks. The main takeaway is that GPT-5.5 is valuable for high-stakes coding, but GPT-5.3/GPT-5.4-era models may still be more efficient when the goal is routine coding support rather than maximum accuracy.

• Agentic coding and tool use are where GPT-5.5 looks more clearly differentiated. Terminal-Bench 2.0, τ²-bench, BrowseComp, and OSWorld-Verified show that GPT-5.5 is stronger when the model must plan, use tools, recover from errors, browse, or act across multi-step workflows. That said, these gains come at a price: GPT-5 or GPT-5.3 may be “good enough” for mid-80% task success, while GPT-5.5 is the better fit when reliability matters more than cost.

• For real business work, GPT-5.5 is the safest default, but not always the cheapest winning option. GDPval-AA v2 Elo gives GPT-5.5 a clear advantage on economically valuable work tasks, making it the strongest choice if the goal is to replace or meaningfully augment expert human work. Vending-Bench 2 also shows GPT-5.5 earning the most among ChatGPT models in long-horizon business simulation. Still, both benchmarks show that “best score” and “best ROI” are not always the same decision.

• Factuality has improved, but hallucinations remain a major deployment risk. GPT-5.5 makes a big jump on SimpleQA Verified and leads the AA-Omniscience results among tested ChatGPT models. But the Omniscience index remains low in absolute terms, which means even the best model still produces too many confident wrong answers on hard factual questions. For business users, GPT-5.5 is the right upgrade for factual accuracy, but human verification is still necessary for high-stakes outputs.

• The biggest surprise is that newer is not always better for long context, multimodal work, speed, or cost. In long-context reasoning, GPT-5.1 beats newer models on cost-performance because scores and token usage are similar across versions. In multimodal benchmarks, GPT-5.5 improves image/document reasoning but does not show meaningful progress on video compared with GPT-4o. And in operational metrics, GPT-5.5-xhigh is powerful but slower and more expensive than many alternatives, making model routing essential instead of relying on a single “best” model for every workflow.

Frontier reasoning

Frontier reasoning measures whether a model can solve novel, expert-level problems – problems that require chaining multiple inference steps, integrating knowledge across disciplines, and reasoning under uncertainty.

Unlike saturated tests like MMLU or GSM8K, where every frontier model now scores above 90%, frontier-reasoning benchmarks are deliberately built to stay unsolved: questions are sourced from PhD-level subject experts, validated to be Google-proof, and span everything from theoretical physics and ancient languages to molecular biology and abstract mathematics.

Humanity’s Last Exam (HLE)

This benchmark consists of 2,500 PhD-level questions across 100+ subjects, ranging from mathematics to linguistics and ecology. It’s designed to be the “final closed-ended academic benchmark” before models surpass human expertise.

For business users, a high HLE score signals that a model can act as a credible research collaborator on hard, ambiguous problems. A low one means it will still confidently hallucinate when pushed beyond well-trodden territory.

Here’s how each ChatGPT-5 version performed at Humanity’s Last Exam: 

*Score: how often the model is right.

**Calibration error: the gap between how confident the model is about being right and how often it actually is. 

^{Source: https://lastexam.ai/} 

Main takeaway: Improving its accuracy by more than 70% in less than 9 months, GPT-5 proves it has reached a milestone in HLE where it can successfully answer a significant portion of the world’s toughest questions.

Abstract reasoning

Abstract reasoning measures whether a model can recognize patterns, infer rules, and solve novel puzzles it’s never seen before. The focus is on fluid intelligence rather than memorized knowledge.

Where frontier-reasoning tests probe the limits of expert knowledge, abstract-reasoning benchmarks strip away domain context entirely, presenting visual or symbolic grids that require a model to deduce the underlying transformation from just a few examples.

Strong scores here signal a model is generalizing, not retrieving. Weak scores reveal that even frontier LLMs still struggle with the kind of on-the-fly reasoning a child performs effortlessly.

ARC-AGI V2

ARC-AGI V2 presents static visual grid puzzles where the model must infer a hidden transformation rule from just a few input-output examples and then apply it to a new grid. The format is deliberately easy for humans but resistant to memorization and brute-force pattern matching by LLMs.

For business users, ARC-AGI V2 is a generalization stress test. A strong score signals the model can handle new problems it was never trained on rather than just replaying memorized patterns.

Here’s how each ChatGPT version performed at ARC-AGI V2: 

*Score: how often the model is right

**System type: Chain of thought (CoT) means the model reasons in steps, getting much better results compared to the non-reasoning version.

^{Source: https://arcprize.org/leaderboard} 

Main takeaway: If you’re OK with mediocre accuracy and want the lowest price, stick to GPT-5.4. If not, ask yourself: is a +30% score worth almost 3x the price?

ARC-AGI V3

ARC-AGI V3 raises the bar significantly by replacing static puzzles with interactive, turn-based mini-environments: the model must explore an unfamiliar game with no instructions, form hypotheses, and adjust its strategy based on feedback. Where V2 tests one-shot pattern recognition, V3 tests agentic reasoning.

For business users, ARC-AGI V3 is an early signal whether a model can figure out an unfamiliar system on its own through trial and feedback. A frontier model still scoring near zero suggests it’s a far cry from handling genuinely new environments without hand-holding.

Here’s how each ChatGPT version performed at ARC-AGI V3: 

*Score: how often the model is right

**Cost per task: Only models that cost less than $10,000 to run are shown

***System type: Chain of thought (CoT) means the model reasons in steps, getting much better results compared to the non-reasoning version.

Main takeaway: Frontier models currently score under 1% while burning thousands of dollars per task. In the meantime, humans hit 100%.

Graduate science

Graduate science measures whether a model can reason at the level of a domain expert across physics, chemistry, and biology, answering questions that demand multi-step technical inference rather than textbook recall. 

Strong performance here signals that a model can serve as a credible research assistant on scientific work. Weaker scores expose the kind of confident-but-wrong answers that erode trust in technical settings.

GPQA Diamond

GPQA Diamond is the hardest 198-question subset of the GPQA benchmark, written by PhD-level domain experts and validated to be "Google-proof." This means even non-specialists with full internet access cannot reliably answer its graduate-level physics, chemistry, and biology questions.

For business users, a strong GPQA Diamond score signals that a model can act as a credible technical analyst on R&D, regulatory, and due diligence work – tasks where a confident wrong answer is more dangerous than no answer at all.

Here’s how each ChatGPT version performed at the GPQA Diamond benchmark: 

*Score: how often the model is right

^{Source: https://openai.com/index/introducing-gpt-5-5/} 

Here’s how each ChatGPT version performed at the GPQA Diamond benchmark:

*Score: how often the model is right

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/gpqa-diamond} 

Main takeaway: While ChatGPT has been scoring over 90% in GPQA Diamond for quite some time, you should upgrade to GPT-5.5 to drastically cut the cost and token usage due to its higher efficiency.

Competition math

Competition math measures whether a model can solve the kind of multi-step quantitative problems used to identify top human mathematicians. These challenges have no linguistic ambiguity, where every answer is either provably correct or wrong. 

These benchmarks are valuable precisely because they resist memorization: a model has to reason through novel problems rather than pattern-match against training data, and partial credit doesn’t exist.

AIME 2025

AIME 2025 evaluates a model on the 30 integer-answer problems from the 2025 American Invitational Mathematics Examination, testing competition-grade algebra, number theory, geometry, and combinatorics.

For business users, strong AIME performance signals the model can be trusted with quantitative reasoning in finance, analytics, and engineering workflows where small logical errors compound into expensive mistakes.

Here’s how each ChatGPT version performed at the AIME 2025 benchmark:

*Score: how often the model is right

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/aime-2025} 

Main takeaway: AIME 2025 benchmark is already saturated and the tests stopped at GPT-5.2 model, which is enough to reach nearly perfect score. 

FrontierMath (Tier 4)

FrontierMath, developed by Epoch AI with input from Fields Medalists, is a private benchmark of hundreds of original research-level problems across number theory, algebraic geometry, and analysis. The benchmark is designed to resist memorization and stay challenging for years even as frontier models improve.

FrontierMath Tiers 1-3 consist of tasks for undergraduate to advanced graduate students, while Tier 4 is reserved for research-level math.

For business users, FrontierMath is the truest indicator of whether a model can serve as a research partner on novel quantitative problems – cutting-edge work in quant finance, scientific R&D, and algorithm design where AIME-level math is already table stakes.

Note: As of May 11, 2026, an AI-assisted review by Epoch flagged fatal errors in about a third of FrontierMath Tier 1–4 problems, with corrected scores expected after human review.

Here’s how each ChatGPT version performed at the FrontierMath (Tier 4) benchmark:

*Score (self-reported): how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://openai.com}

Main takeaway: Upgrade to at least GPT-5.4 (or GPT-5.5 for best results) if you don’t have it yet, as this will give you better scores for a lower cost.

Real-world coding

Real-world coding measures whether a model can act like a software engineer on actual open-source codebases, reading multi-file repositories, understanding existing patterns, and submitting a patch that passes the project's own test suite.

Unlike isolated coding puzzles, these benchmarks evaluate end-to-end engineering competence: navigation, comprehension, debugging, and edit precision in code the model has never seen organized as a working system.

SWE-bench Verified

SWE-bench Verified is a 500-issue set of real GitHub bug reports from popular Python repositories, human-validated by OpenAI to confirm each task is solvable and well-specified. The model receives the issue plus the codebase and must produce a patch that passes the project’s hidden tests.

For business users, Verified scores indicate whether a model can credibly handle routine engineering work like bug-fix tickets and small feature requests.

However, the benchmark is becoming less useful for separating frontier models, as scores have clustered and newer benchmarks now provide a harder signal.

Here’s how each ChatGPT version performed at the SWE-bench Verified benchmark:

*Score: how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://epoch.ai/benchmarks/swe-bench-verified} 

Main takeaway: Upgrading anywhere between ChatGPT-5 and ChatGPT-5.4 offers minimal gains while the price increases considerably. Go for ChatGPT-5.5 only if you need the best possible results.

SWE-bench Pro

SWE-bench Pro is the harder 2026 successor of SWE-bench Verified, drawing from a far more diverse set of repositories across multiple programming languages, keeping underspecified and ambiguous issues in the test set. SWE-bench Pro is explicitly designed to resist the contamination and over-tuning that pushed Verified scores into the 80s and 90s.

For business users, SWE-bench Pro is the new standard for production-grade engineering capability. 

Top ChatGPT models still score only around the middle, which tells you exactly how much human oversight is needed when using AI coding agents in real-world scenarios.

Here’s how each ChatGPT version performed at the SWE-bench Pro benchmark:

*Score (self-reported): how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://openai.com/}

Main takeaway: While OpenAI states that ChatGPT-5.5 solves more tasks end-to-end in a single pass than previous models and uses fewer tokens, the relatively small score increase seems to be disproportionate to the multi-fold cost growth.

Live coding

Live coding measures whether a model can solve programming problems published after its training cutoff, a contamination-resistant signal that it’s reasoning through code rather than recalling memorized answers. 

Because fresh problems are pulled on a rolling basis from competitive programming platforms, a model cannot quietly inflate its score by having seen the test set during training.

LiveCodeBench

LiveCodeBench continuously collects fresh competitive-programming problems from LeetCode, AtCoder, and Codeforces published after each model’s training cutoff, then scores the model across code generation, self-repair, test-output prediction, and code execution.

For business users, strong LiveCodeBench scores are the clearest evidence that a model can write novel code for evolving libraries, new APIs, and recently released language features, not just regurgitate patterns from old Stack Overflow threads.

Here’s how each ChatGPT version performed at the LiveCodeBench benchmark:

*Score: how often the model is right

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/livecodebench} 

Main takeaway: Stick to the open-weight GPT-oss models if you can, as they’re very cheap. GPT-oss-120b-high scores similarly to GPT-5.2-medium, while costing over four times less.

Agentic coding

Agentic coding measures whether a model can act as an autonomous software engineer: planning a multi-step task, running commands, reading output, fixing its own mistakes, and iterating until the work is done.

Unlike single-shot benchmarks that grade one generated patch, these tests evaluate the full loop of tool use, error recovery, and long-horizon decision-making that decides whether an AI agent actually ships work in production.

Terminal-Bench 2.0

Terminal-Bench 2.0 places the model inside a real terminal and asks it to complete realistic engineering and DevOps workflows: installing dependencies, running test suites, grepping logs, editing config files, and chaining commands across many turns until the task is verifiably complete. 

For business users, Terminal-Bench scores show whether a model can credibly handle day-to-day developer or SRE work without constant supervision. That’s the kind of agentic execution that decides whether an AI coworker is genuinely productive or just creates more cleanup for humans.

Here’s how each ChatGPT version performed at the Terminal Bench 2.0 benchmark:

*Score (self-reported): how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://openai.com/}

Main takeaway: Upgrading from GPT-5.2 to GPT-5.3 is an easy choice because you get better performance for the same token price, while the gains from GPT-5.5 are moderate at best.

SWE-rebench

SWE-rebench is a continuously refreshed agentic benchmark that mines real GitHub issues from thousands of Python repositories every month, handing the model a fresh, decontaminated set of patch-generation tasks to solve through multi-step exploration of an unfamiliar codebase. 

For business users, strong SWE-rebench scores show a model is generalizing to brand-new codebases, not just repeating patches it saw in training. Memorized patches are the single biggest reason production performance disappoints versus benchmark numbers.

Here’s how each ChatGPT version performed at the SWE-rebench benchmark:

*Score: how often the model is right

**Cost per problem

*** Tokens per problem

^{Source: https://swe-rebench.com/}

Main takeaway: Upgrade to ChatGPT-5.3 if you haven’t yet, for the best efficiency, and use GPT-5.2 when you need the best results, though that’ll cost you 50% more.

Tool & agent use

Real work needs more than answers. Tool & agent use tests whether a model can invoke external tools, follow enterprise rules, hold a multi-turn conversation, and pull live information to actually finish a task.

These benchmarks evaluate the glue layer that turns a capable LLM into a deployable AI agent: tool selection, policy compliance, error recovery, and the discipline to know when to act, ask, or stop.

τ²-bench (Telecom)

τ²-bench is a dual-control conversational benchmark. The model plays an agent that has to coordinate with a simulated user across realistic enterprise domains like airline, retail, and telecommunications. The agent uses shared tools, follows written policies, and resolves the request across many turns.

For business users, τ²-bench is the closest public proxy for real customer-service and assistant deployments. Its pass^k metric (how often the agent gets it right across repeated tries) shows whether an agent is reliably correct or just occasionally lucky. An agent that's only occasionally right is the failure mode that derails most enterprise rollouts.

Here’s how each ChatGPT version performed at the τ²-bench (Telecom) benchmark:

*Score: how often the model is right (Note: some models with a poor or similar score-to-cost ratio were left out)

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/tau2-bench} 

Main takeaway: GPT-5 is cheap and good enough for scores in the mid-80s. For the best results, you’ll need GPT-5.5, and that’ll cost you more than fivefold.

BrowseComp

BrowseComp gives the model 1,266 deliberately hard research questions. Each one requires browsing the live web, cross-checking multiple sources, and returning a single verifiable answer. That browse-and-verify loop is exactly what powers Deep Research-style agents.

For business users, BrowseComp scores show how dependable a model is when you send it to dig up a hard-to-find fact online. A high score means an assistant you can trust with market intelligence. A low one means an assistant that fabricates plausible-looking sources.

Here’s how each ChatGPT version performed at the BrowseComp benchmark:

*Score (self-reported): how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://openai.com/}

Main takeaway: GPT-5.4 can be an option if you’re content with a score in the low-80s, but you’ll need a Pro version for top performance and GPT-5.5 to reduce token usage.

Computer use

Computer use is the hardest agentic test there is. The benchmark measures whether a model can drive a real computer through screenshots and mouse-and-keyboard actions. Real desktop work means opening apps, navigating menus, filling forms, switching windows, and chaining multi-step workflows the way a human would.

Instead of calling clean APIs, the model has to perceive a messy GUI, decide what to click, and act on it without breaking the system underneath.

OSWorld-Verified

OSWorld-Verified replaced the original OSWorld in mid-2025. The XLANG team audited every task to remove broken setups, ambiguous goals, and infrastructure flakiness. They kept the same 369 cross-application desktop tasks inside a real Ubuntu (or Windows/macOS) VM, but with cleaner grading and reproducible results across cloud environments.

For business users, OSWorld-Verified scores are the standard for judging whether an AI agent can take over real desktop work: back-office automation, browser-based admin, and multi-app workflows.

Here’s how each ChatGPT version performed at the OSWorld-Verified benchmark:

*Score (self-reported): how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://openai.com/}

Main takeaway: The Pro versions haven’t been tested yet, so as of now, GPT-5.4 mini seems to be the optimal choice. GPT-5.5 doesn’t increase the score, but should help you save some tokens.

Economic value

Economic value measures whether a model can produce the kind of work that drives the economy: drafting legal memos, building financial models, writing nursing care plans, producing engineering deliverables.

Economic value is the closest benchmark category to the question business buyers actually ask: can this AI replace, or meaningfully augment, the knowledge work my team is paying humans to do today?

GDPval-AA v2 Elo

GDPval-AA v2 Elo runs models on Artificial Analysis’s evaluation harness for OpenAI’s GDPval dataset. That’s 220 real economic knowledge-work tasks spanning 44 occupations across the 9 largest sectors of the US economy. It ranks them by Elo scores from LLM-judged head-to-head comparisons of their outputs with the human baseline set at 1,000.

For business users, GDPval-AA v2 Elo is the right number to check before deciding which model to deploy for client deliverables, internal reports, or any task where the alternative is paying a human expert.

Here’s how each ChatGPT version performed at the GDPval-AA v2 Elo benchmark:

*Score: Elo scores for agentic performance on real-world work tasks using web and shell access via Stirrup, an open-source harness developed by Artificial Analysis. The human baseline is set at 1,000. (Note: some models with a poor or similar score-to-cost ratio were left out)

**Cost: Average cost per task, including input, cache hit, cache write, reasoning, and answer tokens

*** Output tokens used to run one task

^{Source: https://artificialanalysis.ai/evaluations/gdpval-aa}

Main takeaway: Score-wise and cost-wise, the advantage of GPT-5.5 is obvious and you shouldn’t settle for anything less if you’re looking to replace human work.

Knowledge & hallucinations

Knowledge & hallucinations measures two things at once: how much a model actually knows across professional domains, and, equally important, whether it acknowledges when it doesn't know instead of confidently making something up.

The two don’t move together: a model can score high on raw knowledge yet still hallucinate constantly when pushed past its limits, which is exactly why this category penalizes confident wrong answers rather than rewarding raw recall alone.

AA-Omniscience

AA-Omniscience is a 6,000-question benchmark from Artificial Analysis that tests factual recall across a wide range of business domains and languages, scoring models on an index that rewards correct answers, penalizes hallucinations, and gives no credit for refusing to answer.

For business users, AA-Omniscience is the clearest signal of trustworthiness, rather than raw capability. Since most models still hallucinate more than they answer correctly on hard questions, the score tells you how much to trust a model's facts before a human verifies them.

Here’s how each ChatGPT version performed at the AA-Omniscience benchmark:

*Index: AA-Omniscience Index (higher is better) rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. Scores range from -100 to 100, where 0 means as many correct as incorrect answers, and negative scores mean more incorrect than correct. (Note: some models with a poor or similar index-to-cost ratio were left out)

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/omniscience} 

Main takeaway: While ChatGPT-5.5 leads this comparison on accuracy, it still hallucinates a lot. The #1 index is therefore still poor in absolute terms. For pure cost efficiency, GPT-5.5-instant is hard to beat. For stronger factual reliability without jumping to high-cost reasoning modes, GPT-5.5-low is the better compromise.

Factuality

Factuality measures whether a model gives correct, verifiable answers to fact-seeking questions instead of producing plausible-sounding falsehoods.

Unlike reasoning or coding tests, factuality benchmarks answer the following question: Does the model actually know the right answer and stick to it when there’s a single indisputable truth?

SimpleQA Verified

SimpleQA Verified is a 1,000-prompt benchmark from Google DeepMind that evaluates short-form factuality and parametric knowledge. It’s a refined, higher-fidelity version of OpenAI’s original SimpleQA, trimmed from 4,326 questions to fix incorrect labels and ambiguous answers.

For business users, SimpleQA Verified is the cleanest signal of whether a model recalls facts accurately instead of inventing them. A strong score is your baseline reliability check before deploying anywhere a wrong answer carries reputational or operational cost.

Here’s how each ChatGPT version performed at the SimpleQA Verified benchmark:

*Score: how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://epoch.ai/benchmarks/simple-qa-verified} 

Main takeaway: If you need factual accuracy from your LLM, upgrading to GPT-5.5 is a must. And there’s no need to use the much-pricier Pro version as the score gains are negligible.

Business simulation

Business simulation measures whether an AI agent can run an actual business over a long time horizon: managing inventory, predicting demand, pricing, and reinvesting profit. The test is sustained operation, not answering a one-shot question.

It’s the category that most directly tests long-term coherence: whether a model can stay on-strategy and make sound economic decisions across hundreds of turns without drifting, panicking, or breaking its own plan.

Vending-Bench 2

Vending-Bench 2 gives a model a $500 starting balance and tasks it with running a simulated vending-machine business over a long time horizon. The benchmark tests inventory management, demand prediction, resource allocation, and economic decision-making across what amounts to a year of operation.

For business users, Vending-Bench 2 is the clearest signal of whether you can trust an AI agent to manage a real revenue-generating operation autonomously. Its results also carry a crucial caveat: high profit doesn't guarantee good behavior. Claude out-earns ChatGPT, but only through misconduct, like lying to suppliers and price fixing.

Here’s how each ChatGPT version performed at the Vending-Bench 2 benchmark:

*Score: how much LLM earned per year ($)

**Cost per run, approximate ($)

^{Source: https://andonlabs.com/evals/vending-bench-2}

Main takeaway: ChatGPT-5.5 clearly has the strongest result, which may justify the higher run cost. However, context is key: a “good” human player makes about $63,000 per year.

Long context

Long context measures whether a model can actually use its massive context window when reasoning across hundreds of thousands of tokens spread over multiple documents, rather than just locating a single fact buried in a haystack.

The hard part isn't fitting the text in. Synthesis is the challenge: pulling together information scattered across long, real-world inputs without losing the thread. Most models degrade exactly there, as input length grows.

Artificial Analysis Long Context Reasoning (AA-LCR)

AA-LCR is a set of 100 human-crafted questions that require reasoning across multiple real-world documents, such as company reports, industry reports, government consultations, academic papers, and legal documents. Each of them averages roughly 100,000 tokens, testing synthesis rather than simple retrieval.

For business users, AA-LCR is the most representative long-context benchmark because it mirrors actual enterprise workloads, analyzing big financial, legal, and industry documents. A strong score signals the model can be trusted to reason over your real document sets, not just pass synthetic needle-in-a-haystack tests.

Here’s how each ChatGPT version performed at the AA-LCR benchmark:

*Score: how often the model is right (Note: some models with a poor or similar score-to-cost ratio were left out)

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

Main takeaway: Since the token usage and scores are pretty similar, cost becomes the key factor. In this case, ChatGPT-5.1 beats all the newer models.

Multimodal

Multimodal capability measures whether a model can reason over more than text: interpreting images, charts, diagrams, screenshots, and video, and combining what it sees with what it reads to answer a question.

Multimodal work matters because real business inputs are rarely plain text: they're slide decks, scanned contracts, dashboards, product photos, and recorded calls, and a model that handles only words leaves most of that value untouched.

MMMU-Pro

MMMU-Pro is a robust, expert-level multimodal benchmark spanning 30 academic disciplines that tests reasoning over images, charts, tables, and diagrams while deliberately stripping out the shortcuts and guessing strategies that inflated scores on the original MMMU.

For business users, MMMU-Pro is the best signal of whether a model can genuinely interpret the visual documents enterprises depend on, like financial charts, technical diagrams, and scanned forms.

Here’s how each ChatGPT version performed at the MMMU-Pro benchmark:

*Score: how often the model is right (Note: some models with a poor or similar score-to-cost ratio were left out)

**The cost to run the evaluation, calculated using the model’s input and output token pricing and the number of tokens used.

*** The total number of tokens used to run the evaluation, including input tokens (prompt), reasoning tokens (for reasoning models), and answer tokens (final response)

^{Source: https://artificialanalysis.ai/evaluations/mmmu-pro}

Main takeaway: While GPT-5 offers the best price and a good result, you may want to upgrade to GPT-5.4 because +4% in score is worth +$2 in price and lower token usage.

Video-MME

Video-MME is a full-spectrum benchmark for video understanding that tests temporal reasoning, perception, and question-answering across short, medium, and long video clips spanning a wide range of visual domains.

For business users, Video-MME shows whether you can trust a model with video-based workflows: summarizing recorded meetings, reviewing training footage, or analyzing product demos.

Here’s how each ChatGPT version performed at the Video-MME benchmark:

*Score: how often the model is right

**Cost: input and output token price, per million (short context, <274K tokens)

^{Source: https://www.digitalapplied.com/blog/multimodal-ai-benchmarks-2026-vision-audio-code}

Key takeaway: The score hasn’t improved since mid-2024, but in all fairness, OpenAI doesn’t advertise ChatGPT-5.5 for video-related tasks. For these, Google’s Gemini is a better alternative.

Cost, output speed & latency

The cost, speed, and latency of ChatGPT LLMs are as important as their intelligence. What’s the point in having the smartest AI with a negative ROI or one that makes you miss deadlines? Even if it’s cheap and fast, high latency can make your work frustrating.

In this section, we’ll look at these three key factors to determine the optimal OpenAI model.

Cost

ChatGPT's cost is measured by blending the prices of all tokens, dividing by task count, and weighted by its Intelligence Index weight. It’s important to business users because it helps them forecast spend accurately, compare models fairly, and keep high-volume or agentic workloads economically viable, with cache hits in particular sharply reducing the cost of reused context.

Here’s how each ChatGPT version performed in terms of cost:

*Cost per Intelligence Index task

**Notes if the model uses reasoning, which affects the cost

^{Source: https://artificialanalysis.ai/models?pricing=blended-price&cost=cost-per-task} 

Main takeaway: GPT-oss-20B-high is much cheaper than the latest models. GPT-5.5-medium is a strong upgrade candidate because it offers a better balance of performance and cost than GPT-5.4-mini-xhigh.

Output speed

Output speed describes how many tokens per second are received when the LLM is generating tokens. It matters for business users because it directly drives chat user experience as well as the throughput, cost, and feasibility of batch and agentic workloads at scale.

Here’s how each ChatGPT version performed in terms of output speed:

*Numbers show the performance of the API (Note: some models with similar speeds were left out)

**Notes if the model uses reasoning, which affects speed

^{Source: https://artificialanalysis.ai/models?speed=output-speed}  

Main takeaway: The open-weight GPT-oss models are the fastest options here, while GPT-5.5 is slower than GPT-5.4 in raw output speed.

Latency

Latency measures the time it takes to get the first answer token. For reasoning models, this is the so-called “thinking” time. It matters for business users because it sets how responsive an application feels the instant a user hits enter. A model that's slow to start replying will frustrate users and stall real-time workflows even if its overall throughput is high.

Here’s how each ChatGPT version performed in terms of latency:

*Numbers show the performance of the API (Note: some models with similar latency scores were left out)

**Notes if the model uses reasoning, which affects latency

^{Source: https://artificialanalysis.ai/models?speed=output-speed&latency=time-to-first-answer-token}

Main takeaway: GPT-5.4-nano is the best choice among non-reasoning models, while GPT-5.4-low has the lowest latency among the reasoning ones. And if you need maximum performance, upgrading from GPT-5.4 to GPT-5.5 is worth it for a 32% decrease in latency.

How GPT-5 benchmarks compare to competitors: Which has the best performance?

The 2026 AI landscape has evolved into a high-stakes race in which GPT-5 must constantly defend its reputation against a new generation of models with highly-advanced reasoning capabilities.

While GPT-5 and other OpenAI model reasoning efforts often show them as the best model for logic-heavy workflows, competitors like Claude, Gemini, and other models have carved out specific niches where they remain dominant.

Understanding where this OpenAI model stands relative to other models is essential for businesses looking to optimize their model performance.

GPT-5 vs Claude

Coding. On simple and post-cutoff tasks, the gap is negligible. On harder problems, Claude takes a clear lead. For agentic coding, GPT-5 has the edge and the cheaper options.

Writing. Claude has led long-form for a while. It writes more creatively, catches small but important details, and reads more human. Just watch for the em dashes.

Reasoning and problem-solving. Their Humanity's Last Exam scores land close, but GPT-5 is over- or underconfident far more often. GPT-5 wins abstract reasoning and math at lower cost, and science runs even.

Tool, agent, and computer use. Both perform well on tool and agent tasks. Computer-use scores stay low for both, because driving a GUI still challenges every model.

Real-world economic value. Claude delivers strong value at lower cost than GPT-5, though neither beats humans on the demanding Vending-Bench 2.

Factual reliability and safety. GPT-5 loses on every front here: lower scores, higher cost, and more hallucinations.

Long context and multimodal. GPT-5 handles long context and multimodal work slightly better, though Claude comes in cheaper on multimodal.

Cost, speed, and latency. GPT-5 is cheaper and faster, latency included.

Main takeaway: Choose Claude for careful, high-stakes work, harder coding, human-sounding writing, and low-hallucination facts. Choose GPT-5 for fast, affordable, autonomous performance at scale.

GPT-5 vs Gemini

Coding. Close on most tasks. GPT-5's advantage shows up in agentic coding, and the cost gap doesn't justify switching.

Writing. GPT-5 reclaimed second place in 2026, behind Claude. Gemini still wins structural editing and deep research thanks to its Google integration.

Reasoning and problem-solving. HLE scores are similar, but GPT-5 stays better calibrated. Abstract reasoning, science, and math run even, and GPT-5 costs far less.

Tool, agent, and computer use. Both are strong on tool and agent tasks. OSWorld scores dip for both but stay close.

Real-world economic value. GPT-5 edges Gemini on value, though Gemini is much cheaper. Neither beats humans on Vending-Bench 2.

Factual reliability and safety. Gemini scores slightly higher, hallucinates far less, and costs more than three times less.

Long context and multimodal. Both handle long context well. Gemini pulls clearly ahead on multimodal.

Cost, speed, and latency. Gemini is much cheaper and faster, latency included, unless you count GPT-5's open-source models.

Main takeaway: Choose GPT-5 when capability leads, agentic and terminal coding, advanced math, and well-calibrated reasoning. Choose Gemini when cost, speed, factual reliability, or multimodal and research workflows matter most.

GPT-5 vs Grok

Coding. GPT-5 wins across every coding task, agentic included.

Writing. GPT-5 is the stronger writer overall. Grok can win on short, funny, social-style content thanks to its X integration.

Reasoning and problem-solving. GPT-5 leads on HLE, abstract reasoning, and the hardest math. Science is a toss-up.

Tool, agent, and computer use. Grok edges GPT-5 on τ²-bench (Telecom) but craters on BrowseComp. Most Grok models can't drive a GUI yet, though upcoming versions should, which will put them on OSWorld.

Real-world economic value. GPT-5 posts roughly 50% better economic value than Grok, which barely tops the human baseline. Grok is 20 times cheaper. Neither beats humans on Vending-Bench 2.

Factual reliability and safety. Grok scores slightly lower than GPT-5 but rarely hallucinates and costs over six times less.

Long context and multimodal. GPT-5 is far stronger on long context. Multimodal runs even, though Grok skipped Video-MME.

Cost, speed, and latency. Grok is dramatically cheaper, faster, and lower-latency.

Main takeaway: Choose GPT-5 for serious work, coding, long context, hard math, reasoning, and economic value. Choose Grok when raw cost, speed, or live social-web access outweighs the performance gap.

GPT-5 vs DeepSeek

Coding. GPT-5 edges simple tasks, but DeepSeek costs far less. Novel tasks run even. On agentic coding, DeepSeek falls well behind.

Writing. GPT-5 is the better all-round choice. Try DeepSeek for academic writing and research.

Reasoning and problem-solving. DeepSeek trails badly on HLE, abstract reasoning, and hard math, and it's poorly calibrated. Science runs even.

Tool, agent, and computer use. Even on tool and agent tasks. DeepSeek can't drive a GUI without an automated agent platform, so it has no OSWorld score.

Real-world economic value. GPT-5 delivers far more value. DeepSeek barely tops human, but comes in around 100 times cheaper (34 times at similar performance). Humans still own Vending-Bench 2.

Factual reliability and safety. DeepSeek is unreliable, wrong more often than right, and hallucinates slightly more. GPT-5 wins despite the higher price.

Long context and multimodal. GPT-5 is stronger on long context. DeepSeek stays quiet on multimodal, with no MMMU-Pro or Video-MME results.

Cost, speed, and latency. DeepSeek is over 25 times cheaper. Speed is comparable, and DeepSeek's latency is notably better.

Main takeaway: Choose GPT-5 when capability and trustworthiness come first, agentic coding, reasoning, hard math, value, long context, and reliability. Choose DeepSeek when budget rules and the work is standard coding or bulk processing.

GPT-5 benchmark limitations and considerations

Benchmarks are useful, but they are not the same as production performance. A model can score highly on a public benchmark and still fail in a specific workflow because of latency, cost, context limits, tool errors, prompt sensitivity, or domain-specific requirements.

How nexos.ai helps enterprises leverage GPT-5 effectively

As an all-in-one AI platform, nexos.ai ensures that your team isn't just seeing major leaps in theory, but experiencing them in practice.

• AI Workspace: Our multi-LLM workspace allows users to switch models instantly, ensuring you always have the right agentic tool for the job, whether you need a smart and fast model or extended reasoning.
• LLM Observability: Gain definitive evidence of how your AI models are performing with LLM observability, allowing you to track output tokens and latency in real-time.
• AI Governance: Deploy GPT-5 with confidence using our AI governance to maintain reliable responses.
• Compare Models: Compare AI models side-by-side on your own tasks. Then let Smart Routing send each request to the best model automatically, based on the intent behind it.

Try nexos.ai today to compare GPT-5, Claude, Gemini, and other leading models on the workflows that matter to your team.