Table of contents
Key takeaways
- Gemini 3 Pro topped multiple reasoning tests in Google's launch data.
- Gemini 3 Flash neared Pro scores at a lower price per token.
- Deep Think mode lifted preview scores on the hardest reasoning tests.
- Coding leads shift fast, so avoid locking into one model.
- Match each coding task to the right model to control spend.
What is Gemini 3?
Gemini 3 is Google's third-generation model family, built by Google DeepMind with a focus on deeper reasoning and agentic capabilities. The family launched with three parts:
- Gemini 3 Pro. The launch flagship, highly capable on complex tasks and coding.
- Gemini 3 Flash. The faster, lower-cost model for high-volume work.
- Gemini 3 Deep Think. An enhanced reasoning mode for the hardest problems.
Gemini 3 Pro and Flash have a 1M-token context window and take native multimodal input. Those multimodal capabilities cover text, images, video, audio, PDFs, and other formats. At launch, you could reach both through the Gemini app, the Gemini API, and Vertex AI. Deep Think followed a separate path. Google first limited it to safety testers, then opened it to Google AI Ultra subscribers, and later offered selective API access. Google also launched Google Antigravity, an agent-first development platform, alongside Gemini 3 Pro. If you remember the Gemini vs Gemini Advanced split, the paid tiers now include Google AI Pro and Google AI Ultra.
Gemini 3 benchmarks at a glance
Google reported Gemini 3 Pro outperforming Gemini 2.5 Pro across its major launch benchmarks. The model also led on several headline reasoning tests at launch. Here's what each benchmark measures:
- Humanity's Last Exam. Expert-level academic reasoning.
- GPQA Diamond. Graduate-level science questions.
- ARC-AGI-2. Visual puzzles that test novel problem-solving.
- SWE-bench Verified. Fixing real GitHub issues.
- MMMU-Pro. Multimodal reasoning across text and images.
Gemini 3 Pro, Flash, and Deep Think compare like this across five benchmarks, using Google's published launch figures. [2] Deep Think scores are November 2025 preview results, and Google has since upgraded the mode:
Benchmark | What it tests | Gemini 3 Pro | Gemini 3 Flash | Gemini 3 Deep Think |
|---|---|---|---|---|
Humanity's Last Exam (no tools) | Expert reasoning | 37.5% | 33.7% | 41.0% |
GPQA Diamond | Science knowledge | 91.9% | 90.4% | 93.8% |
ARC-AGI-2 | Visual puzzles | 31.1% | N/A | 45.1% (with code execution) |
SWE-bench Verified | Agentic coding | 76.2% | 78.0% | N/A |
MMMU-Pro | Multimodal reasoning | 81.0% | 81.2% | N/A |
The Deep Think ARC-AGI-2 score uses code execution, so it isn't the same setup as the Pro score.
Deep Think posts the highest score in every row where it appears, and Flash trails Pro by less than four points on Humanity's Last Exam, GPQA Diamond, and MMMU-Pro while beating it on SWE-bench Verified. A single benchmark score rarely predicts how a model handles your codebase or documents. Reading a leaderboard well means knowing how LLM benchmarks work, because the same model can look strong on one test and average on another.
PRO TIP: Treat leaderboards as a shortlist, then test the top candidates on your own prompts.
Gemini 3 Pro benchmarks
Gemini 3 Pro benchmarks cover five areas: reasoning, math, multimodal understanding, long context, and agentic tasks. Together they show the model's ability to solve complex problems. Each area below explains what the score means for your work.
Reasoning benchmarks
According to Google's published results, the model scores 37.5% on Humanity's Last Exam without tools. With search and code execution, the score rises to 45.8%. [2]
On GPQA Diamond, the model's ability to answer graduate-level science questions shows in a 91.9% score.
ARC-AGI-2 is the result that drew the most attention. The model scores 31.1% there, versus 4.9% for Gemini 2.5 Pro. The ARC Prize benchmark tests novel problem-solving, not memorized knowledge, so a jump this big suggests stronger reasoning on unfamiliar problems.
Math benchmarks
The model scores 95% on AIME 2025 without tools and 100% with code execution. On MathArena Apex, it scores 23.4%. [2] MathArena Apex matters because it uses harder, newer competition problems, which reduces the chance of training-data overlap.
Multimodal benchmarks
Gemini 3 Pro's multimodal understanding shows in three tests of multimodal reasoning across images, video, and screens. Each benchmark below measures one of the model's multimodal capabilities.
- MMMU-Pro. The score is 81%. The test measures reasoning across text and images.
- Video-MMMU. The score is 87.6%. [2] The test measures knowledge gained from video.
- ScreenSpot-Pro. The score is 72.7%. The ScreenSpot-Pro benchmark measures screen understanding for computer-use agents, specifically how well a model grounds actions in an interface.
Google's launch examples, such as turning handwritten recipes into a cookbook, show possible applications. The benchmarks don't directly establish success on those workflows, and ScreenSpot-Pro covers interface grounding, not end-to-end task completion.
Long-context benchmarks
On MRCR v2 at 128k context, the model averages 77%. [1] The model supports larger inputs, up to the 1M-token window, but retrieval quality must be tested at the lengths your workload uses. Context limits matter most for long contracts, large codebases, and multi-hour transcripts.
Agentic benchmarks
Agentic tests show whether a model can handle multi step workflows, decision making, and long horizon planning with external tools. Here are the results.
- Vending-Bench 2. A business simulation that tests long horizon planning. The model reported a mean net worth of about $5,478.
- τ²-bench. A tool-use test. Google reported 85.4% at launch. [2] The February 2026 Gemini 3.1 Pro model card lists Retail 85.3% and Telecom 98.0%. [1]
- LMArena text leaderboard. An Elo of 1501 at launch. [2] The ranking reflects user preference.
Arena rankings change weekly and reflect preference, not accuracy.
Gemini 3 Pro coding benchmarks
Gemini 3 Pro coding benchmarks show how the model handled repository fixes, command-line work, competitive programming, and web development. Each result tells a developer something different.
- SWE-bench Verified. The score is 76.2%. [2] The test shows how well a model can generate code that resolves real repository issues.
- Terminal-Bench 2.0. The score is 54.2% in Google's November 2025 launch post. [2] The February 2026 Gemini 3.1 Pro model card lists 56.9% with the Terminus-2 harness. [1] Terminal-Bench measures how well a model operates in a command line.
- LiveCodeBench Pro. The Elo is about 2,439. [1] The test uses competitive programming problems, so it shows how well the model can generate code from scratch under pressure.
- WebDev Arena. The Elo is about 1487. [2] Developers vote on web apps the model builds, which makes it a useful read on vibe coding.
Google positioned Google Antigravity as a place where agents get access to the editor, terminal, and browser, with support for planning, execution, and validation. Google says that setup lets agents plan and execute complex coding tasks, but long agentic jobs still need human review. Strong code execution and consistent tool usage help agents get further on their own, but they don't remove the need for oversight.
Coding scores among frontier models are close. At launch, Claude Sonnet 4.5 (77.2%) and GPT-5.1 (76.3%) were within about one point on SWE-bench Verified in Google's comparison table. Each new model has kept moving the numbers since. The best model for coding this quarter may not hold the spot next quarter, and no single benchmark performance number settles it. The best LLM for coding changes often, and AI for developers works best when your setup can follow it.
Gemini 3 Flash benchmarks
Gemini 3 Flash benchmarks show how much capability you got from the lower-cost, faster model at its December 2025 launch. Google reported that Flash performed close to the Pro version on several tests:
- GPQA Diamond. Flash scores 90.4%.
- Humanity's Last Exam. The score is 33.7% without tools.
- MMMU-Pro. The score is 81.2%.
- SWE-bench Verified. The score is 78%.
On SWE-bench Verified, Flash's reported score is higher than Pro's. That's a good example of why "bigger" doesn't always mean "better for the task."
How Gemini 3 Flash compares with Claude Haiku 4.5 on SWE-bench
Gemini 3 Flash scores 78% on SWE-bench Verified (Google-reported), and Claude Haiku 4.5 scores 73.3% (Anthropic-reported). Both are the smaller, cheaper tiers of their families. The two vendors used different test harnesses, so treat the roughly 5-point gap as directional.
Gemini 3 Flash and Claude Haiku 4.5 compare like this on SWE-bench Verified:
Metric | Gemini 3 Flash | Claude Haiku 4.5 |
|---|---|---|
SWE-bench Verified | 78.0% | 73.3% |
Model tier | Fast, low-cost | Fast, low-cost |
Both models give you strong coding scores at a lower price than their flagship siblings.
How Gemini 3 Flash compares with Gemini 2.5 Pro
Google reported Gemini 3 Flash outperforming Gemini 2.5 Pro, the previous flagship, on most published benchmarks. On GPQA Diamond, Flash scores 90.4% versus 86.4%. On SWE-bench Verified, it scores 78% versus 59.6%.
Flash did this at a lower price. At launch, Gemini 3 Flash cost $0.50 per million text, image, and video input tokens and $3 per million output tokens. Audio input cost $1 per million tokens.
Gemini 3 Deep Think benchmarks
Gemini 3 Deep Think is a mode that spends more compute reasoning through several hypotheses before it answers. Deep Think mode is slower, and Google first offered it to Google AI Ultra subscribers after a safety-testing phase. Google's November 2025 preview results put Deep Think at 41.0% on Humanity's Last Exam without tools, 93.8% on GPQA Diamond, and 45.1% on ARC-AGI-2 with code execution. Google announced a major Deep Think upgrade on February 12, 2026, so these aren't its latest scores. [5]
Deep Think's gains come from combining deeper reasoning with longer thinking time, which means more latency and more cost per answer. Most everyday tasks don't need it. Deep Think works best as a research tool for problems where an extra few points of accuracy pay off, like research questions or tough proofs. Ultra subscribers got direct access first.
How Gemini 3 Pro compares with Gemini 2.5 Pro
Google reported Gemini 3 Pro outperforming Gemini 2.5 Pro across its major launch benchmarks, which makes it a big generational jump. The previous model trails across reasoning, coding, and terminal tasks.
Gemini 3 Pro and Gemini 2.5 Pro compare like this across five benchmarks, using Google's November 2025 launch figures. [3] The Terminal-Bench 2.0 row shows the original 54.2% launch score for Gemini 3 Pro:
Benchmark | Gemini 2.5 Pro | Gemini 3 Pro |
|---|---|---|
Humanity's Last Exam (no tools) | 21.6% | 37.5% |
GPQA Diamond | 86.4% | 91.9% |
ARC-AGI-2 | 4.9% | 31.1% |
SWE-bench Verified | 59.6% | 76.2% |
Terminal-Bench 2.0 | 32.6% | 54.2% |
The biggest gains show up in abstract reasoning (ARC-AGI-2) and agentic coding (SWE-bench and Terminal-Bench). Humanity's Last Exam climbed by almost 16 points, which marks improved reasoning across the board. If you still run Gemini 2.5 Pro in production, these numbers make a strong case for testing a newer Gemini model.
How Gemini 3 compares with GPT-5
Gemini 3 Pro led GPT-5.1 and Claude Sonnet 4.5 on most reasoning tests in Google's launch table, and it trailed slightly on SWE-bench Verified. The table didn't include the original GPT-5, whose own GPT-5 benchmarks come from OpenAI's test settings. The same caution applies to the Grok 4 benchmarks that xAI reported itself.
Gemini 3 Pro, GPT-5.1, and Claude Sonnet 4.5 compare like this in Google's November 2025 launch table, using the same launch-era Terminal-Bench 2.0 score for Gemini 3 Pro:
Benchmark | Gemini 3 Pro | GPT-5.1 | Claude Sonnet 4.5 |
|---|---|---|---|
Humanity's Last Exam (no tools) | 37.5% | 26.5% | 13.7% |
GPQA Diamond | 91.9% | 88.1% | 83.4% |
ARC-AGI-2 | 31.1% | 17.6% | 13.6% |
SWE-bench Verified | 76.2% | 76.3% | 77.2% |
Terminal-Bench 2.0 | 54.2% | 47.6% | 42.8% |
In the launch table, Gemini 3 Pro posts the top score in four of the five rows, while Claude Sonnet 4.5 posts the top score on SWE-bench Verified, with GPT-5.1 within a point. Open-weight models such as DeepSeek-V3.2 also publish scores on overlapping benchmarks. Direct comparisons across vendors' self-reported tables should be read with care.
Gemini 3 Pro speed benchmarks
Independent trackers like Artificial Analysis measure Gemini 3 Pro speed separately from Google's accuracy tables, typically as output tokens per second and time to first token. In its November 2025 launch assessment, Artificial Analysis reported Gemini 3 Pro Preview generating 128 output tokens per second. [4] That put it ahead of GPT-5.1 (high), Kimi K2 Thinking, and Grok 4 in that assessment. The same assessment placed the model first on the Artificial Analysis Intelligence Index, 3 points above GPT-5.1. Those figures describe launch, not current throughput or ranking. Throughput varies by provider, region, and prompt length, and user reports differ, so check live numbers before you plan around them.
Launch pricing adds the cost side. Pro cost $2 input and $12 output per 1M tokens for prompts up to 200k tokens, and $4 and $18 above that. Flash was positioned for high-volume, latency-sensitive work. Pro was built for harder tasks.
What Gemini 3 benchmarks don't tell you
Launch scores don't tell you how the models handle your own code, documents, or edge cases. Safety evaluations and factual accuracy deserve their own checks. Keep these five limits in mind:
- Vendor-reported scores. Most figures come from Google, with its own settings and harnesses.
- Contamination risk. Public test sets can leak into training data.
- Scaffolding differences. Agentic scores depend heavily on the tools and prompts used.
- Factuality gaps. High reasoning scores don't eliminate hallucinations. The model scores 72.1% on SimpleQA Verified and 70.5% on the FACTS Benchmark Suite. [2] Artificial Analysis's November 2025 launch assessment also reported an 88% hallucination rate on AA-Omniscience. [4] That metric counts incorrect answers among all non-correct responses, abstentions included, so it isn't the share of all answers that were wrong.
- Your workload isn't a benchmark. Internal code, docs, and edge cases behave differently.
Good LLM evaluation closes these gaps by testing on your own data. Teams that want to reduce AI hallucination risk should check factual accuracy on their own documents before rollout.
How engineering teams can use Gemini 3 for coding
Engineering teams can use Gemini 3 for coding once they've settled three questions: cost, lock-in, and task fit. Strong scores made it an obvious candidate, but a team rollout needs more than scores. nexos.ai is an all-in-one AI platform that gives your team unified access to leading models, including Gemini. The sections below cover each question.
Tracking Gemini 3 spend per developer
You can track Gemini 3 spend per developer by getting usage and cost data across teams and models. A model can be great, and nobody knows what it costs until the invoice lands. If you're the CTO or engineering lead signing the bill, you need per-developer and per-team spend on Pro. nexos.ai gives you visibility into usage and spend across teams and models, so AI usage tracking stops being a guess. LLM observability adds the detail, and solid AI cost management starts there.
A useful cost view shows:
- Spend per developer. See who uses Pro the most.
- Spend per team or project. Tie costs to the work that earns them.
- Spend per model. See how Pro and Flash usage differ.
Swapping models without rebuilding your setup
You can swap models without rebuilding your setup by keeping workflows, prompts, and agents separate from the model underneath. Today's best model for coding likely won't hold that spot in six months, because each new model often reshuffles the leaderboard. The benchmark tables above already show how close the leaders are. A model-agnostic setup means a model switch doesn't touch your team's work.
Fallbacks protect you too. When one provider has an outage, requests route to another model, so your team isn't stalled. Requests reach the best-fit model automatically with smart LLM routing, and the nexos.ai AI gateway gives your team one entry point to all of it.
PRO TIP: Pick a primary and a backup model for each critical coding workflow before you need one. That way, an outage or a price change is a config switch, not a migration project.

Run every AI model in one Gateway
One secure, lightweight layer to access, route, and manage 200+ models.

Monitor AI usage

Control AI spend

Optimize AI costs
Matching coding tasks to the right model
You should match coding tasks to the right model by sending hard problems to a frontier model and routine work to a cheaper tier. Five common coding tasks map to model tiers like this:
Coding task | Suggested tier | Why |
|---|---|---|
Large refactoring | Frontier (e.g., Pro) | Needs cross-file reasoning |
Tricky debugging | Frontier (e.g., Pro) | Multi-step root-cause analysis |
Boilerplate code | Cheaper (e.g., Gemini 3 Flash) | Predictable patterns |
Unit tests | Cheaper (e.g., Gemini 3 Flash) | Well-defined scope |
Docs and comments | Cheaper (e.g., Gemini 3 Flash) | Low reasoning load |
Routing routine work to a cheaper tier like Gemini 3 Flash and reserving Pro for hard problems can meaningfully cut spend. That split is the core of how to reduce AI costs for work, and the FrugalGPT approach applies the same idea at research scale.
How to test Gemini 3 against other models on your own tasks
You can test Gemini 3 against other models by running real prompts from your team's work across several models and scoring the results. Follow these three steps:
- 1.Pick 10 to 20 real prompts from your team's work.
- 2.Run them through Pro, Gemini 3 Flash, and one or two other models using the same prompts.
- 3.Score the answers on accuracy, speed, and cost.
nexos.ai gives your team access to leading models, including Gemini, so you can run step two in one AI workspace for multiple LLMs. Browse the AI models available in nexos.ai, check nexos.ai pricing, or contact Sales to set up your team.
Sources
[1] Google DeepMind, Gemini 3.1 Pro model card, February 19, 2026 (includes Gemini 3 Pro comparison column): https://deepmind.google/models/model-cards/gemini-3-1-pro/
[2] Google, "A new era of intelligence with Gemini 3" (November 18, 2025): https://blog.google/products-and-platforms/products/gemini/gemini-3/
[3] Google DeepMind, Gemini 3 Pro model evaluation: https://storage.googleapis.com/deepmind-media/gemini/gemini_3_pro_model_evaluation.pdf
[4] Artificial Analysis, "Gemini 3 Pro: Everything you need to know" (November 2025): https://artificialanalysis.ai/articles/gemini-3-pro-everything-you-need-to-know
[5] Google, "Gemini 3 Deep Think: AI model update designed for science" (February 12, 2026): https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-deep-think/
[6] Google AI for Developers, Gemini 3 Pro Preview deprecation notice: https://ai.google.dev/gemini-api/docs/models/gemini-3-pro-preview
Run every AI model in one Gateway
One secure, lightweight layer to access, route, and manage 200+ models.
Run every AI model in one Gateway
One secure, lightweight layer to access, route, and manage 200+ models.