Gemini 3.8 Flash Reality Check: Two Terminal-Bench Scores, 61.6% SWE-Bench Pro and Higher Task Cost

What Google actually released

Google released Gemini 3.8 Flash on September 2, 2026 as a generally available model for coding, agentic workflows, multimodal work and complex knowledge tasks. The public model ID is gemini-3.8-flash. Google also released Gemini 3.8 Flash Cyber, which shares the same foundational intelligence but is a separate restricted deployment for trusted defenders through the Fairwind Program; it should not be treated as an unrestricted API alias for standard 3.8 Flash.

The standard model has a 1,048,576-token input context and a 65,536-token maximum output in Google Cloud documentation. Its supported thinking levels are LOW, MEDIUM and HIGH, with MEDIUM listed as the default in the Enterprise Agent Platform guide. Google explicitly warns that 3.8 Flash can use more tokens than 3.7 Flash because it performs more reasoning and iterative tool calls, and it continues to recommend 3.7 Flash where compute efficiency is the primary constraint.

The introductory Gemini API price is $0.75 per million input tokens and $3.75 per million output tokens through December 31, 2026. Google says the standard price from January 1, 2027 will be $1.50/M input and $7.50/M output.

Two current Google documents publish different Terminal-Bench 2.1 scores

The most important benchmark caveat is not a competitor comparison. It is a discrepancy between two current Google sources.

Google DeepMind's detailed Gemini 3.8 Flash evaluation document reports 89.4% on Terminal-Bench 2.1 for Gemini 3.8 Flash and 85.8% for Gemini 3.7 Flash. The methodology says Gemini results are self-computed and that Terminal-Bench 2.1 is reported using the default Terminus 2 agent harness.

Google Cloud's developer guide, updated September 3, instead lists 90.8% for Gemini 3.8 Flash and 81.6% for Gemini 3.7 Flash on Terminal-Bench 2.1.

Those rows should not be averaged, and neither delta should be quoted without its source. The visible Cloud guide does not disclose enough of the corresponding evaluation recipe to reconcile why the 3.7 and 3.8 values differ from the DeepMind methodology PDF. A benchmark score is a property of a model plus a harness, task revision, reasoning configuration and evaluation date. Until Google publishes a reconciliation, the defensible presentation is that two official Google surfaces currently expose two different Terminal-Bench 2.1 result pairs.

This matters because the apparent improvement changes materially depending on the table: the DeepMind pair is a 3.6-point gain, while the Cloud guide pair is a 9.2-point gain. Reporting only the larger delta would overstate the certainty of the comparison.

DeepSWE v1.1 is not SWE-Bench Pro

Google DeepMind separately reports 73.7% on DeepSWE v1.1 for Gemini 3.8 Flash versus 65.3% for 3.7 Flash. Its methodology states that the 3.8 result is self-computed with a mini-swe-agent harness at high thinking, while comparison rows are taken from the public DataCurve leaderboard where possible.

DataCurve's current DeepSWE v1.1 leaderboard contains 113 original long-horizon engineering tasks. That task count, harness and reasoning level are part of the result. DeepSWE should therefore not be relabeled as SWE-bench, even though both evaluate software-engineering agents.

SWE-Bench Pro is 61.6% in one Google guide, but the row needs more metadata

The Google Cloud developer guide reports 61.6% on SWE-Bench Pro for Gemini 3.8 Flash, compared with 60.4% for Gemini 3.7 Flash. That is a 1.2-point improvement, much smaller than either of the Terminal-Bench deltas.

SWE-Bench Pro itself contains 1,865 tasks across 41 repositories, with a 731-task public set plus held-out and commercial subsets. However, Google's visible 3.8 Cloud guide does not state which Pro subset, harness, turn budget or reasoning protocol produced the 61.6% row. That means the overall benchmark's published dataset size should not be silently converted into the sample size for Google's specific run.

For reproducible comparisons, the missing metadata matters as much as the percentage. A team comparing 61.6% with another vendor's SWE-Bench Pro number should first confirm the exact split and agent scaffold.

SWE-bench Verified remains separate—and no primary 3.8 Flash result was verified here

SWE-bench Verified and SWE-Bench Pro are different benchmarks. The primary Google materials checked for this analysis publish a SWE-Bench Pro number and a DeepSWE v1.1 number, but they do not provide a directly verifiable Gemini 3.8 Flash SWE-bench Verified row with the same level of source and harness detail.

Secondary coding leaderboards may surface Verified claims, but without the underlying run evidence directly verified in this pass, this article leaves the Gemini 3.8 Flash SWE-bench Verified result unknown rather than importing an unattributed or incompletely specified score.

That is not a missing-data problem to paper over. Verified has a different task set and contamination history from Pro, while DeepSWE is a separate 113-task benchmark. Mixing them would create a false ranking.

Independent testing shows why the same token price is not the same task cost

Artificial Analysis independently tested Gemini 3.8 Flash across its three reasoning levels. At high reasoning it reports an Intelligence Index score of 59, up from 56 for Gemini 3.7 Flash high.

The more useful operational finding is cost. Artificial Analysis reports about $0.58 per Intelligence Index task for 3.8 Flash high versus $0.40 for 3.7 Flash high, roughly 40% higher task cost despite the same introductory per-token price. It attributes the increase to about 30% more average output tokens, reaching roughly 48,000 output tokens per task, plus more turns on agentic evaluations.

The same independent test reports approximately 300 output tokens per second at high reasoning, but average time per task rises from 2.2 minutes for 3.7 Flash to 2.5 minutes for 3.8 Flash. With low reasoning, 3.8 Flash falls to about $0.24 per task and 0.8 minutes per task in that evaluation.

These are Artificial Analysis measurements, not Google guarantees, and they are specific to that benchmark suite and provider route. Still, they independently support Google's own warning that 3.8 can "work harder" and consume more tokens. Developers should therefore benchmark cost per successful task, not only token list price or tokens per second.

HLE-Verified illustrates another methodology boundary

Google DeepMind reports 54.9% on HLE-Verified for Gemini 3.8 Flash versus 53.6% for 3.7 Flash. The detailed methodology says this uses the full 1,811-item verified set consisting of 668 verified items from the original HLE plus 1,143 revised items, while excluding 689 original items marked uncertain.

That is substantially better documented than the SWE-Bench Pro row in the Cloud guide. It shows what good benchmark reporting should look like: exact set, exclusions, scoring convention and model ID.

Early user feedback is mixed and should stay anecdotal

Launch-week public feedback broadly echoes the measured quality-versus-effort trade-off, but it is self-selected and uncontrolled.

One Antigravity user reported that 3.8 Flash felt less sycophantic and more diligent, but that medium-difficulty tasks took much longer than on 3.7 Flash and consumed quota faster. Another user reported hallucinations and rapid context loss; replies in the same discussion explicitly disagreed, with some saying they did not observe the problem. A separate user described one Android debugging task that 3.8 Flash solved after GPT-5.6 Sol and Claude Opus 5 had failed.

None of those posts is a benchmark. Prompts, context, tool permissions, serving routes and success criteria were not controlled. They are useful primarily because they identify hypotheses worth measuring: long-running-agent reliability, quota consumption, context retention and time to completed task. No reliable X-based reproduction with pinned model identity, harness and objective metrics was found in this bounded pass, so no X consensus is inferred.

Gemini 3.8 Flash Cyber needs its own evidence labels

Google says the restricted Flash Cyber variant achieved 47.2% pass@1 on CWE-Bench compared with 47.8% for an unnamed leading frontier model, exceeded 70% on an internal 20-programming-language vulnerability-discovery benchmark, and produced 2.6 times more correct Chrome vulnerability patches than larger commercial models tested by Chrome Security.

Those are vendor-reported or partner/internal results. The internal benchmark is not publicly reproducible from the launch post, and Flash Cyber is not generally available through the standard public model selector. The standard Gemini 3.8 Flash SWE-Bench Pro, DeepSWE and Terminal-Bench rows should therefore not be presented as Flash Cyber results, and Cyber's security benchmark results should not be used to rank general coding agents.

Practical takeaway

Gemini 3.8 Flash is a meaningful upgrade for agentic work, but its launch evidence is also a good example of why benchmark hygiene matters. Google publishes strong results, including 73.7% on DeepSWE v1.1, 61.6% on SWE-Bench Pro in the Cloud guide and 54.9% on HLE-Verified. Yet two official Google documents currently disagree on Terminal-Bench 2.1, and the SWE-Bench Pro row lacks enough visible harness/split metadata for a fully reproducible apples-to-apples comparison.

For production evaluation, pin gemini-3.8-flash, record the thinking level, API surface and date, keep DeepSWE v1.1, SWE-bench Verified and SWE-Bench Pro as separate columns, and measure success rate, total output tokens, agent turns, wall time and cost per successful task on the same workload. If efficiency matters more than marginal quality, Google's own documentation and independent task-cost measurements both support keeping 3.7 Flash in the comparison rather than assuming the newer model is automatically cheaper.