Qwen-Drive 1.0 Reality Check: 90.7 NAVSIM, 12% Off-Road and a Rationale Gap

What Qwen-Drive-1.0 actually is

Qwen-Drive-1.0 is a research-oriented autonomous-driving vision-language model from the Qwen team and Huazhong University of Science and Technology. Its technical report appeared as arXiv v1 on August 31, 2026. The exact released model is Qwen/Qwen-Drive-1.0-4B.

The name needs a little unpacking. Qwen-Drive keeps Qwen3.5-4B as the shared vision-language model and attaches two driving-specific components: a bird's-eye-view perception head for 3D detection, occupancy and map segmentation, plus a Planning Expert that generates future ego trajectories. Hugging Face currently labels the complete released model artifact as 5B parameters, while the paper's planning comparison also reports approximately 5.0B parameters excluding LLM token embeddings. So "4B" is best understood as the underlying shared VLM identity, not a claim that every attached driving component keeps the whole system at exactly four billion parameters.

Primary sources:

• Qwen-Drive technical report
• Official GitHub repository
• Official Hugging Face model card

The 90.7 headline is NAVSIM pseudo-closed-loop, not real interactive driving

The strongest headline planning number is 90.7 PDMS on NAVSIM v1.1 navtest for the reward-optimized Qwen-Drive-1.0-RL planner. The imitation-trained SFT model with reasoning scores 88.2.

That is a useful benchmark result, but the evaluation type matters. NAVSIM is pseudo-closed-loop: the planner is queried once, the predicted trajectory is propagated through a vehicle model, and surrounding agents replay their recorded motions rather than reacting to the ego vehicle. The Qwen paper explicitly warns that PDMS should not be treated as a direct proxy for interactive driving quality because the protocol cannot expose the full accumulation of errors during interaction.

The repository also reports a 91.4 best-of-6 NAVSIM score for the RL planner. That number is not an ordinary deployable single-trajectory result. The official evaluation documentation says best-of-6 selects the highest-scoring trajectory using ground truth, making it an oracle upper bound on what an inference-time selector could recover. The directly reported single-output figure is therefore the more conservative 90.7.

This distinction is important whenever model releases advertise both sampled and selected results. A score obtained by choosing the best candidate after seeing ground truth should not be compared as though the production system knew which candidate would succeed.

Closed-loop AlpaSim reveals a different tradeoff

Qwen's most informative safety-style result comes from a genuinely closed-loop evaluation in AlpaSim. The authors use PAI-AV-NuRec v26.02 across 916 scenarios, repeatedly re-querying the planner as the simulated vehicle's actions change later observations.

In that setting, reward optimization cuts Qwen-Drive's off-road rate from 24% to 12%. Its at-fault close-encounter rate changes from 12% to 11%, and the at-fault AlpaSim score rises from 0.27 to 0.37.

But the same table shows a cost: progress falls from 54% to 48%, while the all-event close-encounter rate rises from 38% to 41%. The authors interpret the change as a move toward safer but more conservative behavior.

It also does not make Qwen-Drive an unambiguous closed-loop winner. In the authors' reproduced comparison, Alpamayo-1.5 records an at-fault AlpaSim score of 0.45, above Qwen-Drive-RL's 0.37, while both report an 11% at-fault close-encounter rate. The models also use different visual temporal sampling, so the paper itself discusses this as a possible factor rather than reducing the comparison to one rank.

Open-loop Waymo and PhysicalAI results should stay separate

The paper evaluates multiple planning families, and their scores are not interchangeable.

On the Waymo Open Dataset End-to-End benchmark, Qwen-Drive-RL reports a 7.91 test Rater Feedback Score (RFS). RFS matches a predicted trajectory against human-rated candidate trajectories, so it can reward an acceptable plan that differs from the recorded future.

The same evaluation reports 1.19 m ADE at 3 seconds and 2.67 m at 5 seconds on the Waymo test split for the RL model. These displacement errors measure a different property from the preference-style RFS.

On the NVIDIA PhysicalAI open-loop evaluation, the official repository reports a 644-example split with six trajectories per scene. The SFT-with-reasoning model has 0.37 m average ADE at 3 seconds and 0.34 m minADE; the RL model has 0.42 m average ADE and 0.38 m minADE. The repository states that selecting the minimum-error sample requires ground truth, so minADE is again an oracle-style upper-bound statistic rather than a deployable selector.

This is a good example of why "best benchmark score" is too vague for autonomous-driving systems. Open-loop displacement, preference alignment, pseudo-closed-loop planning and closed-loop interaction answer different questions.

The explanation gap is a first-party admitted limitation

Qwen-Drive can generate textual planning reasoning alongside a trajectory, but the paper does not claim that explanation is always a faithful causal account of the action.

Its limitations section is unusually explicit. The authors say the system can become unstable when multiple causes operate at different timescales, such as a distant red light and an immediate pedestrian hazard. They also state that the generated trajectory does not always adhere to the textual rationale.

That matters because an explanation that sounds plausible is not necessarily a faithful account of why a trajectory was produced. The researchers suggest explicit consistency supervision between rationale and trajectory as future work.

Independent reporting by The Decoder on September 7 highlighted the same limitation alongside the 24%-to-12% off-road improvement. This secondary coverage is consistent with the paper; it is not an independent benchmark rerun.

The training data is large, but reasoning traces cover only part of planning training

For the planning stage, the paper reports about 2.83 million samples drawn from NAVSIM, OpenScene, WOD-E2E and PAI-AV. WOD-E2E contributes 557,000 samples from roughly 2,000 clips and PAI-AV contributes 1.38 million samples from 156,000 clips.

Only 685,000 planning samples, or 24.2%, include an accepted planning-reasoning trace. The remaining 75.8% omit that textual condition.

The team generated reasoning traces using Qwen3.7-Plus and filtered them through a multi-stage audit. That is useful methodology information, but it also means the model's reasoning supervision is partly synthetic and judge-mediated. Users should therefore separate "model can emit a rationale" from "rationale has been independently proven causally faithful."

Access is open-weight, but deployment economics are not standardized

The model card labels Qwen-Drive-1.0-4B Apache-2.0 and provides weights plus code, documentation and demo data. The repository says the package contains roughly:

• 9.1 GB for the shared VLM;
• 2.1 GB for the SFT Planning Expert;
• 2.1 GB for the RL Planning Expert;
• 0.5 GB for the perception head.

The official repository recommends a 24 GB or larger GPU for use.

Hugging Face currently lists the model as 5B parameters and says it is not deployed by any Hugging Face Inference Provider. I did not find an official standardized hosted Qwen-Drive API price, tokens-per-second result, time-to-first-token figure or end-to-end trajectory-planning latency table. Self-hosting cost will depend on hardware, runtime, image resolution, sampling count and which driving head is used.

The underlying Qwen3.5-4B model card lists a native 262,144-token context window, extensible with scaling techniques, but Qwen-Drive's driving benchmarks use their own multi-image temporal layouts and task-specific inputs. It would therefore be misleading to turn the base language-model context number into a claim about practical autonomous-driving history length.

The evaluation documentation does provide one useful inference detail: when multiple candidate trajectories are sampled, the implementation can reuse one VLM pass and perform multiple Planning Expert rollouts. That describes relative computation, not a standardized absolute latency.

There is an unresolved public license question around Waymo-derived training

A current public GitHub issue raises a materially important question that the release pages do not yet resolve.

The Qwen paper says WOD-E2E contributes 557,000 planning training samples. At the same time, Qwen's model card labels the released weights Apache-2.0.

Waymo's current public dataset agreement defines models trained, refined or tuned using the Waymo Open Dataset as derivative IP and says distributed WOD models are subject to the agreement's downstream non-commercial restrictions, including restrictions on vehicle operation, production systems and commercial purposes.

On September 5, a GitHub user opened issue #4 asking whether Qwen obtained separate permission from Waymo or how the Apache-2.0 release should be reconciled with those dataset terms.

This article does not conclude that the release is unlawful or that Qwen lacks additional rights. The public record checked here does not establish whether a separate permission, license or other legal basis exists. But until Qwen or Waymo clarifies the point, developers should not assume that an Apache label by itself answers every downstream right associated with weights trained or refined on WOD data.

Sources:

• Qwen-Drive GitHub issue #4
• Waymo Open Dataset terms

That is a licensing-risk question, not a benchmark result, and it deserves its own evidence column.

Community feedback is still sparse

The official repository has only a small number of public issues so far. Two early issues ask for official fine-tuning or training scripts, and another asks whether larger model sizes are planned. The repository currently exposes model inference/evaluation code and artifacts, but I did not find a complete official SFT/RL training pipeline equivalent to the research procedure described in the paper.

I also searched for a stable, attributable X discussion or controlled third-party rerun tied to the exact Qwen-Drive-1.0-4B checkpoint. I did not find evidence strong enough to quote as a reproducible result or call a public consensus. That absence is better reported than filling the article with unattributed reactions.

The licensing issue is therefore the clearest attributable public feedback found in this pass, but it remains a question from a community member rather than an adjudicated finding.

SWE-bench Verified and SWE-bench Pro are not relevant scores for this release

Qwen-Drive is an autonomous-driving vision-language and planning system. The release does not provide a Qwen-Drive-specific SWE-bench Verified score or SWE-bench Pro score.

Those coding benchmarks should not be imported from the underlying Qwen3.5-4B model, another Qwen release or a coding agent. Fine-tuning and attaching driving-specific modules changes the evaluated system, and software-engineering issue resolution is unrelated to the planning benchmarks above.

For this release, the relevant evidence families are 3D perception, driving VQA, open-loop planning, pseudo-closed-loop NAVSIM and closed-loop AlpaSim.

Evidence level

The strongest evidence available today is still author-run and author-reproduced evaluation backed by unusually detailed public methodology and code. The paper documents benchmark types, sampling, trajectory conversion, closed-loop scenario count and several caveats, which makes its claims easier to audit than a launch graphic alone.

What is still missing is equally important:

• a same-checkpoint, same-harness independent rerun of the main planning results;
• standardized absolute latency and cost measurements;
• broader closed-loop testing under different simulators, camera rigs and weather/traffic distributions;
• validated faithfulness between generated rationales and executed trajectories;
• clarification of the Waymo-derived training and downstream licensing question;
• real-world safety evidence suitable for deployment decisions.

The practical takeaway is not that a 90.7 NAVSIM score proves road readiness. Qwen-Drive-1.0 is a technically interesting open research system because it connects a general VLM to 3D perception and trajectory planning while publishing substantial evaluation detail. Its own evidence also shows why autonomous-driving model comparisons need more than one number: reward optimization improves some safety-style metrics, reduces progress, and leaves a documented gap between explanation and action.