Qwen-Drive-1.0 Reality Check: 90.7 NAVSIM, 0.37 AlpaSim and Oracle-Selection Limits

What Qwen-Drive-1.0 actually is

Qwen-Drive-1.0 is a newly released vision-language foundation model for autonomous-driving research from the Qwen Team and Huazhong University of Science and Technology. The technical report was submitted to arXiv on August 31, 2026, and Qwen's public launch material appeared on September 3. The system keeps Qwen3.5-4B as the shared multimodal backbone and adds two task-specific components: a bird's-eye-view perception head for 3D detection, occupancy and map segmentation, and a Planning Expert that generates future ego trajectories. The original language decoder remains available for general and driving visual question answering.

That architecture makes Qwen-Drive broader than a single trajectory predictor, but it does not make every benchmark directly comparable. Its release mixes driving VQA, 3D perception, open-loop trajectory error, a non-reactive pseudo-closed-loop benchmark and a genuinely closed-loop simulator. The most useful way to read the launch is therefore as a benchmark ladder rather than as one universal “autonomous driving score.”

The 90.7 NAVSIM headline is pseudo-closed-loop, not full interactive driving

The strongest headline planning number is 90.7 PDMS on NAVSIM v1.1 navtest for the reinforcement-learned planner. Qwen also reports 91.4 with best-of-6 selection. Those figures are real results from the team's evaluation, but the methodology matters.

NAVSIM queries the planner once and then propagates the predicted trajectory through a vehicle model while surrounding agents replay recorded motion and do not react to the ego vehicle. The Qwen paper explicitly calls this pseudo-closed-loop and warns that the protocol cannot expose compounding interaction errors in the way a reactive simulator can. The authors also caution that PDMS near the top of the benchmark can increasingly reflect adaptation to the scoring function rather than a direct measure of interactive driving quality.

The 91.4 best-of-6 figure needs an additional label. Qwen's released evaluation code chooses, for every scene, whichever of six sampled trajectories receives the highest PDMS. That choice uses the benchmark's ground-truth/scoring information after generation, so the repository correctly describes it as an oracle upper bound rather than a deployable inference policy. The appropriate apples-to-apples default number is therefore the single-output 90.7 unless another system is evaluated with the same oracle-selection rule.

Closed-loop AlpaSim exposes a different trade-off

Qwen also evaluates 916 scenarios in AlpaSim with PAI-AV-NuRec v26.02, where the planner is repeatedly queried as its actions change subsequent observations. This is the more relevant evidence for compounding-error and recovery behavior.

On that test, Qwen-Drive-1.0-RL reports an 11% at-fault close-encounter rate, 12% off-road rate, 48% progress, and an at-fault AlpaSim score of 0.37. Reinforcement learning materially improves some safety-oriented measures over the SFT planner: off-road rate falls from 24% to 12%, while the at-fault score rises from 0.27 to 0.37. But progress also falls from 54% to 48%, and the all-event close-encounter rate increases from 38% to 41%.

That makes the result more nuanced than “RL makes the car better.” Under the same reproduced setting, Alpamayo-1.5 reports the same 11% at-fault close-encounter rate but 59% progress and a higher 0.45 at-fault AlpaSim score. The Qwen authors characterize their RL policy as safer but more conservative and explicitly recommend reading safety events, progress and AlpaSim score together. A system that simply moves less can sometimes improve selected safety ratios, which is why one aggregate score is not enough.

Open-loop results show that reward optimization is not free

The same trade-off appears in open-loop trajectory metrics. On the leakage-free 700-frame PAI-AV subset, Qwen-Drive-1.0-SFT with reasoning reports 0.42 m average ADE at 3 seconds and 0.39 m minADE, while the RL variant worsens to 0.47 m average ADE and 0.43 m minADE. On the standard 644-example split, SFT with reasoning is 0.37 m average ADE at 3 seconds versus 0.42 m for RL.

The authors describe this as a modest displacement-error cost for improved preference alignment, NAVSIM PDMS and AlpaSim off-road behavior. That is a reasonable engineering trade-off to explore, but it means the RL checkpoint should not be declared unconditionally superior. Teams need to choose metrics that match their deployment objective.

The paper also separates the standard PAI-AV split from a held-out subset because the standard 644-example split overlaps public training data by construction. That disclosure is important. Qwen reports a leakage-free 700-frame subset specifically so generalization is not inferred only from an overlap-prone benchmark.

WOD-E2E has an in-sample validation result and a more informative test result

For the Waymo Open Dataset end-to-end benchmark, Qwen-Drive-1.0-RL reports a 7.91 Rater Feedback Score on the test split, up from 7.78 for the SFT-with-reasoning system. The validation row is much higher at 8.45, but the paper states that validation ratings were used to supervise the reward. The authors therefore treat that validation result as evidence of preference optimization on the training scenarios, not proof of out-of-sample generalization. The held-out test score is the more informative comparison.

This distinction is exactly the kind of detail that can disappear in leaderboard summaries. For agentic or embodied systems, the benchmark split, reward source, simulator version, number of samples and selection rule are part of the result.

Driving VQA is strong, but one judge protocol was changed

Qwen-Drive-1.0-SFT reports 77.8 on LingoQA under the team's common comparison protocol. However, Qwen did not use the benchmark's official LingoJudge for that main table. The paper says it uses Qwen-Plus as the judge because the authors found LingoJudge lenient and inconsistent; under the official LingoJudge protocol, Qwen-Drive obtains 79.4.

Neither score should be silently mixed with results generated under a different judge. The release does re-evaluate comparison systems under the same near-deterministic decoding and judge setup, which is useful for internal consistency, but independent reproduction with the exact prompt, judge version and scoring code would provide stronger evidence.

The training data pipeline is large and partly model-mediated

The paper aggregates 24 public driving vision-language datasets. Qwen3.5-Plus rewrites heterogeneous annotations into a common conversational format, and Qwen3.5-Flash filters whether the rewritten response remains semantically consistent with the source annotation. That filter reduces 5.53 million examples to 3.09 million, a 55.9% retention rate. Qwen additionally constructs planning-reasoning traces using future trajectories and Qwen3.7-Plus, followed by multi-stage model-based auditing.

This is not necessarily a flaw, but it is relevant methodology. Model-generated rewriting, filtering and reasoning traces can improve consistency while also introducing dependencies on the generator and judge models. Reproducing the final weights therefore requires more than downloading the public checkpoint.

Open weights and inference code are available; full training reproduction is not yet packaged

Qwen-Drive-1.0-4B is downloadable from Hugging Face and ModelScope under the Apache 2.0 license. The repository publishes the model implementation, demos and benchmark/evaluation scripts. The README lists roughly 9.1 GB for the shared VLM, 2.1 GB each for the SFT and RL planning experts, and 0.5 GB for the perception head, and recommends a 24 GB or larger GPU for use.

The public repository currently exposes inference and evaluation workflows rather than a complete official end-to-end training package with the exact preprocessing, fine-tuning and reward-optimization scripts used for the paper. Two launch-week GitHub issues independently asked for official fine-tuning/training code so researchers could reproduce or adapt the training pipeline. Those requests are self-selected community feedback, not a measurement of model quality, but they identify a real reproducibility gap in the current release surface.

A public comment in one issue says users can fine-tune the model like Qwen3.5-4B and that official training scripts are not planned, but the commenter is not marked as a repository member or collaborator. It should therefore not be treated as an authoritative Qwen roadmap statement. The verifiable fact is simply that the current repository does not include a complete official training pipeline.

Context, latency and price need careful labels

Because the shared backbone is Qwen3.5-4B, its public configuration uses a native text context limit of 262,144 tokens, and Qwen documents optional YaRN scaling for the base model to about 1.01 million tokens. That is backbone capacity, not a demonstrated Qwen-Drive planning benchmark at million-token context. Driving inference packs multi-camera and temporal visual inputs and has separate memory constraints, so teams should not translate the base model's maximum context into an assumption about practical driving-horizon capacity.

No reproducible Qwen-Drive-specific end-to-end latency number was found in the launch materials reviewed here. Likewise, the release is an open-weight download rather than a separately priced Qwen-Drive hosted API in the sources checked, so there is no verified token price to report for this model. Runtime cost depends on hardware, precision, batch size, number of trajectory samples and which heads are loaded. Unknown latency and hosted pricing are better left blank than guessed.

SWE-bench Verified and SWE-bench Pro are separate—and not applicable here

No Qwen-Drive-1.0 SWE-bench Verified result was found, and no Qwen-Drive-1.0 SWE-bench Pro result was found. These are software-engineering agent benchmarks, while Qwen-Drive is an autonomous-driving VLM/planning system. Its NAVSIM, WOD-E2E, PAI-AV, AlpaSim and driving-VQA results must not be relabeled as coding scores, and results from other Qwen coding models should not be inherited by Qwen-Drive.

Keeping those cells explicitly empty is more informative than manufacturing a cross-domain leaderboard.

What the evidence supports today

Qwen-Drive-1.0 is a substantive open research release: it combines a capable general multimodal backbone with explicit 3D perception and a trajectory expert, publishes weights and evaluation code, and reports competitive results across several levels of driving evaluation. Its strongest launch headline—90.7 NAVSIM PDMS—is meaningful within that benchmark, but it is not a full closed-loop driving score. The 91.4 best-of-6 number is an oracle-selection upper bound, and the AlpaSim results reveal a real safety-versus-progress trade-off that the headline alone does not show.

For practical evaluation, the next useful evidence is an independent reproduction using the exact checkpoint, simulator/version, camera packing, reasoning mode, sample count and selection policy, with single-trajectory and oracle results reported separately. Teams should also measure end-to-end latency, memory, failure/recovery behavior and cost on their own hardware. Until that evidence appears, Qwen-Drive-1.0 is best understood as a promising research foundation—not evidence by itself of production-road readiness.