Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving

A visual token sequence does not explicitly identify the view and timestep of each image, so we provide this information through view and frame tags. The view tags denote eight canonical directions, namely <FRONT VIEW>, <FRONT RIGHT VIEW>, <RIGHT VIEW>, <BACK RIGHT VIEW>, <BACK VIEW>, <BACK LEFT VIEW>, <LEFT VIEW>, and <FRONT LEFT VIEW>. The frame tag frame: $k$ associates each image with timestep $k$.

Input serialization depends on the task. Question answering examples use frame-major order, which places all views at one timestep before those at the next timestep:

Planning examples, including our self-constructed planning-reasoning data, use view-major order:

View-major serialization places consecutive observations from each view adjacent in the token sequence and exposes temporal variation within that view, which is important for control in dynamic environments (Fang et al., 2026). Single-view and single-frame inputs omit the corresponding redundant tags. Both tag types use ordinary vocabulary tokens and require no additional special tokens or architectural modifications.

Given a serialized multimodal input $\mathbf{x}$ and a target response $\mathbf{y}=(y_{1},\ldots,y_{T})$, the VLM predicts each response token conditioned on the input and preceding tokens. We use the standard next-token prediction objective:

We use the same objective for driving-specific and general-purpose vision-language samples.

As shown in Fig. 3(a), the BEV perception head performs single-frame surround-view 3D perception. It receives $N_{v}$ current images and their camera calibrations, where $N_{v}\in\{6,8\}$ in our experiments. The head constructs a shared ego-frame BEV representation for 3D object detection, semantic occupancy prediction, and BEV map segmentation.

The BEV perception head reads two complementary feature streams from each view $i$. The vision encoder feature $\mathbf{F}^{v}_{i}$ captures low-level appearance before the image tokens enter the VLM. After traversing the full VLM, the corresponding image tokens yield the feature $\mathbf{F}^{m}_{i}$, which encodes broader scene context and serves as the semantic source for BEV construction. We denote the feature sequences across views by $\mathbf{F}^{v}=(\mathbf{F}^{v}_{i})_{i=1}^{N_{v}}$ and $\mathbf{F}^{m}=(\mathbf{F}^{m}_{i})_{i=1}^{N_{v}}$. During joint training, the perception losses propagate through $\mathbf{F}^{m}_{i}$, providing an additional gradient path to the vision encoder alongside the direct path through $\mathbf{F}^{v}_{i}$.

To construct an explicit geometric representation, a depth-based view transform (Li et al., 2022c; Philion and Fidler, 2020) lifts the single-scale features $\mathbf{F}^{v}$ into a 3D volume without a feature pyramid. A lightweight depth network comprising residual blocks and an atrous spatial pyramid predicts a per-pixel categorical distribution $\mathbf{D}_{i}$ over $N_{d}$ depth bins without depth supervision. Each voxel center $\mathbf{p}$ within the perception range is projected into view $i$ using the calibration matrix $\mathbf{P}_{i}$, yielding image coordinates $(u_{i},v_{i})$ and depth bin $d_{i}$. Its voxel feature is computed as:

where $\Omega(\mathbf{p})$ contains the views in which $\mathbf{p}$ has a valid image projection. This operation distributes image features along camera rays according to the predicted depth probabilities. The resulting volume $\mathbf{V}$ retains the height dimension for occupancy prediction.

Because $\mathbf{F}^{m}$ is available at a single coarse scale, a simple feature pyramid (Li et al., 2022b) expands it into multiscale features. A query-based BEV transformer (Li et al., 2024b; Yang et al., 2023) then aggregates these features onto the BEV plane. Its queries are initialized using the height-collapsed feature $\bar{\mathbf{V}}$ derived from $\mathbf{V}$, which provides an explicit geometric prior. Each encoder layer alternates self-attention over the BEV grid with deformable cross-attention to the feature pyramid. The resulting ego-frame feature $\mathbf{B}$ integrates geometry from $\mathbf{V}$ with context from $\mathbf{F}^{m}$ and serves all three task-specific branches.

For 3D detection, a DETR-style decoder with deformable attention (Zhu et al., 2021) refines object queries against $\mathbf{B}$. For semantic occupancy, we expand $\mathbf{B}$ along the height dimension and fuse it with $\mathbf{V}$ before a shallow 3D UNet predicts per-voxel semantics. This fusion restores the vertical structure retained in $\mathbf{V}$. For map segmentation, a UNet-style head predicts rasterized map elements on the BEV plane. We jointly optimize the three branches using the perception objective:

Detection follows the set-prediction formulation. The Hungarian algorithm matches object queries to ground-truth boxes, and deep supervision at each decoder layer combines a focal loss (Lin et al., 2017) with an $\ell_{1}$ regression loss:

Following FlashOcc (Yu et al., 2023), the occupancy objective is defined as:

where $\mathcal{L}_{\mathrm{focal}}$ is a class-balanced focal loss, $\mathcal{L}_{\mathrm{geo}}$ and $\mathcal{L}_{\mathrm{sem}}$ are the geometric and semantic scene-class affinity losses of MonoScene (Cao and De Charette, 2022), and $\mathcal{L}_{\mathrm{lov}}$ is the Lovász-softmax loss (Berman et al., 2018). The map objective is $\mathcal{L}_{\mathrm{map}}=100\,\mathcal{L}_{\mathrm{focal}}+\mathcal{L}_{\mathrm{lov}}$.

For motion planning, the Planning Expert predicts future ego motion from the multimodal context encoded by the VLM. We formulate trajectory prediction as conditional generation:

where $\mathbf{s}$ denotes the vehicle sensor inputs and $\ell$ denotes their serialized layout. The variables $\boldsymbol{\tau}_{\mathrm{hist}}$, $\mathbf{n}$, and $\mathbf{e}$ denote the historical ego trajectory, navigation instruction, and current ego state, respectively. The optional textual planning reason $\mathbf{r}$ is set to $\varnothing$ when unavailable. The prompt describes $\ell$, provides $\boldsymbol{\tau}_{\mathrm{hist}}$ and $\mathbf{n}$, and includes $\mathbf{r}$ when available. Each trajectory contains 50 waypoints spanning 5 s at 10 Hz. At waypoint $k$, $x_{k}$ and $y_{k}$ denote the longitudinal and lateral positions in the current ego frame, while $\theta_{k}$ denotes the heading relative to the current ego orientation. For joint training across datasets, we divide $x_{k}$, $y_{k}$, and $\theta_{k}$ by fixed scales of 165 m, 25 m, and $\pi/2$ rad, respectively.

As illustrated in Fig. 3(b), the Planning Expert uses a 32-layer diffusion transformer. The VLM alternates gated linear attention with grouped-query softmax attention (Qwen Team, 2026a). We cache the keys after rotary position embedding (RoPE) and the corresponding values from all eight grouped-query softmax attention layers. Each cache conditions four consecutive Planning Expert layers. Each Planning Expert layer concatenates the cached keys and values with those of the trajectory tokens for joint attention. A trajectory token combines a noisy waypoint with an encoding of $\boldsymbol{\tau}_{\mathrm{hist}}$. Shared adaptive layer normalization injects the flow time, navigation instruction $\mathbf{n}$, and current ego state $\mathbf{e}$. The hidden dimension is 1024, yielding $\sim$1.1B parameters.

We train the Planning Expert by flow matching (Lipman et al., 2023) with an $x$-prediction parameterization that directly estimates the clean trajectory. Let $\boldsymbol{\tau}_{1}$ denote the normalized ground-truth trajectory, and let $\boldsymbol{\tau}_{0}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$ denote a Gaussian noise sample of the same shape. We define the linear interpolation path at flow time $t$ as:

The Planning Expert predicts the clean endpoint $\hat{\boldsymbol{\tau}}_{1}$ rather than the flow velocity or noise. The predicted endpoint induces the flow velocity field $(\hat{\boldsymbol{\tau}}_{1}-\boldsymbol{\tau}_{t})/(1-t)$. This endpoint parameterization reduces sensitivity to sensor noise in trajectories recorded across heterogeneous datasets. To keep this conversion well conditioned, we sample $\tilde{t}\sim\mathrm{Beta}(1.5,1.0)$ and set $t=\min\{\tilde{t},0.9\}$, ensuring $1-t\geq 0.1$. The complete objective combines flow matching with temporal regularization:

where $\mathcal{L}_{\mathrm{fm}}$ is the squared error between the induced flow velocity and the target flow velocity $\boldsymbol{\tau}_{1}-\boldsymbol{\tau}_{0}$. For $j\in\{1,2\}$, $\mathcal{L}_{\Delta^{j}}$ is a Huber penalty that matches the $j$th-order temporal differences of $\hat{\boldsymbol{\tau}}_{1}$ and $\boldsymbol{\tau}_{1}$. Together, these temporal regularizers discourage waypoint jitter and abrupt changes in acceleration.

At inference, Gaussian noise initializes the trajectory tokens. A 10-step Euler solver then integrates the induced flow velocity field to obtain the final trajectory.

We keep the vision encoder and VLM fixed and optimize only the newly initialized BEV perception head with $\mathcal{L}_{\mathrm{perc}}$. Its view transform, BEV transformer, and task decoders learn to construct and decode an ego-frame representation. This stage initializes the newly added module before joint adaptation in Stage 2.

Sec. 3.1 will show that head-only training yields limited perception performance, indicating that the pretrained representations do not directly expose sufficient 3D structure for driving perception. The head-only setting thus probes how readily the pretrained features support explicit 3D prediction. We then optimize the initialized BEV perception head, vision encoder, and VLM together for this capability. Perception samples use $\mathcal{L}_{\mathrm{perc}}$, while vision-language samples use $\mathcal{L}_{\mathrm{ntp}}$.

Each minibatch contains both sample types. Because they activate different task pathways, we provide dummy inputs to inactive branches to maintain a consistent computation graph across distributed workers. We exclude the corresponding dummy outputs from the loss. The BEV perception head uses a learning rate $20\times$ that of the VLM, allowing the task-specific module to adapt more rapidly during joint training. Driving data provide domain-specific supervision, while general-purpose vision-language data help preserve broad visual understanding and instruction-following capabilities. The resulting VLM representations provide the conditions for the Planning Expert.

We keep the vision encoder and VLM fixed and optimize only the Planning Expert. The training mixture contains samples whose prompts include a textual planning reason $\mathbf{r}$ and samples for which $\mathbf{r}=\varnothing$. Both types supervise only the future trajectory through $\mathcal{L}_{\mathrm{plan}}$, with no text-generation objective in this stage. Keeping the conditioning representations fixed separates trajectory learning from changes in the vision-language representations. We refer to the resulting model as Qwen-Drive-1.0-SFT.

Stage 3 trains the Planning Expert to reproduce a single recorded future per scene using $\mathcal{L}_{\mathrm{plan}}$. This imitation objective provides stable trajectory supervision but only partially reflects how a plan is evaluated in practice. A recorded trajectory represents only one of several acceptable futures, so other safe behaviors may be penalized, particularly when the four training sources exhibit different ego-motion distributions. Moreover, trajectory regression does not explicitly capture collision avoidance, drivable-area compliance, progress, or agreement with human preference. This stage therefore optimizes the Planning Expert with task-level rewards that measure these properties. We keep the vision encoder and VLM fixed, confining the adaptation to the Planning Expert and preserving the shared representations learned in Stage 2. We refer to the resulting model as Qwen-Drive-1.0-RL.

These task-level rewards are nondifferentiable through trajectory generation and therefore require sampled rollouts for optimization. The inference sampler in Sec. 2.1 initializes the trajectory tokens from Gaussian noise and then applies deterministic Euler integration. Once the initial noise is drawn, the remaining integration path is deterministic and defines no transition probabilities that a policy gradient could differentiate. We therefore share the initial trajectory noise within each rollout group and introduce stochastic transitions over a contiguous block of the final integration steps. This converts the deterministic flow into a stochastic policy whose transition likelihood depends on the Planning Expert parameters. Indexing the $K=10$ Euler steps from zero, with $t_{k}=k/K$ and $\Delta t=1/K$, we introduce stochasticity only over the final three transitions, $\mathcal{W}=\{7,8,9\}$. Under the endpoint parameterization, perturbations near $t=1$ affect the emitted trajectory more directly, while earlier perturbations are increasingly attenuated by subsequent integration steps. Concentrating exploration near the output therefore yields effective trajectory diversity while limiting deviation from the pretrained flow. Stochastic perturbations can move an intermediate trajectory away from regions favored by the pretrained flow. We therefore construct an approximate restoring score from the Gaussian conditional associated with the interpolation in Eq. 7. Under the Gaussian conditional implied by Eq. 7, we substitute the predicted endpoint $\hat{\boldsymbol{\tau}}_{1}^{(k)}$ for the unknown clean trajectory and obtain the score correction:

This score points toward the conditional center and stabilizes perturbed states. Let $\sigma_{k}$ denote the standard deviation of one discrete stochastic transition. For a continuous diffusion coefficient $g(t)$, the corresponding discrete standard deviation is $\sigma_{k}=g(t_{k})\sqrt{\Delta t}$. The score drift accumulated over one Euler interval is therefore $\frac{1}{2}g(t_{k})^{2}s_{\theta}\Delta t=\frac{1}{2}\sigma_{k}^{2}s_{\theta}$. The resulting transition mean is:

We set $\sigma_{k}=\sigma=0.03$ for $k\in\mathcal{W}$ and $\sigma_{k}=0$ otherwise. Since $\sigma_{k}$ denotes the standard deviation of the discrete transition, $\sigma_{k}^{2}$ already incorporates the integration interval and requires no additional factor of $\Delta t$ in the score correction. In implementation, $1-t_{k}$ is lower-bounded by $\epsilon=0.1$, and $\hat{\boldsymbol{\tau}}_{1}^{(k)}$ is clipped to $[-1,1]$ in normalized coordinates before computing the flow velocity and restoring score.

Independent waypoint noise primarily introduces high-frequency jitter rather than meaningful maneuver diversity, making the resulting samples poorly suited to comparisons of driving quality. We therefore restrict stochastic exploration to a smooth low-frequency temporal subspace. Let $\boldsymbol{\Phi}\in\mathbb{R}^{N\times M}$ contain the first $M=6$ orthonormal cosine modes over the $N=50$ future waypoints, with $\boldsymbol{\Phi}^{\top}\boldsymbol{\Phi}=\mathbf{I}_{M}$. At each stochastic transition, we sample $\mathbf{Z}_{k}\in\mathbb{R}^{M\times 3}$ with independent standard Gaussian entries and update by:

The low-frequency modes vary smoothly over the prediction horizon, so their combinations produce coherent shifts and bends in the trajectory rather than pointwise oscillations. Since $\boldsymbol{\Phi}$ has orthonormal columns, the perturbation of an individual waypoint has an average standard deviation of $\sigma\sqrt{M/N}\approx 0.010$ across waypoints, corresponding to roughly $1.7$ m longitudinally and $0.26$ m laterally per stochastic step. The perturbation lies in the $3M$-dimensional subspace of the full $3N$-dimensional trajectory space. We therefore evaluate a Gaussian likelihood surrogate for the injected stochastic action in the corresponding low-dimensional basis coordinates, rather than treating the transition as a full-rank density in trajectory space. Using $\boldsymbol{\Phi}^{\top}\boldsymbol{\Phi}=\mathbf{I}_{M}$, this likelihood surrogate takes the form:

The implementation averages the squared residual over the $3M$ mode coefficients instead of summing them, which rescales $\mathcal{L}_{\mathrm{rl}}$ by a constant factor of $1/(3M)$ and is absorbed into the learning rate. Since the score correction in Eq. 10 is derived for isotropic diffusion while our perturbation is restricted to a low-dimensional subspace, we interpret it as an approximate restoring correction rather than an exact marginal-preserving transformation.

For each scene, the frozen VLM samples $G=8$ reasoning traces, whose cached keys and values independently condition the $G$ trajectory rollouts from the Planning Expert. The group rollouts share the same initial trajectory noise $\boldsymbol{\tau}^{(0)}$ and sample independent low-frequency perturbations within $\mathcal{W}$, leaving stochastic transitions and sampled reasoning as the sources of within-group diversity. Given rollout rewards $\{R_{i}\}_{i=1}^{G}$, we compute the group-relative advantage as:

where $\sigma_{R}$ is the population standard deviation over the group and $\epsilon_{R}$=1e-8 is used for numerical stability. This group-relative advantage provides a baseline without a learned value function (Shao et al., 2024). During optimization, the sampled states and advantages are treated as constants, while the transition means are recomputed with the current Planning Expert. Only the stochastic transitions in $\mathcal{W}$ contribute to the objective. Writing $k_{w}$ for the $w$-th element of $\mathcal{W}$ in ascending order, we optimize the Planning Expert with a discounted policy gradient over the $W=|\mathcal{W}|$ stochastic steps:

where $\gamma=0.6$ assigns greater credit to the steps closest to the output. Each rollout group is sampled and consumed by a single on-policy update. The objective therefore uses the current model log-likelihood directly and requires no off-policy importance correction.

Multi-source reinforcement learning must accommodate each benchmark’s distinct evaluation criteria. We use the Predictive Driver Model Score (PDMS) for NAVSIM and the Rater Feedback Score for WOD-E2E, and add a shared displacement term to each source so that a single policy receives a comparable learning signal from all three. Appendix A provides the exact reward definitions. Training uses 15K NAVSIM scenes drawn from navtrain and balanced across navigation commands, 15K PhysicalAI-AV (PAI-AV) scenes, and 479 WOD-E2E scenarios with rater-preference annotations.

We evaluate on the official nuScenes validation set and the 16-log OpenScene validation split defined in Sec. 2.3. For 3D detection, we report mean average precision (mAP) and a modified nuScenes detection score (NDS) over the unified seven-class label space. We match a prediction to a ground-truth box when their BEV center distance is below a specified threshold. We compute AP as the normalized area under the precision-recall curve after discarding operating points with precision or recall below 10%. The mAP averages AP over the center-distance thresholds $\{0.5,1,2,4\}$ m and all classes. For NDS, we compute the true-positive error metrics at the 2 m threshold following the official class-specific definitions. Because the unified annotations contain no attribute labels, our modified NDS sets the mean attribute error (mAAE) to zero for all methods.

For semantic occupancy, we report the mean IoU over the unified semantic classes excluding empty (Occ mIoU), and RayIoU (Liu et al., 2024a). Notably, OpenScene stores occupancy in the rear-axle ego frame with an identity LiDAR-to-ego transform. Rays cast from the rear axle would therefore originate at road level and immediately terminate on road-surface voxels. We instead raise the OpenScene ray origin by 1.84 m to match the nuScenes LiDAR mounting height. For BEV map segmentation, we report the mean IoU over the foreground map classes.

To enable fair comparisons, we reproduce the single-frame variants of BEVFormerV2 (Yang et al., 2023), PETR (Liu et al., 2022), and PETRv2 (Liu et al., 2023) on the remapped nuScenes annotations. Our BEVFormerV2 implementation excludes the Group-DETR decoder and its associated 2D losses. We further construct BEVFormerV2^∗ as a unified multi-task variant that jointly performs 3D detection, BEV map segmentation, and semantic occupancy prediction, covering the same three perception tasks as our BEV perception head. We evaluate each architecture with ResNet-50 (He et al., 2016) and SigLIP-Qwen. SigLIP-Qwen denotes the SigLIP-style (Zhai et al., 2023) vision encoder initialized with the same pretrained Qwen3.5-4B weights used by Qwen-Drive-1.0. All comparison methods follow a 24-epoch schedule. Both the comparison methods and our BEV perception head use an input resolution of $896\times 512$. In contrast, our BEV perception head uses no rig-specific camera embeddings, so a single model trains and evaluates across the six-camera nuScenes rig and the eight-camera OpenScene rig.

As shown in Tab. 1, on nuScenes, Qwen-Drive-1.0-SFT establishes the best detection and map-segmentation results, exceeding BEVFormerV2^∗ by 2.01 mAP, BEVFormerV2 by 3.05 NDS, and PETRv2 by 3.37 map mIoU. On OpenScene, it also improves over Head-only (joint) in both detection metrics and map mIoU. Replacing ResNet-50 with the pretrained ViT consistently benefits the dedicated detectors, confirming that vision-language pretraining provides a strong visual initialization. However, a converged head trained on the same SigLIP-Qwen features still trails BEVFormerV2^∗ by 6.34 mAP and 6.91 RayIoU. This contrast indicates that vision-language-pretrained features support visual-text alignment but do not directly expose the 3D structure required for driving perception.

Cross-dataset transfer presents a separate challenge. Despite training on the higher-quality nuScenes annotations, the nuScenes-only head reaches only 16.50 NDS on OpenScene, less than half of its 34.13 nuScenes score. Mixed-source training raises OpenScene NDS to 41.86, but the residual differences between label semantics and annotation pipelines introduce negative transfer on nuScenes. The effect is most pronounced for occupancy, where mIoU decreases by 26.6%. The same source mismatch persists after joint adaptation: Stage 2 substantially recovers nuScenes occupancy but changes both OpenScene occupancy metrics by less than 0.3 points, despite clear improvements in detection and map segmentation. Unlike boxes and rasterized map layers, the machine-generated OpenScene voxel labels retain source-specific semantic and construction artifacts, which limit the benefit of joint adaptation. This observation explains why Qwen-Drive-1.0-SFT leads in detection and map segmentation but not occupancy. Label mapping and offline completion therefore make joint training feasible without fully resolving source ambiguity at the voxel level.

The gains from Stage 2 cannot be explained by continued head optimization alone, since the head-only model had already converged. Once the perception objectives are allowed to update the pretrained ViT encoder and VLM, nuScenes mAP and map mIoU increase by 10.46 and 9.84 points, while OpenScene map mIoU gains a further 4.93 points. Joint adaptation is therefore important for realizing the 3D perception capability of the external head. Together with the ablation in Tab. 8, these results show that the head provides a practical 3D probe and that targeted adaptation produces explicit, inspectable scene predictions while preserving highly competitive vision-language performance. This capability is added without changing the VLM architecture.

Fig. 7 shows that Qwen-Drive-1.0-SFT produces competitive 3D perception results within the camera-visible regions under both the six-view and the eight-view rig. Notably, in Fig. 7(b), our preprocessing does not fully remove floating voxels above the road surface from the machine-generated OpenScene label, while the prediction suppresses these artifacts and correctly recovers the road-surface semantics. This example suggests that supervision across scenes and data sources can reduce sensitivity to residual artifacts in processed labels.

To evaluate motion planning at progressively greater levels of interaction, we consider two open-loop benchmarks, one pseudo-closed-loop benchmark, and one closed-loop simulator.

1) WOD-E2E (Xu et al., 2026b) focuses on challenging long-tail scenarios and provides several human-rated candidate trajectories for each scene. Its primary metric is the Rater Feedback Score (RFS), which matches a prediction to these candidates under a speed-dependent tolerance and assigns the rating of the closest valid match. RFS can therefore recognize an acceptable plan even when it differs from the recorded future. We also report average displacement error (ADE), defined as the mean Euclidean position error relative to the recorded trajectory, at 3 s and 5 s.

2) PAI-AV (NVIDIA, 2025b) requires each method to predict six candidate trajectories per scene. We report their average ADE to measure overall trajectory quality and the minimum ADE (minADE) to determine whether any candidate covers the recorded motion. We evaluate both metrics at 3 s and 5 s. The standard 644-example split overlaps with publicly available training data by official construction. To ensure a fair comparison, we also report results on a leakage-free subset curated from held-out test clips.

3) NAVSIM (Dauner et al., 2024) follows a pseudo-closed-loop protocol. The planner is queried only once, and surrounding agents replay their recorded motion without reacting to the ego vehicle. On NAVSIM v1.1 navtest, we report PDMS and its five components. No collision (NC) and drivable area compliance (DAC) act as multiplicative factors on a weighted combination of ego progress (EP), time-to-collision (TTC), and comfort (Comf.).

4) AlpaSim (NVIDIA et al., 2025) evaluates closed-loop behavior under compounding errors. We use PAI-AV-NuRec (Wu et al., 2025) version 26.02 to simulate 916 scenarios with novel views as the ego vehicle deviates from recorded logs. Following Alpamayo (Wang et al., 2025d), we measure close encounter rate (all-event and at-fault), off-road rate, progress, and AlpaSim score (all-event and at-fault). These metrics quantify close-proximity events, roadway departures, scenario completion, and average distance traveled between events, respectively. The at-fault variants exclude events not caused by the ego vehicle.

We evaluate Qwen-Drive-1.0-SFT with and without the planning-reasoning condition, together with Qwen-Drive-1.0-RL after reinforcement learning. Using 2.83M training samples assembled from public sources, our method attains the highest PDMS on NAVSIM and the highest test-split RFS on WOD-E2E among the compared methods. We analyze the results at increasing levels of interaction, from open-loop prediction to pseudo-closed-loop scoring and closed-loop simulation.

Open-loop. On the long-tail WOD-E2E test split in Tab. 4(b), Qwen-Drive-1.0-SFT with reasoning achieves an RFS of 7.78, slightly exceeding MindVLA-U1 at 7.77 before reinforcement learning. Reasoning raises RFS from 7.76 to 7.78, even though only 142K of the 557K WOD-E2E samples provide reasoning-conditioned supervision. On the validation split in Tab. 4(a), which supplies the RFS annotations used for reinforcement learning, Qwen-Drive-1.0-RL improves RFS from 7.95 to 8.45 while significantly reducing the 3 s and 5 s ADE. The resulting RFS exceeds the human-driver reference of 8.13. Since these annotations supervise the reward, this in-sample result indicates effective optimization of preference alignment on the training scenarios rather than generalization beyond human driving. More importantly, the improvement transfers to the test split. Reinforcement learning raises RFS from 7.78 to 7.91, exceeding the reinforced MindVLA-U1 by 0.04 points. The held-out gain indicates improved alignment with human preference without materially changing displacement from the recorded future.

Complementing the preference-based evaluation on WOD-E2E, PAI-AV assesses both the quality and diversity of six predicted trajectories. We reproduce all comparison methods under the same evaluation setting. As shown in Tab. 5, on the leakage-free subset, Qwen-Drive-1.0-SFT with reasoning attains a 3 s average ADE of 0.42 m, compared with 0.36 m for Alpamayo-1.5. Our minADE is 0.39 m versus 0.17 m, and the narrow gap between average ADE and minADE suggests that our candidates remain concentrated around similar motions. Part of this difference is plausibly attributable to training scale. Alpamayo-1.5 uses 80,000 hours of driving trajectories and 3M CoC reasoning traces, whereas PAI-AV contains 156K clips, corresponding to approximately 900 raw hours before sparse frame sampling. DriveWAM and SimWAM both incorporate future generative supervision, yet exhibit markedly different candidate distributions. DriveWAM achieves a much lower minADE than average ADE, suggesting broader candidate coverage but weaker typical trajectory accuracy. SimWAM substantially improves average ADE to 0.43 m, while its minADE remains close at 0.40 m, indicating more limited diversity. In comparison, Qwen-Drive-1.0-SFT achieves a slightly lower average ADE of 0.42 m with a comparable minADE of 0.39 m. Reinforcement learning increases the PAI-AV errors by only 3 to 5 cm. This modest open-loop trade-off accompanies improved preference alignment on WOD-E2E, higher pseudo-closed-loop PDMS on NAVSIM, and a halving of the closed-loop off-road rate in AlpaSim.

Pseudo-closed-loop. NAVSIM advances one step toward closed-loop evaluation by propagating the predicted trajectory through a vehicle model. However, it does not re-query the planner and keeps surrounding agents non-reactive, so it cannot capture compounding errors. As shown in Tab. 6, before reinforcement learning, Qwen-Drive-1.0-SFT with reasoning reaches 88.2 PDMS and surpasses comparison methods trained only with imitation, including DiffusionDrive at 88.1. This competitive result may draw on capabilities acquired during vision-language pretraining, together with the scale and behavioral diversity of the unified planning data. Reasoning improves PDMS by 0.4 points despite being available for only 78K NAVSIM samples, indicating that the textual condition provides useful context for trajectory generation. Reinforcement learning yields a further 2.5-point gain, raising PDMS to a promising 90.7. These results suggest that reinforcement on NAVSIM is particularly effective at reducing conservative, low-progress behavior. Following a common practice of reporting the best among multiple sampled trajectories, we also select per scene the candidate with the highest PDMS among six samples, which raises PDMS to 91.4 for Qwen-Drive-1.0-RL and 89.3 for Qwen-Drive-1.0-SFT with reasoning. This margin over the single-trajectory output indicates that the sampled set already contains stronger trajectories than the default prediction, suggesting headroom that better inference-time selection could recover. Nevertheless, PDMS should not be treated as a direct proxy for interactive driving quality. Near the upper end of this benchmark, further gains may increasingly reflect adaptation to the scoring function, while the non-reactive protocol cannot reveal how errors accumulate during interaction.

Closed-loop. AlpaSim completes this progression by repeatedly querying the planner as its actions alter subsequent observations, thereby exposing compounding errors and recovery behavior that NAVSIM cannot measure. For a controlled comparison, we reproduce Alpamayo-R1 and Alpamayo-1.5 under the same evaluation setting. As shown in Tab. 7, Qwen-Drive-1.0-RL achieves an at-fault close encounter rate of 11.0%, matching Alpamayo-1.5, while their all-event rates are 41.0% and 37.0%, respectively. Reinforcement learning halves the off-road rate from 24.0% to 12.0%, a rate lower than that of either Alpamayo variant, and raises the at-fault AlpaSim score from 0.27 to 0.37. These safety gains come with lower progress, which decreases from 54.0% to 48.0%, and a modest increase in the all-event close encounter rate. The results indicate a shift toward safer but more conservative behavior. Despite matching Alpamayo-1.5 in at-fault close encounter rate, Qwen-Drive-1.0-RL records lower all-event and at-fault AlpaSim scores of 0.16 and 0.37, compared with 0.23 and 0.45. One possible factor is the temporal sampling of the visual input. AlpaSim repeatedly replans over short intervals and therefore emphasizes rapid responses to recent visual changes. Alpamayo-1.5 observes a dense 0.4 s visual history, whereas our input contains four observations sampled at 0.5 s intervals over 1.5 s. This broader but sparser history may limit responsiveness over the short replanning horizon.

An AlpaSim score should not be interpreted in isolation. DriveWAM (Shi et al., 2026) attains an at-fault score of 0.53, but its progress is only 35.0%. We find that it often remains stationary or advances only briefly. This behavior reduces ego-at-fault and off-road events, which can raise the at-fault score because the metric divides the traveled distance by the corresponding event count. However, a nearly stationary ego vehicle remains susceptible to interactions caused by following traffic. DriveWAM thus records an all-event close encounter rate of 56.0% and an all-event score of 0.10. SimWAM (Zhao et al., 2026b) shows the opposite behavior. We find that its driving policy is considerably more aggressive, frequently accelerating forward while failing to decelerate sufficiently for preceding vehicles or obstacles. This yields relatively high progress of 62.0%, but also leads to an at-fault close encounter rate of 22.0% and an off-road rate of 19.0%, resulting in an at-fault score of only 0.30. In comparison, Qwen-Drive-1.0-RL achieves substantially lower at-fault close encounters and off-road violations of 11.0% and 12.0%, respectively, while maintaining 48.0% progress. These results highlight the importance of jointly considering progress, safety events, and AlpaSim scores when assessing effective closed-loop driving.

Fig. 9 complements the quantitative evaluation with open-loop and closed-loop planning. In Fig. 9(a), Qwen-Drive-1.0-SFT with reasoning grounds its plans in relevant scene evidence. It decelerates for a crossing animal, follows the right-turn-only lane, and adjusts laterally to pass a stopped vehicle. On PAI-AV, the predicted trajectory remains close to the recorded future. Fig. 9(b) shows two AlpaSim rollouts from Qwen-Drive-1.0-RL. In the first rollout, the model follows the lead vehicle through a green light and subsequently stops when the signal turns red. In the second, it follows a slower vehicle, turns right at the intersection, and adjusts its lateral position as another vehicle overtakes. Across both rollouts, the reasoning and trajectory are updated with the evolving traffic state while remaining consistent with the navigation instruction and road geometry.

Fig. 10 further illustrates how reinforcement learning adapts trajectory predictions to the NAVSIM objective. Both variants retain the same high-level left-turn maneuver, while Qwen-Drive-1.0-RL reduces a small lateral deviation from the recorded future. This fine-grained correction better aligns the trajectory with the NAVSIM scoring criteria and is consistent with the PDMS improvement in Tab. 6.

First, we examine how the Stage 2 training mixture in Sec. 2.2 affects vision-language capability, explicit 3D perception, and subsequent planning. Starting from the unadapted Qwen3.5-4B in row i, row ii introduces vision-language training, and row iii additionally enables 3D perception supervision. For a controlled planning comparison, we train a Planning Expert for each variant on WOD-E2E (Xu et al., 2026b) for 15 epochs in Stage 3 and report RFS on the validation split. Tab. 8 shows that vision-language training improves Driving QA Avg. by 6.55 points and produces a substantially larger gain in CoC reasoning, while preserving general vision-language capability. Adding 3D perception supervision keeps all three vision-language aggregates within one point of row ii. Consistent with Sec. 3.1, the BEV perception head serves as a 3D probe by exposing how readily the shared representations support 3D detection, semantic occupancy, and BEV map predictions. Its perception objectives further provide task-specific 3D supervision to the shared visual pathway during joint adaptation. The complete Stage 2 mixture adds an explicit, inspectable 3D perception capability while largely preserving vision-language performance. It also yields the highest RFS after Stage 3, although the margin over the other variants is small. The planning result supports the compatibility of the Stage 2 mixture with subsequent Planning Expert training, but does not establish explicit 3D supervision as the source of the improvement.

We next ablate the data mixture and the shared ADE term used for reinforcement learning, as shown in Fig. 11. With NAVSIM-only training, augmenting the source-specific PDMS reward with ADE raises PDMS from 90.4 to 90.8. Joint training on NAVSIM, WOD-E2E, and PAI-AV produces similar NAVSIM scores of 90.6 and 90.7 without and with the shared ADE term, respectively. The small differences from single-source training suggest limited cross-dataset interference, particularly when each source provides a task-aligned reward. On the WOD-E2E validation split, adding ADE lowers RFS from 8.68 to 8.45 but substantially reduces the 5 s ADE from 2.24 m to 1.27 m. The shared displacement reward anchors preference optimization to the recorded motion and limits excessive trajectory deviation. Based on this trade-off, our final recipe jointly trains on all three sources and includes the shared ADE term for every source.

We further examine how Stage 3 planning performance scales with the amount of trajectory supervision. To isolate the effect of data scale from cross-dataset mixing, we train the Planning Expert exclusively on PAI-AV using 0.17M, 0.35M, 0.69M, 1.04M, and 1.38M training samples. As shown in Fig. 12, both 5 s Avg. ADE and Avg. FDE decrease monotonically as the training set grows. On the standard split, Avg. ADE decreases from 1.34 m to 1.05 m, while Avg. FDE decreases from 4.18 m to 3.24 m. The leakage-free subset remains more challenging but follows the same trend, showing that the gains are not confined to the standard split. Performance continues to improve at 1.38M samples, with no clear sign of saturation at the current data scale.

The quantitative comparison in Sec. 3.1 covers only the two sources that supervise perception. We finally examine whether the perception pathway can operate directly on camera configurations never seen in training. To this end, we sample frames from the eight-camera ring rig of WOD-E2E (Xu et al., 2026b) and the six-camera rig of PAI-AV (NVIDIA, 2025b), whose full surround views are absent from all perception training data. After correcting lens distortion to a pinhole model, we run Qwen-Drive-1.0-SFT directly on both rigs without dataset-specific adaptation. As shown in Fig. 13, the projected boxes are visually plausible across views and remain qualitatively consistent with the corresponding BEV layouts. The occupancy and map outputs also exhibit coherent road surfaces, object regions, drivable areas, lane markings, and road edges. These examples show that the perception pathway remains operational and produces qualitatively coherent outputs on unseen camera rigs. Because neither dataset provides unified ground truth, they do not establish reliable 3D accuracy under the new camera settings. Quantitative evaluation and adaptation with high-quality annotations from the target vehicle platform are required to assess and improve such cross-rig transfer.