Abstract
Counterfactual obstacle insertion exposes blind planning in representative Driving VLAs: their trajectories remain largely unchanged despite a vehicle appearing directly ahead. An empirically calibrated bias–variance analysis identifies how ego-predictable training examples admit a shortcut through ego status and commands. We address this failure with an inverse-kinematics-inspired architecture that anchors trajectory decoding on current and predicted future visual states. First, a next visual state prediction objective that requires the LLM to predict the future visual scene provides dense visual supervision and suppresses shortcut paths. Second, a separate Inverse Kinematics Network (a cross-attention-based conditional diffusion model) that takes only the current and future visual states as input is designed to suppress reliance on ego status and textual shortcuts during trajectory decoding. Our 0.5B model achieves state-of-the-art trajectory planning performance on NAVSIM-v1, NAVSIM-v2, and nuScenes using cameras alone, without LiDAR or a BEV encoder. Against the matched OpenDriveVLA* baseline, it improves NAVSIM PDMS and EPDMS by 19.0 and 20.4 points, respectively. Controlled ablations and intervention analyses support improved use of visual features, with additional benefits from future-state prediction in dynamic situations such as turning.
Key Contributions
- Diagnosing blind planning. Controlled visual interventions reveal that representative Driving VLAs can retain nearly identical trajectories after an obstacle is inserted. A bias–variance analysis characterizes the ego-status shortcut under an empirically calibrated majority-regime assumption.
- Grounding actions in visual states. Next visual state prediction provides dense visual supervision. An Inverse Kinematics Network decodes trajectories from only the current and predicted future visual states, removing direct ego-status and command inputs from the trajectory decoder.
- Strong performance with a 0.5B backbone. Using three front cameras, our model achieves 92.2 PDMS on NAVSIM-v1, 90.6 EPDMS on NAVSIM-v2, and 0.06 m average L2 error on nuScenes under the ST-P3 evaluation protocol.
Overall Methods
Ego status and route commands guide future-state prediction through the LLM, while the trajectory decoder has no direct access to them. The IK Network is a cross-attention conditional diffusion model, with trajectory-loss gradients stopped at its visual conditioning inputs.
“Inverse kinematics” describes the idea of inferring a trajectory between a start and a target visual state; it is an architectural analogy rather than a strict vehicle-dynamics formulation.
Video Overview
Quantitative Results
NAVSIM
Our 0.5B model improves over the matched OpenDriveVLA* baseline by 19.0 PDMS and 20.4 EPDMS points. Selected camera-based VLA comparisons from the paper are shown below.
| Method | Backbone size | NAVSIM-v1 PDMS ↑ |
NAVSIM-v2 EPDMS ↑ |
|---|---|---|---|
| OpenDriveVLA* | 0.5B | 73.2 | 70.2 |
| AutoVLA | 3B | 89.1 | — |
| ReCogDrive | 2B | 89.6 | 83.6 |
| DriveVLA-W0 | 7B | 90.2 | 86.5 |
| DriveFine | 8B | 91.8 | 89.7 |
| Ours (SFT) | 0.5B | 90.3 | 89.3 |
| Ours (SFT → GRPO) | 0.5B | 92.2 | 90.6 |
OpenDriveVLA* shares the backbone scale, training data, hyperparameters, and SFT → GRPO schedule with our full model; the next visual state objective and IK Network differ. Our model uses perspective-camera features without LiDAR or a BEV encoder. A dash indicates a result not included in this comparison.
nuScenes
| Method | 1 s ↓ | 2 s ↓ | 3 s ↓ | Average ↓ |
|---|---|---|---|---|
| DriveVLM | 0.18 | 0.34 | 0.68 | 0.40 |
| EMMA | 0.14 | 0.29 | 0.54 | 0.32 |
| OpenDriveVLA-0.5B | 0.15 | 0.32 | 0.57 | 0.35 |
| OpenDriveVLA-7B | 0.15 | 0.31 | 0.55 | 0.33 |
| Impromptu-VLA-3B | 0.13 | 0.27 | 0.52 | 0.30 |
| Ours-0.5B | 0.04 | 0.06 | 0.10 | 0.06 |
The nuScenes OpenDriveVLA results are reported original baselines, not the NAVSIM matched variant. Under the separate UniAD evaluation protocol, our average L2 error is 0.13 m. See the paper for full metrics and comparisons.
Qualitative Results
Counterfactual Obstacle Insertion
| Model | Deceleration ratio | Avg. displacement change |
|---|---|---|
| Impromptu-VLA 3B | 38.8% | −0.00 m |
| Alpamayo-R1 10B | 54.5% | −0.12 m |
| OpenDriveVLA 0.5B | 44.2% | −0.06 m |
| Ours 0.5B | 63.8% | −1.04 m |
Negative displacement change indicates a shorter predicted 3 s endpoint range, not a measured braking rate. These aggregate results are distinct from the individual examples shown in the figure and videos.
Driving Examples
Obstacle Position and Scale
BibTeX
@misc{park2026grounding,
title = {Grounding Driving VLA via Inverse Kinematics},
author = {Park, Junsung and Shim, Hyunjung},
year = {2026},
note = {Preprint}
}