What Matters in Quality-Aware VLA Pretraining
A Benchmark and Empirical Study

Mingxuan Yan¹, Litian Gong³, Dihong Huang^2,4, Zhixu Li³, Tianyu Zhang³, Wenqian Zhang³, Hefeifei Jiang³, Peihao Li⁵, Jian Zhang⁶, Mengfei Zhao², Yikai Tang⁴, Hai Zhai², Zehao Wang¹, Ruijian Liang², Yanjia Huang⁶, Lin Shao⁷, Changliu Liu⁴, Kaiyu Hang⁸, Zhiwen Fan⁶, Masayoshi Tomizuka⁵, Jianfei Yang⁹, Jiachen Li^1,‡

¹Georgia Institute of Technology ²Axis Robotics ³University of California, Riverside ⁴Carnegie Mellon University ⁵University of California, Berkeley ⁶Texas A&M University ⁷National University of Singapore ⁸Rice University ⁹Nanyang Technological University

^‡Corresponding author

PaperCode (Coming Soon)Dataset (Coming Soon)

AXIS-Bench Figure 1: diverse simulation demonstrations, state-based quality labels, and three findings about quality-aware VLA pretraining.

What Matters in Quality-Aware VLA Pretraining?

Explore how demonstration quality shapes VLA pretraining, from simulation to real-world tasks.

AXIS-Bench Overview

Replayable demonstrations and state-based quality labels connect controlled pretraining studies with real-world validation.

AXIS-Bench system overview: replayable demonstrations, quality labels, controlled simulation comparisons, and a recipe for validation in user-defined real-world workspaces

Real-World Validation in Your Workspace

A reference baseline and sim-to-real pretraining recipe for experiments in your own DROID-compatible workspace.

Findings

See how quality definition, label granularity, and fine-tuning choices affect policy performance.

Finding 01

Episode conditioning improves mean success

Approach-aware episode conditioning improves mean success over plain BC across all three tested VLAs.

Finding 02 / Conditioning

Conditioning favors episode-level quality on average

The preferred granularity depends on both the learning method and the VLA.

Finding 02 / Filtering

Filtering favors chunk-level quality on average

The preferred granularity depends on both the learning method and the VLA.

Finding 03

Keeping quality tags helps fine-tuning

After conditioned pretraining, constant-tag fine-tuning improves mean success.