Initialize in simulation
The LEGS simulator generates 200 whole-body demonstrations per task for base-VLA training and physically grounded play data for broad dynamics coverage.
Whole-body humanoid learning without teleoperation
World Model-Enabled VLA Optimization for Whole-Body Loco-Manipulation
00
Reinforcement learning can improve vision-language-action policies beyond imitation, but humanoid exploration is unsafe and simulator RL can exploit residual sim-to-real gaps. We present WEVO, a teleoperation-free framework for post-training a whole-body humanoid VLA in an action-conditioned video world model. Simulated manipulation and locomotion play data improve contact-dynamics and viewpoint-change prediction. A VLA trained on synthetic demonstrations collects 20 autonomous real robot rollouts per task to align the video WM with deployment observations. The VLA, whole-body controller, and video WM then form a closed-loop RL environment with dense progress rewards. Across three real robot tasks, video WM RL raises mean success from 23.3% to 75.0%, compared with 48.3% for real robot RL. The system leverages complementary environments: simulation provides policy initialization and broad coverage, while the video WM provides a deployment-aligned environment for policy improvement.
01
Sim to dream to real
The LEGS simulator generates 200 whole-body demonstrations per task for base-VLA training and physically grounded play data for broad dynamics coverage.
Each task uses 20 autonomous real robot rollouts, balanced between successes and failures, to align generated observations with deployment.
The current VLA generates 480 on-policy video WM rollouts, while stage-aware progress supplies dense rewards without further robot interaction.
02
LEGS simulator
Full-state planning supplies task data, while autonomous play data covers contact outcomes and locomotion-induced viewpoint changes.
Whole-body action sequences are generated from full-state planning and locomotion and manipulation primitives.
Randomized objects, grasp offsets, slips, misses, and collisions provide broad supervision for contact dynamics.
Approaches, turns, and long-range walking expose the model to large egocentric viewpoint changes.
03
Deployment-aligned video WM
We replay held-out action trajectories and compare autoregressive video WM predictions with temporally aligned ground-truth observations.
Under a matched 45,000-step budget, play data improves average PSNR by 0.93 dB and interaction-region PSNR by 1.17 dB. The larger interaction-region gain indicates stronger modeling around the robot hands and manipulated objects.
04
Closed-loop policy improvement
Each video WM call receives one seed frame and achieved 43-DoF joint states, then generates 12 future views. The terminal generated frame seeds the next call, closing the recurrent VLA–WBC–video WM loop. A progress model converts each trajectory into dense rewards.
05
Real robot evaluation
Video WM and real robot success rise together over 15 RL rounds. In contrast, direct simulator RL reduces real robot success on every task, showing that simulation is effective for initialization but not final policy improvement.
| Method | Pick-and-Place Bottle | Pick-Turn-Place Bottle | Bimanual Move Basket | Mean |
|---|---|---|---|---|
| Baselines | ||||
| Behavior cloning | 40% | 20% | 10% | 23.3% |
| Real robot RL | 60% | 45% | 40% | 48.3% |
| Simulator RL | 30% | 0% | 0% | 10.0% |
| VLAW | 45% | 20% | 35% | 33.3% |
| WMPO | 55% | 30% | 35% | 40.0% |
| RL algorithms in WEVO | ||||
| AWR | 75% | 70% | 70% | 71.7% |
| ReinFlow | 60% | 50% | 45% | 51.7% |
| DICE-RL | 85% | 75% | 65% | 75.0% |
06
Photorealistic simulation trains a whole-body base policy with enough hardware success to collect autonomous deployment experience.
Physically grounded manipulation and locomotion experience improves contact dynamics and consistency under large viewpoint changes.
Limited autonomous successes and failures align the video WM with deployment observations and cross the remaining visual gap.
Dense progress and on-policy training extract useful signal from successful, failed, and suboptimal video WM rollouts.
07
Citation
@article{chen2026wevo,
title = {WEVO: World Model-Enabled VLA Optimization
for Whole-Body Loco-Manipulation},
author = {Chen, Qianzhong and Lin, Muyuan and Sun, Yuyin and
Schwager, Mac and Kuo, Cheng-Hao and
Mart{\'i}n-Mart{\'i}n, Roberto},
year = {2026}
}