MoWAM Swaps Future Video for Motion Prediction in Robot AI

A new research paper describes MoWAM, a robot learning system designed to make so-called World Action Models (WAMs) faster without sacrificing their key advantage: an understanding of how a scene will evolve before the robot acts.

WAMs typically improve robot policies by simulating future dynamics, often by generating video predictions of what the scene will look like after an action. That approach helps robots anticipate outcomes, but generating full video at inference time is computationally expensive. Some systems skip video generation entirely to save compute, but this leaves future dynamics only implicitly baked into the model's observation features — a shortcut that can hurt performance when the robot faces situations outside its training distribution.

MoWAM, as reported by the paper posted to arXiv, takes a middle path. Rather than reconstructing an entire future scene, it predicts structured robot motion — a compact stand-in for the future that captures how the robot is expected to move given the current scene and physical constraints. The system uses a Mixture-of-Transformer architecture that learns visual dynamics during training while jointly predicting motion and action. Critically, once trained, MoWAM can drop video generation completely at inference time while still retaining an explicit representation of what's likely to happen next.

Because the motion representation is compact, it also enables a form of inference-time scaling: the model can sample multiple candidate motion-and-action pairs and use a "motion-aware task-progress verifier" to pick the best one, rather than committing to a single guess.

The authors tested MoWAM on the LIBERO and LIBERO-Plus benchmarks as well as real-world manipulation tasks. According to the abstract, MoWAM achieved strong in-distribution performance, improved robustness on out-of-distribution scenarios, and higher average success rates in real-world tests compared to representative WAM baselines. The paper also reports that performance continued to improve as more candidate motion-action pairs were sampled, suggesting the explicit motion representation is an effective foundation for scaling inference-time compute.

The work was submitted to arXiv's Robotics category on September 17, 2026, by lead author Jiayu Wang. Full experimental details, architecture diagrams, and comparisons against baseline WAMs are available in the paper itself, though only the abstract was available for this summary.