New Method Speeds Up Visual Policy Training for Quadruped Robots

Training robots to move and manipulate objects using only camera or depth data is notoriously expensive, both in computing time and GPU memory, especially when the robot must carefully coordinate contact with the ground or objects. A new paper posted to arXiv proposes a way to make this kind of visual policy learning faster and more robust, as reported by the paper's authors.

The core problem the researchers tackle is a known weakness of a popular training technique called first-order policy gradients (FoPG), which uses differentiable simulation to cut down on training cost. While efficient, FoPG relies on local optimization, which can get stuck in unintended contact patterns — for example, a robot leg awkwardly resting in the wrong spot rather than a proper stepping motion.

To fix this, the team developed Sampling-Guided Policy Search (SGPS), which pairs sampling-based model-predictive control (MPC) with FoPG. In this setup, sampling-based MPC repeatedly refines action targets by trying out many possible action sequences and picking better ones, while FoPG handles fast, short-horizon policy updates. The policy is first initialized using behavior cloning from these sampled actions, and training then alternates between MPC-based refinement and short-horizon FoPG updates, all while the robot's starting states are perturbed and its simulated dynamics are randomized to improve generalization.

A key technical piece of the work is a "decoupled" FoPG formulation that removes the rendering step from the computation graph used during training. This allows the policy to learn directly from depth-camera observations without needing a separate "teacher" policy trained on privileged state information — a common workaround in robot learning that adds extra complexity.

The researchers demonstrated SGPS entirely on a single GPU, training policies for several tasks in simulation: locomotion, obstacle traversal, crate pushing, and bimanual carrying, using simulated Unitree Go2 and G1 robots. According to the paper, experiments confirmed that the MPC-based refinement step meaningfully improved policy quality beyond what initialization and simple tracking alone could achieve.

Most notably, the team distilled a trained policy and deployed it zero-shot — meaning with no additional real-world fine-tuning — onto a physical Unitree Go2. Using only its onboard depth sensor, the real robot was able to autonomously trot, crawl, and clear hurdles, and to switch between these behaviors as needed.

The work, authored by Yilang Liu and collaborators, was submitted to arXiv's Robotics category (cs.RO) alongside Artificial Intelligence and Machine Learning listings, and spans 8 pages with 6 figures. While the paper is a research contribution rather than a commercial product announcement, it points to a practical path for making vision-based locomotion and manipulation policies both cheaper to train and more reliable when they leave simulation for the real world — a persistent bottleneck in legged robotics.

As many robotics teams increasingly rely on the Unitree Go2 as an affordable, widely available research platform, techniques like SGPS could help accelerate progress on visual policies without requiring the massive compute budgets typically associated with photorealistic simulation and rendering.