New Foresight Trick Sharpens 3D Diffusion Policies for Robots

Robots that learn manipulation skills through diffusion-based policies are good at figuring out what move makes sense right now, based on a 3D view of the scene. But knowing what to do at this instant isn't the same as knowing where the interaction is actually headed — and that gap can hurt performance on longer, contact-rich tasks.

A new paper posted to arXiv, titled "Learning Foresight without Explicit Trajectories for 3D Diffusion Policies," proposes a fix called Movement Trend Guidance. Rather than forcing the robot to commit to an explicit future trajectory, the method teaches the policy to compress a short history of observations into a compact latent representation that captures how the interaction is evolving. During training, sparse future gripper states are used to supervise this latent so it learns to reflect meaningful trends. At inference time, no future information is available — only the learned latent is kept, conditioning the policy alongside the robot's current observation.

Architecturally, this latent feeds into the action-generation network in two ways: as a global conditioning signal, and through an additional gated FiLM branch — a technique that modulates a neural network's internal features — applied specifically at the bottleneck of the UNet, the core network used to generate actions in diffusion policies. Notably, the researchers built this on top of DP3 (3D Diffusion Policy), a widely used baseline, while preserving its original dense-action, receding-horizon formulation. The addition costs just 3.52% more parameters than DP3.

The gains reported are substantial. In 50-task mixed training on the RoboTwin2.0 benchmark, the method reached 62.8% success versus 56.1% for baseline DP3. On LIBERO-40, a benchmark of 40 manipulation tasks, the improvement was much larger: 71.93% versus 37.08%. The method was also validated on DexArt, a dexterous manipulation benchmark, and on five real-robot tasks, where it achieved 72.0% success compared to 49.0% for the baseline.

The authors summarize the core insight succinctly: diffusion policies "can benefit substantially from knowing where an interaction is heading, without being told exactly where to move." In other words, foresight doesn't require the robot to plan out and commit to a specific future path — just a compressed sense of trend can meaningfully sharpen its immediate decisions.

The work, submitted by Zhongbo Zhang and collaborators, sits at the intersection of robotics and computer vision, and it fits a broader trend in robot learning: rather than building bigger models or more explicit planners, researchers are finding ways to inject useful structure — like temporal foresight — cheaply into existing architectures. Because the method plugs into DP3's existing formulation with minimal parameter overhead, it could be a relatively easy upgrade for teams already using diffusion-based manipulation policies, provided the reported gains hold up under independent replication across different robot embodiments and task families.