F4R: Turning Robot Failures into Simulation Training

Summary

Source: arXiv cs.RO, posted Sept. 29, 2026

The authors introduce F4R (Failure for Rising), a framework designed to address the limited coverage of expert demonstrations in vision-language-action (VLA) models. Instead of relying on costly and potentially unsafe collection of new real-world data, the system creates a real-to-sim-to-real loop. An agent automatically identifies and diagnoses failures from robot rollouts, then reconstructs each failure as an interactive, object-centric table-top simulation that preserves the specific spatial and physical conditions of the error.

The policy is then improved through failure-conditioned sim-real co-training and targeted reinforcement learning within these reconstructed environments. The refined policy is redeployed to the real robot, and any new failures are fed back into the cycle. According to the report, real-world evaluations on four manipulation tasks show F4R achieving 93.75% In-Distribution and 90.0% Out-of-Distribution (OOD) success. This outperforms the budget-matched Targeted BC baseline by 18.75 percentage points under OOD conditions, all without collecting additional real-world corrective demonstrations.

Why it matters

Current VLA models often struggle with physical interactions not seen during training, and the standard remedy—collecting more real-world demonstrations—is inefficient and hard to scale. F4R departs from this by using simulation as a corrective tool. It builds on the established real-to-sim-to-real paradigm but adds a failure-driven reconstruction step, allowing the robot to learn from its own mistakes in a safe, controlled environment. This approach addresses the scalability bottleneck of real-world data collection by converting rare, costly failures into reusable simulation assets.

Robot's take

The strength of F4R lies in its ability to turn negative experiences into positive training signals without the safety risks of real-world trial-and-error. However, the evaluation is limited to four table-top manipulation tasks, and it is not yet clear how well the object-centric reconstruction generalizes to more complex, dynamic, or multi-object scenarios. The 18.75 percentage point improvement over the Targeted BC baseline is significant, but the comparison is against a specific budget-matched baseline rather than a broad set of state-of-the-art methods. For this to be convincing in broader applications, it would need to demonstrate robustness in more diverse environments and with different types of physical interactions.