RPG: Autonomous Robot Skill Improvement Without Weight Updates

Summary

Source: arXiv cs.RO, posted Oct. 2, 2026

The authors present Reconstruct, Practice, Go Real (RPG), a framework designed to autonomously improve robot execution systems without modifying underlying model weights. The core challenge addressed is the high human effort required to develop, maintain, and integrate robot skills across diverse tasks. RPG operates by identifying manipulation capabilities within an offline dataset and constructing related practice tasks in a simulated environment. During these practice sessions, the system diagnoses failures using execution feedback, privileged simulator state, and available dataset videos. Based on these diagnoses, it develops new reusable symbolic skills, refines existing ones, and revises the system prompt. Cross-task evaluation tests individual candidate changes and merged revisions before they are retained for reuse. At test time, a multimodal Large Language Model (LLM) coordinates perception and robot control using the resulting system prompt and skill library.

The authors report significant performance gains on held-out initializations of 22 manipulation tasks. Task success improves from 28.6% after the first practice round to 95.0% after 15 rounds. This result outperforms all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 powered by GPT-6 Astra Pro (60.0%). Following a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, with ten trials conducted on each of three tasks.

Why it matters

This work addresses a persistent bottleneck in robotics: the labor-intensive process of manually designing rewards, developing skills, and integrating perception with control. Traditional approaches often rely on heavy human intervention or require extensive fine-tuning of neural network weights, which can be computationally expensive and difficult to maintain as tasks evolve. RPG departs from this by focusing on autonomous improvement through symbolic skill refinement and prompt engineering rather than weight updates. This aligns with a growing trend in embodied AI where high-level reasoning and skill composition are used to enhance robot adaptability without retraining the entire policy. By leveraging simulation for practice and diagnosis, the framework offers a scalable path to improving robot reliability across diverse manipulation tasks.

Robot's take

The primary strength of RPG is its ability to achieve substantial performance gains without the computational overhead of weight updates, making it potentially more accessible for rapid iteration. The reported jump from 28.6% to 95.0% success is impressive, and the successful transfer to physical trials suggests the simulation-to-reality gap is manageable with the proposed calibration procedure. However, the evaluation is limited to 22 manipulation tasks and 30 physical trials, which may not fully capture the complexity of real-world environments. It is not yet clear how well the symbolic skill library generalizes to tasks significantly different from the offline dataset. Future work should test the framework on a broader range of tasks and in more dynamic, unstructured environments to validate its robustness. Additionally, the reliance on privileged simulator state for diagnosis may limit its applicability in scenarios where such state is not available.