RAPID Turns a Single Human Demo Into a Working Robot Program

A team of robotics and AI researchers has unveiled RAPID (Robot Agentic Programming from Demonstrations), a framework that lets a robot learn to perform a task by watching a single human demonstration and then writing its own executable program to replicate it, according to a new paper posted on arXiv.

The core idea borrows from the success of AI coding agents, which have shown they can solve complex programming problems by iteratively writing, testing, and revising code. RAPID applies the same loop to robotics: given one visual demonstration, it automatically infers three things it needs to build and debug a program — a testable task specification, a set of action primitives the robot can execute, and an interactive simulated environment in which to run and verify the resulting code.

Rather than simply replaying the exact motions it observed, RAPID builds what the researchers call an object-centric relational program representation. This means the system focuses on the underlying structure of the demonstrated strategy — how objects should move relative to each other — instead of memorizing specific joint trajectories. Concretely, action primitives are expressed as trajectory-optimization programs that produce the intended object-level motion effect, and these primitives are then composed using relational constraints that capture the specific geometry of a scene at runtime. That structure is what allows a program generated from one demonstration to still work when the object's pose, shape, material, or surroundings change.

The team evaluated RAPID in simulation on eight challenging contact-rich nonprehensile manipulation tasks — tasks like pushing, toppling, or repositioning objects without grasping them, which are notoriously difficult to control precisely. They also tested it on general prehensile (grasp-based) manipulation tasks using the LIBERO-Pro benchmark. Beyond simulation, RAPID was deployed on a real Franka robot arm and evaluated on all eight of the nonprehensile tasks.

Across these experiments, the researchers report that RAPID performed strongly and generalized across variations in object pose, shape, material, and environment, without needing new demonstrations for each variation. The paper was authored by a team including Yuyao Liu and was submitted to arXiv's Robotics (cs.RO) category on September 24, 2026, cross-listed under Artificial Intelligence and Computer Vision and Pattern Recognition.

The approach points to a broader trend of applying agentic AI coding techniques — writing, running, and self-correcting code in a loop — directly to physical robot behavior, potentially reducing how much task-specific engineering is needed to turn a single demonstration into a reusable, generalizable robot skill.