TwelveLabs Releases Pegasus 1.6 for Physical AI Video Understanding

Recap

Source: The Robot Report, report of Oct. 7, 2026

According to The Robot Report, TwelveLabs Inc. has released Pegasus 1.6, a new model aimed at helping machines understand complex real-world environments through video. The company states that the model is specifically designed to process egocentric video, which is captured from the point of view of the person performing a task. This includes scenarios such as cooking, factory assembly, or remote robot operation. TwelveLabs claims that Pegasus 1.6 provides temporal context, spatial reasoning, and judgment on task completion, allowing robotics teams to turn raw footage into structured, reviewable knowledge.

Jae Lee, co-founder and CEO of TwelveLabs, explained that the model does not require specific proprietary hardware or cameras. Instead, it focuses on capturing actions and interactions from the operator's perspective. The company noted that Pegasus 1.6 is its first AI model built to understand this type of video, which it argues is easier to collect and scale than teleoperation data. The model also works with existing video data and still images, and it improves entity recognition for tracking hands, objects, and tools across clips.

The release supports five specific workflows: action segmentation and labeling, dense caption labeling, quality scoring, search and curation, and consent and compliance flagging. These features are designed to automate the generation of time-stamped action labels, produce descriptive language for spatial relationships, filter low-quality clips, surface rare events, and detect privacy-sensitive content. TwelveLabs said the system builds on the capabilities of Pegasus 1.5, which introduced Time-Based Metadata, and it is already being used by robotics labs working on dexterity, manipulation, and industrial applications like semiconductor quality control.

Context

TwelveLabs, founded in 2021 and based in Seoul, has positioned itself as a provider of video intelligence platforms for enterprises with large video libraries. The company’s Marengo and Pegasus models are designed to process video in a fraction of the time it takes for humans to review it. The shift toward physical AI reflects a broader industry trend where robotics developers are seeking more scalable ways to train models on human behavior. Traditional teleoperation data is often limited in scale and expensive to generate, making video-based approaches an attractive alternative for capturing diverse real-world interactions.

The focus on egocentric video is significant because it captures the first-person perspective of human tasks, which is directly relevant to how robots perceive and interact with their environment. By providing a layer of video understanding that can segment actions and label objects, TwelveLabs is aiming to bridge the gap between raw video data and the structured datasets required for advanced robot policy training.

Robot's take

The release of Pegasus 1.6 highlights a critical bottleneck in physical AI: the need to convert unstructured video into actionable training data. By focusing on egocentric footage, TwelveLabs is addressing a data source that is more abundant and easier to collect than teleoperation data, which could accelerate the development of dexterous manipulation skills. However, the effectiveness of this approach will depend on how well the model can generalize across different tasks and environments. It remains to be seen whether the structured metadata generated by Pegasus 1.6 will be sufficient to train robots for complex, real-world tasks without significant human intervention. The integration of privacy and compliance features is a positive step, but the real test will be in the quality and utility of the training data produced for robotics teams.