TwelveLabs has released Pegasus 1.6, a video-understanding model designed to turn first-person footage into structured data for physical AI and robotics. The model analyzes what happens in complex real-world scenes, including actions, objects, hand-object interactions, spatial relationships, bystanders, and changes over time. It can process existing video and still images without requiring a specific camera or proprietary hardware.

The release supports five workflows: timestamped action segmentation and labeling, dense captions for robot training, quality scoring, natural-language search and curation, and consent or compliance flagging. TwelveLabs says the system is intended to help robotics teams scale training data for manipulation, packaging, assembly, cleaning, and industrial inspection, while complementing rather than replacing tactile and actuator-level sensors.

https://www.therobotreport.com/pegasus-1-6-brings-video-understanding-physical-ai-says-twelvelabs/