Rerun is a data layer for physical AI that ingests, visualizes, queries, and streams multimodal robotics data to training pipelines.
The tool solves the problem of working with multi-rate, multimodal data from diverse sources—robot logs, human-data rigs, simulators, and web video in formats like MCAP and rrd—by providing a unified substrate. It ingests images, point clouds, transforms, time series, joint states, and video, then renders everything synchronized in realtime within a built-in viewer where users can scrub episodes, compare sensors side-by-side, and watch computer vision pipelines execute live. The same data is queryable via dataframes or SQL and streams directly into training without export jobs or stale copies. The architecture is built in Rust on column-chunk storage designed specifically for multi-rate physical data, with SDKs available in Python, Rust, and C++.
Teams working on robotics, computer vision pipelines, or embodied AI should consider this tool if they need to ingest heterogeneous sensor streams, debug perception systems visually, and feed processed data into training without intermediate export steps. It suits projects that combine multiple data modalities at different sampling rates and require both interactive exploration and programmatic access to the same dataset. The tool is particularly valuable for those running SLAM, hand tracking, motion retargeting, or similar CV operations as part of their data pipeline.
Development activity shows consistent engagement across multiple dimensions. The project maintains active support across three language SDKs, indicating sustained investment in accessibility for different developer communities. Work spans both the core columnar storage system and the viewer interface, suggesting parallel development of infrastructure and user-facing features. The codebase reflects ongoing refinement of how multimodal data flows through the system, from ingestion through visualization to training integration.
