Rerun

Overview

Rerun is the data layer for physical AI. It ingests multi-rate, multimodal data such as images, point clouds, transforms, time series, joint states, and video from robot logs, human-data rigs, simulation, and web video. The built-in viewer renders everything in sync and in realtime, while the same data can be queried with dataframes or SQL and streamed directly into training. It is built in Rust on column-chunk storage purpose-built for multi-rate physical data, with SDKs in Python, Rust, and C++.

Key Features

• Log, query, visualize, and stream multimodal robotics data with an open source SDK.
• Visualize multi-rate, multimodal sequences in sync and in realtime with a time-aware built-in viewer.
• Query recordings with full dataframe or SQL queries down to columns, time ranges, and values.
• Store multi-rate, multimodal data as column-chunks in rrd files, and ingest from formats such as MCAP, rrd, and LeRobot.
• Stream dataset mixes directly to training without an export step, using a column-aware and video-codec-aware dataloader.
• Rerun Hub provides catalog, byte-range indexing, retrieval, query, transform, train, and team sharing infrastructure for Rerun data.

Use Cases

• Debugging robotics systems by visualizing RGB camera feeds, depth images, lidar scans, segmentation outputs, 3D maps, detected objects, and confidence over time.
• Running computer vision pipelines such as SLAM, hand tracking, and motion retargeting as table edits.
• Querying raw, intermediate, and derived data with dataframes or SQL to build datasets for training and evaluation.
• Streaming dataset mixes directly to GPUs for training without export jobs or stale copies.

Getting Started

• Install the Python SDK with pip install rerun-sdk.
• Add the Rust SDK with cargo add rerun.
• Install the Rerun Viewer binary to stream log data over the network or load .rrd data files; it can be installed with pip install rerun-sdk or cargo install rerun-cli --locked --features nasm.
• Run the Rerun Viewer in a browser, or follow the C++, Python, or Rust quick start guides.

Deployment & Requirements

• To stream log data over the network or load .rrd data files, install the rerun binary.
• Only the Python SDK comes bundled with the Viewer; C++ and Rust rely on a separate Viewer install.
• The nasm Cargo feature requires the nasm CLI to be installed and available in your path; skipping it may result in inferior video decoding performance.
• The Rerun Viewer can run in a web browser.

Before You Adopt

• License: Apache-2.0. Review its terms before using, modifying, or distributing the project.
• Rerun is in active development and the API is still evolving, so breaking changes should be expected.
• Viewer performance can degrade when there are too many entities.
• Multi-million point clouds can be slow.
• Rerun Hub, the scalable catalog for robotic data, is currently available only for a few select design partners.