LeRobotDataset v3.0: Chunked Parquet, MP4 Shards and Streaming for Robot Learning Data at Scale (2026)

Open a v3-format robot dataset and pull one frame. You touch a metadata table to learn which shard the episode lives in, a memory-mapped Parquet file for the proprioception row, and one MP4 seek per camera. That is three different access patterns for a single training sample — and it is why LeRobotDataset v3.0 matters far more than a file-layout change should. The old format wrote one Parquet file and one MP4 per episode per camera. At a hundred thousand episodes that is hundreds of thousands of objects, and the bottleneck stops being your GPU and becomes your filesystem’s inode table and your object store’s request rate.

v3.0 trades that away. It packs many episodes into few large files and resolves episode boundaries through relational metadata instead of filenames. The cost of that trade is real, specific, and almost never discussed.

What this covers: the actual on-disk layout with numbers read off live Hub repositories, the per-sample read path, how streaming works and what it costs, the v2.1 migration procedure and its known bugs, and the failure modes that only appear above ten thousand episodes.

Context and Background

Robot-learning data has a shape that neither vision corpora nor time-series stores were designed for. A single episode is a variable-length trajectory carrying two to four synchronised camera streams at 15–30 Hz, a proprioceptive state vector sampled at the same rate, an action vector, and a natural-language task label. The video dominates the bytes; the tabular data dominates the semantics. Any storage format has to serve both without forcing you to choose.

The LeRobot v2.x format answered this with the simplest possible mapping: one Parquet file per episode under data/chunk-000/episode_000000.parquet, and one MP4 per episode per camera under videos/chunk-000/<camera>/episode_000000.mp4. Episode metadata lived in meta/episodes.jsonl, task strings in meta/tasks.jsonl, per-episode statistics in meta/episodes_stats.jsonl. It is a clean design and it works beautifully up to a few thousand episodes.

It stops working at scale for reasons that are arithmetic rather than aesthetic. The DROID collection re-hosted as lerobot/droid_1.0.1 reports 95,658 episodes and 27,630,375 frames in its meta/info.json. Under a per-episode layout with three cameras that is roughly 383,000 files before you count metadata. Git LFS pointers, Hub API listings, os.stat calls at dataset init, S3-style LIST pagination — every one of those scales with file count, not with bytes. Teams hit it as a wall, not a slope.

The alternatives in the wider ML ecosystem had already converged on sharding. RLDS/TFDS, which is how DROID was originally distributed, writes fixed-count TFRecord shards. WebDataset writes tar shards for sequential streaming. Lance offers columnar storage with genuine random access. What none of them model natively is the thing robotics needs most: episode-relative time, and the ability to ask for a window of frames around timestamp t spanning both a Parquet row range and an MP4 byte range.

v3.0 is Hugging Face’s answer, announced on 16 September 2025 and shipped in the lerobot v0.4.0 release tagged 23 October 2025. At the time of writing, v0.5.1 is the most recent tagged release. If you are evaluating this alongside policy architectures, our comparison of GR00T, Gemini Robotics and π0-class VLA models covers the consumers of this data; the official v3.0 documentation covers the producer side.

How the v3.0 Layout Actually Works

LeRobotDataset v3.0 stores many episodes per file instead of one file per episode. Frame-level signals go into chunked Apache Parquet shards, camera streams into per-camera MP4 shards, and a relational metadata table records which shard, which byte offset and which timestamp window each episode occupies. Episode boundaries are resolved by lookup, never by filename.

That one-paragraph summary hides three separate design decisions, and each has consequences you will feel in production.

Figure 1: The v3.0 repository layout — three top-level directories, with the episode metadata table acting as the index that resolves an episode into a data shard and a video shard.

Figure 1 shows the shipped directory structure. meta/ holds four artefacts: info.json with the schema and path templates, stats.json with global normalisation statistics, tasks.parquet mapping task indices to natural-language strings, and meta/episodes/ — itself chunked Parquet — carrying per-episode lengths and offsets. data/ holds the frame-level Parquet shards. videos/ is partitioned first by camera key, then by chunk. The dotted edges are the important part: the episode table is what connects a logical episode to its physical bytes.

Path templates are data, not convention

The most consequential and least remarked-on change is that file paths are declared in meta/info.json rather than hard-coded in the loader. A real v3 dataset carries these two lines:

"data_path":  "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"

Read that carefully. The video template puts {video_key} before the chunk, which inverts the v2.1 ordering of videos/chunk-000/<camera>/. Every tool that glob-matched v2.1 paths breaks, and it breaks silently — a glob that finds nothing returns an empty list rather than an error.

Making paths templated data also means a v3 dataset is self-describing. A reader that honours the template can consume a layout it has never seen. That is a genuine forward-compatibility win, and it is the mechanism that lets the same class read from a local directory, a Hub repository and an HF Storage Bucket without branching.

Shard sizes are byte-budgeted, not count-budgeted

info.json also carries data_files_size_in_mb and video_files_size_in_mb, defaulting to 100 and 500 respectively, plus chunks_size defaulting to 1000 files per chunk directory. The writer accumulates episodes into the current shard until the byte budget is exceeded, then rolls over.

Byte-budgeting rather than count-budgeting is the right call for robotics, because episode lengths vary by an order of magnitude within a single collection session. Count-based shards would produce wildly uneven files; byte-based shards produce even ones. Reading lerobot/droid_1.0.1 confirms the mechanism works: its data shards land between roughly 85.0 MB and 87.7 MB — tight clustering just below the 100 MB ceiling, exactly what a size-triggered rollover produces.

The 5× gap between the data budget and the video budget is not arbitrary. It reflects the actual byte ratio in robot data, and that ratio is larger than most people assume.

The visual-to-tabular ratio is roughly 230 to 1

Take lerobot/svla_so101_pickplace, a 50-episode, 11,939-frame SO-101 dataset at 30 fps with two 480×640 AV1 cameras. Its single data shard is 369,943 bytes. Its two video shards are 45,541,633 and 40,124,489 bytes. Video outweighs tabular data by a factor of about 232.

Work the per-frame numbers. The tabular side stores a 6-float action, a 6-float state and five int64/float32 index columns — about 31 bytes per frame after Parquet compression. The visual side costs roughly 7.2 KB per frame across both cameras, at 480p with a modern codec. That ratio is the single most important number for capacity planning, and it has three direct consequences.

First, compressing your tabular data harder is pointless. Second, your storage bill is a video-encoding decision, not a schema decision — codec, CRF and GOP length move the total by tens of percent, while column pruning moves it by fractions of one percent. Third, and least obvious: because the tabular shards are so small relative to video, you can afford to download all of them even when streaming video. That asymmetry is what makes hybrid local-plus-streaming setups practical, and almost nobody exploits it.

Walking the Read Path End to End

The abstraction v3.0 sells is that dataset[100] returns a dictionary of tensors. The abstraction is honest, but understanding what it costs per sample is the difference between a dataloader that saturates an H100 and one that leaves it 40% idle.

Figure 2: What one __getitem__ call touches — a metadata resolution, a memory-mapped Parquet row read, and one MP4 seek and decode per camera.

Figure 2 traces a single sample. The DataLoader worker holds a global frame index. It resolves that index against the episode metadata table, which yields the owning episode plus its dataset_from_index and dataset_to_index frame offsets and its data/chunk_index and data/file_index. The tabular read is then a memory-mapped row access through PyArrow — cheap, and effectively free on a second pass because the page cache holds it. The expensive half is video: for each camera key, the loader locates the MP4 shard, seeks to the frame’s timestamp within that shard, and decodes.

Metadata is a table, and tables have schemas

The per-episode record in meta/episodes/ carries, at minimum, episode_index, data/chunk_index, data/file_index, dataset_from_index and dataset_to_index — these five appear verbatim in the v2.1→v3.0 conversion source. Alongside them sit the per-camera video coordinates: chunk index, file index, and the from_timestamp/to_timestamp window that bounds the episode inside its shared MP4.

Two things follow. First, dataset_from_index and dataset_to_index are global frame offsets, so the mapping from a global index to an episode is a range lookup over a sorted column — an interval search, not a scan. Second, the video window is stored as timestamps rather than frame numbers, which is the only choice that survives variable-frame-rate encoding but which also means every video read is a seek-by-time, with whatever keyframe-alignment cost that implies.

That second point deserves emphasis. Seeking to an arbitrary timestamp in a long-GOP encode requires decoding from the preceding keyframe forward. A 500 MB shard containing hundreds of concatenated episodes with sparse keyframes turns a “random” frame access into a decode of everything since the last I-frame. This is the precise cost that v3.0’s aggregation introduces and that the per-episode format did not have, because a per-episode MP4 bounded the worst case at one episode’s length.

Metadata itself becomes a scale problem

The streaming design assumes metadata is small — the official write-up puts it at roughly 100 MB for terabyte-scale datasets, and downloads it in full. For most datasets that holds. For lerobot/droid_1.0.1 it does not: meta/episodes/ is seven Parquet files totalling about 592 MB, plus a 1.5 MB tasks.parquet and a 44.6 KB stats.json.

That is a real and under-documented gotcha. Metadata that was a single JSONL file in v2.1 is now itself chunked because it grew past what one file should hold — per-episode statistics for 95,658 episodes across a dozen features is a lot of floats. Any consumer that assumes meta/episodes/ is one file will read 1/7th of DROID and silently produce a dataset with 14% of its episodes. That failure has already been logged against at least one third-party importer, where flattening meta/episodes/chunk-000/file-000.parquet and meta/episodes/chunk-001/file-000.parquet to the same basename caused one to overwrite the other.

Streaming: what is fetched, what is cached

StreamingLeRobotDataset is a drop-in swap for LeRobotDataset that iterates directly from the Hub with no local copy. It is the headline feature of v3.0 and the reason the layout change was worth making.

from lerobot.datasets import StreamingLeRobotDataset

dataset = StreamingLeRobotDataset("lerobot/droid_1.0.1")
for frame in dataset:
    ...

Figure 3: The streaming path — metadata is fetched once, shards are read lazily over HTTPS, frames are decoded on the fly and pass through an in-memory shuffle buffer before a backtrackable iterator assembles time windows.

Figure 3 shows the mechanism. Metadata downloads once. Data and video shards are read lazily over HTTPS through the datasets library’s IterableDataset interface, with video decoded on the fly by torchcodec. Because an iterable gives you next() and nothing else, frame order would otherwise be perfectly sequential — plot retrieved frame index against iteration index and you get a straight line, correlation 1.0. That destroys the i.i.d. assumption behaviour cloning depends on.

The fix is a shuffle buffer of a few thousand frames held in RAM. Frames enter the buffer from randomly chosen shards and are yielded from it in shuffled order, which drops the index/iteration correlation toward zero. buffer_size is therefore a direct randomness-versus-latency dial: a larger buffer gives better mixing and a longer startup stall while it fills.

The startup cost is not a rounding error. Profiling published by the LeRobot team shows streaming training is dominated by stepping the torch.utils.data.DataLoader, which is in turn dominated by the initial buffer fill — both the iteration needed to populate it and the cost of initialising the video-decoder connections. Once the buffer is warm, per-frame throughput is comparable to a memory-resident dataset. The implication for scheduling is blunt: short jobs pay a fixed tax that long jobs amortise, so streaming is a poor fit for hyperparameter sweeps made of many short runs and an excellent fit for one long run.

Time windows without random access

Robot policies rarely consume single frames. Action-chunking methods regress a block of future actions; history-conditioned methods stack past observations. LeRobot expresses this with delta_timestamps, a dictionary of second-offsets per key. In the local dataset this is trivial — index arithmetic. In streaming mode it is not, because you only have next().

The solution is a Backtrackable wrapper in src/lerobot/datasets/utils.py that maintains separate history and lookahead buffers (_back_buf and _ahead_buf) and exposes peek_back(n), peek_ahead(n), prev(), plus can_peek_back() and can_peek_ahead() guards. The guards are the interesting part: they enforce episode boundaries, so a window requested near the start or end of an episode does not silently splice in frames from a different trajectory.

When the requested window runs past a boundary, the dataset returns the available frames plus padding, and a companion mask under <key>.pad_masking. If you write a custom training loop against a streaming dataset and ignore that mask, you will train on padded frames as if they were real — a bug that produces plausible-looking loss curves and a policy that behaves oddly at episode starts. The official numbers also note that enabling delta_timestamps roughly halves streaming throughput, which is what you would expect from multiplying the per-sample video queries.

Multiple workers and multiple GPUs change the calculus

The local and streaming paths parallelise differently, and the difference is easy to get wrong.

With LeRobotDataset, num_workers > 0 in the PyTorch DataLoader gives you independent processes each doing their own memory-mapped Parquet reads and their own video decodes. Because the underlying shards are shared and read-only, the OS page cache is shared too — four workers hitting the same 100 MB data shard pay for one copy in RAM. Video decoding is the part that genuinely scales with workers, since decode is CPU-bound and per-process. This is why the porting guide recommends eight CPUs per task for frame-encoding work: the same reasoning applies in reverse at training time.

With StreamingLeRobotDataset, every worker holds its own shuffle buffer. Four workers with a 10,000-frame buffer each hold 40,000 frames in RAM, not 10,000, and each pays its own fill latency at startup. The buffers also do not coordinate, so total randomisation is better than a single buffer of the same per-worker size but worse than a single buffer of the combined size. Budget memory against num_workers × buffer_size × bytes_per_frame, and remember that bytes_per_frame is dominated by decoded RGB tensors, not by the compressed shard bytes — a 480×640×3 uint8 frame is 921,600 bytes decoded against roughly 3.6 KB on disk, a 250× expansion.

Under distributed data-parallel training the same asymmetry holds. A local dataset shards cleanly by index across ranks. A streaming dataset shards by assigning different shard subsets to different ranks, which means rank-level load balance depends on shard-level episode balance. Byte-budgeted shards contain unequal episode counts by construction, so ranks can finish an epoch at noticeably different times. If you see stragglers in a streaming DDP job, suspect shard skew before you suspect the network.

Capacity Planning and How v3.0 Compares

The 230:1 visual-to-tabular ratio makes storage arithmetic tractable enough to do on a napkin, which is worth doing before you commit to a collection campaign.

Take a plausible fleet: ten SO-101-class arms, two 480p cameras each, 30 fps, collecting six hours of usable demonstration per arm per day. That is 10 × 6 × 3600 × 30 ≈ 6.5 million frames per day. Using the measured svla_so101_pickplace rates — roughly 7.2 KB per frame across two AV1 cameras and about 31 bytes per frame of tabular data — you land near 47 GB per day of video and about 200 MB per day of Parquet. Over a year of five-day weeks that is roughly 12 TB. At the default 500 MB video budget that is about 24,000 video shards; under a per-episode layout with 30-second episodes it would have been on the order of 1.4 million MP4s. That single comparison is the entire argument for v3.0.

The same arithmetic tells you where to push. Halving the camera resolution to 240p cuts the dominant term by roughly a factor of four. Dropping from two cameras to one halves it. Dropping proprioception sampling from 30 Hz to 15 Hz saves almost nothing, because the tabular term was never the problem. Practitioners routinely optimise the wrong one of these three.

Concern	LeRobotDataset v3.0	RLDS / TFDS	WebDataset	Lance
Native episode semantics	Yes — offsets and timestamp windows per episode	Yes — episodes are first-class	No — you encode it yourself	No — you encode it yourself
Random access to a frame	Yes locally; buffer-approximated when streaming	Sequential-first	Sequential-first	Yes, by design
Video handling	MP4 shards, seek-by-timestamp, torchcodec decode	Frames usually embedded per step	Whatever you put in the tar	Blob or per-frame columns
Streaming without download	Yes, Hub-native	Via GCS/TFDS pipelines	Yes, the original use case	Yes, over object storage
Metadata model	Relational Parquet tables	Embedded in the record	Filename conventions	Columnar schema
Best fit	Robot learning on the Hub	TF-centric research pipelines	Very large sequential web-scale corpora	Random-access-heavy multimodal training

Read the table as a statement about defaults rather than capabilities — most of these formats can be bent into most of these shapes. What v3.0 buys is that episode semantics and video windows are modelled rather than convention, which is precisely the part every robotics team otherwise reimplements badly. What it gives up is the unqualified random access that Lance provides, which is why the lerobot-lancedb backends exist as a drop-in alternative for workloads where decode-bound random access dominates.

There is also a recording-side decision that shows up months later as a training-side constraint. The recording CLI exposes --dataset.streaming_encoding, --dataset.encoder_threads and --dataset.rgb_encoder.vcodec, and the codec you pick at collection time determines your seek cost forever after. AV1 at a long GOP produces the smallest archive and the most expensive random seek. H.264 at a short GOP produces a larger archive and cheap seeks. If you expect to train with wide delta_timestamps windows, the short-GOP choice is usually correct even though it costs storage — and re-encoding later with lerobot-edit-dataset --operation.type reencode_videos is possible but is a full rewrite of the video half of the dataset, which is the expensive half.

Trade-offs, Gotchas, and What Goes Wrong

Figure 4: Four symptoms that show up during training, the layout-level cause behind each, and the single remediation they share — verify offsets before trusting a converted dataset.

Figure 4 maps the failures that actually get reported, and every branch traces back to the same root: v3.0 moved truth from filenames into a metadata table, so metadata bugs are now silent data-corruption bugs.

The shard-rollover offset bug. The v2.1→v3.0 converter builds each episode’s metadata record before testing whether the episode fits in the current shard. When it does not fit, the episode is written into the new shard but its already-constructed record still carries the previous data/chunk_index and data/file_index. One episode per rollover ends up pointing at the wrong file. Downstream this surfaces as scrambled dataset_from_index/dataset_to_index values and a pad flag that reads true for nearly every frame past the first few episodes. Nothing errors. The loss curve just refuses to descend.

Shard-hopping on local disk. The streaming reader’s default strategy picks a random shard, materialises one frame, and breaks to pick another random shard. Over the network that is acceptable. On local disk it is a pathological access pattern — thousands of tiny seeks across large Parquet files where a sequential read would be an order of magnitude cheaper. This is a live design discussion in the project, with proposals for a frames_per_shard_visit knob that reads K frames per shard visit while keeping the shuffle buffer. Until that lands, streaming from a locally mounted copy can be slower than streaming from the Hub.

Stats drift after concatenation. Global normalisation statistics live in stats.json and are derived from per-episode statistics. Merge two datasets, delete episodes, or re-split, and those aggregates must be recomputed — but a stale stats.json is a perfectly valid file. Your policy then normalises with the wrong mean and standard deviation. Use lerobot-edit-dataset for merge, split, delete and modify_tasks rather than hand-editing, because it updates meta/tasks.parquet, the task_index column in the data shards, the tasks column in the episode table and total_tasks in info.json together.

Unfinalised writes. v3.0 uses incremental Parquet writing with buffered metadata. You must call dataset.finalize() before push_to_hub(). Skip it and the Parquet footers are never written; the files are corrupt and the dataset will not load. This is the most common self-inflicted wound in new v3 pipelines.

Stale READMEs and stale docs. The auto-generated Hub dataset card is not regenerated by conversion, so a migrated repository can still advertise codebase_version: "v2.1" in its card while meta/info.json correctly says v3.0 — lerobot/svla_so101_pickplace currently shows exactly this. The official docs likewise still describe meta/tasks.jsonl, while shipped datasets carry meta/tasks.parquet. Trust meta/info.json over any prose.

Migrating from v2.1

There is no graceful degradation. Loading a v2.1 dataset with a v3-era lerobot raises a backward-compatibility error stating the format is not compatible and pointing you at the converter. Plan the migration as a project, not a command.

python -m lerobot.datasets.v30.convert_dataset_v21_to_v30 --repo-id=<HF_USER/DATASET_ID>

The converter aggregates episode-0000.parquet, episode-0001.parquet, … into file-0000.parquet, does the same for MP4s, and writes the episode offsets. Three properties of it matter for planning.

It is sequential. The maintainers have said plainly that the script was not built for large-scale datasets and there is an open request to make it distributed. For a multi-terabyte collection you should expect the porting guide’s DROID figures as your order of magnitude: 1.7 TB of RLDS input, roughly 400 GB of LeRobot output, seven-plus days for a single-machine port and three-plus days to upload. The documented path for anything at that scale is SLURM plus datatrove, running per-shard workers and then an aggregation pass.

It has known bugs. Beyond the rollover-offset issue above, conversion is reported to produce broken metadata when the source Parquet embeds raw images rather than referencing video. Convert a sample first, verify, then convert the rest.

Your v2.1 data does not disappear. Datasets migrated on the Hub keep the old revision accessible, so LeRobotDataset(repo_id, revision="v2.1") still resolves the pre-migration state. That is your rollback, and it is also the ground truth you should diff against — converting a small dataset and asserting frame-by-frame equality against its v3 counterpart is the cheapest possible regression test.

Practical Recommendations

Treat v3.0 as a storage-engineering decision with a training-throughput consequence, and make the decision per workload rather than per team.

If your dataset fits comfortably on local NVMe and you will run many epochs, use the local LeRobotDataset. You get true random access, the page cache does the work, and you avoid the buffer-fill tax entirely. If the dataset does not fit, or you will make exactly one pass over it, stream — the download you skip is worth more than the startup stall. If video decode is your measured bottleneck rather than I/O, look at the lerobot-lancedb backends (LeRobotLanceDataset for image datasets, LeRobotLanceVideoDataset for video), which subclass LeRobotDataset and target exactly this case, or re-encode with a shorter GOP using lerobot-edit-dataset --operation.type reencode_videos to make timestamp seeks cheaper.

Measure before you tune. The project ships a profiler that reports both throughput and the frame-index/iteration-index correlation:

python -m lerobot.scripts.profile_streaming --repo-id lerobot/svla_so101_pickplace

A correlation near zero means your buffer is doing its job. A correlation near one means you are training on a nearly sequential stream regardless of what the config says.

Migration and operations checklist

• [ ] Read meta/info.json and confirm codebase_version — never trust the Hub dataset card.
• [ ] Convert one small dataset first; diff it frame-by-frame against revision="v2.1".
• [ ] After conversion, verify that every episode’s data/chunk_index and data/file_index point at a shard that actually contains its frame range, with particular attention to episodes adjacent to shard rollovers.
• [ ] Enumerate every file under meta/episodes/, not just the first, and preserve chunk directories when caching so basenames cannot collide.
• [ ] Call dataset.finalize() before push_to_hub() in any recording or generation script.
• [ ] Budget storage from video, not schema: assume roughly 200:1 visual-to-tabular bytes and size your encoder settings accordingly.
• [ ] If you use delta_timestamps, consume the <key>.pad_masking tensor in your loss.
• [ ] Recompute statistics after any merge, split or episode deletion; prefer lerobot-edit-dataset over manual edits.
• [ ] For streaming jobs, size buffer_size against job length — long runs can afford a big buffer, sweeps cannot.

Frequently Asked Questions

When was LeRobotDataset v3.0 released?

The format was announced on the Hugging Face blog on 16 September 2025, with a companion post on streaming two days later. It shipped in a stable library release with lerobot v0.4.0, tagged 23 October 2025; before that it was available only from a pinned pre-release commit. As of this writing the most recent tagged release is v0.5.1, whose changelog includes further dataset refactoring, notably a split of LeRobotDataset into separate reader and writer classes and a revision-safe Hub cache for downloaded datasets.

Do v2.1 datasets still load in current lerobot?

No. Attempting to load a v2.1 dataset raises a backward-compatibility error explaining that the v3.0 format is not backward compatible and directing you to lerobot.datasets.v30.convert_dataset_v21_to_v30. The conversion is one-way in the tooling, but it is reversible operationally: datasets migrated on the Hub retain the earlier revision, so passing revision="v2.1" to LeRobotDataset still resolves the pre-migration files.

How large are v3.0 shards in practice?

meta/info.json declares the targets. The defaults observed on published datasets are 100 MB for data shards and 500 MB for video shards, with chunks_size of 1000 files per chunk directory. Because rollover is triggered by accumulated bytes, real shards cluster just under the ceiling — lerobot/droid_1.0.1 data shards measure roughly 85–88 MB each. Both budgets are configurable at write time.

Does streaming download anything to disk?

Metadata, yes; bulk data, no. StreamingLeRobotDataset downloads the meta/ tree in full because it is small relative to the payload, then reads Parquet and MP4 shards lazily over HTTPS with torchcodec decoding frames on the fly. Treat the “metadata is negligible” assumption with care at extreme scale: DROID’s meta/episodes/ alone is roughly 592 MB across seven Parquet files, because per-episode statistics for nearly 96,000 episodes are not small.

Why is my streamed training run slow to start?

The shuffle buffer. Streaming yields frames from an in-memory buffer to break the sequential ordering an iterable would otherwise produce, and that buffer must be filled before the first batch emerges. Published profiling attributes the dominant startup cost to exactly this, plus initialising the video-decoder connections. Steady-state throughput afterwards is comparable to a local dataset. Reducing buffer_size shortens the stall at the cost of randomisation quality.

Can I train directly from an HF Storage Bucket instead of a dataset repo?

Yes. StreamingLeRobotDataset accepts repo_type="bucket" to stream from a Hugging Face Storage Bucket using the same code path as a Hub dataset repository, and lerobot-train exposes both through --dataset.streaming=true and --dataset.repo_type=bucket. This is the practical route when your collection pipeline writes shards continuously and you do not want to materialise a versioned dataset repo for every snapshot.

Further Reading

• Diffusion policy for robot manipulation and imitation learning — the policy class that consumes these action-chunk windows.
• VLA models compared: GR00T, Gemini Robotics and π0 — what the data at this scale is actually being used to train.
• Physical Intelligence π0.5 robot foundation model — a concrete case study in large-scale robot pretraining.
• Isaac Lab reinforcement-learning training tutorial — the simulation-side counterpart to real-robot data collection.
• LeRobotDataset v3.0 documentation — the primary specification for layout, streaming and migration.
• Porting large datasets to LeRobot Dataset v3.0 — the DROID walkthrough, including the SLURM and datatrove path for multi-terabyte ports.

By Riju — about