Please publish your failure episodes
Datasets with 99% success rates are marketing, not data. Failures are where the policy gradient actually lives.
#quality#labels
Trending in robotics
A mixed feed of discussions, datasets, robot milestones, research and funded bounties — ranked by what your network is engaging with.
Hot discussions
Scaling laws for embodied data: episodes vs environments
Environment diversity contributes roughly 3x more to out-of-distribution success than raw episode count past 5K per task.
Our home robot mistook a cat for a laundry pile. Twice.
Both episodes are in the dataset, uncut. The classifier confidence was 0.91. Anyone else fighting soft-textured false positives?
CALVIN
Single Arm · 24K episodes · RGB Video + Depth + Joint Positions
Stretch 3
Mobile Robot · 214K episodes · 61 tasks
Reproducing published success rates: results so far
Six papers, real hardware, same tasks. Median gap is 14 points. The two that matched both published their failure episodes.
New Vision-Language-Action model beats larger baselines with 4x less data
Key claim: action chunking with a frozen vision backbone matters more than parameter count below 10M episodes. I reproduced the door subset on real hardware and got within 3 points.
How many episodes until a task actually generalizes?
Our internal curve flattens hard around 8K episodes per task, but only if the environments vary. 8K in one kitchen is worth maybe 1.5K across ten.
Datasets gaining traction
Real-world IndexedOpen X-Embodiment
Source: Open X-Embodiment Collaboration
ManipulationCustomLab
- 1.0M
- 5.6K
- 3
Open Source10 Hz
Real-world IndexedAgiBot World Beta
Source: AgiBot
HouseholdHumanoidHome
- 1.0M
- 5.6K
- 8
Open Source30 Hz
Real-world IndexedLanguage-Table
Source: Google DeepMind Robotics
ManipulationSingle ArmLab
- 442K
- 2.5K
- 2
Open Source10 Hz
Real-world IndexedRoboNet
Source: UC Berkeley RAIL
ManipulationSingle ArmLab
- 162K
- 900
- 2
Open Source5 Hz
Research the community is reading
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Pools 1M+ real robot trajectories from 22 embodiments and 34 labs; co-training raises RT-1-X success by 50% over per-robot baselines.
Open X-Embodiment Collaboration · ICRA 2024 · ▲ 1.9K
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Co-fine-tuning a web-scale VLM on robot trajectories roughly doubles generalization to unseen objects and instructions.
A. Brohan, N. Brown, et al. · CoRL 2023 · ▲ 1.5K
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Denoising diffusion over action sequences improves average success by 46.9% across 15 manipulation benchmark tasks.
C. Chi, S. Feng, Y. Du, et al. · RSS 2023 · ▲ 1.3K
RT-1: Robotics Transformer for Real-World Control at Scale
130K demonstrations over 17 months and 700+ tasks; 97% seen-task success with strong transfer to new distractors and backgrounds.
A. Brohan, N. Brown, et al. · RSS 2023 · ▲ 1.2K
π0: A Vision-Language-Action Flow Model for General Robot Control
Flow matching over action chunks on cross-embodiment data enables laundry folding and table bussing after task-specific fine-tuning.
Physical Intelligence · arXiv preprint · ▲ 1.2K