AI APIs & libraries

GPT models for reasoning, vision and tool calling; often used for task planning and language-conditioned control.

Claude models with long context and tool use, useful for agentic robot planning and code generation.

Multimodal models with native video and image understanding; Gemini Robotics builds on this family.

European LLM provider; several models also ship as open weights you can run on-robot.

Very low-latency inference for open models, handy when a robot loop can't wait on slow responses.

Hosted inference and fine-tuning for many open-weight models.

Run community-published models (vision, segmentation, depth) through a simple API.

Call thousands of Hub models, including robotics policies, without hosting them yourself.

Run LLMs and VLMs locally on a workstation or Jetson with an OpenAI-style API.

PyTorch library and dataset format for imitation learning on real robots (ACT, Diffusion Policy, pi0).

7B open vision-language-action model and fine-tuning code trained on Open X-Embodiment.

Generalist transformer policy, small enough to fine-tune on a single GPU.

Open release of the pi0 / pi0-FAST flow-matching VLA models with training code.

Reference implementation of visuomotor policies trained as diffusion models.

Framework and benchmark for learning from human demonstrations.

Reliable PyTorch implementations of RL algorithms (PPO, SAC, TD3).

Modular RL library with first-class support for Isaac Lab massively parallel envs.

Fast GPU PPO used for legged locomotion training.

GPU-parallel robot learning framework on Isaac Sim.

Fast, accurate physics engine; MJX runs it on GPU/TPU via JAX.

Python-native multi-physics simulator aimed at very fast robot learning.

GPU-parallel manipulation benchmark with rendering for vision policies.

Lightweight, easy-to-install physics sim; good for prototyping.

ROS-integrated simulator for full robot systems and sensors.

Large-scale simulated kitchens for household manipulation tasks.

Promptable image and video segmentation.

Open-vocabulary object detection from text prompts.

Real-time detection, segmentation and pose that runs on edge devices.

Monocular depth estimation from a single RGB camera.

6D pose estimation and tracking of novel objects.

Point cloud and 3D data processing library.

Standard robot software framework for messaging, drivers and tooling.

Motion planning, kinematics and collision checking for arms.

Low-latency dataflow middleware designed for AI-driven robots.

Fast rigid-body dynamics and kinematics.

Log and visualize multimodal robot data (images, point clouds, poses) over time.

Visualization and data management for ROS bags and MCAP recordings.

Episode dataset format behind Open X-Embodiment.

Experiment tracking and model evaluation for training runs.