{"success":true,"data":{"author":"NVIDIA Corporation","bucketAccessType":"public","bucketName":"nvidia/PhysicalAI-Robotics-Manipulation-Augmented","bucketPrefix":"","category":"Robotics","cloudProvider":"huggingface","createdAt":"2026-11-21T22:15:32.629Z","description":"Synthetic robot manipulation demonstrations from NVIDIA in which a single Franka Panda arm stacks three cubes in a fixed order (blue bottom, red middle, green top) in Isaac Lab. Ten human teleoperation demonstrations were expanded to 1,000 trajectories with Isaac Lab's Mimic tool (MimicGen), and the table-camera RGB of every trajectory was then re-rendered with the Cosmos Transfer1 world model for photorealistic domain adaptation, so the same motions exist with plain simulator images and with augmented, realistic-looking images. Each demonstration holds a 7-D action (6-D relative end-effector motion plus gripper), joint positions and velocities, end-effector pose, the three cube poses, and 200x200 table-camera RGB, depth, segmentation and surface-normal images plus wrist-camera RGB. The two sets are single HDF5 files, mimic_dataset_1k.hdf5 and cosmos_dataset_1k.hdf5, readable with h5py and replayable with Isaac Lab's replay_demos.py; a helper script exports MP4 video, and robomimic BC-RNN visuomotor checkpoints are included. About 78 GB (measured 2026-11-21). Use it for behavior cloning, sim-to-real visual transfer and data augmentation studies. NVIDIA releases it under CC-BY-4.0 and states it is ready for commercial use; cite MimicGen (Mandlekar et al., CoRL 2023) and Cosmos-Transfer1 (arXiv:2503.14492).","displayTitle":"Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab)","huggingFaceUrl":"https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented","id":"059e3d01-06e9-5dba-a9f5-b19c3519c48d","license":"CC-BY-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented","objectCountEstimate":104,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/2503.14492","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"nvidia/PhysicalAI-Robotics-Manipulation-Augmented\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"nvidia/PhysicalAI-Robotics-Manipulation-Augmented\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=nvidia, size_categories:1K<n<10K, downloads=493"],"format":"mixed","generatedAt":"2026-11-21T22:31:22.167Z","layout":{"example":"cosmos_dataset_1k.hdf5","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"894844460d0a1a0a000000000008080004001000000000000000000000000000ffffffffffffffff5ac5d9370a000000ffffffffffffffff0000000000000000600000000000000001000000000000008800000000000000a8020000000000000100010001000000180000000000000011001000000000008800000000000000a8020000000000005452454500000100ffffffffffffffffffffffffffffffff00000000000000007b4f46f105000000080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","encoding":"hex","key":"cosmos_dataset_1k.hdf5","truncatedAt":512}],"sampleObjects":["cosmos_dataset_1k.hdf5","hdf5_to_mp4.py","mimic_dataset_1k.hdf5","robomimic_bc_rnn_visuomotor_models/franka_stack_mimic_1k_table_only/20250416141542/config.json","robomimic_bc_rnn_visuomotor_models/franka_stack_mimic_1k_table_only/20250416141542/logs/log.txt"],"sidecars":[{"bytes":"---\nsize_categories:\n- 1K<n<10K\nviewer: false\ntask_categories:\n- robotics\ntags:\n- robotics\n---\n## Dataset Description:\n\nThis is a fully annotated, synthetically generated dataset consisting of 1,000 demonstrations of a single Franka Panda robot arm performing a fixed-order three-cube stacking task in Isaac Lab. The robot consistently stacks cubes in the order: blue (bottom) → red (middle) → green (top).\n\nThe dataset was produced using the following pipeline:\n- Collected 10 human teleoperation demonstrations of the stacking task.\n- Used Isaac Lab’s **Mimic** tool [1] to simulate 1,000 high-quality trajectories in Isaac Sim.\n- Applied **Cosmos Transfer1** model [2] to augment the RGB visuals from the table camera with photorealistic domain adaptation.\n\nEach demonstration includes synchronized multimodal data:\n- RGB videos from both a table-mounted and wrist-mounted camera.\n- Depth, segmentation, and surface normal maps from the table camera.\n- Full low-level robot and object states (joints, end-effector, gripper, cube poses).\n- Action sequences executed by the robot.\n\nThis dataset is ideal for behavior cloning, policy learning, and generalist robotic manipulation research.\n\nThis dataset is ready for commercial use.\n\n## Dataset Owner(s):\n\nNVIDIA Corporation\n\n## Dataset Creation Date:\n\n05/14/2025\n\n## License/Terms of Use:\n\nThis dataset is governed by the Creative Commons Attribution 4.0 International License (CC-BY-4.0).\n\n## Intended Usage:\n\nThis dataset is intended for:\n- Training robot manipulation policies using behavior cloning.\n- Research in generalist robotics and task-conditioned agents.\n- Sim-to-real transfer studies and visual domain adaptation.\n\n## Dataset Characterization:\n\n**Data Collection Method**\n  * Automated\n  * Automatic/Sensors\n  * Synthetic\n\n10 human teleoperated demonstrations were used to bootstrap a Mimic-based simulation [1] in Isaac Sim. All 1,000 demos are generated automatically followed by domain-randomized visual augmentation using Cosmos Transfer1 [2].\n\n**Labeling Method**\n  * Not Applicable\n\n## Dataset Format:\n\nWe provide the Mimic generated 1000 demonstrations and the 1000 Cosmos augmented demonstrations in separate HDF5 dataset files (`mimic_dataset_1k.hdf5` and `cosmos_dataset_1k.hdf5` respectively). Each demo in each file consists of a time-indexed sequence of the following modalities:\n\n**Actions**\n- 7D vector: 6D relative end-effector motion + 1D gripper action\n\n**Observations**\n- Robot states: Joint positions, velocities, and gripper open/close state\n- EEF states: End-effector 6-DOF pose\n- Cube states: Poses (positions + orientations) for blue, red, and green cubes\n- Table camera visuals:\n  - 200×200 RGB\n  - 200×200 Depth\n  - 200×200 Segmentation mask\n  - 200×200 Surface normal map\n- Wrist camera visuals:\n  - 200×200 RGB\n\nThe datasets' trajectories can be replayed in simulation using Isaac Lab (refer to [this script](https://github.com/isaac-sim/IsaacLab/blob/main/scripts/tools/replay_demos.py)). The videos can be extracted in MP4 format using [this script](https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented/blob/main/hdf5_to_mp4.py).\n\n## Dataset Quantification:\n\n**Record Count**\n  * `mimic_dataset_1k`\n    * Number of demonstrations/trajectories: 1000\n    * Number of RGB videos: 2000 (1000 table camera + 1000 wrist camera)\n    * Number of depth videos: 1000 (table camera)\n    * Number of segmentation videos: 1000 (table camera)\n    * Number of normal map videos: 1000 (table camera)\n\n  * `cosmos_dataset_1k`\n    * Number of demonstrations/trajectories: 1000\n    * Number of RGB videos: 2000 (1000 table camera + 1000 wrist camera)\n    * Number of depth videos: 1000 (table camera)\n    * Number of segmentation videos: 1000 (table camera)\n    * Number of normal map videos: 1000 (table camera)\n\n\n**Total Storage**\n  * 69.4 GB\n\n## Reference(s):\n\n```\n[1] @inproceedings{mandlekar2023mimicgen,\n    title={MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations},\n    author={Mandlekar, Ajay and Nasiriany, Soroush and Wen, Bowen a","encoding":"utf8","key":"README.md","name":"README.md","size":5783,"truncated":true}],"sourceObjectKey":"cosmos_dataset_1k.hdf5","starterCell":"from huggingface_hub import hf_hub_download\npath = hf_hub_download(repo_id=\"nvidia/PhysicalAI-Robotics-Manipulation-Augmented\", filename=\"cosmos_dataset_1k.hdf5\", repo_type=\"dataset\")\nprint(path)","starterSamplePath":"cosmos_dataset_1k.hdf5","totalObjects":104},"summary":"Synthetic robot manipulation demonstrations from NVIDIA in which a single Franka Panda arm stacks three cubes in a fixed order (blue bottom, red middle, green top) in Isaac Lab.","title":"Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab)","updatedAt":"2026-11-22T01:05:15.616Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented"}}