{"success":true,"data":{"author":"CaryX AI","bucketAccessType":"public","bucketName":"CaryxAI/everyday-manipulation-3d-raw","bucketPrefix":"","category":"Robotics","cloudProvider":"huggingface","createdAt":"2026-11-21T22:15:33.171Z","description":"An open sample of raw 3D egocentric manipulation data from CaryX AI: 1,513 clips, 10.28 hours and about 280 GiB of everyday two-handed manipulation by 4 participants across 10 tasks in 42 recording sittings, captured with a chest-mounted iPhone Pro running the Record3D app. Each clip is the app's .r3d file (a ZIP) with the audio removed and the sensor streams unmodified: synchronized RGB frames (1440x1920 portrait or 1920x1440 landscape), metric LiDAR depth in float32 meters (192x256 or 256x192, LZFSE-compressed), ARKit per-pixel depth confidence, per-frame ARKit 6-DoF camera poses and per-frame camera intrinsics in a JSON metadata member. 1,027 clips are portrait and 486 landscape, and index.csv and index.jsonl give each clip's resolution and orientation; the card explains how to scale the RGB intrinsics to depth pixels before back-projecting to point clouds. About 300 GB in 1,518 files (measured 2026-11-21). Useful for hand-object interaction, 3D reconstruction and learning robot manipulation from human video. Released under CC BY 4.0, including commercial use with attribution.","displayTitle":"Everyday Manipulation 3D: 1,513 Raw iPhone LiDAR RGB-D Egocentric Clips of Two-Handed Tasks with ARKit Camera Poses (CaryX AI)","huggingFaceUrl":"https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw","id":"be328a4e-9a13-54b7-8442-ce6056795309","license":"CC-BY-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw","objectCountEstimate":1518,"probeStatus":"ok","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"CaryxAI/everyday-manipulation-3d-raw\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"CaryxAI/everyday-manipulation-3d-raw\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=CaryxAI, license:cc-by-4.0, size_categories:1K<n<10K, format:csv, downloads=3440"],"format":"mixed","generatedAt":"2026-11-21T22:31:32.737Z","layout":{"example":"index.csv","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"clip_id,file,episode_id,task,task_phrase,participant_id,day,shoot_day,start_time,device,duration_s,fps,n_frames,rgb_w,rgb_h,depth_w,depth_h,orientation,depth_conf_high,hands_visible,environment,lighting,primary_objects,secondary_objects,description,faces_visible,other_people_visible,screens_or_documents_visible,label_source,size_bytes\r\nEM1-0001,clips/D1/EM1-0001.r3d,EM1-D1-E02,wipe_surface,wiping a table,M1,2026-07-20,D1,14:12:07,12pro,30.97,59.808,1852,1920,1440,256,192,landscape,0.6483,both,dining table,b","encoding":"utf8","key":"index.csv","truncatedAt":512}],"sampleObjects":["clips/D1/EM1-0001.r3d","clips/D1/EM1-0002.r3d","clips/D1/EM1-0003.r3d","clips/D1/EM1-0004.r3d","clips/D1/EM1-0005.r3d"],"sidecars":[{"bytes":"Creative Commons Attribution 4.0 International (CC BY 4.0)\n\nCopyright (c) 2026 CaryX AI\n\nEveryday Manipulation 3D (raw RGB-D) is licensed under the Creative Commons\nAttribution 4.0 International License.\n\nYou are free to:\n\n  Share  — copy and redistribute the material in any medium or format\n  Adapt  — remix, transform, and build upon the material\n           for any purpose, including commercially.\n\nUnder the following terms:\n\n  Attribution — You must give appropriate credit, provide a link to the license,\n  and indicate if changes were made. You may do so in any reasonable manner, but\n  not in any way that suggests the licensor endorses you or your use.\n\n  No additional restrictions — You may not apply legal terms or technological\n  measures that legally restrict others from doing anything the license permits.\n\nFull licence text: https://creativecommons.org/licenses/by/4.0/legalcode\n\n\nSCOPE\n\nThis licence covers the capture data in this repository: RGB frames, ARKit LiDAR\ndepth, per-frame camera pose, and per-frame intrinsics, together with the\naccompanying index.\n\nEach clip is the recording app's original .r3d archive with the audio track\nremoved. No other member was altered: every remaining entry is byte-identical to\nthe file the capture device wrote.\n\nAnnotations built on these clips are released separately and may carry different\nterms. See https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d\n\n\nATTRIBUTION\n\n  CaryX AI. Everyday Manipulation 3D (raw RGB-D). 2026.\n  https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw\n\nContact: founders@caryx.ai\n","encoding":"utf8","key":"LICENSE","name":"LICENSE","size":1610,"truncated":false},{"bytes":"---\nlicense: cc-by-4.0\npretty_name: \"Everyday Manipulation 3D (raw RGB-D)\"\nsize_categories:\n- 1K<n<10K\ntask_categories:\n- robotics\n- video-classification\ntags:\n- egocentric\n- robotics\n- manipulation\n- rgbd\n- lidar\n- depth\n- hand-object-interaction\n- embodied-ai\n- arkit\n- record3d\nconfigs:\n- config_name: default\n  data_files:\n  - split: train\n    path: index.csv\n---\n\n# Everyday Manipulation 3D (raw RGB-D)\n\n**1,513 clips · 10.28 hours · 279 GiB · 4 participants · 10 manipulation tasks · 42 recording sittings**\n\nChest-mounted iPhone Pro capture of everyday two-handed manipulation by\n[CaryX AI](https://caryx.ai). Clips were recorded with\n[Record3D](https://record3d.app/), an iOS app that captures the\niPhone's LiDAR RGB-D stream. Each clip is the app's `.r3d` recording with the\naudio track removed; the sensor streams are unmodified: **synchronised RGB,\nmetric LiDAR depth, per-frame ARKit 6-DoF camera pose, and per-frame camera\nintrinsics.**\n\n## Why this exists\n\nAn open sample of raw 3D egocentric manipulation data for training and building\nmodels, released under CC BY 4.0, including commercial use with attribution.\nFor labels, larger volumes, or questions, reach out to\n[founders@caryx.ai](mailto:founders@caryx.ai).\n\n## What's in a clip\n\nEach `.r3d` is a ZIP containing:\n\n| member | what it is |\n|---|---|\n| `metadata` | JSON: `K` (3×3 intrinsics, column-major), `poses` (per-frame `[qx,qy,qz,qw,tx,ty,tz]`), `frameTimestamps`, `perFrameIntrinsicCoeffs`, `initPose`, `cameraType`, `w`/`h`, `dw`/`dh`, `fps` |\n| `rgbd/<i>.jpg` | RGB frame |\n| `rgbd/<i>.depth` | metric depth, float32 metres, **LZFSE-compressed** |\n| `rgbd/<i>.conf` | ARKit per-pixel confidence, uint8 (0=low, 1=med, 2=high), **also LZFSE-compressed** |\n| `icon` | a small JPEG thumbnail written by the app |\n\n**Both orientations are present.** 1,027 clips are portrait and 486\nare landscape, and the frame dimensions swap accordingly:\n\n| orientation | clips | RGB | depth |\n|---|---:|---|---|\n| portrait | 1,027 | 1440×1920 | 192×256 |\n| landscape | 486 | 1920×1440 | 256×192 |\n\nEvery clip's actual `w`/`h`/`dw`/`dh` are in its own `metadata`, and in\n`index.csv` as `rgb_w`, `rgb_h`, `depth_w`, `depth_h`, `orientation`. **Read\nthem per clip.** A loader that assumes one orientation will transpose every\nframe of the other.\n\n**`K` is in RGB pixel units, so it cannot be applied to depth pixel coordinates\ndirectly.** Depth shares the camera's optical geometry but is sampled on a much\nsmaller grid, so before back-projecting you must either map the depth pixel into\nRGB pixel space, or scale the intrinsics by the resolution ratio:\n\n```python\n# map a depth pixel (px, py) into RGB pixel space, then use K unchanged\nu = px / (dw - 1) * w\nv = py / (dh - 1) * h\nX = (u - cx) / fx * Z;  Y = (v - cy) / fy * Z          # Z = depth_m[py, px]\n\n# equivalently, scale the intrinsics once into depth pixel space\nfx_d, cx_d = fx * dw / w, cx * dw / w\nfy_d, cy_d = fy * dh / h, cy * dh / h\n```\n\nDo not assume `depth[y, x]` corresponds to `rgb[y, x]`.\n\nReading depth needs LZFSE (`pip install pyliblzfse`).\n\n## Tasks\n\n`sweep` 351 clips (1.27 h) · `fold_laundry` 290 clips (2.62 h) ·\n`pick_and_place` 230 clips (0.89 h) · `scoop_grain` 212 clips (1.39 h) ·\n`jar_open_close` 139 clips (0.83 h) · `wash_dishes` 107 clips (2.28 h) ·\n`plug_charger` 68 clips (0.24 h) · `tie_shoelaces` 63 clips (0.30 h) ·\n`pour_liquid` 28 clips (0.25 h) · `wipe_surface` 25 clips (0.21 h)\n\nRecorded by 4 adults, each wearing a chest-mounted iPhone. The\n`participant_id` column in `index.csv` groups clips by wearer.\n\n## Devices and capture conditions\n\n| device | clips | median depth conf | frame rates present |\n|---|---:|---:|---|\n| iPhone 12 Pro | 867 | 87.8% | 15 fps (514), 30 fps (323), 60 fps (13), plus 17 at irregular rates |\n| iPhone 17 Pro | 646 | 69.2% | 60 fps (646) |\n\nClip length: min 3.4 s · median 16.3 s · max 181.9 s.\n\n## Identifiers\n\n| id | example | meaning | stable? |\n|---|---|---|---|\n| `clip_id` | `EM1-0254` | the clip. Assigned once in capture order | **permanent** |\n| `episode_id` | `EM1-D2-E06` | th","encoding":"utf8","key":"README.md","name":"README.md","size":8813,"truncated":true}],"sourceObjectKey":"index.csv","starterCell":"import pandas as pd\ndf = pd.read_csv(\"hf://datasets/CaryxAI/everyday-manipulation-3d-raw/index.csv\")\ndf.head()","starterSamplePath":"clips/D1/EM1-0001.r3d","totalObjects":1518},"summary":"An open sample of raw 3D egocentric manipulation data from CaryX AI: 1,513 clips, 10.28 hours and about 280 GiB of everyday two-handed manipulation by 4 participants across 10…","title":"Everyday Manipulation 3D: 1,513 Raw iPhone LiDAR RGB-D Egocentric Clips of Two-Handed Tasks with ARKit Camera Poses (CaryX AI)","updatedAt":"2026-11-22T00:51:00.959Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/CaryxAI/everyday-manipulation-3d-raw"}}