{"success":true,"data":{"author":"IBM Research and ESA Phi-lab (TerraMind team)","bucketAccessType":"public","bucketName":"ibm-esa-geospatial/TerraMesh","bucketPrefix":"","category":"Earth Observation & Geospatial","cloudProvider":"huggingface","createdAt":"2026-11-14T14:15:15.700Z","description":"Planetary-scale, analysis-ready multimodal dataset for training Earth observation foundation models, released by IBM and ESA Phi-lab and used to pretrain TerraMind. It merges more than 9 million co-registered patches drawn from MajorTOM-Core and SSL4EO-S12 v1.1, each with Sentinel-2 L1C, Sentinel-2 L2A and an RGB composite, Sentinel-1 (either GRD or RTC backscatter), Copernicus DEM, NDVI and ESRI/Impact Observatory land use and land cover. The listing (measured 2026-11-14) shows train/ and val/ splits with one folder per modality (DEM, LULC, NDVI, S1GRD, S1RTC, S2L1C, S2L2A, S2RGB), each holding tar shards of up to 10,240 samples stored as Zarr zip files, plus train_metadata.parquet (about 1.9 GB), val_metadata.parquet, normalisation statistics and a terramesh.py loader; the repository uses about 16.9 TB. Stream with webdataset through the provided loader or open samples with zarr (<= 2.18) and xarray. The card documents fixes to Sentinel-2 radiometric offsets and Sentinel-1 alignment. Released under CC-BY-SA-4.0; cite arXiv:2504.11172.","displayTitle":"TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data","id":"b4660a74-2e7a-5747-af42-ffe227455bdb","license":"CC-BY-SA-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/ibm-esa-geospatial/TerraMesh","objectCountEstimate":6900,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/2504.11172","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"ibm-esa-geospatial/TerraMesh\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"ibm-esa-geospatial/TerraMesh\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=ibm-esa-geospatial, license:cc-by-sa-4.0, size_categories:1M<n<10M, format:parquet, downloads=15533"],"format":"mixed","generatedAt":"2026-11-14T14:15:27.587Z","layout":{"example":"train_metadata.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415ca2f15da084c15d2011500120000e5174c190000006d616a6f72746f6d5f73686172645f30010110312e7461725e1d0000326e1d0000336e1d0000346e1d0000356e1d0000366e1d0000376e1d0000386e1d0000396a1d000431306e1d006e220100316e220100316e220100316e220100316e220100316e220100316e220100316e220100316e220100326e220100326e220100326e220100326e220100326e220100326e220100326e220100326e220100326e220100326e220100336e220100336e220100336e220100336e220100336e220100336e220100336e220100336e220100336e220100336e220100346e220100346e220100346e220100346e220100346e220100346e220100346e220100346e220100346e220100346e220100356e220100356e220100356e220100356e220100356e220100356e220100356e220100356e220100356e220100356e220100366e220100366e220100366e220100366e220100366e220100366e220100366e220100366e220100366e220100366e220100376e220100376e220100376e220100376e220100376e220100376e220100376e220100376e220100376e220100376e220100386e220100386e220100386e220100386e220100386e220100386e220100386e220100386e220100386e220100386e220100396e220100396e220100396e220100396e","encoding":"hex","key":"train_metadata.parquet","truncatedAt":512}],"sampleObjects":["assets/examples.png","assets/one_degree_map.png","assets/radar.png","assets/timestamps.png","terramesh.py"],"sidecars":[{"bytes":"---\nlicense: cc-by-sa-4.0\nsize_categories:\n- 1M<n<10M\npretty_name: TerraMesh\ntags:\n- Earth observation\n- Multimodal\n- Pre-training\ntask_categories:\n- image-feature-extraction\nlibrary_name: webdataset\n# viewer: false\nconfigs:\n- config_name: default\n  data_files:\n  - split: train\n    path: \"train_metadata.parquet\"\n  - split: validation\n    path: \"val_metadata.parquet\"\n---\n\n# TerraMesh\n\nA planetary‑scale, multimodal analysis‑ready dataset for Earth‑Observation foundation models: **TerraMesh** merges data from **Sentinel‑1 SAR, Sentinel‑2 optical, Copernicus DEM, NDVI, and land‑cover** sources into more than **9 million co‑registered patches** ready for large‑scale representation learning.\nYou find more information about the data sampling and preprocessing in our paper: [TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data](https://arxiv.org/abs/2504.11172).\n\n![Examples from TerraMesh](assets%2Fexamples.png)\n\nSamples from the TerraMesh dataset with seven spatiotemporal aligned modalities. Sentinel-2 L2A uses IRRG pseudo-coloring and Sentinel-1 RTC is visualized in db scale as VH-VV-VV/VH. Copernicus DEM is scaled based on the image value range with an additional 10 meter buffer to highlight flat scenes.\n\n### News\n\n- Octomber 2026: Fixed a radiometric offset in reprocessed pre-2022 Sentinel-2 data (thanks Jakob for noticing it) and a few misaligned Sentinel-1 RTC samples (~1.9%).\n- August 2026: Updated data shards: We have (1) increased the number of validation shards for better parallel data loading, and (2) added a 5 km buffer around the sampled MajorTOM validation grid cells (see [val_grid_cells.parquet](https://huggingface.co/datasets/ibm-esa-geospatial/TerraMesh/blob/main/val_grid_cells.parquet), buffer not included) to reduce data leakage. Consequently, we have dropped training samples that intersect the validation area (<2% of training samples).    \n- March 24, 2026: We provide metadata files with a better overview of all samples.\n- March 5, 2026: We fixed a temporal alignment issue in the Sentinel-1 GRD data due to an error in the source dataset SSL4EO-S12 (see [issue](https://github.com/DLR-MF-DAS/SSL4EO-S12-v1.1/issues/4)).\n- August 1, 2025: TerraMesh is released!\n\n---\n\n## Dataset organization\n\nThe archive ships two top‑level splits `train/` and `val/`, each holding one folder per modality. `{train,val}_metadata.parquet` include an overview of the samples and `terramesh.py` includes code for data loading, see [Usage](#Usage).\n```text\nTerraMesh\n├── train\n│   ├── DEM\n│   ├── LULC\n│   ├── NDVI\n│   ├── S1GRD\n│   ├── S1RTC\n│   ├── S2L1C\n│   ├── S2L2A\n│   └── S2RGB\n├── val\n│   ├── DEM\n│   └── ...\n├── train_metadata.parquet\n├── val_metadata.parquet\n└── terramesh.py\n```\n\nEach folder includes up to 889 shard files, containing up to 10240 samples each. Samples from MajorTom-Core are stored in shards with the pattern `majortom_{split}_{id}.tar` while shards with SSL4EO-S12 samples start with `ssl4eos12_`.\nSamples are stored as Zarr Zip files which can be loaded with `zarr` (Version <= 2.18) or `xarray.load_zarr()`. Each sample location includes seven modalities that share the same shard and sample name. Note that each sample only includes one Sentinel-1 version (S1GRD or S1RTC) because of different processing versions in the source datasets.\nEach Zarr file includes aligned metadata as shown by this S1GRD example from sample `ssl4eos12_val_0080385.zarr.zip`:\n\n```\n<xarray.Dataset> Size: 283kB\nDimensions:      (band: 2, time: 1, y: 264, x: 264)\nCoordinates:\n  * band         (band) <U2 16B \"vv\" \"vh\"\n    sample       <U9 36B \"0194630_1\"\n    spatial_ref  int64 8B 0\n  * time         (time) datetime64[ns] 8B 2020-05-03T02:07:17\n  * x            (x) float64 2kB 6.004e+05 6.004e+05 ... 6.03e+05 6.03e+05\n  * y            (y) float64 2kB 4.275e+06 4.275e+06 ... 4.273e+06 4.273e+06\nData variables:\n    bands        (time, band, y, x) float16 279kB -9.461 -10.77 ... -16.67\n    center_lat   float64 8B 3","encoding":"utf8","key":"README.md","name":"README.md","size":17911,"truncated":true},{"bytes":"{\"description\": \"TerraMesh is a global, multimodal analysis\\u2011ready dataset for pre-training Earth Observation foundation models.\", \"citation\": \"@article{blumenstiel2025terramesh,\\n  title={Terramesh: A planetary mosaic of multimodal earth observation data},\\n  author={Blumenstiel, Benedikt and Fraccaro, Paolo and Marsocci, Valerio and Jakubik, Johannes and Maurogiovanni, Stefano and Czerkawski, Mikolaj and Sedona, Rocco and Cavallaro, Gabriele and Brunschwiler, Thomas and Bernabe-Moreno, Juan and others},\\n  journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},\\n  year={2025}\\n}\", \"homepage\": \"https://huggingface.co/datasets/ibm-esa-geospatial/TerraMesh\", \"license\": \"Apache-2.0\", \"features\": {\"zarr.zip\": {\"dtype\": \"binary\", \"_type\": \"Value\"}, \"__key__\": {\"dtype\": \"string\", \"_type\": \"Value\"}, \"__url__\": {\"dtype\": \"string\", \"_type\": \"Value\"}}, \"builder_name\": \"webdataset\", \"dataset_name\": \"train\", \"config_name\": \"default\", \"version\": {\"version_str\": \"1.0.0\", \"major\": 1, \"minor\": 0, \"patch\": 0}}","encoding":"utf8","key":"dataset_info.json","name":"dataset_info.json","size":1067,"truncated":false}],"sourceObjectKey":"train_metadata.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"ibm-esa-geospatial/TerraMesh\")\nprint(ds)","starterSamplePath":"assets/examples.png","totalObjects":6900},"summary":"Planetary-scale, analysis-ready multimodal dataset for training Earth observation foundation models, released by IBM and ESA Phi-lab and used to pretrain TerraMind.","title":"TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data","updatedAt":"2026-11-14T17:32:35.598Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/ibm-esa-geospatial/TerraMesh"}}