{"success":true,"data":{"author":"Multimodal Universe Collaboration (Polymathic AI, UniverseTBD)","bucketAccessType":"public","bucketName":"UniverseTBD/mmu_desi_edr_sv3","bucketPrefix":"","category":"Astronomy & Space","cloudProvider":"huggingface","createdAt":"2026-11-14T01:01:18.918Z","description":"Optical spectra from the Dark Energy Spectroscopic Instrument (DESI) Early Data Release (EDR), Survey Validation 3 (the One-Percent Survey), packaged as part of the Multimodal Universe collection of machine-learning-ready astronomical data. The modality is 1D spectroscopy: each of 1,126,441 objects has a spectrum of 7,781 pixels starting at 3,600 Angstrom with flux, inverse variance, line spread function width and mask arrays, plus the DESI redshift Z, ZERR and ZWARN flag, Galactic extinction and Legacy Surveys g, r and z total and fiber fluxes. The README lists 20 columns in 306 HEALPix partitions and 61.8 GiB on disk. Listing the repository (measured 2026-11-13) shows the main catalog folder mmu_desi_edr_sv3/ (dataset/, hats.properties, partition_info.csv, point_map.fits), the margin catalog mmu_desi_edr_sv3_10arcs/ and collection.properties; about 69 GB in all. Data are a HATS catalog: Apache Parquet partitioned by HEALPix tile, with a 10 arcsecond margin catalog for cross-matching. Open with LSDB (lsdb.open_catalog on the Hugging Face URL, with cone searches and crossmatch), or read the Parquet with pandas, pyarrow, dask or DuckDB. Uses include redshift estimation, spectral classification and cross-modal learning with the Legacy Surveys images. Part of the Multimodal Universe (arXiv:2412.02527); released under CC-BY-4.0, cite the Multimodal Universe paper and the original survey.","displayTitle":"Multimodal Universe: DESI Early Data Release SV3 Spectra","id":"4e33cf76-f213-5f1e-b484-dddd8e13607f","license":"CC-BY-4.0","licenseEvidenceUrl":"https://data.desi.lbl.gov/doc/acknowledgments/","objectCountEstimate":634,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/2412.02527","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"UniverseTBD/mmu_desi_edr_sv3\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"UniverseTBD/mmu_desi_edr_sv3\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=UniverseTBD, license:cc-by-4.0, size_categories:1M<n<10M, format:parquet, downloads=1528"],"format":"mixed","generatedAt":"2026-11-14T01:01:20.786Z","layout":{"example":"mmu_desi_edr_sv3/dataset/Norder=10/Dir=2500000/Npix=2506751.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"504152311504155015624c150a150012000028b52ffd2028410100c8523175cdff8f090315d63ddcff8f098f3e2ecbdcff8f09525bd7e5e1ff8f09381a71baf2ff8f091500151615282c150a1510150615061c1808381a71baf2ff8f091808c8523175cdff8f0916002808381a71baf2ff8f091808c8523175cdff8f09111100000028b52ffd200b590000020000000a010303884600150415e8fd1215d880114c15badf04150012000028b52ffda0745f02003477074ed4cf1a3b2f10685b3a80574f0cc211bc588791e19670dead13bb77bffe478875a02582642bd574654264c25c9e98aaaaaa06b12dce3ad83ad63a5568841e002787072360e18f2aa5702353bef40504601f0391330d4050b65c445fd1a3164e8879f61870216297e22f49539f0802888f22e8d14e8ce09e418846de17a5022f6ad2a809691b886d4fa46a14aaa43c71f6ec2f4c397d08bbb78cd98288c6b47bc14704207cf253d0cb9858c09b8a789f488295adc5ebf33717fa6b82e63d6789309cc017141d88c8e9535a58fb37a554ada6b4a66dc078f22c2cdf3a90020f0fb418a525c47df9bc5f363e9e6221f6ad3a50bee114d63ab84f741039201f64cf4e2fb5a07bc41b588ee4f7542f19157ed79f270f60b44c0cfb0088cee2a3422b1144fa01c15078d3a2b8ebf459d0498e94bc8200bb2ffb99f8e406081e574c10f7d593d1974be25e2c11","encoding":"hex","key":"mmu_desi_edr_sv3/dataset/Norder=10/Dir=2500000/Npix=2506751.parquet","truncatedAt":512}],"sampleObjects":["collection.properties","example_figure.png","mmu_desi_edr_sv3/dataset/Norder=10/Dir=2500000/Npix=2506751.parquet","mmu_desi_edr_sv3/dataset/Norder=4/Dir=0/Npix=1627.parquet","mmu_desi_edr_sv3/dataset/Norder=4/Dir=0/Npix=1628.parquet"],"sidecars":[{"bytes":"---\r\nconfigs:\r\n- config_name: default\r\n  data_dir: mmu_desi_edr_sv3/dataset\r\ntags:\r\n- astronomy\r\nlicense: cc-by-4.0\r\npretty_name: mmu_desi_edr_sv3\r\nsize_categories:\r\n- 1M<n<10M\r\n---\r\n\r\n<div align=\"center\">\r\n<img src=\"example_figure.png\" width=\"600\">\r\n</div>\r\n\r\n# mmu_desi_edr_sv3 HATS Catalog Collection\r\n\r\nThis is the collection of HATS catalogs representing mmu_desi_edr_sv3.\r\n\r\nThis dataset is part of the [Multimodal Universe](https://github.com/MultimodalUniverse/MultimodalUniverse),\r\na large-scale collection of multimodal astronomical data. For full details, see the paper:\r\n[The Multimodal Universe: Enabling Large-Scale Machine Learning with 100TBs of Astronomical Scientific Data](https://arxiv.org/abs/2412.02527).\r\n\r\n### Access the catalog\r\n\r\nWe recommend the use of the [LSDB](https://lsdb.io) Python framework to access HATS catalogs.\r\nLSDB can be installed via `pip install lsdb` or `conda install conda-forge::lsdb`,\r\nsee more details [in the docs](https://docs.lsdb.io/).\r\nThe following code provides a minimal example of opening this catalog:\r\n\r\n```python\r\nimport lsdb\r\n\r\n# Full sky coverage.\r\ncatalog = lsdb.open_catalog(\"https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3\")\r\n# One-degree cone.\r\ncatalog = lsdb.open_catalog(\r\n    \"https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3\",\r\n    search_filter=lsdb.ConeSearch(ra=247.0, dec=43.0, radius_arcsec=3600.0),\r\n)\r\n```\r\n\r\nEach catalog in this collection is represented as a separate [Apache Parquet dataset](https://arrow.apache.org/docs/python/dataset.html) and can be accessed with a variety of tools, including `pandas`, `pyarrow`, `dask`, `Spark`, `DuckDB`.\r\n\r\n### File structure\r\n\r\nThis catalog is represented by the following files and directories:\r\n\r\n- [`collection.properties`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/collection.properties) — textual metadata file describing the HATS collection of catalogs\r\n- [`mmu_desi_edr_sv3`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3) — main HATS catalog directory\r\n  - [`dataset/`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3/dataset/) — Apache Parquet dataset directory for the main catalog\r\n    - ... parquet metadata and data files in sub directories ...\r\n  - [`hats.properties`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3/hats.properties) — textual metadata file describing the main HATS catalog\r\n  - [`partition_info.csv`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3/partition_info.csv) — CSV file with a list of catalog HEALPix tiles (catalog partitions)\r\n  - [`skymap.fits`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3/skymap.fits) — HEALPix skymap FITS file with row-counts per HEALPix tile of fixed order 10\r\n- [`mmu_desi_edr_sv3_10arcs/`](https://huggingface.co/datasets/UniverseTBD/mmu_desi_edr_sv3/mmu_desi_edr_sv3_10arcs) — default margin catalog to ensure data completeness in cross-matching, the margin threshold is 10.0 arcseconds\r\n  - ... margin catalog files and directories ...\r\n\r\n### Catalog metadata\r\n\r\nMetadata of the main HATS catalog, excluding margins and indexes:\r\n\r\n| **Number of rows** | **Number of columns** | **Number of partitions** | **Size on disk** | **HATS Builder** |\r\n| --- | --- | --- | --- | --- |\r\n| 1,126,441 | 20 | 306 | 61.8 GiB | hats-import v0.7.1, hats v0.7.1 |\r\n\r\n\r\n### Catalog columns\r\n\r\nThe main HATS catalog contains the following columns:\r\n\r\n| **Name** |  **`_healpix_29`** | **`spectrum.flux`** | **`spectrum.ivar`** | **`spectrum.lsf_sigma`** | **`spectrum.lambda`** | **`spectrum.mask`** | **`Z`** | **`ZERR`** | **`EBV`** | **`FLUX_G`** | **`FLUX_R`** | **`FLUX_Z`** | **`FLUX_IVAR_G`** | **`FLUX_IVAR_R`** | **`FLUX_IVAR_Z`** | **`FIBERFLUX_G`** | **`FIBERFLUX_R`** | **`FIBERFLUX_Z`** | **`FIBERTOTFLUX_G`** | **`FIBERTOTFLUX_R`** | **`FIBERTOTFLUX_Z`** | **`ra`** | **`dec`** | **`ZWARN`** | **`object_id`** |\r\n| --- |  --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---","encoding":"utf8","key":"README.md","name":"README.md","size":8365,"truncated":true}],"sourceObjectKey":"mmu_desi_edr_sv3/dataset/Norder=10/Dir=2500000/Npix=2506751.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"UniverseTBD/mmu_desi_edr_sv3\")\nprint(ds)","starterSamplePath":"example_figure.png","totalObjects":634},"summary":"Optical spectra from the Dark Energy Spectroscopic Instrument (DESI) Early Data Release (EDR), Survey Validation 3 (the One-Percent Survey), packaged as part of the Multimodal…","title":"Multimodal Universe: DESI Early Data Release SV3 Spectra","updatedAt":"2026-11-14T02:46:20.380Z","visibility":"public","websiteUrl":"https://github.com/MultimodalUniverse/MultimodalUniverse"}}