{"success":true,"data":{"author":"BIOSCAN project (Gharaee, Lowe, Gong et al.; University of Guelph, Vector Institute and partners)","bucketAccessType":"public","bucketName":"bioscan-ml/BIOSCAN-5M","bucketPrefix":"","category":"Life Sciences","cloudProvider":"huggingface","createdAt":"2026-11-14T01:01:20.077Z","description":"Multimodal data for over 5 million insect specimens from the BIOSCAN biodiversity monitoring effort, presented at NeurIPS 2024 (Datasets and Benchmarks). Each specimen has a microscope image, taxonomic labels, its raw DNA barcode nucleotide sequence, an assigned Barcode Index Number (BIN), geographic information and specimen size. Partitions support closed-world species classification (train, val, test), open-world generalization to unseen species of known genera, novelty detection (other_heldout) and a large unlabelled pretrain split for self-supervised learning. Tasks include DNA-based taxonomic classification, zero-shot transfer and cross-modal retrieval between images, DNA and text. Listing the repository (measured 2026-11-13) shows zip archives of images at 256-pixel resolution in two versions, original_256 (37.8 GB) and cropped_256 (39.1 GB), each also split into pretrain, train and eval archives, plus BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip (2.07 GB), a dataset.py loader and a usage demo; about 160 GB in all. Load with Hugging Face datasets using dataset.py (configurations such as cropped_256_eval), or unzip and read metadata with pandas and images with PIL. Released under CC-BY-3.0; cite Gharaee et al. 2024.","displayTitle":"BIOSCAN-5M: Multimodal Insect Images and DNA Barcodes","id":"a3a0f9a6-8483-57b1-b00d-c1d9cee450dd","license":"CC-BY-3.0","licenseEvidenceUrl":"https://zenodo.org/api/records/11973457","objectCountEstimate":13,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/2406.12723","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"bioscan-ml/BIOSCAN-5M\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"bioscan-ml/BIOSCAN-5M\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=bioscan-ml, license:cc-by-3.0, size_categories:1M<n<10M, downloads=1252"],"format":"mixed","generatedAt":"2026-11-14T01:01:24.449Z","layout":{"example":"BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"504b03042d0000000800da60ce585b143483ffffffffffffffff4300140062696f7363616e356d2f6d657461646174612f6a736f6e6c642f42494f5343414e5f354d5f496e736563745f446174617365745f6d657461646174612e6a736f6e6c6401001000ef9992af0100000053bcc94000000000ecbd5b731c3996adf93ebf42d6cf8734bf5fce1b0849509d93cacac9447577d53ca45164a4446b8a2123a9eed6199bff3e1ed8176cc00177a72486c45296a93295625cdc3d141f96afbdf6c6fff37f3d9bfef7ffba7f1efef72f1f6ef717bbbbbbabcb7ff99fcffee5979fccd9bf556579528efff23ffc63eecedf7fb8dec143cefef2d7bf99b6acc7e14417b57cd4fdf97fef6f0e0f79bdbbdf7fb8ba39973ffcf0eed3f5c7f7879faadbfb77b7fb0ffbcbe0e717d7e77777871fffe5e66e77711ffc6c7f7bb9bb3dfcecf9d587fbdd6df0b33fcedf5f5d7f72c7fe6e7f7b7579be0b0efce31bff003cace9c082c7bcdddd7cbccb1df6dd87ddc5d5cefd58fef1e5cdf9ef6faedcc99efdf5a7e7ff533d57f550f4b3879cdf5eec2f778787696badb20afe75f89f99fe37fd9751462b63d4f43363dd6fb4b2faf093c37fdbc3bfa7c7ebc3c3f4e18787274e0fd15a1f7e63dceb1d9e30fd4f4d2f66b53a3cfef056d37fbaf79a5e667adaf4f4e959eef1873f743fc7ffb9b7760737bd03fee87024d34bbae374af68dc5b","encoding":"hex","key":"BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip","truncatedAt":512}],"sampleObjects":["BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip","BIOSCAN_5M_cropped_256.zip","BIOSCAN_5M_cropped_256_eval.zip","BIOSCAN_5M_cropped_256_pretrain.zip","BIOSCAN_5M_cropped_256_train.zip"],"sidecars":[{"bytes":"---\nlicense: cc-by-3.0\nlanguage:\n- en\nsize_categories:\n- 1M<n<10M\npretty_name: BIOSCAN-5M\ntags:\n- Multimodal_dataset\n- Image\n- DNA_barcode\n- Taxonomy\n- Biodiversity\n- LLMs\n- BERT\n- Clustering\n- Multimodal_retrieval_learning\n- Zero_shot_transfer_learning\n- Geo_location\n- Specimen_size\n- Insect\n- Species\n- large_scale_dataset\n- large_scale_foundation_model\nmaintainers:\n- https://huggingface.co/Gharaee\nauthor:\n  name: Zahra Gharaee\n  github: https://github.com/zahrag\n  hf: https://huggingface.co/Gharaee\ndataset_loader_script: dataset.py\ndataset_split_names:\n- pretarin\n- train\n- validation\n- test\n- val_unseen\n- test_unseen\n- key_unseen\n- other_heldout\ndescription: \"Curated multimodal data for 5M insect specimens for biodiversity research.\"\n---\n[![Author: zahrag](https://img.shields.io/badge/author-zahrag-blue)](https://huggingface.co/Gharaee)\n\n# BIOSCAN-5M\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/64cac7e17221ef3c7e2eed1f/ZgP3fd2Z9eVgucsZHYYvA.png)\n\n## Overview\nAs part of an ongoing worldwide effort to comprehend and monitor insect biodiversity, we present the BIOSCAN-5M Insect dataset to the machine learning community. BIOSCAN-5M is a comprehensive dataset containing multi-modal information for over 5 million insect specimens, \nand it significantly expands existing image-based biological datasets by including taxonomic labels, raw nucleotide barcode sequences, assigned barcode index numbers, geographical information, and specimen size. \n\n### Citation\n\nIf you make use of the BIOSCAN-5M dataset and/or its code repository, please cite the following paper:\n\n```\ncite as:\n\n@inproceedings{gharaee2024bioscan5m,\n    title={{BIOSCAN-5M}: A Multimodal Dataset for Insect Biodiversity},\n    booktitle={Advances in Neural Information Processing Systems},\n    author={Zahra Gharaee and Scott C. Lowe and ZeMing Gong and Pablo Millan Arias\n        and Nicholas Pellegrino and Austin T. Wang and Joakim Bruslund Haurum\n        and Iuliia Zarubiieva and Lila Kari and Dirk Steinke and Graham W. Taylor\n        and Paul Fieguth and Angel X. Chang\n    },\n    editor={A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang},\n    pages={36285--36313},\n    publisher={Curran Associates, Inc.},\n    year={2024},\n    volume={37},\n    url={https://proceedings.neurips.cc/paper_files/paper/2024/file/3fdbb472813041c9ecef04c20c2b1e5a-Paper-Datasets_and_Benchmarks_Track.pdf},\n}\n```\n\n## Large-Scale Foundation Model Training for Biodiversity Research\n\n### Dataset partitions\n\n| Partition           | Example Splits                        | Description                                                               |\n|---------------------|---------------------------------------|---------------------------------------------------------------------------|\n| **Closed-world**     | train, val, test                     | Samples with known species names for supervised classification.           |\n| **Open-world**       | key_unseen, val_unseen, test_unseen  | Placeholder species names but known genera, enabling generalization to unseen species. |\n| **Novelty Detection**| other_heldout                        | Unknown species and genus, suitable for open-set detection.                |\n| **Pretraining**      | pretrain                             | Unlabeled data for self-/semi-supervised learning at scale.                |\n\n\n### Supported tasks\n\n| Task                                   | Description                                                                 |\n|----------------------------------------|-----------------------------------------------------------------------------|\n| DNA-based Taxonomic Classification | Predict taxonomic labels from raw DNA barcode sequences.                  |\n| Zero-Shot Transfer Learning        | Evaluate whether unlabeled models can semantically cluster data—across modalities like image and DNA—using learned representations. |\n| Multimodal Retrieval Learning      | Retrieve matching specimens across modalities (e.g., image ↔ DNA ↔ text) via shared embeddings. |\n\n\n#","encoding":"utf8","key":"README.md","name":"README.md","size":9130,"truncated":true}],"sourceObjectKey":"BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip","starterCell":"from huggingface_hub import hf_hub_download\npath = hf_hub_download(repo_id=\"bioscan-ml/BIOSCAN-5M\", filename=\"BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip\", repo_type=\"dataset\")\nprint(path)","starterSamplePath":"BIOSCAN_5M_Insect_Dataset_metadata_MultiTypes.zip","totalObjects":13},"summary":"Multimodal data for over 5 million insect specimens from the BIOSCAN biodiversity monitoring effort, presented at NeurIPS 2024 (Datasets and Benchmarks).","title":"BIOSCAN-5M: Multimodal Insect Images and DNA Barcodes","updatedAt":"2026-11-14T02:38:02.256Z","visibility":"public","websiteUrl":"https://github.com/bioscan-ml/BIOSCAN-5M"}}