{"success":true,"data":{"author":"Hugging Face (HuggingFaceBio, Carbon team)","bucketAccessType":"public","bucketName":"HuggingFaceBio/genomic-niah","bucketPrefix":"","category":"Genomics & Bioinformatics","cloudProvider":"huggingface","createdAt":"2026-11-17T14:12:21.558Z","description":"A long-context retrieval benchmark for genomic language models from Hugging Face, built for the Carbon DNA models. Each example plants a random 24 bp KEY and 24 bp VALUE pair inside a real eukaryotic genome haystack and asks the model to return VALUE given the haystack followed by KEY, so success requires actual long-range retrieval. Four tasks (plain retrieval and three near-duplicate variants with eight distractor keys differing by 4, 2 or 1 bp, inspired by RULER multi-key) cross six context lengths from 4k to 128k 6-mer tokens (about 24 kbp to 786 kbp), giving 24 configurations of 500 examples each (12,000 in total), stratified over five needle depths (10 to 90%) and four kingdoms (animalia, fungi, plantae, protista). Haystacks come from the OpenGenome2 NCBI eukaryotic test and validation splits, using contiguous ACGT runs up to 16k tokens and stitched within-record runs beyond. Data are Parquet under data/, one config per task and length (e.g. niah_16k, niah_neardup_d1_64k), loadable with Hugging Face datasets; about 4 GB. Released under Apache 2.0; cite the Carbon paper (Ben Allal et al., 2026).","displayTitle":"Genomic-NIAH: needle-in-a-haystack long-context retrieval benchmark for DNA language models","huggingFaceUrl":"https://huggingface.co/datasets/HuggingFaceBio/genomic-niah","id":"7413e9e2-c9ea-5349-9d16-957fec78ecd6","license":"Apache-2.0","licenseEvidenceUrl":"https://huggingface.co/datasets/HuggingFaceBio/genomic-niah","objectCountEstimate":28,"probeStatus":"ok","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"HuggingFaceBio/genomic-niah\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"HuggingFaceBio/genomic-niah\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=HuggingFaceBio, license:apache-2.0, size_categories:10K<n<100K, format:parquet, downloads=269"],"format":"mixed","generatedAt":"2026-11-17T14:23:03.325Z","layout":{"example":"data/niah_128k/test-00000-of-00001.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415a09c0115e687014c15e8071500120000904e4c1000000037623165313939633331303664313531091434333865376231316664366665383201143c3663303264383365316437643264626601143c6332346231363939663665636162656301143c3364393962616364393361636339353701143c6439666365616465323535313435656101143c3635313162613134623635326637623901143866373466356432346265303634326609143830646464356665386363373333333001283834306638383431633535356138373005283c3865333332386436326363333536643301283861643336376661626562353931626405b43836393532313838363030633735336305dc38373433633132363438376435643933055038653134343366393564333462376561053c3c35343463363531346465623633326234016438393261633965656461613639653030055034313431613463316263363832373109c83831373032666432353532323665663105503c3666666339613638373438326634303601503c6265363030373066353332376332373501143c33353661626534393766636332393364011438306562393234616430346537316538255438623532346237313364643035313932250438646565353735646238333334323865051408363933214e1c323936643638623705783c336430326236","encoding":"hex","key":"data/niah_128k/test-00000-of-00001.parquet","truncatedAt":512}],"sampleObjects":["assets/modelcard_carbon3b_native.png","assets/modelcard_carbon3b_yarn4x.png","data/niah_128k/test-00000-of-00001.parquet","data/niah_16k/test-00000-of-00001.parquet","data/niah_32k/test-00000-of-00001.parquet"],"sidecars":[{"bytes":"---\nlicense: apache-2.0\ntask_categories:\n  - text-generation\ntags:\n  - dna\n  - genomic\n  - long-context\n  - retrieval\n  - benchmark\n  - niah\nconfigs:\n  - config_name: niah_4k\n    data_files:\n      - split: test\n        path: data/niah_4k/test-*.parquet\n  - config_name: niah_8k\n    data_files:\n      - split: test\n        path: data/niah_8k/test-*.parquet\n  - config_name: niah_16k\n    data_files:\n      - split: test\n        path: data/niah_16k/test-*.parquet\n  - config_name: niah_32k\n    data_files:\n      - split: test\n        path: data/niah_32k/test-*.parquet\n  - config_name: niah_64k\n    data_files:\n      - split: test\n        path: data/niah_64k/test-*.parquet\n  - config_name: niah_128k\n    data_files:\n      - split: test\n        path: data/niah_128k/test-*.parquet\n  - config_name: niah_neardup_d4_4k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_4k/test-*.parquet\n  - config_name: niah_neardup_d4_8k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_8k/test-*.parquet\n  - config_name: niah_neardup_d4_16k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_16k/test-*.parquet\n  - config_name: niah_neardup_d4_32k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_32k/test-*.parquet\n  - config_name: niah_neardup_d4_64k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_64k/test-*.parquet\n  - config_name: niah_neardup_d4_128k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d4_128k/test-*.parquet\n  - config_name: niah_neardup_d2_4k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_4k/test-*.parquet\n  - config_name: niah_neardup_d2_8k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_8k/test-*.parquet\n  - config_name: niah_neardup_d2_16k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_16k/test-*.parquet\n  - config_name: niah_neardup_d2_32k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_32k/test-*.parquet\n  - config_name: niah_neardup_d2_64k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_64k/test-*.parquet\n  - config_name: niah_neardup_d2_128k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d2_128k/test-*.parquet\n  - config_name: niah_neardup_d1_4k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_4k/test-*.parquet\n  - config_name: niah_neardup_d1_8k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_8k/test-*.parquet\n  - config_name: niah_neardup_d1_16k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_16k/test-*.parquet\n  - config_name: niah_neardup_d1_32k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_32k/test-*.parquet\n  - config_name: niah_neardup_d1_64k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_64k/test-*.parquet\n  - config_name: niah_neardup_d1_128k\n    data_files:\n      - split: test\n        path: data/niah_neardup_d1_128k/test-*.parquet\n---\n\n# Genomic-NIAH\n\n**A long-context retrieval benchmark for genomic language models.**\n\nGenomic-NIAH measures whether a DNA model actually *uses* its long context. Each example plants a random (KEY, VALUE) DNA pair inside a real-genome haystack and asks the model to recover VALUE given the haystack followed by KEY. Because both KEY and VALUE are uncorrelated with the surrounding sequence, the only way to succeed is to retrieve the planted pair at long range — turning the abstract question \"does the model use distal context\" into a concrete generative task with a per-example accuracy. This benchmark is inspired from [NIAH](https://github.com/gkamradt/LLMTest_NeedleInAHaystack) and [RULER](https://arxiv.org/abs/2404.06654), popular long context benchmarks for natural language.\n\nThe benchmark provides 4 tasks of varying difficulty, and 24 sub-configurations on a **4 × 6 grid** of task difficulty (plain retrieval and three near-duplicate variants) × context length (from 4k to 128k 6-mer tokens ≈ 24,000 DNA base pairs �","encoding":"utf8","key":"README.md","name":"README.md","size":14444,"truncated":true}],"sourceObjectKey":"data/niah_128k/test-00000-of-00001.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"HuggingFaceBio/genomic-niah\")\nprint(ds)","starterSamplePath":"assets/modelcard_carbon3b_native.png","totalObjects":28},"summary":"A long-context retrieval benchmark for genomic language models from Hugging Face, built for the Carbon DNA models.","title":"Genomic-NIAH: needle-in-a-haystack long-context retrieval benchmark for DNA language models","updatedAt":"2026-11-17T20:10:43.853Z","visibility":"public","websiteUrl":"https://huggingface.co/datasets/HuggingFaceBio/genomic-niah"}}