{"success":true,"data":{"author":"Open Language Data Initiative (OLDI); Meta NLLB team and contributors","bucketAccessType":"public","bucketName":"openlanguagedata/oldi_seed","bucketPrefix":"","category":"Natural Language Processing","cloudProvider":"huggingface","createdAt":"2026-11-21T22:16:38.847Z","description":"OLDI Seed is a multi-way parallel corpus for starting machine translation in languages that lack training data. It has 6,193 sentences sampled from English Wikipedia, each translated into about 45 languages. Many are low-resource languages and scripts of Africa, Asia and the Americas, such as Acehnese in Arabic and Latin script, Moroccan and Egyptian Arabic, Bambara and Shan in Myanmar script. That gives about 576,000 sentence rows in total, one JSONL file per language and script under seed/, with per-language dataset cards and a bibliography of contributing papers. It updates the NLLB Seed data released with No Language Left Behind (Maillard et al., Small Data, Big Impact, ACL 2023). Linguists and MT researchers use it as aligned text in many languages for building translation models, studying cross-lingual transfer and documenting written varieties. Load it with Hugging Face datasets by language config. Because the source is Wikipedia, the data stays under CC BY-SA 4.0, managed by the Open Language Data Initiative. Cite the papers in bibliography.bib.","displayTitle":"OLDI Seed: 6,193 English Wikipedia Sentences Translated into About 45 Low-Resource Languages (Open Language Data Initiative)","huggingFaceUrl":"https://huggingface.co/datasets/openlanguagedata/oldi_seed","id":"423aacfb-740e-506d-a85c-1d720412bffa","license":"CC-BY-SA-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/openlanguagedata/oldi_seed","objectCountEstimate":63,"probeStatus":"ok","referenceUrl":"https://aclanthology.org/2023.acl-long.154","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"openlanguagedata/oldi_seed\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"openlanguagedata/oldi_seed\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=openlanguagedata, license:cc-by-sa-4.0, size_categories:100K<n<1M, format:json, downloads=2102"],"format":"mixed","generatedAt":"2026-11-21T22:31:58.804Z","layout":{"example":"seed/ace_Arab.jsonl","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"7b226964223a302c2269736f5f3633395f33223a22616365222c2269736f5f3135393234223a2241726162222c22676c6f74746f636f6465223a226163686931323537222c2274657874223a22d984d98ad984d98ad8a7d98620d8b6d98ad8a7d986d8a720d8b9d98ad8b3d9872028d9a1d9a420d988daa9d8aad988d8a8d989d8b120d9a1d9a8d9a9d9a320e2809320d9a2d9a720d981d989d8a8d8b1d988d8a7d8b1d98a20d9a1d9a9d9a9d9a32920d986d8a720daa9d989d988d98720d8b3d98ad8afd8b1d988d98920d8a7d8b1d989d988d8aad98ad98720d8a5d985d989d8b1d98adaa9d8a7d88c20d98ad8a7d986d8b920d8a7d8aad98820d988d8b1d989d988d986d8b920d985d989d988d989d98620d981d98ad984d98520d8afd8a7d98620d98ad8a7d986d8b920d8aad988d984d989d98720da86d989d988d8b1d98ad8aad8a720d981d98ad984d985222c2275726c223a2268747470733a5c2f5c2f656e2e77696b6970656469612e6f72675c2f77696b695c2f4c696c6c69616e5f47697368222c226c6173745f75706461746564223a22312e30227d0a7b226964223a312c2269736f5f3633395f33223a22616365222c2269736f5f3135393234223a2241726162222c22676c6f74746f636f6465223a226163686931323537222c2274657874223a22d8b9d98ad8b3d98720d986d8a720daa9d989d988d987","encoding":"hex","key":"seed/ace_Arab.jsonl","truncatedAt":512}],"sampleObjects":["DCO","bibliography.bib","dataset_cards/ary_Arab.md","dataset_cards/ben_Beng.md","dataset_cards/fra_Latn.md"],"sidecars":[{"bytes":"# Changelog\n\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).\n\nIn particular:\n1. Any updates to pre-existing files shall entail a major version bump.\n2. The inclusion of a new language, without changes to other pre-existing data, shall only entail a minor version bump.\n\n## [4.1] - 2026-10-24\n- Added Croatian data (`hrv_Latn`) — details in [Zundanovic et al, 2026](https://www2.statmt.org/wmt26/pdf/2026.wmt-1.200.pdf)\n\n## [4.0] - 2026-03-07\n- Converted the dataset to JSONL to make contributions and visual inspection of files easier.\n- Added a basic data integrity test suite based on FLORES+'s.\n- Updated `lij_Latn` after running a newly developed spellchecker revealed several typos.\n\n## [3.0] - 2025-11-07\n- Participants to the [WMT25 Shared Task](https://www2.statmt.org/wmt25/open-data.html) contributed multiple improvements. Many thanks to all the contributors! Please don't forget to cite the relevant papers listed in [bibliography.bib](bibliography.bib).\n    - Added `fra_Latn` (see [Marmonier et al, 2025](https://aclanthology.org/2025.wmt-1.80/))\n    - Added `kyr_Cyrl`  (see [Jumashev et al, 2025](https://aclanthology.org/2025.wmt-1.84/))\n    - Updated about half of `zgh_Tfng` texts (see [Oktem et al, 2025](https://aclanthology.org/2025.wmt-1.82/))\n\n## [2.0] - 2024-11-08\n\n### Added\n\n- Added `ben_Beng`, `ita_Latn` and `spa_Latn` from the [WMT24 Shared Task](https://www2.statmt.org/wmt24/open-data.html). Many thanks to the contributors!\n- Added `nqo_Nkoo` from <https://github.com/common-parallel-corpora/common-parallel-corpora>.\n\n### Changed\n\n- Retranslated `ary_Arab` after community feedback and additional quality assessment. Data is now more closely aligned with Moroccan Arabic. Many thanks to @MedAymenF for pointing out the issues with the original NLLB-Seed data.\n- Adopted the aligned version of datasets from <https://github.com/common-parallel-corpora/common-parallel-corpora>. Data is now aligned across all languages, i.e. all files use the same ordering of sentences.\n- Data was NFC normalised.\n- Updated `lij_Latn` after additional quality assessment. Data has undergone minor spelling and syntactic fixes.\n- Relabeled `tzm_Tfng` to `zgh_Tfng` after additional quality assessment revealed the data was in Standard Moroccan Tamazight. Many thanks to @MedAymenF for pointing out the issues with the original NLLB-Seed data.\n- Relabeled `grn_Latn` macrolanguage code to the correct code `gug_Latn`.\n\n## [1.0] - 2022-07-11\n\nThe original release of the dataset – described by [NLLB Team et al. (2022)](https://arxiv.org/abs/2207.04672) and [Maillard et al. (2023)](https://aclanthology.org/2023.acl-long.154/) – can be downloaded from <https://tinyurl.com/NLLBSeed>.\n","encoding":"utf8","key":"CHANGELOG.md","name":"CHANGELOG.md","size":2896,"truncated":false},{"bytes":"---\nannotations_creators:\n  - found\nlanguage_creators:\n  - expert-generated\nlicense: cc-by-sa-4.0\npretty_name: OLDI-Seed\ntask_categories:\n  - text-generation\n  - translation\nlanguage:\n  - ace\n  - ary\n  - arz\n  - bam\n  - ban\n  - ben\n  - bho\n  - bjn\n  - bug\n  - crh\n  - dik\n  - dzo\n  - eng\n  - fra\n  - fur\n  - fuv\n  - gug\n  - hne\n  - hrv\n  - ita\n  - kas\n  - knc\n  - lij\n  - lim\n  - lmo\n  - ltg\n  - mag\n  - mni\n  - mri\n  - nqo\n  - nus\n  - pbt\n  - prs\n  - scn\n  - shn\n  - spa\n  - srd\n  - szl\n  - taq\n  - vec\n  - zgh\nconfigs:\n  - config_name: default\n    data_files:\n      - split: train\n        path: seed/*.jsonl\n  - config_name: ace_Arab\n    data_files:\n      - split: train\n        path: seed/ace_Arab.jsonl\n  - config_name: ace_Latn\n    data_files:\n      - split: train\n        path: seed/ace_Latn.jsonl\n  - config_name: ary_Arab\n    data_files:\n      - split: train\n        path: seed/ary_Arab.jsonl\n  - config_name: arz_Arab\n    data_files:\n      - split: train\n        path: seed/arz_Arab.jsonl\n  - config_name: bam_Latn\n    data_files:\n      - split: train\n        path: seed/bam_Latn.jsonl\n  - config_name: ban_Latn\n    data_files:\n      - split: train\n        path: seed/ban_Latn.jsonl\n  - config_name: ben_Beng\n    data_files:\n      - split: train\n        path: seed/ben_Beng.jsonl\n  - config_name: bho_Deva\n    data_files:\n      - split: train\n        path: seed/bho_Deva.jsonl\n  - config_name: bjn_Arab\n    data_files:\n      - split: train\n        path: seed/bjn_Arab.jsonl\n  - config_name: bjn_Latn\n    data_files:\n      - split: train\n        path: seed/bjn_Latn.jsonl\n  - config_name: bug_Latn\n    data_files:\n      - split: train\n        path: seed/bug_Latn.jsonl\n  - config_name: crh_Latn\n    data_files:\n      - split: train\n        path: seed/crh_Latn.jsonl\n  - config_name: dik_Latn\n    data_files:\n      - split: train\n        path: seed/dik_Latn.jsonl\n  - config_name: dzo_Tibt\n    data_files:\n      - split: train\n        path: seed/dzo_Tibt.jsonl\n  - config_name: eng_Latn\n    data_files:\n      - split: train\n        path: seed/eng_Latn.jsonl\n  - config_name: fra_Latn\n    data_files:\n      - split: train\n        path: seed/fra_Latn.jsonl\n  - config_name: fur_Latn\n    data_files:\n      - split: train\n        path: seed/fur_Latn.jsonl\n  - config_name: fuv_Latn\n    data_files:\n      - split: train\n        path: seed/fuv_Latn.jsonl\n  - config_name: gug_Latn\n    data_files:\n      - split: train\n        path: seed/gug_Latn.jsonl\n  - config_name: hne_Deva\n    data_files:\n      - split: train\n        path: seed/hne_Deva.jsonl\n  - config_name: hrv_Latn\n    data_files:\n      - split: train\n        path: seed/hrv_Latn.jsonl\n  - config_name: ita_Latn\n    data_files:\n      - split: train\n        path: seed/ita_Latn.jsonl\n  - config_name: kas_Arab\n    data_files:\n      - split: train\n        path: seed/kas_Arab.jsonl\n  - config_name: kas_Deva\n    data_files:\n      - split: train\n        path: seed/kas_Deva.jsonl\n  - config_name: knc_Arab\n    data_files:\n      - split: train\n        path: seed/knc_Arab.jsonl\n  - config_name: knc_Latn\n    data_files:\n      - split: train\n        path: seed/knc_Latn.jsonl\n  - config_name: lij_Latn\n    data_files:\n      - split: train\n        path: seed/lij_Latn.jsonl\n  - config_name: lim_Latn\n    data_files:\n      - split: train\n        path: seed/lim_Latn.jsonl\n  - config_name: lmo_Latn\n    data_files:\n      - split: train\n        path: seed/lmo_Latn.jsonl\n  - config_name: ltg_Latn\n    data_files:\n      - split: train\n        path: seed/ltg_Latn.jsonl\n  - config_name: mag_Deva\n    data_files:\n      - split: train\n        path: seed/mag_Deva.jsonl\n  - config_name: mni_Beng\n    data_files:\n      - split: train\n        path: seed/mni_Beng.jsonl\n  - config_name: mri_Latn\n    data_files:\n      - split: train\n        path: seed/mri_Latn.jsonl\n  - config_name: nqo_Nkoo\n    data_files:\n      - split: train\n        path: seed/nqo_Nkoo.jsonl\n  - config_name: nus_Latn\n    data_files:\n      - split: train\n        path: seed/nus_Latn.jsonl\n  - config_name: pbt_Arab\n    data_files:\n      - split: train\n        path: seed/pbt_Arab.","encoding":"utf8","key":"README.md","name":"README.md","size":16869,"truncated":true}],"sourceObjectKey":"seed/ace_Arab.jsonl","starterCell":"from datasets import load_dataset\nds = load_dataset(\"openlanguagedata/oldi_seed\")\nprint(ds)","starterSamplePath":"dataset_cards/ary_Arab.md","totalObjects":63},"summary":"OLDI Seed is a multi-way parallel corpus for starting machine translation in languages that lack training data.","title":"OLDI Seed: 6,193 English Wikipedia Sentences Translated into About 45 Low-Resource Languages (Open Language Data Initiative)","updatedAt":"2026-11-22T01:05:30.903Z","visibility":"public","websiteUrl":"https://oldi.org"}}