{"success":true,"data":{"author":"Heinzinger, Weissenow, Gomez Sanchez, Henkel, Steinegger and Rost (Rostlab, TUM; Seoul National University)","bucketAccessType":"public","bucketName":"Rostlab/ProstT5Dataset","bucketPrefix":"","category":"Structural Biology","cloudProvider":"huggingface","createdAt":"2026-11-14T14:15:14.608Z","description":"Training data for ProstT5, the bilingual protein language model that translates between amino acid sequence and Foldseek 3Di structure strings (Heinzinger et al., NAR Genomics and Bioinformatics 2024). Per the card, the set starts from the AlphaFold Protein Structure Database: 214 million UniProtKB sequences were clustered with MMseqs2 into 52 million clusters, Foldseek clustered these further into 18.8 million structural clusters, and quality filtering removed low-pLDDT, short and highly repetitive 3Di entries, leaving about 17 million training proteins plus 474 validation and 474 test representatives split by whole clusters. Rows are tokenized with the ProstT5 tokenizer: input_id_x holds 3Di tokens and input_id_y holds amino acid tokens, and the card gives the token-to-character mapping (amino acids 3 to 27, 3Di letters 128 to 147, special tokens for direction). Measured 2026-11-14: 133 parquet files under data/ (about 5.1 GB listed, repository storage about 25 GB). Load with datasets.load_dataset('Rostlab/ProstT5Dataset') or pyarrow. Useful for structure-aware protein language models, inverse folding style translation, remote homology search with 3Di, and comparing AlphaFold-predicted versus PDB structure statistics. The card states an MIT licence from Rostlab; the underlying AlphaFold DB structures are CC BY 4.0, which governs here. Cite ProstT5 and AlphaFold DB.","displayTitle":"ProstT5 Dataset: Paired Amino Acid and 3Di Structure Sequences","doi":"10.1093/nargab/lqae150","id":"2a435277-d822-56bf-a4d2-c0dbe2fc1ee0","license":"CC-BY-4.0","licenseEvidenceUrl":"https://alphafold.ebi.ac.uk/download","objectCountEstimate":135,"probeStatus":"ok","referenceUrl":"https://doi.org/10.1093/nargab/lqae150","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"Rostlab/ProstT5Dataset\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"Rostlab/ProstT5Dataset\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=Rostlab, license:mit, size_categories:10M<n<100M, format:parquet, downloads=151"],"format":"mixed","generatedAt":"2026-11-14T14:15:17.078Z","layout":{"example":"data/test-00000-of-00001-b109fa020c25190c.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415d00215c6014c152a1500120000a8010487000901008309070400930d08008a0d0800810d08008d0d08008e0d0800850d0800800d080491000901009209070400860d0800890d0800880d08008f0d0800840d0800010d08008b0d0800820d083c8c000000000000009000000000000000150015ccf004158ac5042c15a8a2081510150615061c180893000000000000001808010000000000000016002808930000000000000018080100000000000000000000a6b8022c9e08000003fe88020103fe540d047c4e0103fe6e0103feb6010103fef8030103fe680103fef60c0103fe760103fed0012c00aa011c00480d3100ae0112105e0103fe84010900e40105009c010500cc013c009a012a10740103fea40109106c0103fe80011700a6010e20360103fe5a0103fef0011234440103fe5c0103fe8c0a0103fec8011f00ac010500560185014e009c010e00f2010a00f2010a00b8010510640103feda0109009e0142248c070103fe7c0103feb20113083001030942009e0134018200ce011700b2012500a6010500bc010f012600b80121018c18340103fe580103050c0084011e0532211800e4010e01bd053510720103fe8e011200e8010500fc01052c3a0103fe7a0103fec204010305cb255300b2015a011710380103fede012810400103fe46119a00e6011f00b0011700f401c0107e0103febc0113","encoding":"hex","key":"data/test-00000-of-00001-b109fa020c25190c.parquet","truncatedAt":512}],"sampleObjects":["data/test-00000-of-00001-b109fa020c25190c.parquet","data/train-00000-of-00131-4351a6135cdeb312.parquet","data/train-00001-of-00131-95b5a435d3a45a25.parquet","data/train-00002-of-00131-eb06174e568914e8.parquet","data/train-00003-of-00131-ad954d19e0d5383e.parquet"],"sidecars":[{"bytes":"---\ndataset_info:\n  features:\n  - name: input_id_x\n    sequence: int64\n  - name: input_id_y\n    sequence: int64\n  splits:\n  - name: test\n    num_bytes: 1087504\n    num_examples: 474\n  - name: valid\n    num_bytes: 1124160\n    num_examples: 474\n  - name: train\n    num_bytes: 65391887792\n    num_examples: 17070828\n  download_size: 810671738\n  dataset_size: 65394099456\nlicense: mit\ntask_categories:\n- text-generation\ntags:\n- biology\nsize_categories:\n- 10M<n<100M\n---\n# Dataset Card for \"ProstT5Dataset\"\n\n* **Contributors:** Michael Heinzinger and Konstantin Weissenow, Joaquin Gomez Sanchez and Adrian Henkel, Martin Steinegger and Burkhard Rost\n* **Licence:** MIT\n\n## Table of Contents\n- [Overview](#overview)\n- [Dataset Description](#dataset-description)\n    - [Data Collection and Annotation](#data-collection-and-annotation)\n    - [Data Splits](#data-splits)\n- [Dataset Structure](#dataset-structure)\n    - [Data Fields](#data-fields)\n    - [Data Instances](#data-instances)\n- [Data Considerations](#data-considerations)\n    - [Social Impact of Dataset](#social-impact-of-dataset)\n    - [Discussion of Biases](#discussion-of-biases)\n    - [Other Known Limitations](#other-known-limitations)\n- [Licensing Information](#licensing-information)\n- [Citation Information](#citation-information)\n- [Contributions](#contributions)\n\n## Overview\nThe ProstT5Dataset is a curated collection of *tokenized* protein sequences and their corresponding structure sequences (3Di). \nIt is derived from the [AlphaFold Protein Structure Database](https://alphafold.ebi.ac.uk/) and includes various steps of clustering and quality filtering.\nTo capture 3D information of the sequence, the [3Di structure string representation](https://www.nature.com/articles/s41587-023-01773-0#Sec2) is leveraged. This format\ncaptures the spatial relationship of each residue to its neighbors in 3D space, effectively translating the 3D information of the sequence.\nThe sequence tokens are generated using the [ProstT5 Tokenizer](https://huggingface.co/Rostlab/ProstT5).\n\n## Data Fields\n- **input_id_x** (3Di Tokens): Corresponding tokenized 3Di structure representation sequences derived from the proteins.\n- **input_id_y** (Amino Acid Tokens): Tokenized amino acid sequences of proteins.\n\n## Dataset Description\n\n![image/png](https://cdn-uploads.huggingface.co/production/uploads/62c412251f45e8bdb2b05855/BgiKOoFUGjlHDPjbxJWOX.png)\nWe compare basic protein properties (sequence length, amino acid composition, 3Di-distribution) between our \ndataset (training, validation, test sets) and proteins obtained from the [Protein Data Bank (PDB)](https://www.rcsb.org/). Key findings include similar amino acid distributions across datasets, \nan overrepresentation of certain 3Di-tokens (d, v, p) and helical structures in AlphaFold2 predictions compared to PDB, and a tendency for shorter protein \nlengths in this dataset (average 206-238) relative to PDB proteins (average 255). The analysis also highlights the relationship between \n3Di states and secondary structures, with a notable distinction in strand-related tokens between datasets.\n\n## Data Collection and Annotation\nThe dataset began with the AlphaFold Protein Structure Database , undergoing a two-step clustering process and one step of quality filtering:\n1. *First Clustering:* 214M UniprotKB protein sequences were clustered using MMseqs2, resulting in 52M clusters based on pairwise sequence identity.\n2. *Second Clustering:* Foldseek further clustered these proteins into 18.8M clusters, expanded to 18.6M proteins by adding diverse members.\n3. *Quality Filtering:* Removed proteins with low pLDDT scores, short lengths, and highly repetitive 3Di-strings. The final training split contains 17M proteins.\n\n## Data Splits\nData splits into train, test, and, validation were created by moving whole clusters (after quality filtering - see above), to either of the sets. \nFor validation and test, we only kept representatives to avoid bias towards large families. \nThis resulted in 474 proteins for test, 474 proteins for validation and around 17M proteins for training.\n\n#","encoding":"utf8","key":"README.md","name":"README.md","size":6267,"truncated":true}],"sourceObjectKey":"data/test-00000-of-00001-b109fa020c25190c.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"Rostlab/ProstT5Dataset\")\nprint(ds)","starterSamplePath":"data/test-00000-of-00001-b109fa020c25190c.parquet","totalObjects":135},"summary":"Training data for ProstT5, the bilingual protein language model that translates between amino acid sequence and Foldseek 3Di structure strings (Heinzinger et al., NAR Genomics and…","title":"ProstT5 Dataset: Paired Amino Acid and 3Di Structure Sequences","updatedAt":"2026-11-14T17:50:20.354Z","visibility":"public","websiteUrl":"https://github.com/mheinzinger/ProstT5"}}