{"success":true,"data":{"author":"Google Research (Geva et al.)","bucketAccessType":"public","bucketName":"google-research-datasets/discofuse","bucketPrefix":"discofuse-wikipedia","category":"Natural Language Processing","cloudProvider":"huggingface","createdAt":"2026-11-22T01:09:15.542Z","description":"DiscoFuse is a large-scale dataset for discourse-based sentence fusion: given two input sentences, a model must merge them into one fluent sentence, typically by inserting or adjusting a discourse connective (but, although, because), an anaphoric pronoun or a relative clause. Examples were created by applying a rule-based splitting method to single sentences, so each item pairs two separate sentences with the original fused sentence and labels for the discourse type and connective. The full release also has a sports-news part crawled from the web, which this row excludes. This row is the Wikipedia portion only (DiscoFuse-Wiki), built from English Wikipedia text: 16.6 million examples in random 98/1/1 train, development and test splits plus balanced versions of each, in 15 Parquet files (about 3.9 GB). It is used to train and evaluate text-editing and sentence-fusion models and to pre-train for discourse connective prediction. Load it with the Hugging Face datasets library (config discofuse-wikipedia) or read the Parquet files with pandas. Licensed by Google Research under Creative Commons Attribution-ShareAlike 3.0 (CC BY-SA 3.0); cite Geva et al., DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion, NAACL 2019.","displayTitle":"DiscoFuse Wikipedia: 16 Million Sentence Fusion Examples with Discourse Connectives from English Wikipedia (Google Research)","huggingFaceUrl":"https://huggingface.co/datasets/google-research-datasets/discofuse","id":"7370e431-1f30-5375-b912-4f3070ea830f","license":"CC-BY-SA-3.0","licenseEvidenceUrl":"https://github.com/google-research-datasets/discofuse","objectCountEstimate":49,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/1902.10526","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/discofuse\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"google-research-datasets/discofuse\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=google-research-datasets, license:cc-by-sa-3.0, size_categories:10M<n<100M, format:parquet, downloads=1037"],"format":"mixed","generatedAt":"2026-11-22T01:10:09.918Z","layout":{"example":"discofuse-sport/test-00000-of-00001.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415ec051580054c15421500120000f6021803000000616e6401073c627574050000002c20616e6400000000090d0c627574090516186265636175736501165c73696e63650b0000006f7468657277697365202c070000000d232806000000706c7573202c02012404720a054020616c74686f75676805012904757409321c2c207768696c6504052204736f016a24686f7765766572202c08019b0d350921046f72015b14756e6c657373051e106e79776179092a1c696e64656564202c014320696e7374656164202c016c4c6e6f77202c0d000000667572746865726d6f7265091a70616e64202c0c000000636f6e76657273656c79202c0e000000666f7220014414616e6365202c017404746801cf012c247265676172646c657373111a04656e09461c696e20746865206501561c10000000696e206f01714820776f726473202c07000000736f2074686174150015f00b15fa0b2c15d00f1510150615061c36002806756e6c6573731800000000f805f07d03000000d00f01067f401008c20000c0300cc3300c83200003310cc44000442008c3500083600c01100c83000cc2200cc33004c1210880100c000220c0402040320888100c00410803100c8a200c04220c003100030110c0802c08030c011200c3200c40231083200404000481400c08300c03300c03000cc3200c033000010cf47502300c83","encoding":"hex","key":"discofuse-sport/test-00000-of-00001.parquet","truncatedAt":512}],"sampleObjects":["discofuse-sport/test-00000-of-00001.parquet","discofuse-sport/train-00000-of-00030.parquet","discofuse-sport/train-00001-of-00030.parquet","discofuse-sport/train-00002-of-00030.parquet","discofuse-sport/train-00003-of-00030.parquet"],"sidecars":[{"bytes":"---\nannotations_creators:\n- machine-generated\nlanguage_creators:\n- found\nlanguage:\n- en\nlicense:\n- cc-by-sa-3.0\nmultilinguality:\n- monolingual\nsize_categories:\n- 10M<n<100M\nsource_datasets:\n- original\ntask_categories:\n- text2text-generation\ntask_ids: []\npaperswithcode_id: discofuse\npretty_name: DiscoFuse\ntags:\n- sentence-fusion\ndataset_info:\n- config_name: discofuse-sport\n  features:\n  - name: connective_string\n    dtype: string\n  - name: discourse_type\n    dtype: string\n  - name: coherent_second_sentence\n    dtype: string\n  - name: has_coref_type_pronoun\n    dtype: float32\n  - name: incoherent_first_sentence\n    dtype: string\n  - name: incoherent_second_sentence\n    dtype: string\n  - name: has_coref_type_nominal\n    dtype: float32\n  - name: coherent_first_sentence\n    dtype: string\n  splits:\n  - name: train\n    num_bytes: 14736176073\n    num_examples: 43291020\n  - name: test\n    num_bytes: 151655243\n    num_examples: 445521\n  - name: validation\n    num_bytes: 150206657\n    num_examples: 440902\n  download_size: 9422142544\n  dataset_size: 15038037973\n- config_name: discofuse-wikipedia\n  features:\n  - name: connective_string\n    dtype: string\n  - name: discourse_type\n    dtype: string\n  - name: coherent_second_sentence\n    dtype: string\n  - name: has_coref_type_pronoun\n    dtype: float32\n  - name: incoherent_first_sentence\n    dtype: string\n  - name: incoherent_second_sentence\n    dtype: string\n  - name: has_coref_type_nominal\n    dtype: float32\n  - name: coherent_first_sentence\n    dtype: string\n  splits:\n  - name: train\n    num_bytes: 6377885028\n    num_examples: 16310585\n  - name: test\n    num_bytes: 64007750\n    num_examples: 163657\n  - name: validation\n    num_bytes: 65681627\n    num_examples: 168081\n  download_size: 3929336540\n  dataset_size: 6507574405\nconfigs:\n- config_name: discofuse-sport\n  data_files:\n  - split: train\n    path: discofuse-sport/train-*\n  - split: test\n    path: discofuse-sport/test-*\n  - split: validation\n    path: discofuse-sport/validation-*\n- config_name: discofuse-wikipedia\n  data_files:\n  - split: train\n    path: discofuse-wikipedia/train-*\n  - split: test\n    path: discofuse-wikipedia/test-*\n  - split: validation\n    path: discofuse-wikipedia/validation-*\n---\n\n# Dataset Card for \"discofuse\"\n\n## Table of Contents\n- [Dataset Description](#dataset-description)\n  - [Dataset Summary](#dataset-summary)\n  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)\n  - [Languages](#languages)\n- [Dataset Structure](#dataset-structure)\n  - [Data Instances](#data-instances)\n  - [Data Fields](#data-fields)\n  - [Data Splits](#data-splits)\n- [Dataset Creation](#dataset-creation)\n  - [Curation Rationale](#curation-rationale)\n  - [Source Data](#source-data)\n  - [Annotations](#annotations)\n  - [Personal and Sensitive Information](#personal-and-sensitive-information)\n- [Considerations for Using the Data](#considerations-for-using-the-data)\n  - [Social Impact of Dataset](#social-impact-of-dataset)\n  - [Discussion of Biases](#discussion-of-biases)\n  - [Other Known Limitations](#other-known-limitations)\n- [Additional Information](#additional-information)\n  - [Dataset Curators](#dataset-curators)\n  - [Licensing Information](#licensing-information)\n  - [Citation Information](#citation-information)\n  - [Contributions](#contributions)\n\n## Dataset Description\n\n- **Repository:** https://github.com/google-research-datasets/discofuse\n- **Paper:** [DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion](https://arxiv.org/abs/1902.10526)\n- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)\n- **Size of downloaded dataset files:** 6.04 GB\n- **Size of the generated dataset:** 21.55 GB\n- **Total amount of disk used:** 27.59 GB\n\n### Dataset Summary\n\nDiscoFuse is a large scale dataset for discourse-based sentence fusion.\n\n### Supported Tasks and Leaderboards\n\n[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)\n\n### La","encoding":"utf8","key":"README.md","name":"README.md","size":9706,"truncated":true}],"sourceObjectKey":"discofuse-sport/test-00000-of-00001.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/discofuse\")\nprint(ds)","starterSamplePath":"discofuse-sport/test-00000-of-00001.parquet","totalObjects":49},"summary":"DiscoFuse is a large-scale dataset for discourse-based sentence fusion: given two input sentences, a model must merge them into one fluent sentence, typically by inserting or…","title":"DiscoFuse Wikipedia: 16 Million Sentence Fusion Examples with Discourse Connectives from English Wikipedia (Google Research)","updatedAt":"2026-11-22T03:00:59.059Z","visibility":"public","websiteUrl":"https://github.com/google-research-datasets/discofuse"}}