{"success":true,"data":{"author":"Google Research (Kwiatkowski et al.)","bucketAccessType":"public","bucketName":"google-research-datasets/natural_questions","bucketPrefix":"","category":"Natural Language Processing","cloudProvider":"huggingface","createdAt":"2026-11-22T01:09:15.405Z","description":"Natural Questions (NQ) is an open-domain question answering benchmark built from real, anonymized queries that users issued to Google Search. For each question an annotator was shown an entire English Wikipedia article and marked a long answer (usually a paragraph or table) and, where one exists, a short answer span or a yes/no answer; some questions have no answer on the page. Because systems must read the whole page rather than a selected passage, NQ is a standard test for reading comprehension, passage retrieval and long-document machine reading. This Hugging Face release has 307,373 training examples and 7,830 development examples, each with the full article HTML, tokens, the question, and the long and short answer annotations, stored as about 300 Parquet shards (about 58 GB) under default/ and dev/. Load it with the Hugging Face datasets library (load_dataset('google-research-datasets/natural_questions')) or read the Parquet files with pandas or pyarrow. Released by Google Research under Creative Commons Attribution-ShareAlike 3.0 (CC BY-SA 3.0); cite Kwiatkowski et al., Natural Questions: a Benchmark for Question Answering Research, TACL 2019.","displayTitle":"Natural Questions (NQ): 307,373 Real Google Search Questions with Long and Short Answers Annotated on Full Wikipedia Pages (Google Research)","huggingFaceUrl":"https://huggingface.co/datasets/google-research-datasets/natural_questions","id":"126007a3-cf44-5b7a-bf0a-63d86de870f0","license":"CC-BY-SA-3.0","licenseEvidenceUrl":"https://huggingface.co/datasets/google-research-datasets/natural_questions","objectCountEstimate":303,"probeStatus":"ok","referenceUrl":"https://doi.org/10.1162/tacl_a_00276","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/natural_questions\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"google-research-datasets/natural_questions\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=google-research-datasets, license:cc-by-sa-3.0, size_categories:10K<n<100K, format:parquet, downloads=40375"],"format":"mixed","generatedAt":"2026-11-22T01:10:09.616Z","layout":{"example":"default/train-00000-of-00287.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"50415231150415a0ed0215b8a0024c15d00f1500120000d0b601b81300000034353439343635323432373835323738373835140000002d32353433333838303032313636313633323532012f4835393835333535303431333833313637313833092f14393735313732011c1c3633303535373938051844313035323333343833333530323532383439095f4836323532333433333532383636383932393435055f443834373538393230353336393435363536300117482d37373933393833333630323139353037353101174432393636343030303232353032343030313905454833353432353936343639323931323139393636012e483532393730393431383233353732303535363105a3443734323931303531313332333937363936390974403130353735333635323135393333343332092f0435320160203339353536333435330109054744333132363030363633323530333937353931058d303439333138383034313631323121070038055e443234373331393939373330373733343938300946483734373737383132323439393739303334313401bc2031303330363530333621751431303837363005754838373838363136353935363731343139303239012f44353231363031323233363234353137373133298e4433303830393430303738323435393337393809d44432333034353837303532373939","encoding":"hex","key":"default/train-00000-of-00287.parquet","truncatedAt":512}],"sampleObjects":["default/train-00000-of-00287.parquet","default/train-00001-of-00287.parquet","default/train-00002-of-00287.parquet","default/train-00003-of-00287.parquet","default/train-00004-of-00287.parquet"],"sidecars":[{"bytes":"---\nannotations_creators:\n- no-annotation\nlanguage_creators:\n- crowdsourced\nlanguage:\n- en\nlicense:\n- cc-by-sa-3.0\nmultilinguality:\n- monolingual\nsize_categories:\n- 100K<n<1M\nsource_datasets:\n- original\ntask_categories:\n- question-answering\ntask_ids:\n- open-domain-qa\npaperswithcode_id: natural-questions\npretty_name: Natural Questions\ndataset_info:\n- config_name: default\n  features:\n  - name: id\n    dtype: string\n  - name: document\n    struct:\n    - name: html\n      dtype: string\n    - name: title\n      dtype: string\n    - name: tokens\n      sequence:\n      - name: end_byte\n        dtype: int64\n      - name: is_html\n        dtype: bool\n      - name: start_byte\n        dtype: int64\n      - name: token\n        dtype: string\n    - name: url\n      dtype: string\n  - name: question\n    struct:\n    - name: text\n      dtype: string\n    - name: tokens\n      sequence: string\n  - name: long_answer_candidates\n    sequence:\n    - name: end_byte\n      dtype: int64\n    - name: end_token\n      dtype: int64\n    - name: start_byte\n      dtype: int64\n    - name: start_token\n      dtype: int64\n    - name: top_level\n      dtype: bool\n  - name: annotations\n    sequence:\n    - name: id\n      dtype: string\n    - name: long_answer\n      struct:\n      - name: candidate_index\n        dtype: int64\n      - name: end_byte\n        dtype: int64\n      - name: end_token\n        dtype: int64\n      - name: start_byte\n        dtype: int64\n      - name: start_token\n        dtype: int64\n    - name: short_answers\n      sequence:\n      - name: end_byte\n        dtype: int64\n      - name: end_token\n        dtype: int64\n      - name: start_byte\n        dtype: int64\n      - name: start_token\n        dtype: int64\n      - name: text\n        dtype: string\n    - name: yes_no_answer\n      dtype:\n        class_label:\n          names:\n            '0': 'NO'\n            '1': 'YES'\n  splits:\n  - name: train\n    num_bytes: 143039948860\n    num_examples: 307373\n  - name: validation\n    num_bytes: 3451288641\n    num_examples: 7830\n  download_size: 56843626971\n  dataset_size: 146491237501\n- config_name: dev\n  features:\n  - name: id\n    dtype: string\n  - name: document\n    struct:\n    - name: title\n      dtype: string\n    - name: url\n      dtype: string\n    - name: html\n      dtype: string\n    - name: tokens\n      sequence:\n      - name: token\n        dtype: string\n      - name: is_html\n        dtype: bool\n      - name: start_byte\n        dtype: int64\n      - name: end_byte\n        dtype: int64\n  - name: question\n    struct:\n    - name: text\n      dtype: string\n    - name: tokens\n      sequence: string\n  - name: long_answer_candidates\n    sequence:\n    - name: start_token\n      dtype: int64\n    - name: end_token\n      dtype: int64\n    - name: start_byte\n      dtype: int64\n    - name: end_byte\n      dtype: int64\n    - name: top_level\n      dtype: bool\n  - name: annotations\n    sequence:\n    - name: id\n      dtype: string\n    - name: long_answer\n      struct:\n      - name: start_token\n        dtype: int64\n      - name: end_token\n        dtype: int64\n      - name: start_byte\n        dtype: int64\n      - name: end_byte\n        dtype: int64\n      - name: candidate_index\n        dtype: int64\n    - name: short_answers\n      sequence:\n      - name: start_token\n        dtype: int64\n      - name: end_token\n        dtype: int64\n      - name: start_byte\n        dtype: int64\n      - name: end_byte\n        dtype: int64\n      - name: text\n        dtype: string\n    - name: yes_no_answer\n      dtype:\n        class_label:\n          names:\n            '0': 'NO'\n            '1': 'YES'\n  splits:\n  - name: validation\n    num_bytes: 3451288639\n    num_examples: 7830\n  download_size: 1337126358\n  dataset_size: 3451288639\nconfigs:\n- config_name: default\n  data_files:\n  - split: train\n    path: default/train-*\n  - split: validation\n    path: default/validation-*\n- config_name: dev\n  data_files:\n  - split: validation\n    path: dev/validation-*\n---\n\n# Dataset Card for Natural Questions\n\n## Table of Contents\n- [Dataset Description](#dataset-description)\n  - [Dataset Summary](#dataset-summary)\n  - [Supported T","encoding":"utf8","key":"README.md","name":"README.md","size":13681,"truncated":true}],"sourceObjectKey":"default/train-00000-of-00287.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/natural_questions\")\nprint(ds)","starterSamplePath":"default/train-00000-of-00287.parquet","totalObjects":303},"summary":"Natural Questions (NQ) is an open-domain question answering benchmark built from real, anonymized queries that users issued to Google Search.","title":"Natural Questions (NQ): 307,373 Real Google Search Questions with Long and Short Answers Annotated on Full Wikipedia Pages (Google Research)","updatedAt":"2026-11-22T04:06:16.721Z","visibility":"public","websiteUrl":"https://github.com/google-research-datasets/natural-questions"}}