{"success":true,"data":{"author":"Google Research (Austin et al.)","bucketAccessType":"public","bucketName":"google-research-datasets/mbpp","bucketPrefix":"","category":"Computer Science","cloudProvider":"huggingface","createdAt":"2026-11-22T01:09:15.755Z","description":"MBPP (Mostly Basic Python Problems) is a program synthesis benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers, covering programming fundamentals and standard library functionality. Each problem has a short natural language task description, a reference code solution and three automated assert test cases, so generated code can be scored by execution (pass@k). A hand-verified subset of 427 problems is provided as the sanitized configuration. The authors' split uses task IDs 11 to 510 for testing, 1 to 10 for few-shot prompts, 511 to 600 for validation and 601 to 974 for training. The repository holds two folders of Parquet files, full/ and sanitized/, each with train, validation, test and prompt splits (well under 1 MB). It is one of the standard benchmarks for code generation by large language models, alongside HumanEval. Load with the Hugging Face datasets library (load_dataset('google-research-datasets/mbpp')). Licensed CC BY 4.0 by Google Research; cite Austin et al., Program Synthesis with Large Language Models, 2021.","displayTitle":"MBPP (Mostly Basic Python Problems): 974 Crowd-Sourced Python Programming Tasks with Reference Solutions and Unit Tests (Google Research)","huggingFaceUrl":"https://huggingface.co/datasets/google-research-datasets/mbpp","id":"bc986b33-48e5-5d3b-9d23-c0bf4e3f0060","license":"CC-BY-4.0","licenseEvidenceUrl":"https://huggingface.co/datasets/google-research-datasets/mbpp","objectCountEstimate":10,"probeStatus":"ok","referenceUrl":"https://arxiv.org/abs/2108.07732","region":"us","schemaCard":{"accessHints":[{"language":"python-datasets","snippet":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/mbpp\")"},{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"google-research-datasets/mbpp\", repo_type=\"dataset\")"}],"caveats":["HuggingFace metadata: author=google-research-datasets, license:cc-by-4.0, size_categories:1K<n<10K, format:parquet, downloads=288786"],"format":"mixed","generatedAt":"2026-11-22T01:10:11.846Z","layout":{"example":"full/prompt-00000-of-00001.parquet","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleHeaders":[{"bytes":"504152311504155015544c15141500120000289c0100000002000000030000000400000005000000060000000700000008000000090000000a0000001500152015242c15141510150615061c18040a000000180401000000160028040a000000180401000000000000103c0200000014010405103254769800000026f6011c150219351000061918077461736b5f69641502161416e60116ee01267826081c18040a000000180401000000160028040a00000018040100000000192c15041500150200150015101502000000150415fa0e15c2094c15141500120000bd07f05290000000577269746520612066756e6374696f6e20746f2066696e6420746865206d696e696d756d20636f7374207061746820746f20726561636820286d2c206e292066726f6d2028302c20302920666f7220013640676976656e20636f7374206d6174726978050c385b5d5b5d20616e64206120706f7369056c002809450069092e0122042e4d7e94004473696d696c617220656c656d656e74732066018319784074776f207475706c65206c697374732e361d5114707974686f6e32ec006c6964656e74696679206e6f6e2d7072696d65206e756d626572732e661d3a00664e1f01386c61726765737420696e74656765720d8b00612d01017f0c206f66200d4f70207573696e67206865617020717565756520616c676f726974686d2e","encoding":"hex","key":"full/prompt-00000-of-00001.parquet","truncatedAt":512}],"sampleObjects":["full/prompt-00000-of-00001.parquet","full/test-00000-of-00001.parquet","full/train-00000-of-00001.parquet","full/validation-00000-of-00001.parquet","sanitized/prompt-00000-of-00001.parquet"],"sidecars":[{"bytes":"---\nannotations_creators:\n- crowdsourced\n- expert-generated\nlanguage_creators:\n- crowdsourced\n- expert-generated\nlanguage:\n- en\nlicense:\n- cc-by-4.0\nmultilinguality:\n- monolingual\nsize_categories:\n- n<1K\nsource_datasets:\n- original\ntask_categories:\n- text2text-generation\ntask_ids: []\npretty_name: Mostly Basic Python Problems\ntags:\n- code-generation\ndataset_info:\n- config_name: full\n  features:\n  - name: task_id\n    dtype: int32\n  - name: text\n    dtype: string\n  - name: code\n    dtype: string\n  - name: test_list\n    sequence: string\n  - name: test_setup_code\n    dtype: string\n  - name: challenge_test_list\n    sequence: string\n  splits:\n  - name: train\n    num_bytes: 176879\n    num_examples: 374\n  - name: test\n    num_bytes: 244104\n    num_examples: 500\n  - name: validation\n    num_bytes: 42405\n    num_examples: 90\n  - name: prompt\n    num_bytes: 4550\n    num_examples: 10\n  download_size: 236069\n  dataset_size: 467938\n- config_name: sanitized\n  features:\n  - name: source_file\n    dtype: string\n  - name: task_id\n    dtype: int32\n  - name: prompt\n    dtype: string\n  - name: code\n    dtype: string\n  - name: test_imports\n    sequence: string\n  - name: test_list\n    sequence: string\n  splits:\n  - name: train\n    num_bytes: 63453\n    num_examples: 120\n  - name: test\n    num_bytes: 132720\n    num_examples: 257\n  - name: validation\n    num_bytes: 20050\n    num_examples: 43\n  - name: prompt\n    num_bytes: 3407\n    num_examples: 7\n  download_size: 115422\n  dataset_size: 219630\nconfigs:\n- config_name: full\n  data_files:\n  - split: train\n    path: full/train-*\n  - split: test\n    path: full/test-*\n  - split: validation\n    path: full/validation-*\n  - split: prompt\n    path: full/prompt-*\n  default: true\n- config_name: sanitized\n  data_files:\n  - split: train\n    path: sanitized/train-*\n  - split: test\n    path: sanitized/test-*\n  - split: validation\n    path: sanitized/validation-*\n  - split: prompt\n    path: sanitized/prompt-*\n---\n\n# Dataset Card for Mostly Basic Python Problems (mbpp)\n\n## Table of Contents\n- [Dataset Card for Mostly Basic Python Problems (mbpp)](#dataset-card-for-mostly-basic-python-problems-(mbpp))\n  - [Table of Contents](#table-of-contents)\n  - [Dataset Description](#dataset-description)\n    - [Dataset Summary](#dataset-summary)\n    - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)\n    - [Languages](#languages)\n  - [Dataset Structure](#dataset-structure)\n    - [Data Instances](#data-instances)\n    - [Data Fields](#data-fields)\n    - [Data Splits](#data-splits)\n  - [Dataset Creation](#dataset-creation)\n    - [Curation Rationale](#curation-rationale)\n    - [Source Data](#source-data)\n      - [Initial Data Collection and Normalization](#initial-data-collection-and-normalization)\n      - [Who are the source language producers?](#who-are-the-source-language-producers)\n    - [Annotations](#annotations)\n      - [Annotation process](#annotation-process)\n      - [Who are the annotators?](#who-are-the-annotators)\n    - [Personal and Sensitive Information](#personal-and-sensitive-information)\n  - [Considerations for Using the Data](#considerations-for-using-the-data)\n    - [Social Impact of Dataset](#social-impact-of-dataset)\n    - [Discussion of Biases](#discussion-of-biases)\n    - [Other Known Limitations](#other-known-limitations)\n  - [Additional Information](#additional-information)\n    - [Dataset Curators](#dataset-curators)\n    - [Licensing Information](#licensing-information)\n    - [Citation Information](#citation-information)\n    - [Contributions](#contributions)\n\n## Dataset Description\n- **Repository:** https://github.com/google-research/google-research/tree/master/mbpp\n- **Paper:** [Program Synthesis with Large Language Models](https://arxiv.org/abs/2108.07732)\n\n### Dataset Summary\nThe benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in t","encoding":"utf8","key":"README.md","name":"README.md","size":9058,"truncated":true}],"sourceObjectKey":"full/prompt-00000-of-00001.parquet","starterCell":"from datasets import load_dataset\nds = load_dataset(\"google-research-datasets/mbpp\")\nprint(ds)","starterSamplePath":"full/prompt-00000-of-00001.parquet","totalObjects":10},"summary":"MBPP (Mostly Basic Python Problems) is a program synthesis benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers, covering…","title":"MBPP (Mostly Basic Python Problems): 974 Crowd-Sourced Python Programming Tasks with Reference Solutions and Unit Tests (Google Research)","updatedAt":"2026-11-22T04:01:51.591Z","visibility":"public","websiteUrl":"https://github.com/google-research/google-research/tree/master/mbpp"}}