Dataset Card for Mostly Basic Python Problems (mbpp) Table of Contents Dataset Card for Mostly Basic Python Problems (mbpp)) Table of Contents Dataset Description Dataset Summary Supported Tasks and Leaderboards Languages Dataset Structure Data Instances Data Fields Data Splits Dataset Creation Curation Rationale Source Data Initial Data Collection and Normalization Who are the source language producers? Annotations Annotation process Who are the annotators? Personal and Sensitive Information Considerations for Using the Data Social Impact of Dataset Discussion of Biases Other Known Limitations Additional Information Dataset Curators Licensing Information Citation Information Contributions Dataset Description Repository: https://github.com/google research/google research/tree/master/mbpp Paper: Program Synthesis with Large Language Models Dataset Summary The benchmark consists of around 1,000 crowd sourced Python programming problems, designed to be solvable by entry level programmers, covering programming fundamentals, standard library functionality, and so on. Each problem consists of a task description, code solution and 3 automated test cases. As described in the paper, a subse…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy