APPS Dataset Dataset Description APPS is a benchmark for code generation with 10000 problems. It can be used to evaluate the ability of language models to generate code from natural language specifications. You can also find APPS metric in the hub here codeparrot/apps metric. Languages The dataset contains questions in English and code solutions in Python. Dataset Structure How to use it You can load and iterate through the dataset with the following two lines of code for the train split: Each sample consists of a programming problem formulation in English, some ground truth Python solutions, test cases that are defined by their inputs and outputs and function name if provided, as well as some metadata regarding the difficulty level of the problem and its source. If a sample has non empty input output feature, you can read it as a dictionary with keys inputs and outputs and fn name if it exists, and similarily you can parse the solutions into a list of solutions as shown in the code above. You can also filter the dataset for the difficulty level: Introductory, Interview and Competition. Just pass the list of difficulties as a list. E.g. if you want the most challenging problems, yo…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy