Mostly Basic Python Problems (MBPP) is a benchmarkdataset used to evaluate a model's ability to generate correct Python code for short programming tasks. It typically measures whether generated programs satisfy provided tests or expected behavior.
A dataset for evaluating an LLM's proficiency in generating Python code.
Mostly Basic Python Problems
provides about 1,000 crowd-sourced programming problems.
Each problem in the dataset contains:
- A task description
- Solution code
- Three automated test cases