Also known as: Math pass@k
Math-pass@k is an evaluation metric measuring whether a model can solve a math problem within k attempts. It is a task-specific form of pass@k, used to assess mathematical reasoning performance across multiple generated solutions.
A metric to determine an LLM's accuracy in solving a math problem within K attempts. For example, math-pass@2 measures an LLM's ability to solve math problems within two attempts. An accuracy of 0.85 on math-pass@2 indicates that an LLM was able to solve math problems 85% of the time within two attempts. math-pass@k is identical to the pass@k metric, except that the term math-pass@k is specifically used for math evaluation.