Fraction of successes is an evaluation metric that divides the number of generated outputs judged successful by the total number of generated outputs. It is used when success can be defined by a task-specific criterion rather than by a single exact answer.
A metric for evaluating an ML model's generated text. The fraction of successes is the number of "successful" generated text outputs divided by the total number of generated text outputs. For example, if a large language model generated 10 blocks of code, five of which were successful, then the fraction of successes would be 50%. Although fraction of successes is broadly useful throughout statistics, within ML, this metric is primarily useful for measuring verifiable tasks like code generation or math problems.