Average precision at k is a metric for evaluating ranked results for a single query or prompt. It averages the precision values observed at the ranks where relevant items appear within the top k results. The metric is commonly used in recommendation, retrieval, and search evaluation.
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations. Average precision at k is, well, the average of the precision at k values for each relevant result. The formula for average precision at k is therefore: \\\\\\[\\\\\average precision at k\\ = \\\\1\\n\ \\\_\i=1\\\^n \\\\\precision at k for each relevant item\ \ \\\\\\] where: - \\\\(n\\\\) is the number of relevant items in the list. Contrast with recall at k. [!NOTE] Note: Average precision at k evaluates the output for a single prompt. Use mean average precision at k to evaluate the quality of a model's output across many different prompts. [!NOTE] Note: Some people abbreviate average precision at k to simply average precision. Suppose a large language model is given the following query: And the large language model returns the following list: 1. The General 2. Mean Girls 3. Platoon 4. Bridesmaids 5. Citizen Kane 6. This is Spinal Tap Four of the movies in the returned list are very funny (that is, they are relevant) but two movies are dramas (not relevant). The following table details the results: Movie Precision at k | ------| The General 1.0 | Mean Girls 1.0 | Platoon not relevant | Bridesmaids 0.75 | Citizen Kane not relevant | This is Spinal Tap 0.67 | The number of relevant results is 4. Therefore, you can calculate the average precision at 6 as follows: