Beginner
What is model evaluation?
Evaluation is the process of measuring the quality, behavior, or performance of a model, system, or change against defined criteria. In machine learning it may use validation and test data, while LLM evaluation may also include safety, factuality, robustness, and user-impact assessments.
What is a benchmark in AI evaluation?
A benchmark is a standardized test, dataset, task, or evaluation procedure used to measure and compare the performance of AI systems.
How are precision and recall different?
Use Precision when false positives are costly; use Recall when missed positives are costly.
Why can accuracy be misleading?
Accuracy is a classification metric equal to the proportion of predictions that are correct among all predictions made. It is useful in some evaluation settings but can be misleading when classes are imbalanced, where precision, recall, or other metrics may better reflect model performance.