Also known as: LLM Evaluations (evals) Β· LLM Evaluations evals
LLM evaluations, or evals, are structured methods for assessing the performance, safety, reliability, and task suitability of large language models. They may use benchmarks, metrics, human review, adversarial tests, or domain-specific test sets to compare models and detect regressions.
A set of metrics and benchmarks for assessing the performance of large language models (LLMs). At a high level, LLM evaluations: - Help researchers identify areas where LLMs need improvement. - Are useful in comparing different LLMs and identifying the best LLM for a particular task. - Help ensure that LLMs are safe and ethical to use. See Large language models (LLMs) in Machine Learning Crash Course for more information.