Also known as: evaluation (eval)
Оценка — это процесс измерения качества, поведения или производительности модели, системы или изменения относительно заданных критериев. В машинном обучении она может использовать validation и test data, а оценка LLM может также включать безопасность, фактичность, робастность и анализ влияния на пользователя.
The process of measuring a model's quality or comparing different models against each other. To evaluate a supervised machine learning model, you typically judge it against a validation set and a test set. Evaluating a LLM typically involves broader quality and safety assessments.
An eval, short for "evaluation", is a type of experiment in which logged or synthetic queries are sent through two Search stacks--an experimental stack that includes your change and a base stack without your change. Evals produce diffs and metrics that let you evaluate the impact, quality, and other effects of your change on search results and other parts of the Google user experience. Evals are used during tuning, or iterations, on your change. They are also used as part of launching a change to live user traffic.
A side-by-side comparison of Model Monitoring and Evaluation. It explains how continuous observation of a deployed model differs from measuring a model, system, or change against defined criteria.
A side-by-side comparison of Evaluation and Benchmark. Understand how the concepts differ, when each term applies, and why the distinction matters for AI governance, evaluation, or system design.