Also known as: evaluation (eval)
Evaluation ist der Prozess, Qualität, Verhalten oder Leistung eines Modells, Systems oder einer Änderung anhand definierter Kriterien zu messen. Im maschinellen Lernen kann sie Validierungs- und Testdaten verwenden, während LLM-Evaluation auch Sicherheit, Faktentreue, Robustheit und Auswirkungen auf Nutzer umfassen kann.
The process of measuring a model's quality or comparing different models against each other. To evaluate a supervised machine learning model, you typically judge it against a validation set and a test set. Evaluating a LLM typically involves broader quality and safety assessments.
An eval, short for "evaluation", is a type of experiment in which logged or synthetic queries are sent through two Search stacks--an experimental stack that includes your change and a base stack without your change. Evals produce diffs and metrics that let you evaluate the impact, quality, and other effects of your change on search results and other parts of the Google user experience. Evals are used during tuning, or iterations, on your change. They are also used as part of launching a change to live user traffic.
A side-by-side comparison of Model Monitoring and Evaluation. It explains how continuous observation of a deployed model differs from measuring a model, system, or change against defined criteria.
A side-by-side comparison of Evaluation and Benchmark. Understand how the concepts differ, when each term applies, and why the distinction matters for AI governance, evaluation, or system design.