Валидация — это оценка обученной или частично обученной модели для проверки производительности, обобщающей способности и пригодности для intended purpose. Она может включать проверку predictive quality, настройку non-learnable parameters и подтверждение того, что поведение модели соответствует заданным требованиям.
The initial evaluation of a model's quality. Validation checks the quality of a model's predictions against the validation set. Because the validation set differs from the training set, validation helps guard against overfitting. You might think of evaluating the model against the validation set as the first round of testing and evaluating the model against the test set as the second round of testing.
Validation checks the quality of a model's inferences against the validation set. This involves defining metrics to measure the generated content's quality, speed, adherence to instructions, and safety. Validation often uses labeled data (input prompts and expected outputs) to compare the model's inferences with the ground truth. Metrics like F1 score (for classification) and ROUGE-L score (for summarization) might be used. The process also includes testing edge cases and unusual scenarios to ensure robustness. For deployed models, continuous monitoring and capturing of common data points and edge cases help improve future validation efforts.
‘confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled’ ISO/IEC 22989.