Validation is the evaluation of a trained or partially trained model to assess performance, generalization, and suitability for its intended purpose. It may include checking predictive quality, tuning non-learnable parameters, and confirming that model behavior meets specified requirements.
The initial evaluation of a model's quality. Validation checks the quality of a model's predictions against the validation set. Because the validation set differs from the training set, validation helps guard against overfitting. You might think of evaluating the model against the validation set as the first round of testing and evaluating the model against the test set as the second round of testing.
Validation checks the quality of a model's inferences against the validation set. This involves defining metrics to measure the generated content's quality, speed, adherence to instructions, and safety. Validation often uses labeled data (input prompts and expected outputs) to compare the model's inferences with the ground truth. Metrics like F1 score (for classification) and ROUGE-L score (for summarization) might be used. The process also includes testing edge cases and unusual scenarios to ensure robustness. For deployed models, continuous monitoring and capturing of common data points and edge cases help improve future validation efforts.