An autorater is a model or automated system used to evaluate the quality of another model's response or output. It may compare responses, apply scoring criteria, or support evaluation pipelines such as side-by-side model assessment.
An autorater is a language model that evaluates the quality of model responses given an original inferenceprompt. It's used in the AutoSxSpipeline to compare the inferences of two models and determine which model performed the best. For more information, see The autorater.