Also known as: Automatic Side-by-side (AutoSxS) Β· AutoSxS Β· Automatic Side-by-side AutoSxS Β· Automatic Side by side [AutoSxS]
Automatic Side-by-side, or AutoSxS, is a model-assisted evaluation approach that compares two large language models or sets of generated outputs against the same prompts. An autorater judges which response is better according to defined criteria, enabling scalable side-by-side evaluation that can approximate human preference testing in some settings.
Automatic side-by-side (AutoSxS) is a model-assisted evaluation tool that compares two large language models (LLMs) side by side. It can be used to evaluate the performance of either generative AI models in Vertex AI Model Registry or pregenerated inferences. AutoSxS uses an autorater to decide which model gives the better response to a prompt. AutoSxS is available on demand and evaluates language models with comparable performance to human raters.