SuperGLUE is a benchmark suite of language understanding tasks used to evaluate advanced natural language processing models. It includes multiple datasets designed to test reasoning, comprehension, inference, and contextual understanding.
An ensemble of datasets for rating an LLM's overall ability to understand and generate text. The ensemble consists of the following datasets: - Boolean Questions (BoolQ) - CommitmentBank (CB) - Choice of Plausible Alternatives (COPA) - Multi-sentence Reading Comprehension (MultiRC) - Reading Comprehension with Commonsense Reasoning Dataset (ReCoRD) - Recognizing Textual Entailment (RTE) - Words in Context (WiC) - Winograd Schema Challenge (WSC) For details, see SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.