Also known as: CB
CB, или CommitmentBank, — это benchmark-датасет, используемый для оценки способности языковых моделей выводить степень приверженности говорящего или автора утверждению. Он проверяет понимание убеждения, следования и контекста в коротких фрагментах текста.
A dataset for evaluating an LLM's proficiency in determining whether the author of a passage believes a target clause within that passage. Each entry in the dataset contains: - A passage - A target clause within that passage - A Boolean value indicating whether the passage's author believes the target clause For example: - Passage: What fun to hear Artemis laugh. She's such a serious child. I didn't know she had a sense of humor. - Target clause: she had a sense of humor - Boolean: True, which means the author believes the target clause CommitmentBank is a component of the SuperGLUE ensemble.