Also known as: COPA
A dataset for evaluating how well an LLM can identify the better of two alternative answers to a premise. Each of the challenges in the dataset consists of three components: - A premise, which is typically a statement followed by a question - Two possible answers to the question posed in the premise, one of which is correct and the other incorrect - The correct answer For example: - Premise: The man broke his toe. What was the CAUSE of this? - Possible answers: 1. He got a hole in his sock. 2. He dropped a hammer on his foot. - Correct answer: 2 COPA is a component of the SuperGLUE ensemble.