Also known as: Boolean Questions (BoolQ) · Boolean Questions BoolQ
Boolean Questions, или BoolQ, — набор данных, используемый для оценки способности языковой модели отвечать на вопросы «да» или «нет». Каждый пример обычно включает вопрос, поддерживающий отрывок и булев ответ. Он используется в бенчмарках понимания прочитанного и понимания естественного языка.
A dataset for evaluating an LLM's proficiency in answering yes-or-no questions. Each of the challenges in the dataset has three components: - A query - A passage implying the answer to the query. - The correct answer, which is either yes or no. For example: - Query: Are there any nuclear power plants in Michigan? - Passage: ...three nuclear power plants supply Michigan with about 30% of its electricity. - Correct answer: Yes Researchers gathered the questions from anonymized, aggregated Google Search queries and then used Wikipedia pages to ground the information. For more information, see BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions. BoolQ is a component of the SuperGLUE ensemble.