Also known as: WIC
WIC, или Words in Context, — это dataset для оценки того, может ли model determine, имеет ли одно и то же слово одинаковое значение в двух different sentences. Каждый example содержит two sentences, target word и Boolean answer, indicating whether the meaning is shared. Это компонент benchmark suite SuperGLUE.
A dataset for evaluating how well an LLM uses context to understand words that have multiple meanings. Each entry in the dataset contains: - Two sentences, each containing the target word - The target word - The correct answer (a Boolean), where: - True means the target word has the same meaning in the two sentences - False means the target word has a different meaning in the two sentences For example: - Two sentences: - There's a lot of trash on the bed of the river. - I keep a glass of water next to my bed when I sleep. - The target word: bed - Correct answer: False, because the target word has a different meaning in the two sentences. For details, see WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations. Words in Context is a component of the SuperGLUE ensemble.