Also known as: WIC
WIC, or Words in Context, is a dataset for evaluating whether a model can determine if the same word has the same meaning in two different sentences. Each example provides two sentences, a target word, and a Boolean answer indicating whether the meaning is shared. It is a component of the SuperGLUE benchmark suite.
A dataset for evaluating how well an LLM uses context to understand words that have multiple meanings. Each entry in the dataset contains: - Two sentences, each containing the target word - The target word - The correct answer (a Boolean), where: - True means the target word has the same meaning in the two sentences - False means the target word has a different meaning in the two sentences For example: - Two sentences: - There's a lot of trash on the bed of the river. - I keep a glass of water next to my bed when I sleep. - The target word: bed - Correct answer: False, because the target word has a different meaning in the two sentences. For details, see WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations. Words in Context is a component of the SuperGLUE ensemble.