Also known as: WIC
WIC, o Words in Context, es un conjunto de datos para evaluar si un modelo puede determinar si la misma palabra tiene el mismo significado en dos oraciones diferentes. Cada ejemplo proporciona dos oraciones, una palabra objetivo y una respuesta booleana que indica si el significado es compartido. Es un componente del benchmark SuperGLUE.
A dataset for evaluating how well an LLM uses context to understand words that have multiple meanings. Each entry in the dataset contains: - Two sentences, each containing the target word - The target word - The correct answer (a Boolean), where: - True means the target word has the same meaning in the two sentences - False means the target word has a different meaning in the two sentences For example: - Two sentences: - There's a lot of trash on the bed of the river. - I keep a glass of water next to my bed when I sleep. - The target word: bed - Correct answer: False, because the target word has a different meaning in the two sentences. For details, see WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations. Words in Context is a component of the SuperGLUE ensemble.