Attention es un mecanismo de red neuronal que asigna distintos niveles de importancia a partes de una entrada al producir una salida. Ayuda a los modelos a centrarse en tokens, características o regiones relevantes y es un componente central de las arquitecturas transformer. Attention puede reducir la carga de representar toda la información de entrada por igual al ponderar el contexto según la relevancia para la tarea.
A mechanism used in a neural network that indicates the importance of a particular word or part of a word. Attention compresses the amount of information a model needs to predict the next token/word. A typical attention mechanism might consist of a weighted sum over a set of inputs, where the weight for each input is computed by another part of the neural network. Refer also to self-attention and multi-head self-attention, which are the building blocks of Transformers. See LLMs: What's a large language model? in Machine Learning Crash Course for more information about self-attention.