Also known as: Lost in the middle Effect
The lost-in-the-middle effect is the tendency of large language models to use information near the beginning or end of a long context more effectively than information placed in the middle. It can reduce answer accuracy when critical evidence is embedded deep inside a long prompt or document.
An LLM's tendency to use information from the start and end of a long context window more effectively than information from the middle. That is, given a long context, the lost-in-the-middle effect causes accuracy to be: - Relatively high when the relevant information to form a response is near the beginning or end of the context. - Relatively low when the relevant information to form a response is in the middle of the context. The term comes from Lost in the Middle: How Language Models Use Long Contexts.