A replay buffer is a memory store used in reinforcement learning, especially Deep Q-Network methods, to hold past state transitions. Training samples are drawn from the buffer to reduce temporal correlation and improve learning stability.
In DQN-like algorithms, the memory used by the agent to store state transitions for use in experience replay.