Also known as: Auto regressive Model
An auto-regressive model generates or predicts each output by conditioning on its own previous outputs or earlier elements in a sequence. In language modeling, an auto-regressive model predicts the next token based on preceding tokens and then repeats the process step by step. Many transformer-based large language models use this approach for text generation.
A model that infers a prediction based on its own previous predictions. For example, auto-regressive language models predict the next token based on the previously predicted tokens. All Transformer-based large language models are auto-regressive. In contrast, GAN-based image models are usually not auto-regressive since they generate an image in a single forward-pass and not iteratively in steps. However, certain image generation models are auto-regressive because they generate an image in steps.