A masked language model is a language model trained to predict missing or masked tokens within a sequence. Because it can use surrounding context on both sides of the masked token, this approach is commonly associated with bidirectional language models.
A language model that predicts the probability of candidate tokens to fill in blanks in a sequence. For example, a masked language model can calculate probabilities for candidate word(s) to replace the underline in the following sentence: The ____ in the hat came back. The literature typically uses the string "MASK" instead of an underline. For example: The "MASK" in the hat came back. Most modern masked language models are bidirectional.