Regularization is a set of techniques that constrain or modify model training to improve generalization and reduce overfitting. Methods such as L1 regularization, L2 regularization, dropout, and early stopping discourage models from relying too heavily on training-specific patterns.
Regularization penalizes a linear model for being too complicated. There are two types of regularization:
A set of techniques such as dropout, early stopping, and L1 and L2 regularization to reduce overfitting and underfitting when training a learning algorithm.
Any mechanism that reduces overfitting. Popular types of regularization include: - L~1~ regularization - L~2~ regularization - dropout regularization - early stopping (this is not a formal regularization method, but can effectively limit overfitting) Regularization can also be defined as the penalty on a model's complexity. Regularization is counterintuitive. Increasing regularization usually increases training loss, which is confusing because, well, isn't the goal to minimize training loss? Actually, no. The goal isn't to minimize training loss. The goal is to make excellent predictions on real-world examples. Remarkably, even though increasing regularization increases training loss, it usually helps models make better predictions on real-world examples. See Overfitting: Model complexity in Machine Learning Crash Course for more information.