Регуляризация — набор методов, которые ограничивают или изменяют обучение модели для улучшения generalization и снижения overfitting. Методы, такие как L1 regularization, L2 regularization, dropout и early stopping, уменьшают зависимость модели от patterns, специфичных для training data.
Regularization penalizes a linear model for being too complicated. There are two types of regularization:
Any mechanism that reduces overfitting. Popular types of regularization include: - L~1~ regularization - L~2~ regularization - dropout regularization - early stopping (this is not a formal regularization method, but can effectively limit overfitting) Regularization can also be defined as the penalty on a model's complexity. Regularization is counterintuitive. Increasing regularization usually increases training loss, which is confusing because, well, isn't the goal to minimize training loss? Actually, no. The goal isn't to minimize training loss. The goal is to make excellent predictions on real-world examples. Remarkably, even though increasing regularization increases training loss, it usually helps models make better predictions on real-world examples. See Overfitting: Model complexity in Machine Learning Crash Course for more information.