Staged training is a training strategy in which a model is trained through a sequence of discrete phases. Each stage may change architecture, data, objectives, or parameters to improve efficiency, stability, or final model quality.
A tactic of training a model in a sequence of discrete stages. The goal can be either to speed up the training process, or to achieve better model quality. An illustration of the progressive stacking approach is shown below: - Stage 1 contains 3 hidden layers, stage 2 contains 6 hidden layers, and stage 3 contains 12 hidden layers. - Stage 2 begins training with the weights learned in the 3 hidden layers of Stage 1. Stage 3 begins training with the weights learned in the 6 hidden layers of Stage 2. !Three stages, which are labeled Stage 1, Stage 2, and Stage 3. Each stage contains a different number of layers: Stage 1 contains 3 layers, Stage 2 contains 6 layers, and Stage 3 contains 12 layers. The 3 layers from Stage 1 become the first 3 layers of Stage 2. Similarly, the 6 layers from Stage 2 become the first 6 layers of Stage 3. See also pipelining.