Auxiliary loss is an additional loss function used alongside a model's main loss function during training. It can help improve optimization, accelerate early learning, or strengthen gradient flow to earlier layers, particularly in deep neural networks where vanishing gradients may occur.
A loss function---used in conjunction with a neural network model's main loss function---that helps accelerate training during the early iterations when weights are randomly initialized. Auxiliary loss functions push effective gradients to the earlier layers. This facilitates convergence during training by combating the vanishing gradient problem.