Also known as: Rectified Linear Unit (ReLU) · Rectified Linear Unit ReLU
An activation function with the following behavior: - If input is negative or zero, then the output is 0. - If input is positive, then the output is equal to the input. For example: - If the input is -3, then the output is 0. - If the input is +3, then the output is 3.0. Here is a plot of ReLU: !A cartesian plot of two lines. The first line has a constant y value of 0, running along the x-axis from -infinity,0 to 0,-0. The second line starts at 0,0. This line has a slope of +1, so it runs from 0,0 to +infinity,+infinity. ReLU is a very popular activation function. Despite its simple behavior, ReLU still enables a neural network to learn nonlinear relationships between features and the label.