Also known as: Rectified Linear Unit (ReLU) Β· Rectified Linear Unit ReLU
Rectified Linear Unit, or ReLU, is an activation function that outputs zero for negative inputs and returns the input itself for positive inputs. It is widely used in neural networks because it is simple, computationally efficient, and helps support nonlinear learning.
An activation function with the following behavior: - If input is negative or zero, then the output is 0. - If input is positive, then the output is equal to the input. For example: - If the input is -3, then the output is 0. - If the input is +3, then the output is 3.0. Here is a plot of ReLU: !A cartesian plot of two lines. The first line has a constant y value of 0, running along the x-axis from -infinity,0 to 0,-0. The second line starts at 0,0. This line has a slope of +1, so it runs from 0,0 to +infinity,+infinity. ReLU is a very popular activation function. Despite its simple behavior, ReLU still enables a neural network to learn nonlinear relationships between features and the label.