Функция активации — функция, используемая в узле нейронной сети для преобразования взвешенных входов в выход, передаваемый последующим слоям. Вводя нелинейное поведение, функции активации позволяют нейронным сетям моделировать сложные зависимости, а не только простые линейные закономерности.
In artificial neural networks, the activation function of a node defines the output of that node given an input or set of inputs.
A function that enables neural networks to learn nonlinear (complex) relationships between features and the label. Popular activation functions include: - ReLU - Sigmoid The plots of activation functions are never single straight lines. For example, the plot of the ReLU activation function consists of two straight lines: !A cartesian plot of two lines. The first line has a constant y value of 0, running along the x-axis from -infinity,0 to 0,-0. The second line starts at 0,0. This line has a slope of +1, so it runs from 0,0 to +infinity,+infinity. A plot of the sigmoid activation function looks as follows: !A two-dimensional curved plot with x values spanning the domain -infinity to +positive, while y values span the range almost 0 to almost 1. When x is 0, y is 0.5. The slope of the curve is always positive, with the highest slope at 0,0.5 and gradually decreasing slopes as the absolute value of x increases. In a neural network, activation functions manipulate the weighted sum of all the inputs to a neuron. To calculate a weighted sum, the neuron adds up the products of the relevant values and weights. For example, suppose the relevant input to a neuron consists of the following: ---| input weight | -1.3 | 0.6 | 0.4 | The weighted sum is therefore: Suppose the designer of this neural network chooses the sigmoid function to be the activation function. In that case, the neuron calculates the sigmoid of -2.0, which is approximately 0.12. Therefore, the neuron passes 0.12 (rather than -2.0) to the next layer in the neural network. The following figure illustrates the relevant part of the process: !An input layer with three features passing three feature values and three weights to a neuron in a hidden layer. The hidden layer calculates the raw value (-2.0), and then passes the raw value to the activation function. The activation function calculates the sigmoid of the raw value and passes the result (0.12) to the next layer of the neural network. See Neural networks: Activation functions in Machine Learning Crash Course for more information.