Activation Functions for Deep Learning
What are Activation Functions? At the heart of every artificial neural networks there lies a linear transformation followed by an activation function. Artificial Neural Networks (ANNs) learn to map its input information to useful outputs information. These input output relations are mostly nonlinear in nature which are described by complex non-linear functions. Activation functions make ANNs capable of modeling and mimicking almost any complex function in the world. Major Components of an artificial neuron An artificial neuron has three basic components The synaptic links or connecting paths that provide weights w ji , to the input values s i for i =1,... d ; An adder that sums the weighted input values to compute the activation value x j . An activation function f (also called a squashing function) that maps the activation x j to f ( x j ) = y j , the output value of the neuron. The individual computation element in the artificial neural network is known as the neuron unit ,