Single layer training algorithm

Network architecture

Consider the following single layer architecture:

where:

• \(x_i\) are the inputs of the network
• \(o_j\) are the outputs of the network
• \(f(\sigma)\) is the activation function
• \(w_{ij}\) is the weight connecting input \(x_i\) to neuron \( j \) (i.e. associated to output \(o_j\))

Algorithm

Here is the trainning algorothm for a single layer neural network:

Initialize weights \( w_{ij} \) with arbitrary values
Repeat
         Pick a training example <\( x \),\( \check{o}\)> (x is the input, \( \check{o} \) is the expected output)
         Compute the sum for each neuron: \( S_j = \sum\limits_{i=1}^N w_{ij}x_i \)
         Compute outputs (\( o \)) of the network: \( o_j = f(S) \)
         For each output, compute the error: \( \delta_j = ( \check{o}_j - o_j ) \)
         Update each synaptic weights of the network: \( w_{ij} = w_{ij} + \eta.\delta_j.x_i.\frac{df(S)}{dS} \)
Until the training set is empty

See also

• Neural networks curve fitting
• Datasets for deep learning
• Gradient descent example
• How popular are neural networks over the years?
• Install TensorFlow and Keras for Linux
• Learning rule demonstration
• Linear regression example
• Most popular activation functions for deep learning
• Most relevant deep learning research papers
• Neural Network Perceptron
• Simplest neural network with TensorFlow
• Simplest perceptron
• Single layer classification example
• Gradient descent for neural networks
• Single layer limitations
• Neural networks

---