Linear regression example

In this example, we want to approximate the following scatter plot with a single layer neural network. Blue points are the training set given by an input \( x_i \) and an expected output \( y′_i \). The red line is the output of the network \( y=f(x) \) after training.

Linear regression example with a neural network

The following perceptron will be used for the single layer network:

Architecture of the neural network for the linear regression problem

\( x \) is the input. The activation function is given by \( f(x)=x \).

Update rule

As explained on the previous page, the weights will be updated according to this formula:

$$ w_i'= w_i + \eta(y'-y)x_i $$

Let's detail for each weight \( w_1 \) and \( w_2 \):

$$ w_1'= w_1 + \eta(y'-y)x $$ $$ w_2'= w_2 + \eta(y'-y) $$

Download & source code

See also

Last update : 01/30/2021