# Neural networks curve fitting

This page presents a neural network curve fitting example. This example shows and details how to create nonlinear regression with TensorFlow.

The following has been performed with the following version:

• Python 3.6.9 64 bits
• Matplotlib 3.1.1
• TensorFlow 2.1.0

Try the example online on Google Colaboratory.

## Problem definition

The goal of this example is to approximate a nonlinear function given by the following equation:

$$y = 0.1.x.\cos(x)$$

The blue dots are the training set, the red line is the output of the network:

## Source code

Each line is explained in the next section. Source code and example can be run online on Google Colaboratory

## Explanation

First, we import the libraries:

• numpy for arrays and matrices
• matplotlib for displaying charts
• Tensorflow for neural networks
• math for the cosinus function
import numpy as np
import matplotlib.pyplot as plt
from tensorflow import keras
from google.colab import files
import tensorflow as tf
import math

Then, we create the training data. x_data composed of 1000 points, and normal noise is added to the y-coordinate of each point:

# Create noisy data
x_data = np.linspace(-10, 10, num=1000)
y_data = 0.1*x_data*np.cos(x_data) + 0.1*np.random.normal(size=1000)
print('Data created successfully')


Here is the training set:

Once our training dataset is built, we can create our network:

• First layer is a single linear unit layer (for the input)
• Second layer is a 64 units RELU layer
• Third layer is a 64 units RELU layer
• Last layer is a single linear unit (for the output)

RELU is probably not the best choice for this application, but it works fine. ELU should provide smotther results.

# Create the model
model = keras.Sequential()
model.add(keras.layers.Dense(units = 1, activation = 'linear', input_shape=[1]))
model.add(keras.layers.Dense(units = 64, activation = 'relu'))
model.add(keras.layers.Dense(units = 64, activation = 'relu'))
model.add(keras.layers.Dense(units = 1, activation = 'linear'))

# Display the model
model.summary()

The model is compiled with the following optimization parameters:

• Optimization algorithm is Adam (optimizer="adam"), more info here.
• Loss is the regression loss based on Mean Square Error (loss='mse'). More information about metrics on this page.

Once the model is defined, let's train our network:

• x_data is the input
• y_data is the expected output
• epochs=100 means our network will be trained 100 times with our dataset
• verbose=1 display progression and loss in the console.
# Training
model.fit( x_data, y_data, epochs=100, verbose=1)

It should display something like (loss should decrease):

Train on 1000 samples
Epoch 1/100
1000/1000 [==============================] - 0s 321us/sample - loss: 0.2125
Epoch 2/100
1000/1000 [==============================] - 0s 49us/sample - loss: 0.1914
Epoch 3/100
1000/1000 [==============================] - 0s 50us/sample - loss: 0.1932
Epoch 4/100
1000/1000 [==============================] - 0s 60us/sample - loss: 0.1922

...

Epoch 97/100
1000/1000 [==============================] - 0s 59us/sample - loss: 0.0180
Epoch 98/100
1000/1000 [==============================] - 0s 53us/sample - loss: 0.0188
Epoch 99/100
1000/1000 [==============================] - 0s 54us/sample - loss: 0.0161
Epoch 100/100
1000/1000 [==============================] - 0s 55us/sample - loss: 0.0147

Once trainning is over, we can predict and display the output for each input:

# Compute the output
y_predicted = model.predict(x_data)

# Display the result
plt.scatter(x_data[::1], y_data[::1])
plt.plot(x_data, y_predicted, 'r', linewidth=4)
plt.grid()
plt.show()

Here is the result:

## Source code

You can try this example online on Google Colaboratory