FriconiX
Free collection of beautiful vector icons for your web pages.

Let's consider the function $$f: \mathbb{R^2} \mapsto \mathbb{R}$$ given by:

$$f(x,y) = (x-2)^2 + 2(y-3)^2$$

Here is a 3D surface plot of this function:

## Optimization

We want to apply the gradient descent algorithm to find the minima. Steps are given by the following formula:

$$X_{n+1} = X_n - \alpha \nabla f(X_n)$$

Let's start by calculating the gradient of $$f(x,y)$$:

$$\nabla f(X) = \begin{pmatrix} \frac{df}{dx} \\ \frac{df}{dy} \end{pmatrix} = \begin{pmatrix} 2x-4 \\ 4y-12 \end{pmatrix}$$

The coordinates will be updated according to:

$$x_{n+1} = x_{n} - \alpha(2x_{n} - 4)$$ $$y_{n+1} = y_{n} - \alpha(4y_{n} - 12)$$

In the following example, we arbitrary placed the starting point at coordinates $$X_0=(30,20)$$. Here is an illustration of the convergence to $$X_200=(2,3)$$ after 200 iterations: