Ex. 1 - Find the minimum of \(f(x,y) = x^2 + xy+ y^2 in (x,y) \in R^2\)

Solution:

We can extend the optimization procedure for univariate functions to multivariate functions using partial derivatives and relevant conditions.

The stationary conditions are \(\displaystyle \frac{\partial f}{\partial x}\) = \(2x + y = 0\) and \(\displaystyle \frac{\partial f}{\partial y}\) = \(x + 2y = 0\)

From the second condition we get either \(x = -2y\) or \(y = −x/2\)

Substituting \(x = -2y\) into the first condition gives us

\(2(-2y) + y = 0\)

\(-4y + y = 0\)

\(-3y = 0\)

or \(y = 0\)

Similarly, substituting \(y = −x/2\) into the first condition gives us \(2x + −x/2 = 0\)

\(3x/2 = 0\)

\(x = 0\)

So \(x = 0\) and \(y = 0\).

We need to define the Hessian martix from these second partial derivatives to determine whether the value corresponds to a maximum or a minimum.

\[\begin{bmatrix}fxx & fxy\\ fyx & fyy \end{bmatrix}\]

=

\[\begin{bmatrix}2 & 1\\ 1 & 2 \end{bmatrix}\]

\(fxx = 2x + y = 2\)

\(fxy = 2x + y = 1\)

\(fyx = x + 2y = 1\)

\(fyy = x + 2y = 2\)

H is positive definite which means that the point (0,0) is a minimum.

Ex.2 - For \(f(x) = x^4\) in R, it has the global minimum at x = 0. Find its new minimum if a constraint \(x^2 >= 1\) is added.

Solution:

\(f(x) = x^4 and g(x) = x^2\)

\(f(x) + \mu[g(x)]^2\) = \(x^4 + \mu[x^2]^2\) = \(x^4 + \mu[x^4]\)

\(f'(x)\) = \(4x^3 + 4\mux^3\) = 0 = \(4x^3(1 + \mu) = 0\)

X can be either 1 or -1. The new minimum is x = 1.

Ex. 3 - Use a Lagrange multiplier to solve the optimization problem \(minf(x,y) = x^2 + 2xy + y^2\), subject to \(y = x^2 − 2\)

Solution:

\(L = f(x,y) + \lambda h(x,y)\) \(h(x,y) = x^2 − 2 − y\)

\(L = x^2 + 2xy + y2 + \lambda (x^2 − 2 − y)\)

\(\frac{\partial l}{\partial x} = 2x + 2y + 2\lambda x = 0\) \(\frac{\partial l}{\partial y} = 2x + 2y - \lambda = 0\) \(\frac{\partial l}{\partial y} = x^2 - 2 - y = 0\)

Similarly, \(2x + 2y -\lambda = 0\) which gives \(\lambda = 2(x+y)\)

Plugging \(\lambda = 2(x+y)\) into the first condition gives us

\(2x + 2y+ 2(2(x+y))x = 0\)

\(2x + 2y + 4x^2 + 4xy = 0\)

\(2(x + y + 2x^2 + 2xy) = 0 which gives y = (−2x^2 - x) / (2x + 1)\)

Substitute \(y = −2x^2 −x / (2x + 1)\) into the third condition gives us

\(x^2 − 2 − (-2x^2 − x)/(2x + 1) = 0\) when simplified gives \(x = 1 or x = - 2\)

Plugging x = 1 into the third condition gives us y = -1 and plugging in x = -2 gives us y = 2. We get two points (1, -1) and (-2, 2).

The optimality for \(min f(x,y) = x^2 + 2xy + y^2\) is (1, -1) with fmin = 0 and (-2, 2) with fmin=0