1) Newton’s Method on \[f(x)=3x^4-4x^3)12\] on range [-10,10]

For the question below, we need to find x_0 when we set the function equal to zero \[f(x)=\frac{1}{4}x^4-\frac{1}{3}x^3=0\]

Looping through our interval, we will notice the lowest x_0 is at x_0=1 \[\frac{1}{4}1^4-\frac{1}{3}1^3=\frac{-1}{12}\]

We can write the formula of newton’s method \[x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}\]

However, we are trying to find the minimum of f(x). For this we need, we need to use the newton’s method for optimization

\[x_{n+1}=x_n-\frac{f'(x_n)}{f''(x_n)}\]

and transform it to fit our problem \[f'(x)=x^3-x^2\] \[f''(x)=3x^2-2x\] \[x_{n+1}=x-\frac{x^3-x^2}{3x^2-2x}\]

For the x_0 value, we can use a initial value of 1 and loop through the method to find the lowest value produced. We seem to

#initializing the functions for f(x) and its newton method
fx<-function(x){x-((x^3-x^2)/(3*x^2-2*x))}
f<-function(x){(1/4)*x^4-(1/3)*x^3}
fxprev<-c(1)

#For the loop of 10 iterations
for (n in 1:10){
  #At n=1, lets find the x0 root and solve for f(x0)
  if (n==1){
    f0<-f(fxprev[1])
    print(sprintf("At x0== %f,f(x)= %f",fxprev[1],f0))
  }else{
    #For all nex titeration, take the last value of xn and solve f(x)
    fxprev[n]<-fx(fxprev[n-1])
    f0<-f(fxprev[n])
    print(sprintf("At xn== %f,f(xn)= %f",fxprev[n],f0))
  }
  
}
## [1] "At x0== 1.000000,f(x)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"
## [1] "At xn== 1.000000,f(xn)= -0.083333"

2) Explore Optimize in r for the following function

It appears the smallest value in f(x)=(1/3)x^4-(1/3) is at -0.083

#We can optimize our xn function to find the step with the smallest minimum value possible
xmin<-optimize(f,c(-10,10))
xmin
## $minimum
## [1] 0.9999986
## 
## $objective
## [1] -0.08333333

3) Find the minimum of \[f(x,y)=(x-11)^2+100(y-x^2)^2\] at domain -10<-x, y<=10

Following the optimization for the newton method for a multivariate function, we can solve for the x0 and x1 with the Hessian matrix.

\[f(x,y)=(x-11)^2+100(y-x^2)^2=100x^4-200x^2y+100y^2+x^2-22x+121\] \[\frac{df}{dx}=400x^3-400xy+2x-22\] \[\frac{d^2f}{dx^2}=1200x^2-400y+2\] \[\frac{df}{dy}=-200x^2+200y\] \[\frac{d^2f}{dy^2}=200\] \[\frac{d^2f}{dydx}=-400x\] \[\frac{d^2f}{dxdy}=-400x\] \[H=|\binom{1200x^2+2-400y -400x}{200 -400x}\]

We can compute the hessian matrix with r and not manually go through all those steps. One issue with optimization is the amount of step it take to optimize manually with the Hessian matrix. Although the steps and the given domain does make this process more straightforward, the amount of variables and computation would be faster if via programmaing!

#Using the pracma's hessian function
library(pracma)
## Warning: package 'pracma' was built under R version 4.1.3
f<- function(n) {
    x <- n[1]; y <- n[2]
    return((x-1)^2+100*(y-x^2)^2)
}

#Let's use the initial value at (0,0)
#Looking at the resulting matrix, the function f(x,y) will converge
hessian(f, c(0,0))
##          [,1] [,2]
## [1,] 2.000003    0
## [2,] 0.000000  200

4) Using optimr for the following question

library(optimr)
## Warning: package 'optimr' was built under R version 4.1.3
#Selecting the current method as on the r document it handles optimizations
optimr(c(-10,10),f,method="L-BFGS-B")
## $par
## [1] 0.9997999 0.9995999
## 
## $value
## [1] 4.002673e-08
## 
## $counts
## function gradient 
##       71       71 
## 
## $convergence
## [1] 0
## 
## $message
## [1] "CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH"