The secant method for finding roots of nonlinear equations is a common and popular variation of the Newton-Raphson method that has been used for several millennia before the invention of Newton-Raphson (Papakonstantinou, J. as cited in Wikipedia). The secant method is an iterative method that takes two initial guesses of the root, compared to just one guess required for Newton-Raphson. Due to the extra computations as a result of requiring two approximations, the secant method often converges more slowly compared to Newton-Raphson; however, it is usually more stable. The secant method has another advantage over NR as it does not need the derivative of the function in question to be known or assumed to be easily calculated. The secant method uses secant lines (hence the need for two initial starting values) to find the root of a function while the Newton-Raphson method approximates the root with a tangent line.

The Secant Method

The derivation of the secant method is relatively straightforward and will not be discussed here. Wikipedia features a clear and well-written section on the derivation of the method. Given two initial values, \(x_0\) and \(x_1\), the first approximation of the root, \(x_2\), is defined as:

\[ x_2 = x_1 - f(x_1) \bigg/ \frac{f(x_1) - f(x_0)}{x_1 - x_0} \]

The process is repeated using each iterated value to draw a new secant line that moves closer and closer to the function’s root. This iterative process can thus be generalized as:

\[ x_{n+1} = x_n - f(x_n) \bigg/ \frac{f(x_n) - f(x_{n-1})}{x_n - x_{n-1}} \]

Example of the Secant Method

Consider the function \(cos(x)\) bounded by the interval [-1, 4]. Approximate the root of the function located within the interval using the secant method. Use \(x_0 = 0\) and \(x_1 = 3\) as starting values. As before, it is a good idea to plot the function to see how it behaves.

cos.func <- function(x) {
  cos(x)
}

curve(expr = cos.func, col='blue', lwd=2, lty=2, xlim=c(-1,4))
abline(h=0)
abline(v=0)

The root of the function within the interval appears to be somewhere around \(x = 1.50\). Plot a secant line using the starting values given above. Solve the equation for the given points to find the associated \(y\) value of the starting points.

x0 <- 0
x1 <- 3
a <- cos(x0)
b <- cos(x1)

Draw the secant line using the determined points.

curve(expr = cos.func, col='blue', lwd=2, lty=2, xlim=c(-1,4))
abline(h=0)
abline(v=0)
points(x0, a, cex=1.25, pch=21, bg='blue', col='blue')
points(x1, b, cex=1.25, pch=21, bg='blue', col='blue')
segments(x0, a, x1, b, col='red', lwd = 2)

The secant line crosses the x-axis close to where the root of the function exists. Solve for the new point using the secant method general equation above starting at \(x_2\):

\[ x_{2} = x_1 - f(x_1) \bigg/ \frac{f(x_1) - f(x_0)}{x_1 - x_0} = 3 - cos(3) \bigg/ \frac{cos(3) - cos(0)}{3 - 0} = 1.507543 \]

The new value \(x_2\) then replaces \(x_1\) and \(x_1\) replaces \(x_0\). The function is evaluated at each new point and a new secant line is drawn.

x1 <- 3
x2 <- 1.507543
a <- cos(x1)
b <- cos(x2)

curve(expr = cos.func, col='blue', lwd=2, lty=2, xlim=c(-1,4))
abline(h=0)
abline(v=0)
points(x1, a, cex=1.25, pch=21, bg='blue', col='blue')
points(x2, b, cex=1.25, pch=21, bg='blue', col='blue')
segments(x1, a, x2, b, col='red', lwd = 2)

The next iteration replaces the new values for the values in the previous iteration.

\[ x_{3} = x_2 - f(x_2) \bigg/ \frac{f(x_2) - f(x_1)}{x_2 - x_1} = 1.507543 - cos(1.507543) \bigg/ \frac{cos(1.507543) - cos(3)}{1.507543 - 3} = 1.597117 \]

Calculate the new secant line points.

x2 <- 1.507543
x3 <- 1.597117
a <- cos(x2)
b <- cos(x3)

curve(expr = cos.func, col='blue', lwd=2, lty=2, xlim=c(-1,4))
abline(h=0)
abline(v=0)
points(x2, a, cex=1.25, pch=21, bg='blue')
points(x3, b, cex=1.25, pch=21, bg='blue')
segments(x2, a, x3, b, col='red', lwd = 2)

Notice the secant line quickly converges around the root of the function. As the iterations continue, the secant line will move even closer to the root until it reaches a desired level of tolerance.

The Secant Method in R

The NLRoot package provides several methods for finding the root of an equation.

library(NLRoot)

Consider the equation examined in a previous post on the Newton-Raphson method, \(x^3 - 2x - 5\).

func <- function(x) {
  x^3 - 2 * x - 5
}

Plot the function.

curve(func, xlim=c(-3,3), col='blue', lwd=1.5, lty=2)
abline(h=0)
abline(v=0)

The root appears to be located around \(x = 2\), so we will use starting values \(x_0 = 1\) and \(x_1 = 3\). The SMfzero function in the NLRoot package performs the secant method of root-finding.

SMfzero(func, 1, 3)
## [1] 2.094551
## [1] -3.771081e-08
## [1] "finding root is successful"

The function determined the root exists at \(2.094551\), the same result that was found with the Newton-Raphson method in a previous example.

We can also take it one step further and write our own function to implement the secant method in R. The function takes three inputs, the function in question, and the two starting values.

secant.method <- function(f, x0, x1, tol = 1e-9, n = 500) {
  for (i in 1:n) {
    x2 <- x1 - f(x1) / ((f(x1) - f(x0)) / (x1 - x0)) # Calculate the new x value
    if (abs(x2 - x1) < tol) { # If the difference between the new value and the previous value is small enough, end iteration and output root.
      return(x2)
    }
    # If the root was not determined in the previous iteration, update the values and proceed to the next iteration.
    x0 <- x1
    x1 <- x2 
  }
}

secant.method(func, 1, 3)
## [1] 2.094551

The function reports the same root as the SMfzero and the Newton-Raphson method.

One More Example of the Secant Method

Consider the function \(f(x) = e^{2x} - x - 6\)

func2 <- function(x) {
  exp(2 * x) - x - 6
}

Plotting the equation reveals how the function behaves.

curve(func2, col = 'blue', lty = 2, lwd = 2, xlim=c(-5,5), ylim=c(-5,5), ylab='f(x)')
abline(h=0)
abline(v=0)

The root appears to exist in the interval [0, 2], which we can use as starting values.

SMfzero(func2, 0, 2)
## [1] 0.97087
## [1] 5.514042e-09
## [1] "finding root is successful"
secant.method(func2, 0, 2)
## [1] 0.97087

The SMfzero function and our function agree on the root of the equation.

Summary

The secant method is a popular and powerful algorithm for finding the root of an equation. The method has several advantages over the Newton-Raphson approach as the secant method does not require the function derivative and is less prone to erratic swings between iterations. However, the secant method struggles compared to Newton-Raphson regarding iterations required. Most implementations of root-finding algorithms in the most commonly used statistical packages use a combination of Newton-Raphson and the secant method to find the root of an equation.

References

Secant method (2016). In Wikipedia. Retrieved from https://en.wikipedia.org/wiki/Secant_method

The Newton-Raphson method. Retrieved from https://www.math.ubc.ca/~anstee/math104/104newtonmethod.pdf

THE SECANT METHOD. Retrieved from http://homepage.divms.uiowa.edu/~atkinson/ftp/ENA_Materials/Overheads/sec_3-3.pdf