Finite Difference Method

Numerical differentiation is a method of approximating the derivative of a function ff at particular value xx. Often, particularly in physics and engineering, a function may be too complicated to merit the work necessary to find the exact derivative, or the function itself is unknown, and all that is available are some points xx and the function evaluated at those points. Numerical differentiation, of which finite differences is just one approach, allows one to avoid these complications by approximating the derivative. The central difference approximation is more accurate than forward and backward differences and should be used whenever possible.

Differencing Digitized Data Series

Consider the following set of data points:

x f(x) 0.0 0.00000 0.2 0.74140 0.4 1.37180

The function is unknown; however, we would still like to approximate the function’s derivative at the values of x.

x <- c(0.0, 0.2, 0.4)
fx <- c(0.00000, 0.74140, 1.3718)

finite.differences <- function(x, y) {
  if (length(x) != length(y)) {
    stop('x and y vectors must have equal length')
  }
  
n <- length(x)
# Initialize a vector of length n to enter the derivative approximations
fdx <- vector(length = n)
}

Forward differencing function

finite.differences <- function(x, y) {
if (length(x) != length(y)) {
stop('x and y vectors must have equal length')
}
n <- length(x)
# Initialize a vector of length n to enter the derivative approximations
fdx <- vector(length = n)
# Iterate through the values using the forward differencing method
for (i in 2:n) {
fdx[i-1] <- (y[i-1] - y[i]) / (x[i-1] - x[i])
}
# For the last value, since we are unable to perform the forward differencing method 
# as only the first n values are known, we use the backward differencing approach
# instead. Note this will essentially give the same value as the last iteration 
# in the forward differencing method, but it is used as an approximation as we 
# don't have any more information
fdx[n] <- (y[n] - y[n - 1]) / (x[n] - x[n - 1])
return(fdx)
}

Solution from forward differencing function

Using the function, perform each differencing method on the x values and the evaluated points f(x).

finite <- finite.differences(x, fx)
finite
## [1] 3.707 3.152 3.152

Data from a function

Let’s assume the data from earlier was taken from the function e^x - 2x^2 + 3x - 1

f <- function(x) {
  return(exp(x) - 2 * x^2 + 3 * x - 1)
}

Central differencing function

central.difference <- function(f, x) {
  steps <- c(0.1, 0.01, 0.001, 0.0001, 0.00001, 0.000001, 0.0000001)
  n <- length(steps)
  fdx <- vector(length = n)
  for (h in 1:n) {
    fdx[h] <- (f(x + 0.5 * steps[h]) - f(x - 0.5 * steps[h])) / steps[h]
  }
  return(fdx)
}

Solution by Central differencing method

for (i in x) {
  print(central.difference(f, i))
}
## [1] 4.000417 4.000004 4.000000 4.000000 4.000000 4.000000 4.000000
## [1] 3.421912 3.421408 3.421403 3.421403 3.421403 3.421403 3.421403
## [1] 2.892446 2.891831 2.891825 2.891825 2.891825 2.891825 2.891825

Example Function and Its Derivative

The example function is e^x - 2x^2 + 3x - 1 The derivative of the function is e^x - 4x + 3

Plot both the function and its derivative to get a visualization of where the function’s derivative is at the values of x.

fdx <- function(x) {
  return(exp(x) - 4 * x + 3)
}

library(ggplot2)
## Warning: package 'ggplot2' was built under R version 3.4.2
ggplot(data = data.frame(x = 0), mapping = aes(x = x)) + 
  stat_function(fun = f, size = 1.25, alpha = 0.75) +
  stat_function(fun = fdx, size = 1.25, color = 'blue', alpha = 0.75) +
  xlim(-3,3)

Exact vs Approximation

calculated derivative’s values for each x and compare them with our previously approximated values.

actual <- vector(length = length(x))
central.approx <- c(4.00000, 3.421403, 2.891825)
for (i in 1:length(x)) {
  actual[i] <- fdx(x[i])
}
approx.df <- data.frame(cbind(actual, central.approx, actual - central.approx, finite, actual - finite))
colnames(approx.df) <- c('Actual Values', 'Central Difference Approx', 'Central Differences Error', 'Finite Differences', 'Finite Differences Error')
approx.df
##   Actual Values Central Difference Approx Central Differences Error
## 1      4.000000                  4.000000              0.000000e+00
## 2      3.421403                  3.421403             -2.418398e-07
## 3      2.891825                  2.891825             -3.023587e-07
##   Finite Differences Finite Differences Error
## 1              3.707                0.2930000
## 2              3.152                0.2694028
## 3              3.152               -0.2601753