Ch2.3.1 Round-Off Error & Machine Epsilon

• Another characteristic of number storage is machine error.
• Most numbers \( x \) on number line have no machine representation.
• Machine error takes into account spacing between floating point numbers.

• As we know, floating point cannot represent every number between 0 and 1.79769313486232e+308.

mantissa <- rep(1,52)
powers <- seq(from = 1, to = 52, by = 1)
f <- sum(mantissa*(1/2)^(powers))
2^(1023)*(1 + f)

[1] 1.797693e+308

• Calculate smallest positive floating point number:

c <- 0
f <- 2^(-52)
2^(c - 1023)*(1 + f)

[1] 1.112537e-308

• The machine error, frequently just called machine \( \epsilon \), captures the spacing of numbers within the floating point space.
• Machine \( \epsilon \) at \( x \) actually depends on \( x \), and is written \( \epsilon_x \)

• Machine \( \epsilon \) is understood to be for \( x = 1 \), with \( \epsilon = \epsilon_1 \).

• Thus machine \( \epsilon \) is not the smallest positive number that can be represented in floating point format.
• Smallest number, from a few slides back:

[1] 1.112537e-308

• Rather \( \epsilon \) = gap between 1 and next greater number.

• Thus the smallest interval \( [a,b] \) on the binary number line starting at \( a = 1 \) would be

[a, b] = [1, 1 + epsilon]

• Machine \( \epsilon \) is defined as the smallest value such that

\[ 1 + \epsilon > 1 \]

• Mathematically, \( 1 + x > 1 \) for any \( x > 0 \).
• But with the number spacing within floating point, that is not necessarily the case on a computer.
  
• Thus machine \( \epsilon \) is the threshold value for representation.

• In R, we can find machine \( \epsilon \) as follow:

(eps <- .Machine$double.eps)

[1] 2.220446e-16

• Let's see more digits:

eps

[1] 2.220446e-16

print(1 + eps,digits = 20)

[1] 1.0000000000000002

• Let's see if

\[ 1 + 0.5\epsilon \ngtr 1 \]

print(1 + eps/2,digits = 20)

[1] 1

• It looks like R is rounding up on this example:

eps

[1] 2.220446e-16

print(1 + 0.6*eps,digits = 20)

[1] 1.0000000000000002

• In mathematics, addition is associative:

\[ a + (b + c) = (a + b) + c \]

• However, due to limitations on machine number representation, computer addition isn't always associative.

• As an example, see if you can explain these results:

print((1 + eps/2) + eps/2, digits = 20)

[1] 1

print(1 + (eps/2 + eps/2), digits = 20)

[1] 1.0000000000000002

• For general \( x \), machine epsilon is smallest \( \epsilon_x > 0 \) such that

\[ x + \epsilon_x > x \]

• For \( x = 8 \), we can infer here that \( \epsilon_8 > \epsilon \):

print(8 + eps, digits = 20)

[1] 8

• The library \( \textsf{pracma} \) contains a function, \( \textsf{eps} \), that provides the value of \( \epsilon_x \) for any given value of \( x \).

library(pracma)
eps(8)

[1] 1.776357e-15

(e <- eps(8))

[1] 1.776357e-15

print(8 + e, digits = 20)

[1] 8.0000000000000018

• The value of \( \epsilon_x \) is directly proportional to the value of \( x \).

c(eps(4), eps(8))

[1] 8.881784e-16 1.776357e-15

2*0.8882

[1] 1.7764

• Most values of \( x \) on the number line continuum cannot be represented precisely on a computer.
• This is due the tolerance limitations specified by \( \epsilon_x \) and/or the impossibility of storing an infinite number of decimal digits.

c(1/4,22/7,pi,sqrt(2))

[1] 0.250000 3.142857 3.141593 1.414214

• Let \( x \) = actual number
• Let \( x'= \mathrm{fl}(x) \) = floating point representation of \( x \).
• Then the round-off error \( E_R \) for a number \( x \) is given by
  

\[ E_R = |x - \mathrm{fl}(x)| \]