VARIANCE REDUCTION - ANTITHETIC VARIATES

In simplest terms (for just a single pair of values), \(X_1\) is the first variable, and \(X_2\) is the second, let \(X_1 = x\) and \(X_2 = 1 - x\)

The mean will always be \((\frac{X_1 + X_2}{2}) = \frac{1}{2}\), then the variance (difference in mean, squared, then summed), will equal zero. In other words, \(X_1\) and \(X_2\) have a perfect negative correlation, being that if \(X_1\) increases, \(X_2\) will decrease. If the covariance of two variables is \(\leq 0\), the variance is reduced.

In graphical terms, a plot of 10 random variables uniformly distributed, and the \(1-x\) values are below:

In order to estimate \(\mu\) for a function \(h\), we can generate random variables for \(X_1\), and $1 - $ the same distribution to get \(X_2\). Applying to the function, we would have \(X_1 = h(U_1,...,U_i)\), and \(X_2 = h(1-U_1,...1-U_i)\). Since both sets have the same distribution, they both have the same expected value (\(E(X)\)). Because of the negative correlation (\(1 - x\)), both vectors \(X_1\) and \(X_2\) are negatively correlated.

The function below will compare calculating the expected value with standard methods, and then using antithetic variables:

antithetic <- function(fx, n=5000){
  U <- runif(n)
  Ex <- mean(fx(U))
  
  U2 <- runif((1/2) * n)  # smaller sample required
  
  X_1 <- fx(U2)
  X_2 <- fx(1-U2)
  
  theta_hat <- mean((X_1 + X_2) / 2)
  
  reg_var <- var(fx(U))
  anti_var <- .25 * (var(X_1) + var(X_2) + (2 * cov(X_1, X_2)))
  
  print(paste("Expected value for F(x) is", round(Ex, 5), ", with variance of", round(reg_var, 6)))

  print(paste("Expected value for F(x) using antithetic variables is", 
              round(theta_hat, 5), ", with variance of", round(anti_var, 6)))
  
}

Now we’ll test the antithetic function with a simple \(F(x)\):

func <- function(x){sqrt((1-x^2))}  # establish a function

antithetic(func)
## [1] "Expected value for F(x) is 0.78259 , with variance of 0.050125"
## [1] "Expected value for F(x) using antithetic variables is 0.78741 , with variance of 0.006473"

The two exepected values are fairly close, but the second one generated with antithetic variables is closer to the true value, and the variance is much smaller.

In simulation, reducing the variance will increase the precision (and smaller confidence intervals), and will require less iterations, reducing runtime, processing, and costs.