Variance Reduction

STRATIFIED SAMPLING:

Stratified sampling reduced variance by dividing the sample space by space to strata and then estimating the yield in each stratum. Stratificaton is most effective when the means of the strata are widely disperded. Stratification is effective for integrands that are monotone functions.

Below is an example from the Rizzo text book which represents a more than 98% reduction in variance.

M <- 10000 #number of replicates
 k <- 10 #number of strata
 r <- M / k #replicates per stratum
 N <- 50 #number of times to repeat the estimation
 T2 <- numeric(k)
 estimates <- matrix(0, N, 2)
 g <- function(x) {
   exp(-x - log(1+x^2)) * (x > 0) * (x < 1) 
 } 
 for (i in 1:N) {
  estimates[i, 1] <- mean(g(runif(M)))
  for (j in 1:k)
   T2[j] <- mean(g(runif(M/k, (j-1)/k, j/k)))
  estimates[i, 2] <- mean(T2)
}

 estimates
##            [,1]      [,2]
##  [1,] 0.5192331 0.5245617
##  [2,] 0.5260135 0.5247552
##  [3,] 0.5237585 0.5248763
##  [4,] 0.5235200 0.5250666
##  [5,] 0.5263455 0.5246224
##  [6,] 0.5271371 0.5250201
##  [7,] 0.5223080 0.5247277
##  [8,] 0.5246409 0.5245928
##  [9,] 0.5209376 0.5252851
## [10,] 0.5248925 0.5249405
## [11,] 0.5288811 0.5242819
## [12,] 0.5255642 0.5251893
## [13,] 0.5269780 0.5250086
## [14,] 0.5216509 0.5245829
## [15,] 0.5249779 0.5245405
## [16,] 0.5214423 0.5245560
## [17,] 0.5258964 0.5251164
## [18,] 0.5181427 0.5249557
## [19,] 0.5240705 0.5246774
## [20,] 0.5264203 0.5251544
## [21,] 0.5219942 0.5250020
## [22,] 0.5274886 0.5249143
## [23,] 0.5239167 0.5249297
## [24,] 0.5269598 0.5248087
## [25,] 0.5256139 0.5249333
## [26,] 0.5209839 0.5246705
## [27,] 0.5281579 0.5241916
## [28,] 0.5218649 0.5245477
## [29,] 0.5255202 0.5248561
## [30,] 0.5247472 0.5248974
## [31,] 0.5230380 0.5246755
## [32,] 0.5273611 0.5250998
## [33,] 0.5210379 0.5243538
## [34,] 0.5250622 0.5248902
## [35,] 0.5236462 0.5251442
## [36,] 0.5209504 0.5245459
## [37,] 0.5250036 0.5247309
## [38,] 0.5219275 0.5248548
## [39,] 0.5262322 0.5251849
## [40,] 0.5277489 0.5251797
## [41,] 0.5232700 0.5249306
## [42,] 0.5242462 0.5245654
## [43,] 0.5266421 0.5247999
## [44,] 0.5231808 0.5246308
## [45,] 0.5240861 0.5249560
## [46,] 0.5234178 0.5246084
## [47,] 0.5204671 0.5250664
## [48,] 0.5243466 0.5248203
## [49,] 0.5263923 0.5245337
## [50,] 0.5265444 0.5246729
#The result of this simulation produces the following estimates.

apply(estimates, 2, mean) 
## [1] 0.5242932 0.5248101
apply(estimates, 2, var) 
## [1] 6.002877e-06 6.293945e-08