0

set.seed(128)

1 Law of Large Numbers (LLN) describes the statistical property of mean value, for a random variable X, we can only observe the realization of the random variable in real life, such as rolling a dice, we can get the sample data by rolling the data for several times, then we can estimate the mean of X through the sample mean, LLN tells us that when the number of sample increases to infinity, the sample mean will converge to the population mean, i.e., the expectation value of X.

2 Central limit theorem (CLT) describes the behavior of the sample mean of a group of sample obtained from an identical probability distribution. Suppose we can get several sample from a distribution, then we can calculate the mean of each sample, the distribution of the mean would approxiamete to a normal distribution when the number of sample increase to infinity, the mean and the variance of the distribution are determined by the population mean and variance as well as n.

3 Both of the LLN and CLT describe the statistical property related to mean of sample. Both of the theorems are conducted when n increases to infinity. LLN describes the sample mean converges the population mean when the number of observations goes to infinity, while CLT describes the distribution of sample mean, not for a single group of sample, but the behavior of infinity group of sample.

4 We pick Student t distribution. It is a continuous probability distribution with a parameter about degrees of freedom. The distribution is symmetry around 0.

myt <- rt(10000,df=3)
myt[1:16]            
##  [1]  0.588809876  2.065316265 -1.888904053  0.005046143  1.593622708
##  [6]  0.592943574 -1.005645860  0.869799628 -0.638859773 -0.401896369
## [11]  1.433680794 -9.339177305  0.241684198  1.037160661 -0.343944912
## [16]  1.653851389
mu <- mean(myt)      
mu
## [1] -0.02079518
sigma <- sd(myt)     
sigma
## [1] 1.825781
library("psych")
describe(myt)        
##    vars     n  mean   sd median trimmed  mad   min   max range  skew kurtosis
## X1    1 10000 -0.02 1.83  -0.01   -0.01 1.13 -34.2 22.49 56.68 -2.04    53.24
##      se
## X1 0.02
### Plot out the distribution  
hist(x    = myt, 
     main = "Histogram of t Distribution",
     xlab = ""
     )

5

# fill in the matrix with 0s
z <- matrix(data = rep(x     = 0, 
                       times = 10000
                       ), 
            nrow = 10000, 
            ncol = 1)

z[1:16]
##  [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
describe(z)
##    vars     n mean sd median trimmed mad min max range skew kurtosis se
## X1    1 10000    0  0      0       0   0   0   0     0  NaN      NaN  0
for (i in 1:10000){ 
    z[i,] <- mean(sample( x        = myt, 
                          size     = 100,    
                          replace  = TRUE    
                        )
                )
  }

z[1:16]
##  [1] -0.093458828 -0.198098500  0.155881067 -0.241643531  0.150797416
##  [6] -0.221640062 -0.348617917 -0.071726518 -0.172282090 -0.001374101
## [11]  0.128548894 -0.102690024 -0.125057820  0.275398373 -0.036964487
## [16] -0.226498348
describe(z)
##    vars     n  mean   sd median trimmed  mad   min  max range  skew kurtosis se
## X1    1 10000 -0.02 0.18  -0.02   -0.02 0.17 -0.95 0.69  1.64 -0.22     0.51  0
hist(z, xlab = "", main = "Histogram of Sample Mean (n = 100)")

z <- matrix(data = rep(x     = 0, 
                       times = 40000
                       ), 
            nrow = 10000, 
            ncol = 4)
n <- c(2, 6, 30, 1000) # let n be the sample size


for (j in 1:4){        # indexing columns of matrix, where each column will represent different sample size
 
   for (i in 1:10000){  # indexes the rows of matrix
  
       z[i,j] <- mean(sample( x       = myt,  # compute mean and assign
                              size    = n[j], 
                              replace = TRUE
                            )
                      )
    }
}
colnames(z) <- c("Sample size=2", "Sample size=6", "Sample size=30", "Sample size=1000") 
summary(z) 
##  Sample size=2        Sample size=6       Sample size=30     Sample size=1000  
##  Min.   :-17.075802   Min.   :-8.172275   Min.   :-1.82598   Min.   :-0.26630  
##  1st Qu.: -0.580602   1st Qu.:-0.399001   1st Qu.:-0.21657   1st Qu.:-0.05759  
##  Median :  0.005895   Median : 0.005267   Median :-0.01644   Median :-0.01939  
##  Mean   : -0.003783   Mean   :-0.012187   Mean   :-0.02456   Mean   :-0.01980  
##  3rd Qu.:  0.610164   3rd Qu.: 0.392075   3rd Qu.: 0.18705   3rd Qu.: 0.01855  
##  Max.   : 13.431731   Max.   : 4.482109   Max.   : 1.60880   Max.   : 0.20725
par(mfrow=c(3,2))

length(myt)
## [1] 10000
hist(x    = myt, 
     main = "Histogram of t Distribution, N=10,000",
     xlab = ""
     )


for (k in 1:4){
    hist(x    = z[,k],    
         main = "Histogram of Sample Mean of t Distribution", 
         xlim = c(-12, 12), 
         xlab = paste0("Sample Size ", n[k], " (Column ", k, " from matrix)") 
         )
  }  

z[1:16] 
##  [1]  0.89668406  0.28095111  0.64489329 -0.51907234  0.96063187  0.27741641
##  [7] -0.69251530  0.03012412 -0.51481915  0.37349566 -1.43860664 -0.25820790
## [13] -0.62708450 -0.30070045 -1.98991847 -8.05868298
mu      
## [1] -0.02079518
colMeans(z)            
##    Sample size=2    Sample size=6   Sample size=30 Sample size=1000 
##     -0.003782978     -0.012186675     -0.024561579     -0.019796273
sigma   
## [1] 1.825781
apply(X = z,
      MARGIN = 2, 
      FUN = sd)    
##    Sample size=2    Sample size=6   Sample size=30 Sample size=1000 
##       1.35035693       0.76032311       0.33098580       0.05750532
apply(X = z,
      MARGIN = 2,
      FUN = sd) * c(sqrt(2), 
                    sqrt(6), 
                    sqrt(30), 
                    sqrt(1000)
                    ) 
##    Sample size=2    Sample size=6   Sample size=30 Sample size=1000 
##         1.909693         1.862404         1.812884         1.818478
sigma
## [1] 1.825781

The CLT holds according to our experiment. From the histogram, we can find as the number of observations increases, the distribution of sample mean is more centered, and the standard derivation is getting smaller.