set.seed(seed = 42)
library(psych)
  1. Law of Large Numbers (LLN)

The LLN is a fundemental rule in probability and statistics. It states that if you repeat an experiment a large number of times, take result of each experiment, and then take the average of the results, the average should be very close to the expected value (true population mean). The more times the experiment is repeated, the closer the result would be to the expected value.It describes the behaviour of the sample mean of a random var as sample size increases. The larger the sample size, the better it represent the population as a whole.

There are 2 versions of the LLN: Strong LLN (SLLN), and Weak LLN (WLLN). The SLNN - states that as the sample size increases, the sample mean will converge almost surely (probability of 1) to the expected value of a random var. As the sample size grows to infinity, the sample mean converges almost surely to the true population mean. WLLN- states that as the sample size increases, the sample mean will converge in probability to the expected value of the random var. That means, that for large enough samples, the probability that the sample mean is far from the expected value is close to zero. The more data you collect, the sample mean becomes a better estimate of the true population mean.

  1. Central Limit Theorem

The CLT is also a fundamental concept in probability and statistics. It states that the distribution of sample means drawn from a population will tend towards having a normal, bell shaped, distribution, as the sample size increases, regardless of the shape of the original population distribution. CLT addresses the issues when the population is not normally distributed.It has few rules follow: the mean of the sampling distribution is equal to the true population mean (mu), the standard deviation of the sample means (called the standard error) equals to the population standard deviation (sigma), divided by the square root of the sample size (n), and the sample size should be above 30 (rule of thumb) for the theorem to work well. CLT allows researchers to use normal distribution even when the population data is skewed, uniform, or unknown. It also enables statistical tests like confidence intervals, hypothesis testing, and t-tests.

  1. Similarities between CLT and LLN
  1. Both CLT and LLN assume independent trials and identically distributed random variables.

b.Both analyze the behavior of the sample mean (xbar) over a growing sample size (n > 30 or n goes to infinity)

c.Both help estimate unknown population parameters using observable sample statistics.

Differences between CLT and LLN

  1. The focus of LLN is about the center of the sampling distribution, where the sample mean lands. The focus of the CLT is the shape of the sampling distribution, how the sample means scatter around the center.

  2. LLN converges to a single value(population mean), while CLT converges to a probability distribution (normal distribution).

  3. The main use of LLN is to prove that sample estimates are reliable and consistent, while CLT enables hypothesis testing and confidence interval construction.

  1. I picked the Chi-square Distribution

Chi-square distribution is a continuous probability distribution created by summing the squares of independent standard normal random variables. This kind of distribution is encountered very often in statistics, especially in the estimation of variance and in hypothesis testing. It takes a single parameter, k (or v), representing the degrees of freedom, which is the number of squared variables being added.

It has few key properties. The shape of this distribution is right-skewed (positively skewed). As k increases, the distribution becomes more symmetrical and looks more like a normal distribution.The mean of this distribution is k, and the variance is 2k. The values of this distribution is always zero or positive (x>=0).

5A. Apply the CLT on the sample mean of chi-square distribution

#?rchisq
# create chi-square distribution
mychisq <- rchisq(n=10000,df = 3)

#mychisq[1:16]

# Mean and SD of population
mu <- mean(mychisq)
mu
## [1] 3.030427
sigma <-sd(mychisq)
sigma
## [1] 2.493191
# plot histogram of original distribution
hist(mychisq,main="Histogram of Chi-Square distribution")

# create empty matrix
z <- matrix(data = rep(x=0,
                       times = 10000),
            nrow = 10000,
            ncol = 1
            )

z[1:16]
##  [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
describe(z)
##    vars     n mean sd median trimmed mad min max range skew kurtosis se
## X1    1 10000    0  0      0       0   0   0   0     0  NaN      NaN  0
# take a sample of 100 observations from the chi-square distribution, find its mean, and populate matrix with the means

for (i in 1:10000){
  z[i,] <- mean(sample(x=mychisq,size=100,replace=TRUE
                       )
                )
}
z[1:16]
##  [1] 3.343727 3.382647 2.994124 3.082784 2.807453 2.975541 2.833017 3.073498
##  [9] 3.267591 2.880994 2.801553 3.004579 3.126758 2.917830 3.357801 3.529126
describe(z)
##    vars     n mean   sd median trimmed  mad  min  max range skew kurtosis se
## X1    1 10000 3.03 0.25   3.03    3.03 0.25 2.08 3.97  1.89 0.16     0.04  0
hist(z, xlab = "", main = "Histogram of Sample Mean (n = 100)")

# comparing sample means and original mean
mean(z)
## [1] 3.031156
mu
## [1] 3.030427
# comparing sample sd and original sd 
sd(z)
## [1] 0.2502305
sigma/sqrt(100)
## [1] 0.2493191

The original chi-squared distribution is right-skewed, but the distribution of the 10000 sample means is approximately normal and bell shaped. The means of the of the original distribution and the sample means is almost identical. The standard deviation is also almost identical to the standard error.

5B. Lets apply the CLT on the sample 25th percentile

# 25th percentile of the original chi-square distribution
percentile25 <- quantile(mychisq,probs=0.25)
percentile25
##      25% 
## 1.214219
#create empty matrix
z_25 <- matrix(data= rep(x=0,times=10000),nrow=10000,ncol=1)

#z_25[1:16]

# take a sample of 100 observations from the chi-square distribution, find the 25th percentile of each sample, and populate matrix with the percentiles

for (i in 1:10000){
  z_25[i,] <- quantile(sample(x=mychisq,size=100,replace=TRUE
                       ),probs = 0.25
                )
}
z_25[1:16]
##  [1] 0.9762898 1.0188976 1.2588955 1.3833882 1.1791772 1.4503353 1.2909793
##  [8] 1.0758138 1.2454605 1.1521961 1.0654656 1.3527360 1.1576579 1.1997344
## [15] 1.1320316 1.5667987
hist(z_25,xlab="25th percentile",main="Sampling Distribution of the 25th Percentile (n = 100)")

#compare mean to samples 25th percentile
mean(z_25)
## [1] 1.234449
percentile25
##      25% 
## 1.214219

The original population’s 25th percentile is close to the center of the 10000 sample 25th percentile. Their sampling distribution is much more symmetric, and bell shaped, than the original chi-square distribution.

The CLT holds in my simulations in both cases. The sampling distribution becomes approximately normal, even though the original chi-square distribution is right skewed. The mean of the sample means is close to the original population mean, and the standard deviation of the sample means follows the standard error formula.