asymptotic_v1.knit

class: center, middle
# Asymptotic Properties of OLS
### Dr. Francisco J. Cabrera-Hernández
#### Econometría
#### Maestría en Economía
Primavera 2025
#####CIDE Santa Fe, Ciudad de México.

---
## Introduction

We now investigate OLS asymptotic properties.

Applies to the CEF model and the projection model.

`$$Y = X'\beta + e$$`
$$ \beta = (E[XX'])^{-1} E[XY] $$

We maintain assumptions:

1. `$(Yi,Xi), i=1,...,n,$` are i.i.d.
2. `$E[Y^2] < \infty$`
3. `$E||X||^2 < \infty$`
4. `$Qxx= E[XX']$` is positive definite.

---
## Consistency

**Definition 1**. A sequence of random vectors `$Z_n \in \mathbb{R}^k$`  converge in probability to `$Z$` as `$n \to \infty$`

`$$lim_{n \to \infty} \mathbb{P}[||Z_n-Z||<\delta]=1$$`
Z is the probability limit of `$Z_n$`

$$
Z_n \to_p Z \quad \text{as} \quad n \to \infty
$$
For a random vector, this holds if and only if each element in the vector converges in probability to its limit.

---
## Consistency

**Definition 2**. `$Z_n$` are random vectors with distributions `$F_n(u) = \mathbb{P}[Z_n \le u]$`

`$u \in \mathbb{R}^k$` is afixed vector.

For all `$u$` at which  `$F_n(u) = \mathbb{P}[Z_n \le u]$` is continuous `$F_n(u) \to F(u)$` as `$n \to \infty$`.

We say that `$Z_n \to_d Z$` or **converges in distribution** to Z as `$n \to \infty$`

`$Z$` and `$F(U)$` are called the asymptotic distributions, large sample distribution, or the limit distribution of `$Z_n$`.

[Some nice code](https://github.com/fcabrerahz/EconometricsME/blob/main/Code/10_distribution_convergence.R)

---
## Consistency

**Weak Law of Large Numbers (WLLN)**

If `$Y_i \in \mathbb{R}^k$` are i.i.d. and `$E||Y|| < \infty$` then as `$n \to \infty$`

`$$\bar{Y} = {1 \over n} \sum_{i=1}^{n} Y_i \to_p E[Y]$$`
The sample mean `$\bar{Y}$` in probability to  the true population expectation `$\mu$`.

An estimator `$\hat{\theta}$` is **consistent** if `$\hat{\theta} \to \theta$` as `$n \to \infty$`.

---
## Central Limit Theorem

If `$Y_i \in \mathbb{R}^k$` are i.i.d. and `$E||Y||^2 < \infty$` then as `$n \to \infty$`

`$$\sqrt{n}(\bar{Y}-\mu) \to_d N(0,V)$$`
Where `$\mu = E[Y]$` and `$V= E[(Y-\mu)(Y-\mu)']$`

The central limit theorem shows that distribution sample mean is approximately normal in large samples. It allows for singular `$V$`.

---
## Summary

- Being i.i.d. does not imply normality.

- A sample can be i.i.d. from a non-normal distribution, such as Uniform, Exponential, Bernoulli, Poisson.

- The Law of Large Numbers (LLN) and the Central Limit Theorem (CLT), hold under the i.i.d. assumption, even when  distribution not normal.

- But the limit in CLT is normal.

- Normality is typically required only for small-sample exact inference, such as when using t-statistics.

- In large samples, asymptotic normality of estimators emerges, even from non-normal data, as long as the observations are i.i.d. and regularity conditions are met (e.g. consistency and identification!)

---
## Continuos Mapping Theorem (CMT)

Makes use of convergence in probability and convergence in distribution.

Let `$Z_n \in \mathbb{R}^k$` and `$g(u)$`: `$\mathbb{R}^k \to \mathbb{R}^q$`:

If `$Z_n \to_p c$` as `$n \to \infty$` and `$g(u)$` is continuous at `$c$` then `$g(Z_n) \to_p g(c)$` as `$n \to \infty$`.

If a sequence of random vectors `$Z_n$` converge in probability to a constant vector `$c$`, and you apply a continuous function `$g$` to each `$Z_n$`, then the transformed sequence `$g(Z_n)$` will also converge in probability to `$g(c)$`.

Needed for deriving the asymptotic distributions of estimators after transformation (e.g., log, inverse, square root, etc.).

---

## Illustration of the Continuous Mapping Theorem (CMT)

Let: 
`$$Z_n \sim \mathcal{N}(1, 1/n),$$`
`$$Z_n \xrightarrow{p} 1 \quad \text{as } n \to \infty.$$`
A continuous function:

$$
g(u) = \log(u),
$$

* which is continuous at `$u = 1$`.

Then, by the CMT:

$$
\log(Z_n) \xrightarrow{p} \log(1) = 0.
$$

[Beautiful Code](https://github.com/fcabrerahz/EconometricsME/blob/main/Code/11_CMT.R)

---
## Continuos Mapping Theorem (CMT)

If `$Z_n \to_d Z$` as `$n \to \infty$` and `$g$`: `$\mathbb{R}^m \to \mathbb{R}^k$` has the set of discontinuity points `$D_g$` such that `$\mathbb{P}[Z\in D_g]=0$` then `$g(Z_n) \to_d g(Z)$` as `$n \to \infty$`.

Differentiable functions of asymptotically normal random estimators are asymptotically normal.

This version of the Continuous Mapping Theorem applies when we have **convergence in distribution** rather than in probability.

Allows applying a `$g(\cdot)$` — even with discontinuities — to a converging sequence of random variables, **as long as the limiting random `$Z$` does not land on the discontinuity set `$D_g$` with positive probability**.

---
## Consistency of Least Square Estimators

OLS estimator can be written as a continuous function of a set of sample moments.

The WLLN shows that sample moments converge in probability to population moments.

The CMT states that continuous functions preserve convergence in probability.

OLS is a function of sample moments `$\hat{Q}_{XX}^{-1}$` `$\hat{Q}_{XY}$`:

`$$\hat{\beta} = ({1 \over n} \sum_{i=1}^n X_iX_i')^{-1}({1 \over n} \sum_{i=1}^n Y_iX_i) = \hat{Q}^{-1}_{XX}\hat{Q}_{XY}$$`
---
## Consistency of Least Square Estimators

Using **WLLN** these sample moments converge in probability to their population expectations.

As `$(Y_i,X_i)$` are i.i.d, any function of these is i.i.d. including  `$X_iX_i'$` and  `$Y_iX_i$`.

With finite expectations, as `$n \to \infty$`:

`$$\hat{Q}_{XX} = {1 \over n} \sum_{i=1}^n X_iX_i' \to_p \mathbb{E}[XX'] = Q_{XX}$$`

`$$\hat{Q}_{XY} = {1 \over n} \sum_{i=1}^n X_iY_i' \to_p \mathbb{E}[XY] = Q_{XY}$$`

---
## Consistency of Least Square Estimators

By the **CMT** we are allowed to combine these above equations:

`$$\hat{\beta}=\hat{Q}^{-1}_{XX}\hat{Q}_{XY} \to_p Q_{XX}^{-1}Q_{XY}^{} = \beta$$`
OLS estimator converges in probability to the projection coefficient vector `$\beta$` as the sample size `$n$` gets large.

Because:

`$$\hat{\beta}=g(\hat{Q}_{XX},\hat{Q}_{XY})$$`

and `$g$` is a continuous function of `$Q_{XX}$` and `$Q_{XY}$` at all values of the arguments, such that `${Q}^{-1}_{XX}$` exists.

This justifies the use of the CMT.

---
## Consistency of Least Square Estimators

A different demonstration: 
`$$\hat{\beta} - \beta = \hat{Q}^{-1}_{XX}\hat{Q}_{Xe}$$`

Where: 
`$$\hat{Q}_{Xe}= {1 \over n} \sum_{i=1}^nX_ie_i$$`

WLLN imply: 
`$$\hat{Q}_{Xe} \to_p \mathbb{E}[Xe]=0$$`

`$$\hat{\beta} - \beta = \hat{Q}^{-1}_{XX}\hat{Q}_{Xe} \to_p \hat{Q_{XX}}^{-1}0=0$$`

This is: `$\hat{\beta} \to_p \beta$` as `$n \to \infty$`. Thus, `$\hat{\beta}$` is consistent for `$\beta$`.

---
##Estimadores insesgados `$(\hat\beta_1)$` n=1000