In order to use AR and MA models the data have to be “well behaved.” Formally, the data need to be “stationary.”
Then X is “mean stationary” if the expected value of X at a particular time does not depend upon the particular time period in which it is observed. Thus, the unconditional expectation of X is not a function of the time period t:
\[ E(Xt)=E(X1)=E(X2)= \cdot \cdot \cdot =E(X)= \mu \] Likewise, X is said to be “variance stationary” if its variance is not a function of time, so that: \[ Var(Xt)=Var(X1)=Var(X2)= \cdot \cdot \cdot =Var(X)= \sigma \] Finally, X is “covariance stationary” if the covariance of X with its own lagged values depends only upon the length of the lag, but not on the specific time period nor on the direction of the lag. Symbolically, for a lag-length of one,
\[Cov(X_t, X_{t+1})=Cov(X_{t−1},X_t) \] An example of a time-series that is not covariance stationary is one where there is seasonality, with the size of the seasonality fluctuating over time.
Show whether the purely random process
\[ X_t = β_0 + e_t \]
with
\[ e_t ∼ iidN (0, 1)\]
is mean stationary, variance stationary, and covariance stationary
\[ E(X_t) = E( β_0 + e_t) = E( β_0) + E(e_t) = β_0 \] ### Variance \[ Var(X_t) = Var( β_0 + e_t) = Var(e_t) = \sigma^2 \] # AR(1) Models
\[ X_t = \beta X_{t-1} + e_t \]
If we were to estimate this model, we’d regress X on itself (lagged one period). This is why the model is called an “autoregressive model with lag one” or “AR(1)” for short. An autoregression is a regression of a variable on itself.
library(fpp3)## -- Attaching packages -------------------------------------------- fpp3 0.4.0 --## v tibble 3.1.2 v tsibble 1.0.1
## v dplyr 1.0.7 v tsibbledata 0.3.0
## v tidyr 1.1.3 v feasts 0.2.2
## v lubridate 1.7.10 v fable 0.3.1
## v ggplot2 3.3.5## -- Conflicts ------------------------------------------------- fpp3_conflicts --
## x lubridate::date() masks base::date()
## x dplyr::filter() masks stats::filter()
## x tsibble::intersect() masks base::intersect()
## x tsibble::interval() masks lubridate::interval()
## x dplyr::lag() masks stats::lag()
## x tsibble::setdiff() masks base::setdiff()
## x tsibble::union() masks base::union()AR models explicitly have lagged dependent variables (LDVs). This implies that, even if the errors are iid (independent and identically distributed) and serially uncorrelated, OLS estimates of the parameters will be biased.
\[ X_t = \beta X_{t-1} + e_t \]
\[\hat{\beta}_{OLS}= \frac{Cov(X_t,X_{t-1})}{Var(X_{t-1})}= \frac{\sum X_tX_{t-1}}{\sum X_{t-1}^2}=\frac{\sum(\beta X_{t-1} +e_t)X_{t-1}}{\sum X_{t-1}^2}\] \[\hat{\beta}_{OLS}= \frac{\sum \beta X_{t-1}^2}{\sum X_{t-1}^2}+ \frac{\sum e_tX_{t-1}}{\sum X_{t-1}^2}\]
Thus we can see that the OLS estimate $ _{OLS}$ is equal to the true value of \(\beta\) plus some bias. Fortunately this bias shrinks in larger samples (that is, the estimate is said to be “consistent.”)
If the errors are autocorrelated then the problem is worse. OLS estimates are biased and inconsistent. That is, the problem of bias doesn’t go away even in infinitely large samples.
If a time-series X is known to be, or is well represented by, an AR(1) process, then we might ask: How does X respond to shocks? Does it dampen over time, or is the effect of the shock persistent?
In general, an \(AR(1)\) process can be expressed as:
\[ X_t = \beta X_{t-1} +e_t = \beta^t X_0 + \sum_{1=1}^{t} \beta^{t-i}e_i \] The impulse response \(k\) periods after the shock is \[IRF(k) = \beta^k \] As we move farther away from \(X_0\), the effect of \(X_0\) becomes diminished since it is multiplied by \(\beta^t\). All of the shocks have an effect, but their impact becomes negligible the farther back in time they occur.
In general, a process is said to be \(AR(p)\) if
\[ X_t = β_1X_{t−1} + β_2X_{t−2} + \cdot \cdot \cdot + β_pX_{t−p} + e_t\] Usually, economic theory is silent on the number of lags to include. The matter is usually an econometric one: AR models with more lags can accommodate richer dynamics.
Paul Samuelson’s multiplier-accelerator model is an example of \(AR(2)\) process from economic theory. Beginning with the GDP accounting identity for a closed economy with no governmental expenditure,
\[Y_t = C_t + I_t\]
Samuelson adds the classic Keynesian consumption function with autonomous consumption \((β_0)\) and marginal propensity to consume \((β_1)\),
\[ C_t = β_0 + β_1Y_{t−1}\] and an equation that models investment as depending on the amount of growth in consumption from last period to today.
\[ I_t = β_2 (C_t − C_{t−1}) + e_t \] These three equation imply an \(AR(2)\) model:
\[ Y_t = β_0 + β_1(1 + β_2)Y_{t−1} − β_1β_2Y_{t−2} + e_t\] \[ Y_t = α_0 + α_1Y_{t−1} + α_2Y_{t−2} + e_t \]
with the α’s properly defined. Samuelson’s model accommodates different kinds of dynamics: dampening, oscillating, etc. . . depending on the estimated parameters.
Let’s practice estimating AR(p) models using the dataset ARexamples.dta. The dataset consists of a three variables (X, Y and Z) and a time variable.
arima X, ar(1/4) nocons \[ X_t = β_1X_{t−1} + β_2X_{t−2} + β_3X_{t−3} + β_4X_{t−4} + e_t \]
arima X, ar(1 2 4) nocons \[ X_t = β_1X_{t−1} + β_2X_{t−2} + β_4X_{t−4} + e_t\]
arima X, ar(2 4) nocons \[ X_t = β_2X_{t−2} + β_4X_{t−4} + e_t \]
\(X_t = β_1X_{t−1} + β_2X_{t−2} + β_3X_{t−3} + β_4X_{t−4} + e_t\) arima X, ar(1/4) nocons
\(X_t = β_1X_{t−1} + β_2X_{t−2} + β_3X_{t−3} + e_t\) arima X, ar(1/3) nocons
\(X_t = β_1X_{t−1} + β_4X_{t−4} + e_t\) arima X, ar(1 4) nocons
Using the ARexamples.dta dataset, graph the last 100 observations of \(X\) over time. Using all of the observations, estimate the \(AR(1)\) model, \[X_t = β_1X_{t−1} + e_t\] Verify that the coefficient is approximately: \(β^1 ≈ 0.50\).
Using the ARexamples.dta dataset, graph the last 100 observations of Y over time. Using all of the observations, estimate the AR(2) model, \[Y_t = β_1Y_{t−1} + β_2Y_{t−2} + e_t\] Verify that the coefficients are approximately: \(βˆ1 ≈ 0.70\) and \(βˆ2 ≈ 0.20\).
Using the ARexamples.dta dataset, graph the last 100 observations of Z over time. Using all of the observations, estimate the AR(3) model, \[Z_t = β_1Z_{t−1} + β_2Z_{t−2} + β_3Z_{t−3} + e_t\] Verify that the coefficients are approximately: \(βˆ1 ≈ 0.60\), \(βˆ2 ≈ 0.20\), and \(βˆ3 ≈ 0.10\).
Using the ARexamples.dta dataset, estimate the AR(3) model, \[Z_t = β_1Z_{t−1} + β_3Z_{t−3} + e_t\]. Notice that this is a restricted model, where the coefficient on the second lag is set to zero. Verify that the estimated coefficients are approximately: \(βˆ1 ≈ 0.70\), and \(βˆ3 ≈ 0.20\)
Calculate by hand the IRFs out to five periods for the following AR models: (a) \[X_t = 0.5X_{t−1} + e_t\] \[ X_1 = 0.5X_{0} + e_1 = 0.5 \cdot 0 + 1 = 1 \] \[ X_2 = 0.5X_{1} + e_2 = 0.5 \cdot 1 + 0 = 0.5 \] \[ X_3 = 0.5X_{2} + e_3 = 0.5 \cdot 0.5 + 0 = 0.25 \] \[ X_4 = 0.5X_{3} + e_4 = 0.5 \cdot 0.25 + 0 = 0.125\] \[ X_5 = 0.5X_{4} + e_5 = 0.5 \cdot 0.125 + 0 = 0.0625 \]
\[X_t = −0.5X_{t−1} + e_t\] \[X_1 = −0.5X_{0} + e_1 = −0.5 \cdot 0 + 1 = 0 + 1 = 1 \] \[X_2 = −0.5X_{1} + e_2 = −0.5 \cdot 1 + 0 = -0.5\] \[X_3 = −0.5X_{2} + e_3 = −0.5 \cdot -0.5 + 0 = 0.25\] \[X_4 = −0.5X_{3} + e_4 = −0.5 \cdot 0.25 + 0 = -0.125\] \[X_5 = −0.5X_{4} + e_5 = −0.5 \cdot -0.125 + 0 = 0.0625\]
\[X_t = 0.5X_{t−1} − 0.10X_{t−2} + e_t\] \[X_1 = 0.5X_{0} − 0.10X_{-1} + e_1 =0.5 \cdot 0 -0.10 \cdot 0 + 1 = 1 \] \[X_2 = 0.5X_{1} − 0.10X_{0} + e_2 = 0.5 \cdot 1 -0.10 \cdot 0 + 0 = 0.5 \] \[X_3 = 0.5X_{2} − 0.10X_{1} + e_3 = 0.5 \cdot 0.5 -0.10 \cdot 1 + 0 = 0.15\] \[X_4 = 0.5X_{3} − 0.10X_{2} + e_4 = 0.5 \cdot 0.15 -0.10 \cdot 0.5 + 0 = 0.025\] \[X_5 = 0.5X_{4} − 0.10X_{3} + e_5 = 0.5 \cdot 0.025 -0.10 \cdot 0.15 + 0 = -0.0025 \]
\[X_t = 0.10 + 0.5X_{t−1} − 0.20X_{t−2} + e_t\] \[X_1 = 0.10 + 0.5X_{0} − 0.20X_{-1} + e_1 = 0.10 + 0.5 \cdot 0 -0.20 \cdot 0 + 1 = 1.10\] \[X_2 = 0.10 + 0.5X_{1} − 0.20X_{0} + e_2 = 0.10 + 0.5 \cdot 1.10 -0.20 \cdot 0 + 0 = 0.65\] \[X_3 = 0.10 + 0.5X_{2} − 0.20X_{1} + e_3 = 0.10 + 0.5 \cdot 0.65 -0.20 \cdot 1.10 + 0 = 0.205 \] \[X_4 = 0.10 + 0.5X_{3} − 0.20X_{2} + e_4 = 0.10 + 0.5 \cdot 0.205 -0.20 \cdot 0.65 + 0 = 0.0725\] \[X_5 = 0.10 + 0.5X_{4} − 0.20X_{3} + e_5 = 0.10 + 0.5 \cdot 0.0725 -0.20 \cdot 0.205 + 0 = 0.09525 \]
\[X_t = X_{t−1} + e_t\] \[X_1 = X_{0} + e_1 = 0 + 1 = 1\] \[X_2 = X_{1} + e_2 = 1 + 0 = 1\] \[X_3 = X_{2} + e_3 = 1 + 0 = 1\] \[X_4 = X_{3} + e_4 = 1 + 0 = 1\] \[X_5 = X_{4} + e_5 = 1 + 0 = 1\]
IT DOESN’T APEAR TO BE STATIONARY. IT KEEPS GROWING
\[ X_1 = 5 \] \[ X_2 = -10 \] \[ X_3 = 15 \] \[ X_4 = 6.75 \] \[ X_5 = 11.5625 \] \[ X_6 = 0.75X_{t−5}+0.50X_{t−4}+0.10X_{t−3}+e_6= 0.75(11.5625)+0.50(6.75)+0.10(15)+0= 13.54688\] \[X_7 = 0.75X_{t−6}+0.50X_{t−5}+0.10X_{t−4}+e_7= 0.75(13.54688)+0.50(11.5625)+0.10(6.75)+0= 16.61641\] \[X_8 = 0.75X_{t−7}+0.50X_{t−6}+0.10X_{t−5}+e_8= 0.75(13.54688)+0.50(11.5625)+0.10(6.75)+0= 20.39199\] \[X_9 = 0.75X_{t−6}+0.50X_{t−5}+0.10X_{t−4}+e_7= 0.75(20.39)+0.50(16.61)+0.10(13.54)+0= 24.95688\] \[X_10 = 0.75X_{t−6}+0.50X_{t−5}+0.10X_{t−4}+e_7= 0.75(24.95)+0.50(20.39)+0.10(16.61)+0= 30.5753\]
\[X_t = 0.6171596X_{t-1}+0.1851499X_{t-2}+0.1050058X_{t-3}\]
\[X_{3001} = 0.6171596X_{3000}+0.1851499X_{2999}+0.1050058X_{2998} = 0.6362278\]
\[X_{3002} = 0.6171596X_{3001}+0.1851499X_{3000}+0.1050058X_{2999} = 0.6167556\]
\[X_{3003} = 0.6171596X_{3002}+0.1851499X_{3001}+0.1050058X_{3000} = 0.5597\] \[X_{3004} = 0.6171596X_{3003}+0.1851499X_{3002}+0.1050058X_{3001} = 0.5264241\]