Question 1 (Shumway and Stoffer, Problem 3.8.

ARMA(1, 1) Verification of calculation

Given the ARMA(1, 1) process:

\[x_t = \phi x_{t-1} + \theta w_{t-1} + w_t,\]

where \(|\phi| < 1\), the autocovariance function \(\gamma(h)\) for \(h \geq 2\) satisfies:

\[\gamma(h) - \phi\gamma(h - 1) = 0,\]

and the initial conditions are given by:

\[\gamma(0) = \frac{\sigma^2_w(1 + 2\theta\phi + \theta^2)}{1 - \phi^2},\] \[\gamma(1) = \frac{\sigma^2_w(1 + \theta\phi)(\phi + \theta)}{1 - \phi^2}.\]

From these, the ACF \(\rho(h)\) for \(h \geq 1\) is:

\[\rho(h) = \frac{(1 + \theta\phi)(\phi + \theta)}{1 + 2\theta\phi + \theta^2}\phi^{h-1}.\]

ACF Forms for ARMA(1, 0) and ARMA(0, 1)

  • ARMA(1, 0) (AR(1)): The ACF is given by \(\rho(h) = \phi^h\).

  • ARMA(0, 1) (MA(1)): The ACF for lag 1 is \(\rho(1) = \frac{\theta}{1 + \theta^2}\), and for \(h > 1\), \(\rho(h) = 0\).

library(ggplot2)

# Parameters
phi <- 0.6
theta <- 0.9
lags <- 1:20  # Number of lags for ACF calculation

# ARMA(1,1)
acf_arma11 <- ((1 + theta * phi) * (phi + theta) / (1 + 2 * theta * phi + theta^2)) * phi^(lags - 1)

# ARMA(1,0) - equivalent to AR(1)
acf_arma10 <- phi^lags

# ARMA(0,1) - equivalent to MA(1)
acf_arma01 <- rep(0, length(lags))
acf_arma01[1] <- theta / (1 + theta^2)  # Non-zero only for lag 1

acf_data <- data.frame(
  Lag = rep(lags, 3),
  ACF = c(acf_arma11, acf_arma10, acf_arma01),
  Series = factor(rep(c("ARMA(1,1)", "ARMA(1,0)", "ARMA(0,1)"), each = length(lags)))
)

# Plot
ggplot(acf_data, aes(x = Lag, y = ACF, color = Series)) +
  geom_line() +
  geom_point() +
  theme_minimal() +
  ggtitle("ACF of ARMA(1,1), ARMA(1,0), and ARMA(0,1) Series") +
  xlab("Lag") +
  ylab("ACF")

Conclusion:

The behaviors of AR(1), MA(1), and ARMA(1, 1) processes through their ACFs. While AR(1) and MA(1) processes show clear, patterns (geometric decay and a sharp cutoff after lag 1, respectively), the ARMA(1, 1) process presents a more complex pattern due to the influence of both AR and MA components.

The challenge in distinguishing ARMA(1, 1) processes from AR(1) processes based solely on the ACF, as both exhibit geometrically decaying ACF patterns. This tells about the importance of also considering the partial autocorrelation function (PACF) for model identification, as the PACF can provide clearer difference between processes that have similar ACF patterns. Thus we shoudl use PACF too here.

Question 2 (Shumway and Stoffer, Problem 3.9.)

library(forecast)
## Registered S3 method overwritten by 'quantmod':
##   method            from
##   as.zoo.data.frame zoo
# Parameters
n <- 100
phi <- 0.6
theta <- 0.9

set.seed(123)  # For reproducibility
arma11_series <- arima.sim(model = list(ar = phi, ma = theta), n = n)
arma10_series <- arima.sim(model = list(ar = phi), n = n)
arma01_series <- arima.sim(model = list(ma = theta), n = n)

par(mfrow=c(2, 3))  

# ARMA(1,1)
Acf(arma11_series, main="Sample ACF for ARMA(1,1)")
Pacf(arma11_series, main="Sample PACF for ARMA(1,1)")

# ARMA(1,0)
Acf(arma10_series, main="Sample ACF for ARMA(1,0)")
Pacf(arma10_series, main="Sample PACF for ARMA(1,0)")

# ARMA(0,1)
Acf(arma01_series, main="Sample ACF for ARMA(0,1)")
Pacf(arma01_series, main="Sample PACF for ARMA(0,1)")

# Reset plot layout
par(mfrow=c(1, 1))

Expected Results according to the table

ARMA(1,1): The sample ACF should tail off, and the PACF should also tail off, due to the mixed AR and MA components.

ARMA(1,0) (AR(1)): The ACF is expected to tail off, reflecting the AR component’s influence. The PACF should cut off after lag 1, corresponding to the AR order.

ARMA(0,1) (MA(1)): The ACF should cut off after lag 1, indicating the order of the MA component. The PACF will tail off, reflecting the less direct influence of past values.

Comparison:

ARMA(1, 1): The ACF and the PACF both tails off.

AR(1): The ACF tails off, however the PACF does not cuts off.

MA(1): The ACF does not cut off at lag1, however the PACF tails off.

Conclusion

The reason why we could be getting this different behavior than expected behavior could be due to the sample size as well as the alues of phi and theta. We might want to change them to influence the series.

Question 3 Show that the value at lag 2 of the partial ACF of the MA(1) process

ANSWER IN SHEETS (PDF Attached)