ACF vs PACF in Time Series Analysis

Author

Arvind Sharma

1 Core Concepts

1.1 What is Autocorrelation?

Autocorrelation measures how correlated a time series is with delayed versions of itself at different time lags. Think of it as asking: “How similar is today’s value to yesterday’s, last week’s, or last month’s?”

1.2 ACF (Autocorrelation Function)

Definition: Measures the linear relationship between a time series and its lagged values, capturing both direct and indirect relationships.

Key Characteristics:

  • Shows the total correlation at each lag

  • Includes indirect effects transmitted through intermediate lags

  • Gradually decays for stationary AR processes

  • Cuts off sharply for MA processes

Mathematical Formula: \[\rho(k) = \frac{\text{Cov}(Y_t, Y_{t-k})}{\sqrt{\text{Var}(Y_t) \times \text{Var}(Y_{t-k})}}\]

1.3 PACF (Partial Autocorrelation Function)

Definition: Measures the direct linear relationship between a time series and its lagged values, after removing the influence of all intermediate lags.

Key Characteristics: - Shows only the direct correlation at each lag

  • Controls for the effects of all shorter lags

  • Cuts off sharply for AR processes after lag p

  • Gradually decays for MA processes

The Intuitive Difference: ACF vs PACF

Think of autocorrelation like gossip spreading through a network:

  • ACF (Autocorrelation Function): Measures total influence - both direct gossip AND gossip that traveled through other people first
  • PACF (Partial Autocorrelation Function): Measures direct influence only - what’s the correlation if we ignore all the middlemen?

2 Identification Rules (The Golden Rules)

2.1 For AR(p) Processes

  • ACF: Gradually decays (exponential or sinusoidal pattern)

  • PACF: Cuts off sharply after lag p

2.2 For MA(q) Processes

  • ACF: Cuts off sharply after lag q

  • PACF: Gradually decays (exponential or sinusoidal pattern)

2.3 For ARMA(p,q) Processes

  • Both ACF and PACF: Gradually decay after their respective cutoff points

3 Setup and Data Preparation

# Load required packages
library(forecast)
library(tseries)
library(ggplot2)
library(gridExtra)
library(knitr)
# library(DT)

# Set seed for reproducibility
set.seed(123)

3.1 AR Process Examples

3.1.1 AR(2) Process Simulation

Let’s simulate and analyze an AR(2) process: \(y_t = 0.6y_{t-1} - 0.3y_{t-2} + \epsilon_t\)

# Simulate AR(2) process
ar2_series <- arima.sim(model = list(ar = c(0.6, -0.3)), n = 500)

# Create comprehensive plots
par(mfrow = c(2, 2))
plot(ar2_series, main = "AR(2) Time Series", ylab = "Value", xlab = "Time")
acf(ar2_series, main = "ACF of AR(2)", lag.max = 20)
pacf(ar2_series, main = "PACF of AR(2)", lag.max = 20)

# Add theoretical ACF for comparison
ar2_theoretical_acf <- ARMAacf(ar = c(0.6, -0.3), lag.max = 20)
plot(0:20, ar2_theoretical_acf, type = "h", main = "Theoretical ACF of AR(2)", 
     xlab = "Lag", ylab = "ACF")

Observation: Notice how the PACF cuts off after lag 2, while the ACF gradually decays. This is the signature pattern of an AR(2) process.

3.1.2 AR(1) Process for Comparison

# Simulate AR(1) process
ar1_series <- arima.sim(model = list(ar = c(0.7)), n = 500)

par(mfrow = c(2, 2))
plot(ar1_series, main = "AR(1) Time Series")
acf(ar1_series, main = "ACF of AR(1)")
pacf(ar1_series, main = "PACF of AR(1)")

3.2 MA Process Examples

3.2.1 MA(2) Process Simulation

Let’s simulate and analyze an MA(2) process: \(y_t = \epsilon_t + 0.7\epsilon_{t-1} + 0.4\epsilon_{t-2}\)

# Simulate MA(2) process
ma2_series <- arima.sim(model = list(ma = c(0.7, 0.4)), n = 500)

par(mfrow = c(2, 2))
plot(ma2_series, main = "MA(2) Time Series")
acf(ma2_series, main = "ACF of MA(2)")
pacf(ma2_series, main = "PACF of MA(2)")

# Theoretical ACF for MA(2)
ma2_theoretical_acf <- ARMAacf(ma = c(0.7, 0.4), lag.max = 20)
plot(0:20, ma2_theoretical_acf, type = "h", main = "Theoretical ACF of MA(2)",
     xlab = "Lag", ylab = "ACF")

Observation: The ACF cuts off after lag 2, while the PACF gradually decays - the opposite pattern from AR processes.

3.3 ARMA Process Example

3.3.1 ARMA(1,1) Process

# Simulate ARMA(1,1) process
arma11_series <- arima.sim(model = list(ar = c(0.5), ma = c(0.3)), n = 500)

par(mfrow = c(2, 2))
plot(arma11_series, main = "ARMA(1,1) Time Series")
acf(arma11_series, main = "ACF of ARMA(1,1)")
pacf(arma11_series, main = "PACF of ARMA(1,1)")

3.4 Model Identification Function

# Function for systematic model identification
identify_model <- function(ts_data, series_name = "Time Series") {
  cat("=== Model Identification for", series_name, "===\n\n")
  
  # 1. Stationarity test
  adf_result <- adf.test(ts_data)
  cat("ADF Test p-value:", round(adf_result$p.value, 4), "\n")
  cat("Series is", ifelse(adf_result$p.value < 0.05, "stationary", "non-stationary"), "\n\n")
  
  # 2. Generate plots
  par(mfrow = c(2, 2))
  plot(ts_data, main = paste(series_name, "- Time Plot"))
  acf(ts_data, main = paste(series_name, "- ACF"))
  pacf(ts_data, main = paste(series_name, "- PACF"))
  
  # 3. Model suggestions
  cat("Model Identification Guidelines:\n")
  cat("- If PACF cuts off at lag p and ACF decays: AR(p)\n")
  cat("- If ACF cuts off at lag q and PACF decays: MA(q)\n")
  cat("- If both decay gradually: ARMA(p,q)\n\n")
}

3.4.1 Practical Example: Applying the Function

# Test the identification function on our simulated series
identify_model(ar2_series, "AR(2) Series")
=== Model Identification for AR(2) Series ===
ADF Test p-value: 0.01 
Series is stationary 
Model Identification Guidelines:
- If PACF cuts off at lag p and ACF decays: AR(p)
- If ACF cuts off at lag q and PACF decays: MA(q)
- If both decay gradually: ARMA(p,q)

4 Advanced Diagnostics

4.1 Information Criteria Comparison

AR(2) corresponds to our earlier simulated series. Let’s compare different models using AIC and BIC. We expect AR(2) to perform best.

# Compare different models for AR(2) series
models_to_test <- list(
  "AR(1)" = c(1, 0, 0),
  "AR(2)" = c(2, 0, 0),
  "AR(3)" = c(3, 0, 0),
  "MA(1)" = c(0, 0, 1),
  "MA(2)" = c(0, 0, 2),
  "ARMA(1,1)" = c(1, 0, 1),
  "ARMA(2,1)" = c(2, 0, 1)
)

# Calculate AIC and BIC for each model
results <- data.frame(
  Model = names(models_to_test),
  AIC = NA,
  BIC = NA
)

for (i in 1:length(models_to_test)) {
  fit <- arima(ar2_series, order = models_to_test[[i]])
  results$AIC[i] <- AIC(fit)
  results$BIC[i] <- BIC(fit)
}

# Display results
kable(results, digits = 2, caption = "Model Comparison for AR(2) Series")
Model Comparison for AR(2) Series
Model AIC BIC
AR(1) 1446.05 1458.69
AR(2) 1397.93 1414.79
AR(3) 1399.81 1420.88
MA(1) 1406.87 1419.51
MA(2) 1408.16 1425.02
ARMA(1,1) 1408.39 1425.25
ARMA(2,1) 1399.68 1420.76 
# Find best models
best_aic <- results$Model[which.min(results$AIC)]
best_bic <- results$Model[which.min(results$BIC)]

cat("\nBest model by AIC:", best_aic)

Best model by AIC: AR(2)
cat("\nBest model by BIC:", best_bic)

Best model by BIC: AR(2)

4.2 Residual Analysis

Expect the residuals of the best-fitting model to behave like white noise.

# Fit the best model and check residuals
best_model <- arima(ar2_series, order = c(2, 0, 0))

# Residual diagnostics
par(mfrow = c(2, 2))
plot(residuals(best_model), main = "Residuals")
acf(residuals(best_model), main = "ACF of Residuals")
pacf(residuals(best_model), main = "PACF of Residuals")
qqnorm(residuals(best_model))
qqline(residuals(best_model))

# Ljung-Box test
ljung_test <- Box.test(residuals(best_model), lag = 10, type = "Ljung-Box")
cat("\nLjung-Box test p-value:", round(ljung_test$p.value, 4))

Ljung-Box test p-value: 0.9051
cat("\nResiduals are", ifelse(ljung_test$p.value > 0.05, "white noise (good!)", "not white noise"))

Residuals are white noise (good!)

5 Summary Table: Pattern Recognition

ACF/PACF Pattern Recognition Guide
Process ACF_Pattern PACF_Pattern Key_Insight
AR(p) Gradual decay Cuts off after lag p PACF tells AR order
MA(q) Cuts off after lag q Gradual decay ACF tells MA order
ARMA(p,q) Gradual decay after lag q Gradual decay after lag p Both patterns present
Random Walk Very slow decay Large spike at lag 1 Non-stationary signal
White Noise No significant lags No significant lags No autocorrelation

6 Real-world Application Tips

6.1 1. Dealing with Non-stationarity

ADF test and differencing:

# Simulate a random walk (non-stationary)
rw_series <- cumsum(rnorm(200))

par(mfrow = c(2, 2))
plot(rw_series, main = "Random Walk (Non-stationary)")
acf(rw_series, main = "ACF of Random Walk")

# Difference to make stationary
diff_rw <- diff(rw_series)
plot(diff_rw, main = "Differenced Series (Stationary)")
acf(diff_rw, main = "ACF of Differenced Series")

cat("ADF test on random walk:", round(adf.test(rw_series)$p.value, 4))
ADF test on random walk: 0.9082
cat("\nADF test on differenced series:", round(adf.test(diff_rw)$p.value, 4))

ADF test on differenced series: 0.01

6.2 2. Seasonal Patterns

Seasonal patterns can complicate ACF/PACF interpretation:

# Create series with seasonal pattern
t <- 1:100
seasonal_series <- 2 * sin(2 * pi * t / 12) + rnorm(100, 0, 0.5)

par(mfrow = c(2, 2))
plot(seasonal_series, main = "Seasonal Time Series")
acf(seasonal_series, lag.max = 36, main = "ACF (Extended for Seasonality)")
pacf(seasonal_series, lag.max = 36, main = "PACF (Extended for Seasonality)")

7 Key Takeaways

  1. ACF captures total correlation (direct + indirect effects)
  2. PACF isolates direct correlation (controlling for intermediate lags)
  3. Use both together for comprehensive model identification
  4. Always check stationarity before interpreting patterns
  5. Validate with residual analysis after model fitting
  6. Consider information criteria for model selection
  7. Look for seasonal patterns in extended ACF/PACF plots

7.1 Exercise for Practice

Try identifying the following simulated series:

# Mystery series - can you identify the process?
mystery_series <- arima.sim(model = list(ar = c(0.8), ma = c(-0.4)), n = 300)

par(mfrow = c(1, 3))
plot(mystery_series, main = "Mystery Series - What process is this?")
acf(mystery_series, main = "ACF of Mystery Series")
pacf(mystery_series, main = "PACF of Mystery Series")

cat("Hint: Look at the patterns in both ACF and PACF!")
Hint: Look at the patterns in both ACF and PACF!
cat("\nAnswer: This is an ARMA(1,1) process with φ=0.8 and θ=-0.4")

Answer: This is an ARMA(1,1) process with φ=0.8 and θ=-0.4

8 Appendix

8.1 Understanding the Spike at Lag 0

The spike at lag 0 is ALWAYS 1.0 - this is not random! Here’s why:

Lag 0 = Correlation with itself

  • At lag 0, we’re asking: “How correlated is today’s value with… today’s value?”

  • The answer is always perfectly correlated = 1.0

Why is this important?

  • It’s your reference point - all other correlations are relative to this

  • If lag 0 weren’t 1.0, something would be seriously wrong with your data!

  • It helps you gauge the magnitude of other correlations

8.2 Intuitive Explanation: Why PACF “Regresses Out” Intermediate Lags

To identify the order of an Autoregressive (AR) process, we must distinguish between cumulative correlation and direct impact. This note uses two analogies to explain why the Autocorrelation Function (ACF) lingers while the Partial Autocorrelation Function (PACF) “shuts off.”

Case Study: Imagine a sequence of events where each day’s misery is a direct result of the previous day’s misfortune.

The Series (\(X_t\))

  • \(X_t\) (Day 1): No GPA, no Job, “Totally Screwed.”
  • \(X_{t-1}\) (Day 2): You missed a major exam.
  • \(X_{t-2}\) (Day 3): You received a massive hospital bill.
  • \(X_{t-3}\) (Day 4): You were in a traffic accident.
  • \(X_{t-4}\) (Day 5): You “Felt Low” (the initial shock).

8.2.1 The ACF: The “Snowball” Effect / Domino Chain

The ACF at lag \(k\) measures the total correlation between \(X_t\) and \(X_{t-k}\).

  • Logic: It tracks the “memory” of the series. Even if “Felt Low” (\(X_{t-4}\)) isn’t the direct reason you are unemployed today, it initiated the chain reaction (Accident \(\rightarrow\) Bill \(\rightarrow\) Exam \(\rightarrow\) Screwed).
  • Result: The ACF at lag 4 will be high. It captures the cumulative linear dependence moving through the entire domino chain.
  • “Your bad mood 4 days ago still correlates with today because the whole disaster chain propagated forward.”

Day 5        Day 4        Day 3        Day 2        Day 1
(Felt Low) → (Accident) → (Bill) → (Exam) → (Screwed)
     \____________________________________________/
                  TOTAL ASSOCIATION

8.2.2 The PACF: The “Direct Laser”

The PACF at lag \(k\) measures the correlation between \(X_t\) and \(X_{t-k}\) after removing the influence of all intervening lags (\(1, 2, \dots, k-1\)).

  • Logic: It asks: “Given that I already know you missed your exam, have a hospital bill, and had an accident, does knowing you ‘Felt Low’ 5 days ago provide any NEW information about why you are screwed today?”
    • “After accounting for accident, bill, and exam…, did your Day-5 mood still independently wreck your Day-1 life?”

Day 5        Day 4        Day 3        Day 2        Day 1
(Felt Low) → [BLOCKED] → [BLOCKED] → [BLOCKED] → (Screwed)
     \____________________________________________/
                 DIRECT EFFECT ONLY

\[ Corr(Y_t​,Y_{t−4}​∣Y_{t−1}​,Y_{t−2}​,Y_{t−3}​) \]

  • Mathematical Intuition: In an \(AR(1)\) process: \[X_t = \phi X_{t-1} + \epsilon_t\] The PACF at lag 2, 3, and 4 will be zero because \(X_{t-1}\) already contains all the information from the past that is relevant to the present.

8.2.3 The Academic Alternative: The Telephone Game

The Telephone Game is the standard formal alternative for identifying AR processes.

Feature The Telephone Game Analogy
The Series A message passed from Person \(A \rightarrow B \rightarrow C \rightarrow D\).
ACF (Lag 3) Measures how much Person D’s message resembles Person A’s. Correlation is high because the words evolved step-by-step.
PACF (Lag 3) Measures how much Person D “copied” Person A directly, after accounting for what Person C said.
Interpretation In a perfect \(AR(1)\), Person D only hears Person C. The PACF for lags 2 and 3 is zero because D has no direct contact with A or B.

8.2.4 Summary for Model Identification

Tip

  • ACF = “Does the past echo?” ; PACF = “Which past day actually caused today?”

  • ACF tells you how memory flows ;PACF tells you where memory originates

  • ACF measures persistence ; PACF identifies structure

  1. AR(p) Identification: The PACF is the “hero.” It will show significant spikes only for the actual number of direct steps (\(p\)) and then “shut off” to zero.
  2. MA(q) Identification: The ACF is the “hero.” It will show significant spikes only for the number of “shocks” (\(q\)) currently affecting the system, then cut off.

The Golden Rule of Identification: > * If the PACF hits a wall, it’s an AR model (count the spikes for \(p\)). If the ACF hits a wall, it’s an MA model (count the spikes for \(q\)). If both linger and decay, it’s time to start testing ARMA combinations.