Regression Analysis

• Model Summary:

  • R2 : 0.586.
  • Adjusted R2 : 0.5821.
  • The model explains approximately 58.6% of the variance in Bitcoin prices.While this is a substantial proportion, there is still a significant amount of variance that is not explained by the model.
  • The p-value for the Date coefficient is < 2e-16, indicating a significant relationship.

# Create a plot of the linear model on top of the time series dataset line plot with scatter data points
ggplot(BitCoin_df, aes(x = Date, y = Price)) + 
  geom_point() + 
  geom_line() + 
  geom_smooth(method = "lm", se = FALSE, color = "red") +
  labs(x = "Time", y = "Monthly Average BTC Price") +
  theme_minimal() + 
  ggtitle("Linear Regression Model on Time Series Data")

# Perform residual analysis and create a line & scatter plot of the residuals. Explain the outcome
residuals <- resid(linear_model)
plot(BitCoin_df$Date, residuals,xlab="Time", ylab="Residuals", type = "o", col = "blue", main = "Residuals of Linear Model")
abline(h=0.05, col= "red",lty=2)

Residual Analysis:

• Pattern in Residuals: The residual plot shows patterns, suggesting non-linearity and autocorrelation. Ideally, residuals should be randomly scattered around zero without any discernible pattern.

# Create a histogram plot of the residuals. Explain the outcome
hist(residuals, xlab="Residuals", breaks = 30, col = "lightblue", main = "Histogram of Residuals")

Histogram of Residuals: The histogram indicated that the residuals are not normally distributed and do not follow a bell shaped with the peak at zero. Normality of residuals is an assumption of linear regression. We can see from the histogram that there is skewness on the right hand side. And maximum residual exists majorly on the left hand side of the histogram.

# Create ACF & PACF plots of residuals. Explain the outcome
acf(residuals)

pacf(residuals)

ACF and PACF of Residuals: The ACF and PACF plots shows significant autocorrelation in the residuals. This can be seen from the gradual decrease in the ACF plot, indicating that the residuals are not independent. This suggests that the model has not adequately captured all the underlying patterns and dependencies in the data.

# Create QQ plot of residuals. Explain the outcome
qqnorm(residuals)
qqline(residuals, col = "red")

QQ Plot: The QQ plot showed that the residuals deviate from the line, suggesting that they are not normally distributed.

# Perform Shapiro-Wilk test on residuals. Explain the outcome
shapiro.test(residuals)

    Shapiro-Wilk normality test

data:  residuals
W = 0.85983, p-value = 1.215e-08

Shapiro-Wilk Test: The Shapiro-Wilk test returned a p-value of 1.215e-08, confirming that the residuals are not normally distributed.

Model Appropriateness: While the linear regression model indicates a statistically significant relationship between Date and Bitcoin Price, several assumptions of linear regression are violated:

• The residuals are not normally distributed.
• There is significant autocorrelation in the residuals.
• The residuals show patterns indicating non-linearity.

Given these violations, the linear regression model may not be the most appropriate for accurately modeling and forecasting Bitcoin prices.

Quadratic Regression

• Model Summary:
  • 𝑅2 : 0.5885.
  • Adjusted R2 : 0.5806.
  • Similar𝑅2 to the linear model but includes a non-significant quadratic term.

Model Appropriateness:

The quadratic term is not significant (p-value = 0.432), Suggesting that the quadratic model does not significantly improve the fit compared to the linear model. Based on the model summary and the characteristics of Bitcoin price data,the non-significance of the quadratic term and the moderate R-squared value suggest that this model does not capture the complexity of the data adequately.

Best Models Based on AIC and BIC:

First Model: Model 012 (ARIMA(0,1,2)) is the best choice according to both AIC and BIC, indicating a strong candidate for selection.

Second Model: Model 311 (ARIMA(3,1,1)) has the second lowest AIC and a relatively low BIC.

Therefore, while both AIC and BIC point to Model 012 as the best fit, Model 311 is also a reasonable choice, especially if predictive performance is a higher priority than model simplicity.

model_012 = Arima(stationary, order = c(0, 1, 2))
model_311 = Arima(stationary, order = c(3, 1, 1))

a012 <- accuracy(model_012)
a311 <- accuracy(model_311)

df_models <- data.frame(
rbind(a012, a311)
)
colnames(df_models) <- c("ME", "RMSE", "MAE", "MPE", "MAPE", "MASE", "ACF1")
rownames(df_models) <- c("ARIMA(0,1,2)",  "ARIMA(3,1,1)")

kable(df_models, digits = 2, formats="html", row.names = TRUE) %>%
  kable_styling(full_width = F, font_size = 12, position = "center")

Findings From the Table: The ARIMA(3,1,1) model has a slightly lower RMSE (4254.90 vs. 4312.32) and MAE (2423.61 vs. 2490.54) compared to the ARIMA(0,1,2) model, indicating better overall accuracy in terms of these metrics. The ACF1 values for both models are close to zero (-0.03 for ARIMA(0,1,2) and -0.01 for ARIMA(3,1,1)), suggesting minimal autocorrelation in the residuals, which is desirable. Despite these differences, both models exhibit high MAPE values, indicating substantial forecasting errors. Overall, ARIMA(3,1,1) performs slightly better in terms of error metrics, but both models have similar performance in terms of residual autocorrelation.

res_012 <- rstandard(model_012)
res_311 <- rstandard(model_311)

#Model 012
par(mfrow = c(1, 1))

# Plot 1: Time series plot of standardized residuals
plot(res_012,
     xlab = "Year Index", ylab = "Residuals",
     main = "Time series plot of standardized residuals-Model 012")
lines(res_012, col = "blue")
abline(h = 0.05, col = "red", lty = 2)

#Model 311
par(mfrow = c(1, 1))

# Plot 1: Time series plot of standardized residuals
plot(res_311,
     xlab = "Year Index", ylab = "Residuals",
     main = "Time series plot of standardized residuals- Model 311")
lines(res_311, col = "blue")
abline(h = 0.05, col = "red", lty = 2)

Both model 012 and 311 show stationarity.

#Histogram 012
hist(res_012,breaks=20, col="skyblue",
     xlab="Residuals",ylab="Frequency",
     main="Histogram of Residuals Model-012")

#Histogram 311
hist(res_311,breaks=20, col="skyblue",
     xlab="Residuals",ylab="Frequency",
     main="Histogram of Residuals Model-311")

Both show normal distribution and very similar spread.

# ACF & PACF of Residuals 012
acf_012 <- acf(res_012)

pacf_012 <- pacf(res_012)

ACF: Both models have residuals with minimal autocorrelation, indicated by most spikes being within the 95% confidence bounds. Given the slightly lower residual variance and error metrics for ARIMA(3,1,1) compared to ARIMA(0,1,2), the ARIMA(3,1,1) model is considered slightly better.

# ACF & PACF of Residuals 311
acf_311 <- acf(res_311)

pacf_311 <- pacf(res_311)

PACF: Both PACF plots indicate minimal partial autocorrelation, with most spikes within the confidence bounds for both models. Given the slightly better performance metrics and lower residual variance observed earlier, the ARIMA(3,1,1) model is considered slightly better overall.

#QQ Plot of Residuals 012
qqnorm(res_012)
qqline(res_012, col="red")

#QQ Plot of Residuals 311
qqnorm(res_311)
qqline(res_311, col="blue")

Both Q-Q plots show that the residuals of the ARIMA(0,1,2) and ARIMA(3,1,1) models are approximately normally distributed with slight deviations at the tails.

# Shapiro-Wilk Test on Residuals 012
print(shapiro.test(res_012))

    Shapiro-Wilk normality test

data:  res_012
W = 0.85275, p-value = 7.244e-09

# Shapiro-Wilk Test on Residuals 311
print(shapiro.test(res_311))

    Shapiro-Wilk normality test

data:  res_311
W = 0.84209, p-value = 2.889e-09

Both model’s residuals fail the Shapiro-Wilk normality test, indicating significant non-normality. The ARIMA(0,1,2) model has a slightly higher W statistic than the ARIMA(3,1,1) model, suggesting that its residuals are closer to normality.

Despite this, when considering previous analyses (lower residual variance and better fit metrics), the ARIMA(3,1,1) model is still considered slightly better overall.

Linear Regression:

Pros: Simple to implement and interpret.

Cons: Limited by linearity assumption, residuals show non-normality and autocorrelation.

Quadratic Regression:

Pros: Can model slight curvature in the trend.

Cons: The quadratic term was not significant, similar performance to linear regression.

ARIMA:

Pros: Captures both trend and seasonality, well-suited for time series data.

Cons: More complex to implement and interpret, requires careful model identification and validation.