AS2-3: Detecting Flu Epidemics via Search Engine Query Data

---

Section-1

【1.1】 Looking at the time period 2004-2011, which week corresponds to the highest percentage of ILI-related physician visits? Which week corresponds to the highest percentage of ILI-related query fraction?

Ans: 2009-10-18 - 2009-10-24

TR[which.max(TR$ILI),]

                       Week      ILI Queries
303 2009-10-18 - 2009-10-24 7.618892       1

TR[which.max(TR$Queries),]

                       Week      ILI Queries
303 2009-10-18 - 2009-10-24 7.618892       1

【1.2】 Let us now understand the data at an aggregate level. Plot the histogram of the dependent variable, ILI. What best describes the distribution of values of ILI?

Ans: Most of the ILI values are small, with a relatively small number of much larger values (in statistics, this sort of data is called “skew right”).

【1.3】 Plot the natural logarithm of ILI versus Queries. What does the plot suggest?.

Ans: There is a positive, linear relationship between log(ILI) and Queries.

---

Section-2

【2.1】 Based on the plot we just made, it seems that a linear regression model could be a good modeling choice. Based on our understanding of the data from the previous subproblem, which model best describes our estimation problem?

Ans: log(ILI) = intercept + coefficient x Queries, where the coefficient is positive

【2.2】 Let’s call the regression model from the previous problem (Problem 2.1) FluTrend1 and run it in R. Hint: to take the logarithm of a variable Var in a regression equation, you simply use log(Var) when specifying the formula to the lm() function.

What is the training set R-squared value for FluTrend1 model (the “Multiple R-squared”)?

Ans: 0.709

TR1 = lm(log(TR$ILI)~Queries, data = TR)
summary(TR1)

Call:
lm(formula = log(TR$ILI) ~ Queries, data = TR)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.76003 -0.19696 -0.01657  0.18685  1.06450 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.49934    0.03041  -16.42   <2e-16 ***
Queries      2.96129    0.09312   31.80   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.2995 on 415 degrees of freedom
Multiple R-squared:  0.709, Adjusted R-squared:  0.7083 
F-statistic:  1011 on 1 and 415 DF,  p-value: < 2.2e-16

【2.3】 For a single variable linear regression model, there is a direct relationship between the R-squared and the correlation between the independent and the dependent variables. What is the relationship we infer from our problem? (Don’t forget that you can use the cor function to compute the correlation between two variables.)

Ans: R-squared = Correlation^2

cor(log(TR$ILI), TR$Queries)

[1] 0.8420333

0.709 / cor(log(TR$ILI), TR$Queries)

[1] 0.8420095

---

Section-3

【3.1】 What is our estimate for the percentage of ILI-related physician visits for the week of March 11, 2012? (HINT: You can either just output FluTest$Week to find which element corresponds to March 11, 2012, or you can use the “which” function in R. To learn more about the which function, type ?which in your R console.)

Ans: 2.187378

PredTS[which(TS$Week=="2012-03-11 - 2012-03-17")]

      11 
2.187378 

【3.2】 What is the relative error betweeen the estimate (our prediction) and the observed value for the week of March 11, 2012? Note that the relative error is calculated as (Observed ILI - Estimated ILI)/Observed ILI

Ans: 0.04623827

(TS$ILI[11]-PredTS[11])/TS$ILI[11]

        11 
0.04623827 

【3.3】 What is the Root Mean Square Error (RMSE) between our estimates and the actual observations for the percentage of ILI-related physician visits, on the test set?

Ans:0.7490645

RMSE

[1] 0.7490645

---

Section-4

【4.1】 In these commands, the value of -2 passed to lag means to return 2 observations before the current one; a positive value would have returned future observations. The parameter na.pad=TRUE means to add missing values for the first two weeks of our dataset, where we can’t compute the data from 2 weeks earlier.

How many values are missing in the new ILILag2 variable?

Ans: 2

ILILag2 = lag(zoo(TR$ILI), -2, na.pad=TRUE)
TR$ILILag2 = coredata(ILILag2)
sum(is.na(ILILag2))

[1] 2

【4.2】 Use the plot() function to plot the log of ILILag2 against the log of ILI. Which best describes the relationship between these two variables?

Ans: There is a strong positive relationship between log(ILILag2) and log(ILI).

plot(log(TR$ILILag2),log(TR$ILI))

【4.3】 Train a linear regression model on the FluTrain dataset to predict the log of the ILI variable using the Queries variable as well as the log of the ILILag2 variable. Which coefficients are significant at the p=0.05 level in this regression model? What is the R^2 value of the FluTrend2 model?

Ans: Intercept, Queries, and log(ILILag2). R-sq = 0.9063

TR2 = lm(log(ILI) ~ Queries + log(ILILag2), TR)
summary(TR2)

Call:
lm(formula = log(ILI) ~ Queries + log(ILILag2), data = TR)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.52209 -0.11082 -0.01819  0.08143  0.76785 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -0.24064    0.01953  -12.32   <2e-16 ***
Queries       1.25578    0.07910   15.88   <2e-16 ***
log(ILILag2)  0.65569    0.02251   29.14   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1703 on 412 degrees of freedom
  (2 observations deleted due to missingness)
Multiple R-squared:  0.9063,    Adjusted R-squared:  0.9059 
F-statistic:  1993 on 2 and 412 DF,  p-value: < 2.2e-16

【4.4】 On the basis of R-squared value and significance of coefficients, which statement is the most accurate?

Ans: FluTrend2 is a stronger model than FluTrend1 on the training set.

---

Section-5

【5.1】 So far, we have only added the ILILag2 variable to the FluTrain data frame. To make predictions with our FluTrend2 model, we will also need to add ILILag2 to the FluTest data frame (note that adding variables before splitting into a training and testing set can prevent this duplication of effort).

Modify the code from the previous subproblem to add an ILILag2 variable to the FluTest data frame. How many missing values are there in this new variable?

Ans: 2

TS$ILILag2 = dplyr::lag(TS$ILI,2)
is.na(TS$ILILag2) %>% sum

[1] 2

【5.2】 In this problem, the training and testing sets are split sequentially – the training set contains all observations from 2004-2011 and the testing set contains all observations from 2012. There is no time gap between the two datasets, meaning the first observation in FluTest was recorded one week after the last observation in FluTrain. From this, we can identify how to fill in the missing values for the ILILag2 variable in FluTest.

Which value should be used to fill in the ILILag2 variable for the first observation in FluTest? Which value should be used to fill in the ILILag2 variable for the second observation in FluTest?

Ans: The ILI value of the second-to-last observation in the FluTrain data frame. The ILI value of the last observation in the FluTrain data frame.

【5.3】 Fill in the missing values for ILILag2 in FluTest. In terms of syntax, you could set the value of ILILag2 in row “x” of the FluTest data frame to the value of ILI in row “y” of the FluTrain data frame with “FluTest\(ILILag2[x] = FluTrain\)ILI[y]”. Use the answer to the previous questions to determine the appropriate values of “x” and “y”. It may be helpful to check the total number of rows in FluTrain using str(FluTrain) or nrow(FluTrain).

What is the new value of the ILILag2 variable in the first row of FluTest? What is the new value of the ILILag2 variable in the second row of FluTest?

Ans: 1.852736, 2.12413

nrow(TR)

[1] 417

TS$ILILag2[2] = TR$ILI[417]
TS$ILILag2[1] = TR$ILI[416]
TS$ILILag2[1]

[1] 1.852736

TS$ILILag2[2]

[1] 2.12413

【5.4】 Obtain test set predictions of the ILI variable from the FluTrend2 model, again remembering to call the exp() function on the result of the predict() function to obtain predictions for ILI instead of log(ILI).

What is the test-set RMSE of the FluTrend2 model?

Ans: 0.2942029

PredTS = exp(predict(TR2, TS))
SSE = sum((PredTS- TS$ILI)^2)
RMSE = sqrt(SSE/nrow(TS))
RMSE

[1] 0.2942029

【5.5】 Which model obtained the best test-set RMSE?

Ans: FluTrend2. The test-set RMSE of FluTrend2 is 0.294, as opposed to the 0.749 value obtained by the FluTrend1 model.