Question 1: Applications of Time Series

Time series analysis can be used for analyzing any measurements that change with time. It is different from other kinds of data because of the natural order imposed due to time. Measurements taken with time can have some unique features besides randomness like trend and seasonality. Time series analysis provides tools that help us deal with these properties of time series data.

Some of the key applications of Time Series data are in the following.

Economic and Social data For handling data of census, economics, trade, business and stock market Time Series Analysis provides a systematic way of handling these kinds of data. These all have major impact in the life of human beings and society.

Science and Engineering Time Series has connection with Signal Processing and is useful because natural phenomenon like weather or measurements in physics and chemistry can change with time. Also in a field like robotics data from sensors are time series.

Medicine and Healthcare Measurements of patients on indicators of health and other biological data are useful to develop futurstic diagnostic and treatment methods because the data obtained either by manual measurement or by electronic devices are all time series.

The main goals of Time Series is to forecast future values of variables of interest and infer about parameters of the stochastic model that is generating the time series and is therefore useful in every sphere of human life.

Question 2: ARMA(1,1)

Drawing from ARMA(1,1)

To draw from ARMA(1,1) we can use the recurrence formula and the given values of the parameters to iteratively generate the Time Series. But we will use to some burn-in as the first value of the time series I am choosing as zero. For this I generate 1000 iid samples from standard normal and generate 1000 values of this time series. Then I choose the last 40 as my ARMA(1,1) sample.

set.seed(123)
z <- rnorm(1000, 0, 1)
x <- vector()
x[1] <- 0
for (i in 2:1000) {
  x[i] <- 0.1*x[i-1] + z[i] + 0.4*z[i-1]
}
arma <- x[961:1000]

Displaying the values below.

data.frame(list(values = arma))
        values
1   1.19238169
2   2.73252588
3   1.34980332
4   0.24439505
5   0.13971875
6   0.12237244
7  -1.80832904
8  -2.59134907
9  -1.00512470
10 -0.71259544
11 -0.24894997
12 -2.11420876
13 -2.55460248
14 -1.95642299
15  0.34982255
16 -0.66908446
17 -1.30581840
18 -0.37051360
19 -0.32784819
20 -0.01546619
21  2.36208223
22  0.03362935
23 -0.75194817
24 -0.71414212
25  1.23427743
26 -0.04109901
27 -0.39399077
28  0.71489978
29 -0.67224711
30  1.16451489
31  1.45428423
32 -0.65847322
33 -0.04095216
34 -0.03557124
35  0.22434879
36  0.05775119
37  1.04030108
38 -0.81886386
39 -1.14494324
40 -0.57273168

Graphing the time series

library(ggplot2)
qplot(y = arma, x = 1:40, geom =c("point", "line"))

Centering the sample and creating lagged values up till lag 7.

x <- arma - mean(arma)
x1 <- c(x[2:40], 0)
x2 <- c(x[3:40], rep(0,2))
x3 <- c(x[4:40], rep(0,3))
x4 <- c(x[5:40], rep(0,4))
x5 <- c(x[6:40], rep(0,5))
x6 <- c(x[7:40], rep(0,6))
x7 <- c(x[8:40], rep(0,7))

Displaying the centered and lagged values of the sample

I have padded the missing values with zero for ease of calculation later.

data.frame(list(values = x, at_lag_1 = x1, at_lag_2 = x2, at_lag_3 = x3, at_lag_4 = x4, at_lag_5 = x5, at_lag_6 = x6, at_lag_7 = x7))
        values    at_lag_1    at_lag_2    at_lag_3    at_lag_4    at_lag_5
1   1.37008587  2.91023006  1.52750750  0.42209924  0.31742293  0.30007662
2   2.91023006  1.52750750  0.42209924  0.31742293  0.30007662 -1.63062486
3   1.52750750  0.42209924  0.31742293  0.30007662 -1.63062486 -2.41364489
4   0.42209924  0.31742293  0.30007662 -1.63062486 -2.41364489 -0.82742052
5   0.31742293  0.30007662 -1.63062486 -2.41364489 -0.82742052 -0.53489126
6   0.30007662 -1.63062486 -2.41364489 -0.82742052 -0.53489126 -0.07124579
7  -1.63062486 -2.41364489 -0.82742052 -0.53489126 -0.07124579 -1.93650458
8  -2.41364489 -0.82742052 -0.53489126 -0.07124579 -1.93650458 -2.37689830
9  -0.82742052 -0.53489126 -0.07124579 -1.93650458 -2.37689830 -1.77871881
10 -0.53489126 -0.07124579 -1.93650458 -2.37689830 -1.77871881  0.52752673
11 -0.07124579 -1.93650458 -2.37689830 -1.77871881  0.52752673 -0.49138028
12 -1.93650458 -2.37689830 -1.77871881  0.52752673 -0.49138028 -1.12811422
13 -2.37689830 -1.77871881  0.52752673 -0.49138028 -1.12811422 -0.19280942
14 -1.77871881  0.52752673 -0.49138028 -1.12811422 -0.19280942 -0.15014401
15  0.52752673 -0.49138028 -1.12811422 -0.19280942 -0.15014401  0.16223800
16 -0.49138028 -1.12811422 -0.19280942 -0.15014401  0.16223800  2.53978642
17 -1.12811422 -0.19280942 -0.15014401  0.16223800  2.53978642  0.21133353
18 -0.19280942 -0.15014401  0.16223800  2.53978642  0.21133353 -0.57424399
19 -0.15014401  0.16223800  2.53978642  0.21133353 -0.57424399 -0.53643794
20  0.16223800  2.53978642  0.21133353 -0.57424399 -0.53643794  1.41198162
21  2.53978642  0.21133353 -0.57424399 -0.53643794  1.41198162  0.13660517
22  0.21133353 -0.57424399 -0.53643794  1.41198162  0.13660517 -0.21628659
23 -0.57424399 -0.53643794  1.41198162  0.13660517 -0.21628659  0.89260396
24 -0.53643794  1.41198162  0.13660517 -0.21628659  0.89260396 -0.49454293
25  1.41198162  0.13660517 -0.21628659  0.89260396 -0.49454293  1.34221907
26  0.13660517 -0.21628659  0.89260396 -0.49454293  1.34221907  1.63198841
27 -0.21628659  0.89260396 -0.49454293  1.34221907  1.63198841 -0.48076904
28  0.89260396 -0.49454293  1.34221907  1.63198841 -0.48076904  0.13675202
29 -0.49454293  1.34221907  1.63198841 -0.48076904  0.13675202  0.14213295
30  1.34221907  1.63198841 -0.48076904  0.13675202  0.14213295  0.40205297
31  1.63198841 -0.48076904  0.13675202  0.14213295  0.40205297  0.23545537
32 -0.48076904  0.13675202  0.14213295  0.40205297  0.23545537  1.21800526
33  0.13675202  0.14213295  0.40205297  0.23545537  1.21800526 -0.64115968
34  0.14213295  0.40205297  0.23545537  1.21800526 -0.64115968 -0.96723906
35  0.40205297  0.23545537  1.21800526 -0.64115968 -0.96723906 -0.39502750
36  0.23545537  1.21800526 -0.64115968 -0.96723906 -0.39502750  0.00000000
37  1.21800526 -0.64115968 -0.96723906 -0.39502750  0.00000000  0.00000000
38 -0.64115968 -0.96723906 -0.39502750  0.00000000  0.00000000  0.00000000
39 -0.96723906 -0.39502750  0.00000000  0.00000000  0.00000000  0.00000000
40 -0.39502750  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000
      at_lag_6    at_lag_7
1  -1.63062486 -2.41364489
2  -2.41364489 -0.82742052
3  -0.82742052 -0.53489126
4  -0.53489126 -0.07124579
5  -0.07124579 -1.93650458
6  -1.93650458 -2.37689830
7  -2.37689830 -1.77871881
8  -1.77871881  0.52752673
9   0.52752673 -0.49138028
10 -0.49138028 -1.12811422
11 -1.12811422 -0.19280942
12 -0.19280942 -0.15014401
13 -0.15014401  0.16223800
14  0.16223800  2.53978642
15  2.53978642  0.21133353
16  0.21133353 -0.57424399
17 -0.57424399 -0.53643794
18 -0.53643794  1.41198162
19  1.41198162  0.13660517
20  0.13660517 -0.21628659
21 -0.21628659  0.89260396
22  0.89260396 -0.49454293
23 -0.49454293  1.34221907
24  1.34221907  1.63198841
25  1.63198841 -0.48076904
26 -0.48076904  0.13675202
27  0.13675202  0.14213295
28  0.14213295  0.40205297
29  0.40205297  0.23545537
30  0.23545537  1.21800526
31  1.21800526 -0.64115968
32 -0.64115968 -0.96723906
33 -0.96723906 -0.39502750
34 -0.39502750  0.00000000
35  0.00000000  0.00000000
36  0.00000000  0.00000000
37  0.00000000  0.00000000
38  0.00000000  0.00000000
39  0.00000000  0.00000000
40  0.00000000  0.00000000

Calculating the ACVF values till lag 7

gamma0 <- (1/40)*sum(x*x)
gamma <-(1/40)*c(sum(x*x1), sum(x*x2), sum(x*x3), sum(x*x4), sum(x*x5), sum(x*x6), sum(x*x7))
data.frame(list(ACVF_at_lags_upto_7 = gamma))
  ACVF_at_lags_upto_7
1          0.62602572
2          0.14552586
3          0.16652840
4          0.20448565
5          0.14423253
6          0.06817725
7         -0.20283076

Calculating the ACF values till lag 7

rho <- gamma/gamma0
data.frame(list(ACF_at_lags_upto_7 = rho))
  ACF_at_lags_upto_7
1          0.4544433
2          0.1056398
3          0.1208860
4          0.1484398
5          0.1047010
6          0.0494911
7         -0.1472385

Graphing Sample ACF

df <-  data.frame(x= 1:7, y = rho)
ggplot(df, aes(x = 1:7, y = rho)) + geom_bar(stat="identity")

Confidence Interval for ACF at lag 1

Using Bartlett’s formula the estimated value of ACF at lag 1 follows Normal distribution with mean equal to estimate of ACF at lag 1 and variance, w given by the formula below.

w <- (rho[2] + 1 - 2*(rho[1])^2)^2 + (((rho[1])*(0.1^2) + rho[1] - 0.2*(rho[1])^2)^2)/(1-0.01)
w
[1] 0.6559202

The 95% Confidence Interval we get as below.

c(rho[1] - 1.96*sqrt(w/40), rho[1] + 1.96*sqrt(w/40) )
[1] 0.2034561 0.7054305

We see that it contains the model value of rho[1] using the given parameter values of phi, rho and sigma.

(0.4 + 0.1)*(1+ 0.4*0.1)/(1 + 2*0.4*0.1 + 0.4^2)
[1] 0.4193548

This lies in the COnfidence Interval calculated above.

Question 3: Uniform(0,9) iid Sample

set.seed(123)
sample <- runif(40, min = 0, max = 9)

Graphing the time series

library(ggplot2)
qplot(y = sample, x = 1:40, geom =c("point", "line"))

Sample mean and centered sample

x <- sample - mean(sample) 
x
 [1] -2.48568389  2.02086464 -1.39308928  2.87327506  3.39032398 -4.66387308
 [7] -0.32093218  2.95788982 -0.11096645 -0.96434896  3.53761853 -0.99387417
[13]  1.02425414  0.07981904 -4.14755943  3.02454316 -2.85909197 -4.69534577
[19] -2.12259510  3.51665127  2.93197227  1.16134908  0.69067975  3.87454641
[25]  0.82747062  1.30289264 -0.17728735  0.27339661 -2.47144394 -3.74985875
[31]  3.59333652  3.04680983  1.14246593  2.08532518 -4.85235842 -0.77371784
[37]  1.75225426 -3.12621015 -2.21025251 -2.98924951

Creating lagged samples

x1 <- c(x[2:40], 0)
x2 <- c(x[3:40], rep(0,2))
x3 <- c(x[4:40], rep(0,3))
x4 <- c(x[5:40], rep(0,4))
x5 <- c(x[6:40], rep(0,5))
x6 <- c(x[7:40], rep(0,6))
x7 <- c(x[8:40], rep(0,7))

Displaying the centered and lagged values of the sample

data.frame(list(values = x, at_lag_1 = x1, at_lag_2 = x2, at_lag_3 = x3, at_lag_4 = x4, at_lag_5 = x5, at_lag_6 = x6, at_lag_7 = x7))
        values    at_lag_1    at_lag_2    at_lag_3    at_lag_4    at_lag_5
1  -2.48568389  2.02086464 -1.39308928  2.87327506  3.39032398 -4.66387308
2   2.02086464 -1.39308928  2.87327506  3.39032398 -4.66387308 -0.32093218
3  -1.39308928  2.87327506  3.39032398 -4.66387308 -0.32093218  2.95788982
4   2.87327506  3.39032398 -4.66387308 -0.32093218  2.95788982 -0.11096645
5   3.39032398 -4.66387308 -0.32093218  2.95788982 -0.11096645 -0.96434896
6  -4.66387308 -0.32093218  2.95788982 -0.11096645 -0.96434896  3.53761853
7  -0.32093218  2.95788982 -0.11096645 -0.96434896  3.53761853 -0.99387417
8   2.95788982 -0.11096645 -0.96434896  3.53761853 -0.99387417  1.02425414
9  -0.11096645 -0.96434896  3.53761853 -0.99387417  1.02425414  0.07981904
10 -0.96434896  3.53761853 -0.99387417  1.02425414  0.07981904 -4.14755943
11  3.53761853 -0.99387417  1.02425414  0.07981904 -4.14755943  3.02454316
12 -0.99387417  1.02425414  0.07981904 -4.14755943  3.02454316 -2.85909197
13  1.02425414  0.07981904 -4.14755943  3.02454316 -2.85909197 -4.69534577
14  0.07981904 -4.14755943  3.02454316 -2.85909197 -4.69534577 -2.12259510
15 -4.14755943  3.02454316 -2.85909197 -4.69534577 -2.12259510  3.51665127
16  3.02454316 -2.85909197 -4.69534577 -2.12259510  3.51665127  2.93197227
17 -2.85909197 -4.69534577 -2.12259510  3.51665127  2.93197227  1.16134908
18 -4.69534577 -2.12259510  3.51665127  2.93197227  1.16134908  0.69067975
19 -2.12259510  3.51665127  2.93197227  1.16134908  0.69067975  3.87454641
20  3.51665127  2.93197227  1.16134908  0.69067975  3.87454641  0.82747062
21  2.93197227  1.16134908  0.69067975  3.87454641  0.82747062  1.30289264
22  1.16134908  0.69067975  3.87454641  0.82747062  1.30289264 -0.17728735
23  0.69067975  3.87454641  0.82747062  1.30289264 -0.17728735  0.27339661
24  3.87454641  0.82747062  1.30289264 -0.17728735  0.27339661 -2.47144394
25  0.82747062  1.30289264 -0.17728735  0.27339661 -2.47144394 -3.74985875
26  1.30289264 -0.17728735  0.27339661 -2.47144394 -3.74985875  3.59333652
27 -0.17728735  0.27339661 -2.47144394 -3.74985875  3.59333652  3.04680983
28  0.27339661 -2.47144394 -3.74985875  3.59333652  3.04680983  1.14246593
29 -2.47144394 -3.74985875  3.59333652  3.04680983  1.14246593  2.08532518
30 -3.74985875  3.59333652  3.04680983  1.14246593  2.08532518 -4.85235842
31  3.59333652  3.04680983  1.14246593  2.08532518 -4.85235842 -0.77371784
32  3.04680983  1.14246593  2.08532518 -4.85235842 -0.77371784  1.75225426
33  1.14246593  2.08532518 -4.85235842 -0.77371784  1.75225426 -3.12621015
34  2.08532518 -4.85235842 -0.77371784  1.75225426 -3.12621015 -2.21025251
35 -4.85235842 -0.77371784  1.75225426 -3.12621015 -2.21025251 -2.98924951
36 -0.77371784  1.75225426 -3.12621015 -2.21025251 -2.98924951  0.00000000
37  1.75225426 -3.12621015 -2.21025251 -2.98924951  0.00000000  0.00000000
38 -3.12621015 -2.21025251 -2.98924951  0.00000000  0.00000000  0.00000000
39 -2.21025251 -2.98924951  0.00000000  0.00000000  0.00000000  0.00000000
40 -2.98924951  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000
      at_lag_6    at_lag_7
1  -0.32093218  2.95788982
2   2.95788982 -0.11096645
3  -0.11096645 -0.96434896
4  -0.96434896  3.53761853
5   3.53761853 -0.99387417
6  -0.99387417  1.02425414
7   1.02425414  0.07981904
8   0.07981904 -4.14755943
9  -4.14755943  3.02454316
10  3.02454316 -2.85909197
11 -2.85909197 -4.69534577
12 -4.69534577 -2.12259510
13 -2.12259510  3.51665127
14  3.51665127  2.93197227
15  2.93197227  1.16134908
16  1.16134908  0.69067975
17  0.69067975  3.87454641
18  3.87454641  0.82747062
19  0.82747062  1.30289264
20  1.30289264 -0.17728735
21 -0.17728735  0.27339661
22  0.27339661 -2.47144394
23 -2.47144394 -3.74985875
24 -3.74985875  3.59333652
25  3.59333652  3.04680983
26  3.04680983  1.14246593
27  1.14246593  2.08532518
28  2.08532518 -4.85235842
29 -4.85235842 -0.77371784
30 -0.77371784  1.75225426
31  1.75225426 -3.12621015
32 -3.12621015 -2.21025251
33 -2.21025251 -2.98924951
34 -2.98924951  0.00000000
35  0.00000000  0.00000000
36  0.00000000  0.00000000
37  0.00000000  0.00000000
38  0.00000000  0.00000000
39  0.00000000  0.00000000
40  0.00000000  0.00000000

Calculating sample ACVF

gamma0 <- (1/40)*sum(x*x)
gamma <-(1/40)*c(sum(x*x1), sum(x*x2), sum(x*x3), sum(x*x4), sum(x*x5), sum(x*x6), sum(x*x7))
data.frame(list(ACVF_at_lags_upto_7 = gamma))
  ACVF_at_lags_upto_7
1          0.06000661
2         -1.18064155
3          0.71728770
4         -0.90109716
5          0.06365633
6         -0.53282804
7         -1.50739544

Sample ACF

rho <- gamma/gamma0
data.frame(list(ACF_at_lags_upto_7 = rho))
  ACF_at_lags_upto_7
1        0.008626289
2       -0.169723881
3        0.103114152
4       -0.129537798
5        0.009150957
6       -0.076597036
7       -0.216696595

Graphing Sample ACF

df <-  data.frame(x= 1:7, y = rho)
ggplot(df, aes(x = 1:7, y = rho)) + geom_bar(stat="identity")

Test based on CI

95% Confidence Interval for sample correlation is as below.

c((-1.96)/sqrt(40), (1.96)/sqrt(40))
[1] -0.3099032  0.3099032

We can see that the values of ACF at all lags up to lag 7 fall in this confidence interval so we cannot reject the iid hypothesis.

Portmanteau Test

We calculate the value of the Portmanteau statistic and compare it with the upper 0.95 cutoff point of the chi-square distribution with 7 degrees of freedom which has value = 14.06714.

qchisq(0.95,7)
[1] 14.06714

The value of the simple Portmanteau statistic is as given below.

(Q <- 40*sum(rho^2))
[1] 4.368058

As the value of Q is much less than 95% cutoff of the chi-square distribution with 7 degrees of freedom so we cannot reject the iid Null Hypothesis.

Ljung and Box

vec <- c(39, 38, 37, 36, 35, 34, 33)

(Q_LB <- 40*42*sum((rho^2)/vec)) 
[1] 5.227068

This is also much less than the upper 95% of the chi-square distribution with 7 degrees of freedom so we cannot reject Null Hypothesis.

McLeod and Li

This is a test based on squared values of the sample.

y <- sample^2
y <- y - mean(y)
y1 <- c(y[2:40], 0)
y2 <- c(y[3:40], rep(0,2))
y3 <- c(y[4:40], rep(0,3))
y4 <- c(y[5:40], rep(0,4))
y5 <- c(y[6:40], rep(0,5))
y6 <- c(y[7:40], rep(0,6))
y7 <- c(y[8:40], rep(0,7))
ygamma0 <- (1/40)*sum(y*y)
ygamma <-(1/40)*c(sum(y*y1), sum(y*y2), sum(y*y3), sum(y*y4), sum(y*y5), sum(y*y6), sum(y*y7))
yrho <- gamma/gamma0

(Q_ML <- 40*42*sum((yrho)^2/vec))
[1] 5.227068

This is also much less than the upper 95% of the chi-square distribution with 7 degrees of freedom so we cannot reject Null Hypothesis.

Rank test

We can do a rank based test by finding pairs of observation that obey the time ordering which detects linear trend in the time series. We see that we get a p-value of 0.6 which indicates there is high probability that the Null Hypothesis is correct. So we cannot reject the iid Null Hypothesis.

library(trend)
mk.test(x)

    Mann-Kendall trend test

data:  x
z = -0.5243, n = 40, p-value = 0.6001
alternative hypothesis: true S is not equal to 0
sample estimates:
            S          varS           tau 
 -46.00000000 7366.66666667   -0.05897436

End of report