T K Chakrabarty
2026-04-02
We have seen a number of examples of time series in our last two chapters. We can now say that a time series is a collection of observations sequentially in time.
Our interest will not be in such series that are deterministic but rather in those whose values behave according to the laws of probability.
As such, each observation \(x_t\) at time t, of a time series is a realization of a random variable \(X_t\). In this chapter, we will discuss the fundamentals involved in the statistical analysis of time series. To begin, we must be more careful in our definition of a time series. Actually, a time series is a special type of stochastic process.
A time series is a stochastic process \(\{X_t|t\in T\}\), a collection of random variables \(X_t\) sequentially over a time index set \(T\). If \(T\) takes on values on the set \(T=\{0, 1, 2,...\}\) or \(T=\{0,\pm 1,\pm 2,...\}\), we refer to as discrete parameter time series.In case, \(T=(-\infty ,\infty )\) or \(T=(0, \infty)\), the series is continuous parameter process.
A time series model for the observed data \(\{x_t\}\) is a specification of the joint distributions (or possibly only the means and covariances) of a sequence of random variables \(\{X_t\}\) of which \(\{x_t\}\) is postulated to be a realization.
A complete probabilistic time series model for the sequence of random variables \(\{X_1,X_2, . . .\}\) is avoided.
Instead we specify only the first- and second-order moments of the joint distributions, i.e., the expected values \(E(X_t)\) and the expected products \(E(X_{t+h}X_t ), t = 1, 2, . . ., h = 0, 1, 2, . . .\), focusing on properties of the sequence \(\{Xt\}\) that depend only on these. Such properties of \(\{X_t\}\) are referred to as second-order properties.
We now discuss various measures that describe the general behavior of a time series process as it evolves over time. While defining these measures, we shall be restricting our attention only to the second-order properties as stated before.
Let \(\{X_t\}\) be a time series with \(E(X_t)^2<\infty\).
The mean function of \(\{Xt\}\) is \(\mu_X(t)= E(X_t)\).
The covariance function of \(\{X_t\}\) is
\(\gamma_X(r, s) = Cov(X_r,X_s)\)
\(= E[(X_r-\mu_X(r))(X_s-\mu_X(s))]\)
for all integers r and s.
A time series with finite variance process where
the mean value function, \(\mu_X(t)\), is constant and is independent of t, and
the autocovariance function, \(\gamma(r,s)\) depends on times s and t only through their time difference or lag \(|s-r|=h\).
In view of the condition (ii), whenever we use the term covariance function with reference to a stationary time series \(\{X_t\}\), we shall mean the function \(\gamma_X\) of one variable, defined by
\(\gamma_X(h) = \gamma_X(t+h,t)\).
The function \(\gamma_X(.)\) will be referred to as the autocovariance function and \(\gamma_X(h)\) as its value at lag h.
Let \(\{X_t\}\) be a stationary time series. The autocovariance function (ACVF) of \(\{X_t\}\) at lag h is defined as \[\begin{equation} \gamma_X(h) =\gamma_X(t + h, t)= Cov(X_{t+h},X_t). \end{equation}\] The autocorrelation function (ACF) of \(\{X_t\}\) at lag h is \[\begin{equation} \rho_X(h) = \frac{\gamma_X(h)}{\gamma_X(0)}= Cor(X_{t+h},X_t). \end{equation}\]
correlation measures the linear association between a pair of variables, and is obtained by standardising the covariance, by dividing the covariance by the standard deviations of the variables.
A value of +1 or -1 indicates an exact linear association with the pairs falling on a straight line of positive or negative slope respectively.
In time series, observations tend to be serially correlated, the measure of linear dependence is called as autocorrelation.
The adjective “auto” ,which means self, is used to refer to the relation between the same variable at different time points.
Because it is a correlation, we have \(1\le \rho(h)\le 1\) for all \(h\), enabling one to assess the relative importance of a given autocorrelation value by comparing with the extreme values \(-1\) and \(1\).
In this section, we will learn the properties of the autocovariance and autocorrelation functions for stationary time series. If a time series is weakly stationary, then the autocovariance function only depends on h. Thus, for stationary processes, we denote this autocovariance function by \(\gamma(h)\). Similarly, the autocorrelation function for a stationary process is given by \(\rho(h)=\frac{\gamma(h)}{\gamma(0)}\) . The autocovariance function of a stationary time series satisfies the following properties:
Theorem
\(\gamma(0)\ge 0\).
\(|\gamma(h)|\le\gamma(0)\)for all \(h\).
\(\gamma(.)\) is even, i.e., \(\gamma(h)=\gamma(-h)\) for all h.
The function \(\gamma(.)\) is positive semidefinite. That is, for any set of time points \(t_1, t_2,...,t_k \in T\) and and all real \(a_1, a_2,...,a_k\), we have \[\begin{equation} \sum_{i=1}^{k}\sum_{j=1}^{k}a_i\gamma(t_i-t_j)a_j\ge0. \end{equation}\]
Proof. The first property is simply the statement that \(\gamma(0)=Var(X_t)\ge0\), the second is an immediate consequence of the fact that correlations are less than or equal to 1 in absolute value (or the Cauchy–Schwarz inequality), and the third is established by observing that \[\begin{equation*} \gamma(h)=Cov(X_{t+h},X_t)=Cov(X_t,X_{t+h})=\gamma(-h). \end{equation*}\] To prove (4), let \(W=\sum_{i=1}^{k}a_iX_(t_i)\). Now, \[\begin{align*} Var(W)&\ge 0\\ aD(X)a^T&\ge 0\\ \sum_{i=1}^{k}\sum_{j=1}^{k}a_i\gamma(t_i-t_j)a_j&\ge 0 \end{align*}\] and the result follows.
The equation (3.3) is equivalent to the following autocovariance matrix \[\begin{align} \Gamma_k & = \begin{pmatrix} 1 & \gamma_1 & \cdots & \gamma_k \\ \gamma_1 & 1 & \cdots & \gamma_{k-1} \\ \vdots & \vdots & \vdots & \vdots \\ \gamma_k & \gamma_{k-1} & \cdots & 1 \\ \end{pmatrix}, \end{align}\] is positive semidefinite for each k.