Bahadur Representation

Theorem by RR Bahadur (1966)

Let \(F(x)=\mathbb{P}\left(X_{n} \leq x\right)\) be the marginal distribution function of \(X_{n}\) and let \(f\) be its density. For \(0<p<1\), denote by \(\xi_{p}=\) \(\inf \{x: F(x) \geq p\}\) the \(p\) th quantile of \(F\). Given a sample \(X_{1}, \ldots, X_{n}\), let \(\hat{\xi}_{p, n}\) be the \(p\) th \((0<p<1)\) sample quantile and define the empirical distribution function

\[ F_{n}(x)=\frac{1}{n} \sum_{i=1}^{n} \mathbf{1}_{X_{i} \leq x} \]

For simplicity we also refer to \(\hat{\xi}_{p, n}\) as the \(p\) th quantile of \(F_{n}\).
\(\xi_p\) be a fixed point such that F(\(\xi_p\)) = p
It is assumed that F is at least twice differentiable at some neighbourhood of \(\xi_p\), that \(F^{\prime\prime}(x)\) is bounded in the neighbourhood and \(F^{\prime}(\xi_p)\) = \(f(\xi_p)\) > 0. These assumptions imply, in particular, that 0 < p < 1 and \(\xi_p\) is the unique \(p^{th}\) quantile of \(F\).

We are interested in finding asymptotic representations of \(\hat{\xi}_{p, n}\).

Weaker version

Theorem from JK Ghosh (1971)

Suppose \(F^{\prime}\left(\xi_{p}\right)\) exists and is strictly positive. Then,

where, \(R_{n}=o_{P}\left(n^{-1 / 2}\right)\)

We will prove the weaker version of the Bahadur representation and later use it in problems regarding finding asymptotic distributions of few specific functions.

Necessary lemma

Lemma from JK Ghosh (1971)

Suppose \(\left\{W_{n}: n \geqslant 1\right\}\) is a uniformly tight sequence of random variables. Also assume that the sequence of random variables \(\left\{V_{n}: n \geqslant 1\right\}\) satisfies the following condition: for all \(c \in \mathbb{R}\) and all \(\epsilon>0\),

\[ \lim _{n \rightarrow \infty} \mathbf{P}\left(V_{n} \leqslant c, W_{n} \geqslant c+\epsilon\right)=0 \text { and } \quad \lim _{n \rightarrow \infty} \mathbf{P}\left(V_{n} \geqslant c+\epsilon, W_{n} \leqslant c\right)=0 \]

Then, \(\left(V_{n}-W_{n}\right) \xrightarrow{P} 0\).

Proof of the lemma

Given that, \(\left\{W_{n}: n \geqslant 1\right\}\) is uniformly tight. Then, there exists M > 0, such that we can restrict \(W_{n}\) to [-M , M] with arbitrarily high probability.
Write [-M , M] = \(\bigcup_{j=1}^{K} [a_j , a_{j+1}]\) such that -M = \(a_1 < a_2 < . . . < a_K < a_{K+1}\) = M and |\(a_{j+1} - a_j\)| < \(\epsilon\)
It follows that \(\forall\) - \(\infty\) < a < b < \(\infty\) and \(\forall\) \(\epsilon\) > 0,
\(P(a \leq W_n \leq b, V_n \leq a - \epsilon\) or \(V_n \geq b + \epsilon) \rightarrow\) 0 as \(n \rightarrow \infty\)
\(=>\) \(P(a \leq W_n \leq b, V_n \epsilon (- \infty, a - \epsilon) \bigcup [b + \epsilon, \infty]) \rightarrow 0\) as \(n \rightarrow \infty\)
Again,

\[ \begin{aligned} \mathbf{P}\left(\left|V_{n}-W_{n}\right|>2 \epsilon\right) & =\mathbf{P}\left(\left|V_{n}-W_{n}\right|>2 \epsilon,\left|W_{n}\right|>M\right)+\mathbf{P}\left(\left|V_{n}-W_{n}\right|>2 \epsilon,\left|W_{n}\right| \leqslant M\right) \\ & \leqslant \delta+\sum_{j=1}^{K} \mathbf{P}\left(\left|V_{n}-W_{n}\right|>2 \epsilon, W_{n} \in\left[a_{j}, a_{j+1}\right]\right) \\ & =\delta+\sum_{j=1}^{K}\left[\mathbf{P}\left(V_{n}>W_{n}+2 \epsilon, W_{n} \in\left[a_{j}, a_{j+1}\right]\right)+\mathbf{P}\left(V_{n}<W_{n}-2 \epsilon, W_{n} \in\left[a_{j}, a_{j+1}\right]\right)\right] \\ & \leqslant \delta+\sum_{j=1}^{K}\left[\mathbf{P}\left(V_{n}>a_{j+1}+\epsilon, a_{j} \leqslant W_{n} \leqslant a_{j+1}\right)+\mathbf{P}\left(V_{n}<a_{j}-\epsilon, a_{j} \leqslant W_{n} \leqslant a_{j+1}\right)\right] \end{aligned} \]

Now, each term in the above summand goes to zero as \(n \rightarrow \infty\). Since \(\delta>0\) and \(\epsilon>0\) are arbitrary, hence the claim follows. Thus, \(\left(V_{n}-W_{n}\right) \xrightarrow{P} 0\).

Proof

Define, \(G_{n}(x)=1-F_{n}(x)\) and \(G(x)=1-F(x)\). For each \(n \geqslant 1\) and each \(t \in \mathbb{R}\), define the following random variables:

\[ V_{n}=\sqrt{n}\left(\hat{\xi}_{p, n}-\xi_{p}\right), \\ W_{n}=\frac{\sqrt{n}\left(G_{n}\left(\xi_{p}\right)-G\left(\xi_{p}\right)\right)}{F^{\prime}\left(\xi_{p}\right)},\\ Z_{t, n}=\frac{\sqrt{n}\left(G_{n}\left(\xi_{p}+n^{-1 / 2} t\right)-G\left(\xi_{p}+n^{-1 / 2} t\right)\right)}{F^{\prime}\left(\xi_{p}\right)} \]

Note that

\[ \left[V_{n} \leqslant t\right] \Rightarrow\left[Z_{t, n} \leqslant t_{n}\right], \quad \text { where } \quad t_{n}=\frac{\sqrt{n}\left(F\left(\xi_{p}+n^{-1 / 2} t\right)-p\right)}{F^{\prime}\left(\xi_{p}\right)} \]

due to given assumption, \(p=F\left(\xi_{p}\right)\) and hence \(t_{n} \rightarrow t\).

Proof (contd.)

Note that, if \(t>0\),

\[ \begin{aligned} &\mathbf{E}\left(Z_{t, n}-W_{n}\right)^{2}\\ &=\mathbf{E}[\frac{\sqrt{n}(G_n(\xi_p+\frac{t}{\sqrt{n}})) - G_n(\xi_p+\frac{t}{\sqrt{n}})}{F^{\prime}(\xi_{p})} - \frac{\sqrt{n}(G_n(\xi_p)) - G_n(\xi_p)}{F^{\prime}(\xi_{p})}]^2\\ &=\frac{n}{(F^{\prime}(\xi_{p}))^{2}} \cdot\mathbf{E}[(F_n(\xi_{p}+n^{-1 / 2} t)-F_n(\xi_{p})) -(F(\xi_{p}+n^{-1 / 2} t)-F(\xi_{p}))]^2\\ & =\frac{n}{\left(F^{\prime}\left(\xi_{p}\right)\right)^{2}} \cdot \mathbf{E}\left(\frac{1}{n} \sum_{i=1}^{n} \mathbf{1}\left(X_{i} \in\left(\xi_{p}, \xi_{p}+n^{-1 / 2} t\right)\right)-\mathbf{P}\left(X_{1} \in\left(\xi_{p}, \xi_{p}+n^{-1 / 2} t\right)\right)\right)^{2} \\ & =\frac{n}{(F^{\prime}(\xi_{p}))^{2}} \cdot \frac{1}{n} \cdot \mathbf{P}(X_{1} \in(\xi_{p}, \xi_{p}+n^{-1 / 2} t))\cdot(1 - \mathbf{P}(X_{1} \in(\xi_{p}, \xi_{p}+n^{-1 / 2} t)))\\ &=\frac{1}{(F^{\prime}(\xi_{p}))^{2}} \cdot[(F(\xi_{p}+n^{-1 / 2} t)-F(\xi_{p}))(1 -(F(\xi_{p}+n^{-1 / 2} t)-F(\xi_{p})))]\\\\ &\rightarrow 0, \quad \text { as } n \rightarrow \infty \end{aligned} \]

due to continuity of \(F\) at \(\xi_{p}\).
Hence, for any \(\epsilon>0\),

\[ \mathbf{P}\left(\left|Z_{t, n}-W_{n}\right|>\epsilon\right) \leqslant \frac{1}{\epsilon^{2}} \cdot \mathbf{E}\left(Z_{t, n}-W_{n}\right)^{2} \rightarrow 0\ as\ n\ \rightarrow \infty \]

Combining similar arguments for \(t<0\), we claim that for any \(t \in \mathbb{R},\left|Z_{t, n}-W_{n}\right| \xrightarrow{P} 0\).

Proof (contd.)

Now, by iid CLT, \(W_{n}\) converges in distribution to a normal random variable and hence \(\left\{W_{n}: n \geqslant 1\right\}\) will be uniformly tight.

Then, for any \(t \in \mathbb{R}, \epsilon>0\) and large enough \(n\),

\[ \begin{aligned} \mathbf{P}\left(V_{n} \leqslant t, W_{n} \geqslant t+\epsilon\right) & =\mathbf{P}\left(Z_{t, n} \leqslant t_{n}, W_{n} \geqslant t+\epsilon\right) \\ & \leqslant \mathbf{P}\left(Z_{t, n} \leqslant t+\epsilon / 4, W_{n} \geqslant t+\epsilon\right), \quad\left(\text { since, } t_{n} \leqslant t+\epsilon / 4, \text { for large } n\right) \\ & \leqslant \mathbf{P}\left(\left|Z_{t, n}-W_{n}\right|>3 \epsilon / 4\right)\\ & \rightarrow 0\ as\ n\rightarrow \infty \end{aligned} \]

Similarly, we can show that,
\(\mathbf{P}\left(V_{n} \geqslant t+\epsilon, W_{n} \leqslant t\right) \rightarrow 0\ as\ n\rightarrow \infty\)
Thus both the conditions of the lemma are satisfied.
Therefore, \(\boxed{V_{n}-W_{n} \xrightarrow{P} 0}\) as \(n\rightarrow \infty\).

Proof (contd.)

Now let, \(U_{n}=V_{n}-W_{n}\), we have

\[ \begin{aligned} U_{n}=V_{n}-W_{n} & =\sqrt{n}\left(\hat{\xi}_{p, n}-\xi_{p}\right)-\frac{\sqrt{n}\left(G_{n}\left(\xi_{p}\right)-G\left(\xi_{p}\right)\right)}{F^{\prime}\left(\xi_{p}\right)} \\ & =\sqrt{n}\left(\hat{\xi}_{p, n}-\xi_{p}\right)-\frac{\sqrt{n}\left(1-F_{n}\left(\xi_{p}\right)-\left(1-F\left(\xi_{p}\right)\right)\right)}{F^{\prime}\left(\xi_{p}\right)} \\ & =\sqrt{n}\left(\hat{\xi}_{p, n}-\xi_{p}\right)-\frac{\sqrt{n}\left(p-F_{n}\left(\xi_{p}\right)\right)}{F^{\prime}\left(\xi_{p}\right)} \\ \Rightarrow \hat{\xi}_{p, n} & =\xi_{p}+\frac{p-F_{n}\left(\xi_{p}\right)}{F^{\prime}\left(\xi_{p}\right)}+R_{n}, \quad \text { where, } R_{n}=U_{n} / \sqrt{n} . \end{aligned} \]

Since, \(U_{n}=o_{P}(1)\), it implies \(R_{n}=o_{P}\left(n^{-1 / 2}\right)\). This proves the result.

Interquartile range

Let \(X_1, X_2,. . . , X_n\) \(\stackrel{\text{iid}}{\sim}\) F. A measure of dispersion/spread of F is the inter-quartile range, IQR = \(Q_3 - Q_1\), where \(Q_1\) = \(\xi_{0.25, n}\) and \(Q_3\) = \(\xi_{0.75, n}\) are the first and the third quartiles, respectively.

Now we are interested in finding the asymptotic distribution of the IQR.
We know that, joint distribution of any two quantiles is given by

\[ \left(\begin{array}{c} \sqrt{n}\left(\hat\xi_{p_{1}, n}-\xi_{p_{1}}\right) \\ \sqrt{n}\left(\hat\xi_{p_{2}, n}-\xi_{p_{2}}\right) \end{array}\right) \xrightarrow{D} N_2(0, \Sigma) \] where, \(\Sigma\) = \[ \left( \begin{array}{cc} \frac{p_1(1-p_1)}{{f(\xi_{p_1})}^2} & \frac{(p_1 \wedge p_2) - p_1p_2}{f(\xi_{p_1})f(\xi_{p_2})}\\ \frac{(p_1 \wedge p_2) - p_1p_2}{f(\xi_{p_1})f(\xi_{p_2})} & \frac{p_2(1-p_2)}{{f(\xi_{p_2})}^2} \end{array} \right) \]

and \(\hat{\xi}_{p_{1}, n} = \xi_{p_{1}, n} + \frac{p_1 - \hat {F_n}(\xi_{p_{1}})}{\sqrt{p_1(1 - p_1)}}\)
and \(\hat{\xi}_{p_{1}, n} = \xi_{p_{1}, n} + \frac{p_1 - \hat {F_n}(\xi_{p_{1}})}{\sqrt{p_1(1 - p_1)}}\)

Here \(p_1\) = 0.75 and \(p_2\) = 0.25,
So we get,

\[ \left(\begin{array}{c} \sqrt{n}\left(\hat\xi_{0.75, n}-\xi_{0.75}\right) \\ \sqrt{n}\left(\hat\xi_{0.25, n}-\xi_{0.25}\right) \end{array}\right) \xrightarrow{D} N_2(0, \Sigma) \]

Where,  \(\Sigma\) =
\[ \left( \begin{array}{cc} \frac{3}{{16f(\xi_{0.75})}^2} & \frac{1}{16f(\xi_{0.75})f(\xi_{0.25})}\\ \frac{1}{16f(\xi_{0.75})f(\xi_{0.25})} & \frac{3}{{16f(\xi_{0.25})}^2} \end{array} \right) \]

IQR (contd.)

Now consider the transformation:

\[ f\left(\begin{array}{c} x \\ y \end{array}\right) = x - y \]

So,
\(\frac{\delta f}{\delta x}\) = 1 and \(\frac{\delta f}{\delta y}\) = -1

and hence,
\(\sqrt{n}(\hat{IQR} - IQR) \xrightarrow{D} N(0, {\sigma}^2)\)
where,

\({\sigma}^2\) = \(\frac{1}{16}[\frac{3}{{f(\xi_{0.75})}^2} + \frac{3}{{f(\xi_{0.25})}^2} - \frac{2}{f(\xi_{0.25})f(\xi_{0.75})}]\)

Central order statistics

Let \(X_{(n,k_n)}\) be a central order statistic such that \(\sqrt{n}|\frac{k_n}{n} - p| \rightarrow 0\ as\ n \rightarrow \infty\). Suppose that the cdf F of \(X_1, X_2\), . . . is differentiable at \(\xi_p\) and \(F^{\prime}(\xi_{p}) = f(\xi_p) > 0\)

We will show that,
\[\begin{equation*} X_{(n,k_n)}=\xi_{p}+\frac{p-\hat{F}_{n}(\xi_{p})}{F^{\prime}(\xi_{p})}+Y_{n} \end{equation*}\]
where \(Y_{n}=o_{P}\left(n^{-1 / 2}\right)\ as\ n\rightarrow\infty\)

Order statistics(contd.)

We know that
\[\begin{equation*} \boxed{\sqrt{n}(X_{(n,k_n)} - \xi_p) \xrightarrow{D} \mathcal{N}(0, \frac{p(1-p)}{{F^{\prime}({\xi_p})}^2})} \end{equation*}\]

So,  \(X_{(n,k_n)} \xrightarrow{P} \xi_p\ as\ n\rightarrow \infty\)
and,
\(\sqrt{n}(X_{(n,k_n)} - \xi_p) = o_{P}(1)\)

Again,
\(\hat{\xi}_{p, n} \xrightarrow{P} \xi_p\ as\ n \rightarrow \infty\)
and,
\(\sqrt{n}(\hat{\xi}_{p, n} - \xi_p) = o_{P}(1)\)

From these two, it implies that,
\(X_{(n,k_n)} - \hat{\xi}_{p, n} = o_{P}(\frac{1}{\sqrt{n}})\)

Again from the weaker version,
\(\hat{\xi}_{p, n} =\xi_{p}+\frac{p-F_{n}\left(\xi_{p}\right)}{F^{\prime}\left(\xi_{p}\right)}+R_{n}, \quad \text { where, } R_{n}=o_{P}\left(n^{-1 / 2}\right)\)

Using this, we have
\(X_{(n,k_n)} - Z_{n} =\xi_{p}+\frac{p-F_{n}\left(\xi_{p}\right)}{F^{\prime}\left(\xi_{p}\right)}+R_{n}, \quad \text { where, } R_{n}=o_{P}\left(n^{-1 / 2}\right),\ Z_{n}=o_{P}\left(n^{-1 / 2}\right)\)

So, ultimately we obtain,  \(X_{(n,k_n)} =\xi_{p}+\frac{p-F_{n}\left(\xi_{p}\right)}{F^{\prime}\left(\xi_{p}\right)}+Y_{n}, \quad \text { where, } Y_{n}=o_{P}\left(n^{-1 / 2}\right)\)