Inference 2: Testing of Hypothesis

Introduction

Hypothesis of testing deals with evaluating the feasibility of two competing statements about the underlying population based on a random sample drawn from the same.

Lecture 1: Some Definitions and Concepts:

Definition: [Hypothesis]

A statement or conjecture about the population or population parameters.

Definition: [Null Hypothesis]

The two contradictory statements (hypotheses) in a hypothesis testing problem are called the null hypothesis, denoted by \(H_{0}\), and the alternative hypothesis, denoted by \(H_{1}\).

The testing of hypothesis mainly deals with accepting or rejecting the null hypothesis \(H_{0}\) given the data, in view of \(H_{1}\) as the alternative.

Example 1: Suppose \(X_{1}, \ldots, X_{n}\) is a random sample from \(N(\mu,1)\) distribution, and we are interested in testing \(H_{0}: \mu=0\) against \(\mu>0\). While testing the hypothesis \(H_{0}\) we will collect evidence from the data in support of \(H_{0}\), in view of \(H_{1}\) as the alternative. Thus if the sample indicates that the population mean is much larger than zero, then we reject \(H_{0}\), otherwise we accept \(H_{0}\). Note that, we are not bothered if the sample indicates that the population mean is much smaller than zero.

Definition: [One-sided or Two-sided alternatives]

Suppose we are testing \(H_{0}:\theta=\theta_{0}\). The possible alternatives can be \(H_{1,1}:\theta=\theta_{1}\) where \(\theta_{1}>\theta_{0}\), \(H_{1,2}:\theta=\theta_{1}\) where \(\theta_{1}<\theta_{0}\), \(H_{1,3}:\theta>\theta_{0}\), \(H_{1,4}:\theta<\theta_{0}\), \(H_{1,5}:\theta \neq \theta_{0}\), etc. The first four alternatives are one-sided, where as \(H_{1,5}\) is a two-sided alternative.

Definition: [Simple and Composite Hypotheses]

Under a hypothesis \(H\), if the population distribution is completely specified, then \(H\) is called a simple hypothesis, otherwise it is called a composite hypothesis.

Suppose under \(H\), it is specified that the underlying parameter vector \(\theta\in \Theta_{0}\), then \(H\) is simple if \(\Theta_{0}\) is singleton, and it is composite otherwise.

Example 1 (continue): Suppose \(X_{1}, \ldots, X_{n}\) is a random sample from \(N(\mu,1)\) distribution, and we are interested in testing \(H_{0}: \mu=0\) against \(H_{1}: \mu>0\). Here \(H_{0}\) is simple, but \(H_{1}\) is composite.

Definition: [Hypothesis Test]

A hypothesis test is a set of rules that indicates which sample values lead to acceptance of \(H_{0}\), and which sample values lead to rejection of \(H_{0}\). Basically, a testing procedure partitions the sample space into two regions: one, called acceptance region, leads to acceptance of \(H_{0}\), and the other, called critical region, leads to rejection of \(H_{0}\). In other words, if the observed sample falls in the critical region then \(H_{0}\) is rejected, otherwise \(H_{0}\) is accepted.

Definition: [Critical and Acceptance Regions]

Let the support of \({\bf x}\) be \(S_{x}\subseteq \mathbb{R}^{n}\). A subset \(C\) of \(S_{x}\) (or, \(\mathbb{R}^{n}\)) such that if the data \({\bf x}\in C\) then \(H_{0}\) is rejected, is called the critical region. A subset \(A\) of \(S_{x}\) (or, \(\mathbb{R}^{n}\)) such that if the data \({\bf x}\in A\) then \(H_{0}\) is accepted, is called the acceptance region. Note that \(S_{x}\subseteq C\cup A\).

Sometimes it is convenient to define the test in terms of a function from \(\phi:S_{x}\to [0,1]\), such that \[ \phi({\bf x}) =1 \quad \text{if}~~ {\bf x} \in C, \quad \text{and}\quad \phi({\bf x}) =0 \quad \text{if}~~ {\bf x} \in A.\] Such a function is called a test function.

Definition: [Type I and Type II errors]

In following a test procedure two types of errors can occur. One may reject the null hypothesis when it is indeed true, or one may accept the null hypothesis when it is indeed false. The first type of error is called Type I error, and the second type of error is called Type II error.

True State \(\rightarrow\) Decision \(\downarrow\)	\(H_0\) is true	\(H_{1}\) is true
Accept \(H_{0}\)	No error	Type II error
Reject \(H_{0}\)	Type I error	No error

Definition: [Probabilities of Type I and Type II errors]

The probability of type I error is \(P({\bf X} \in C \mid H_{0})\), where \(C\) is the critical region. The probability of type II error is \(P({\bf X} \in \bar{C} \mid H_{1})\).

Example 1 (continue): In the above example, suppose we construct the following test procedure based on \(n\) samples: If the sample mean is greater than or equal to \(6\) then we reject \(H_{0}\). Then the probability of type I error is \(P\left[\bar{X} \geq 6 \mid \bar{X}\sim N(0,1/n)\right]\).

Definition: [Power Function]

Suppose we want to test \(H_{0}: \theta \in \Theta_{0}\) against \(H_{1}: \theta \in \Theta_{1}\), where \(\Theta_{0}\cup \Theta_{1}=\Theta\), based on a random sample \({\bf X}\). Suppose further that a test function \(\phi\) with the critical region \(C\) is proposed. Then the power function of the test \(\phi\) is \[\beta_{\phi}(\theta) =E_{\theta}(\phi)= P_{\theta} \left( {\bf X} \in C\right).\]

When \(\theta\in \Theta_{0}\), then \(\beta_{\phi}(\theta)\) provides the probability of type I error at \(\theta\), and when \(\theta\in \Theta_{1}\), \(\beta_{\phi}(\theta)\) provides the complement of probability of type II error (power) at \(\theta\).

Remark 1: Ideally one would like a test procedure for which the probabilities of both the types of errors are minimized. However, in practice if one tends to minimize the probability of one error then the probability of the other error increases. Thus the procedure used in practice is to bound the maximum probability of the type I error to a pre-assigned level \(\alpha\) which is small enough, and then to minimize the probability of type II error. The pre-assigned threshold of maximum probability of type I error \(\alpha\) is called the level of significance, or just level of the test.

Definition: [Level of Significance]

Let \(\phi\) be a test function for testing \(H_{0}: \theta\in \Theta_{0}\) against \(H_{1}: \theta\in \Theta_{1}\). Then \(\phi\) is called a level-\(\alpha\) test, or a test with level of significance \(\alpha\) if \[ E_{\theta} \left[ \phi({\bf X}) \right] \leq \alpha, \quad \text{for all} \quad \theta\in\Theta_{0}. \tag{*}\] In other words, a level-\(\alpha\) test satisfies \[\sup_{\theta\in \Theta_{0}}\beta_{\phi}(\theta)\leq \alpha. \tag{**}\]

Definition: [Size of a Test]

Let \(\phi\) be a test function for testing \(H_{0}: \theta\in \Theta_{0}\) against \(H_{1}: \theta\in \Theta_{1}\). Then the size of \(\phi\) is \(\sup_{\theta\in \Theta_{0}}\beta_{\phi}(\theta)\).

Example 2: Suppose \(X_{1}, \ldots, X_{10}\) be a random sample of size \(10\) from \(\mathtt{Bernoulli}(p)\), and consider the testing problem \(H_{0}:p=0.5\) against \(H_{1}:p=0.75\). Obviously, one would reject \(H_{0}\) if more number of heads appear. What would be the exact test procedure? Consider the following test procedures and the corresponding probabilities of type I and powers.

Test	Procedure	P(Type I error)	Power=1-P(Type II error)
\(\phi_{1}\)	Reject if 8 or more heads appear	\(\sum_{x=8}^{10} \binom{10}{x} (0.5)^{10}\) \(\approx 0.0547\)	\(\sum_{x=8}^{10} \binom{10}{x} (0.75)^{x} (0.25)^{10-x}\) \(\approx 0.5256\)
\(\phi_{2}\)	Reject if 9 or more heads appear	\(\sum_{x=9}^{10} \binom{10}{x} (0.5)^{10}\) \(\approx 0.0107\)	\(\sum_{x=9}^{10} \binom{10}{x} (0.75)^{x} (0.25)^{10-x}\) \(\approx 0.2440\)
\(\phi_{3}\)	Reject if 10 heads appear	\(\binom{10}{10} (0.5)^{10}\) \(\approx 0.001\)	\(\binom{10}{10} (0.75)^{10} \approx 0.0563\)

Remark 2: In view of the above remark, the generally recommended test procedure can be stated as follows: Suppose we want to test \(H_{0}: \theta \in \Theta_{0}\) against \(H_{1}: \theta \in \Theta_{1}\) at level \(\alpha\). Then we will only consider tests satisfying the level-\(\alpha\) conditions (\(\star\)), equivalently (\(\star \star\)). Then among the tests satisfying (\(\star\)), we will consider the test which has highest power.

Definition: [Most Powerful (MP) Test]

Suppose we are interested in testing \(H_{0}: \theta \in \Theta_{0}\) against \(H_{1}: \theta \in \Theta_{1}\) at level \(\alpha\), and \(\Phi_{\alpha}\) is the class of tests satisfying the level-\(\alpha\) condition (i.e., for any test \(\phi\in \Phi_{\alpha}\), (\(\star\star\)) is satisfied). A test \(\phi_{0}\in \Phi_{\alpha}\) is called most powerful test against an alternative \(\theta_{1}\in\Theta_{1}\) if \[ \beta_{\phi_{0}} (\theta_{1}) \geq \beta_{\phi} (\theta_{1})\quad \text{for all } \phi\in \Phi_{\alpha}.\]

Definition: [Uniformly Most Powerful (UMP) Test]

Suppose we are interested in testing \(H_{0}: \theta \in \Theta_{0}\) against \(H_{1}: \theta \in \Theta_{1}\) at level \(\alpha\), and \(\Phi_{\alpha}\) is the class of tests satisfying the level-\(\alpha\) condition. A test \(\phi_{0}\in \Phi_{\alpha}\) is called uniformly most powerful test if \[ \beta_{\phi_{0}} (\theta_{1}) \geq \beta_{\phi} (\theta_{1})\quad \text{for all } \phi\in \Phi_{\alpha}, \quad \text{unifromly in}~ \theta\in\Theta_{1} ~(\text{i.e., for all}~ \theta\in \Theta_{1}).\]

Example 2 (continued): Suppose we want to find the MP test for testing \(H_{0}\) against \(H_{1}\) in example 2 at level \(0.05\). Of course the test \(\phi_{1}\) does not satisfy the level condition. Both \(\phi_{2}\) and \(\phi_{3}\) satisfy the level condition. However, \(\phi_{2}\) has higher power than \(\phi_{3}\). So, \(\phi_{2}\) should be preferred over \(\phi_{3}\). Is \(\phi_{2}\) the most powerful test?

Suppose I consider a test as follows:

• If the number of heads is \(9\) or more, then \(H_{0}\) is rejected.

• If the number of heads is \(8\), then select a random number \(U\) from \(\text{Uniform}(0,1)\). If the realized value of \(U\), say \(u\), satisfies \(u<0.85\), then reject \(H_{0}\), otherwise accept \(H_{0}\).

• If the number of heads is \(7\) or less, then accept \(H_{0}\).

Does this test satisfy the level condition? What is the power of this test?

We may write the test in terms of a test function as follows: \[ \phi_{4}({\bf x})=\left\{ \begin{array}{lll} 1 &\quad \text{if}~ {\bf 1}^{\top}{\bf x}=9,10,\\ 0.85 &\quad \text{if}~ {\bf 1}^{\top}{\bf x}=8 \\0 & \quad \text{otherwise.} \end{array}\right.\] Checking the level condition: The probability of type I error is \[P(\text{Reject }H_{0}\mid H_{0})=P\left(\sum_{i=1}^{10} X_{i} \in \{9,10\}\mid p=0.5\right)+ P\left(\text{Reject}~H_{0}, \sum_{i=1}^{10} X_{i}=8 \mid p=0.5 \right)\] The last probability can be calculated as follows: \[ P\left(\text{Reject}~H_{0}, \sum_{i=1}^{10} X_{i}=8 \mid p=0.5, U< 0.85 \right)\times P(U< 0.85) + P\left(\text{Reject}~H_{0}, \sum_{i=1}^{10} X_{i}=8 \mid p=0.5, U\geq 0.85 \right)\times P(U \geq 0.85)\]

\[ =0.85 \times P\left(X_{i}=8 \mid p=0.5 \right). \] Thus, \(P(\text{Type I error})=0.0481\), which also satisfies the level condition.

Next, consider the power of the test. A similar calculation would lead to \[\text{Power}= 1- P\left( \text{Type II error}\right) = P\left(\text{Reject }H_{0} \mid H_{1} \right)=0.4834,\] which is much higher than the power of \(\phi_{2}\). Is \(\phi_{4}\) the MP test? No, because one may further adjust the test function for the case \(\sum x_{i}=8\) to get a better power, at the cost of reduced probability of type I error. One may continue to make adjustment as long as the level condition is valid.

Definition: [Randomized and Non-randomized Tests]

A test function of the form \(I_{C}({\bf x})\) is called a non-randomized test. Any other test function (a function from the sample space \(S_{x}\) to \([0,1]\)) corresponds to a randomized test.

Lecture 2: Neyman-Pearson Lemma

Theorem 1 (Neyman-Pearson Lemma)

Consider the problem of testing simple vs simple hypotheses, \(H_{0}:{\bf X} \sim f_{0}({\bf x})\) against \(H_{1}:{\bf X} \sim f_{1}({\bf x})\), using a test \(\phi\) with critical region \(C\) that satisfies \[ {\bf x} \in C \quad \text{if} \quad f_{1}({\bf x}) >k f_{0}({\bf x}), \quad \text{and} \quad {\bf x} \in \bar{C} \quad \text{if} \quad f_{1}({\bf x}) <k f_{0}({\bf x}),\tag{*}\] for some \(k\geq 0\), and \[\alpha = P({\bf X}\in C\mid H_{0})=E_{H_{0}} \left[ \phi({\bf X})\right].\tag{**}\]

• Any test that satisfies (*) and (**) is a UMP level \(\alpha\) test.

• If these exists a test satisfying (*) and (**) with \(k>0\), then every UMP level \(\alpha\) test is a size \(\alpha\) test, and every UMP level \(\alpha\) test satisfies (*) except perhaps on a set \(A\) where \(P({\bf X}\in A\mid H_{0})=P({\bf X}\in A\mid H_{1})=0\).

[Proof of Theorem 1]

Example 3: Let \(X\) be a random variable with p.m.f. under \(H_{0}\) and \(H_{1}\) are as follows. Find an UMP level \(\alpha=0.03\) test using Neyman-Pearson Lemma.

\(x\)	1	2	3	4	5	6
\(f_{0}(x)\)	0.01	0.01	0.01	0.01	0.01	0.95
\(f_{1}(x)\)	0.05	0.04	0.03	0.02	0.01	0.85

Example 4: Let \(X\sim N(0,1)\) under \(H_{0}\) and \(X\sim \mathtt{Cauchy}(0,1)\) under \(H_{1}\). Find an UMP level \(\alpha=0.05\) test using Neyman-Pearson Lemma.

Corollary 1:

Consider the hypothesis testing problem in Theorem 1, and let \(T({\bf X})\) be a sufficient statistic for \(\theta\) with \(g_{T}(t\mid\theta_{i})\) being the pdf or pmf of \(T({\bf X})\) under \(\theta_{i}\), \(i=0,1\). Then any test based on \(T\) with critical region \(S\) is a UMP level \(\alpha\) test if it satisfies \[t \in S \quad \text{if} \quad g_{T}(t\mid \theta_{1}) >k g_{T}(t\mid \theta_{0}), \quad \text{and} \quad t \in \bar{S} \quad \text{if} \quad g_{T}(t\mid \theta_{1}) <k g_{T}(t\mid \theta_{0}),\] for some \(k\geq 0\), and \[\alpha = P(T({\bf X})\in S\mid H_{0}).\] Example 5: Let \(X_{1},\ldots , X_{n}\) be a random sample from \(N(\mu,\sigma^{2})\). Test \(H_{0}: \mu=\mu_{0},\sigma=\sigma_{0}\) against \(H_{1}: \mu=\mu_{1},\sigma=\sigma_{0}\).

Example 6: Let \(X_{1},\ldots , X_{n}\) be a random sample from \(N(\mu,\sigma^{2})\). Test \(H_{0}: \mu=\mu_{0},\sigma=\sigma_{0}\) against \(H_{1}: \mu=\mu_{0},\sigma=\sigma_{1}\).

Example 7: Let \(X_{1},\ldots , X_{n}\) be a random sample from \(\mathrm{Poisson}(\lambda)\). Test \(H_{0}: \lambda=\lambda_{0}\) against \(H_{1}: \lambda=\lambda_{1}\).

Definition: [Test Statistic]

As the above corollary suggests, typically, a hypothesis test is specified in terms of the values of a statistic \(T({\bf X})\). This statistic is called test statistic.

Definition: [\(p\)-value]

The p-value is the probability, assuming the null hypothesis to be true, of seeing data at least as extreme as the experimental data. What ‘at least as extreme’ means depends on the experimental design. The smaller the \(p\)-value, the more extreme the outcome and the stronger the evidence against \(H_{0}\).

Lecture 3: Families with Monotone Likelihood Ratio:

Neyman-Pearson’s lemma prescribes a method of finding UMP test for simple vs simple hypothesis. The next step would be to find an UMP test for composite hypotheses. However, an UMP test for composite \(H_{0}\) and/or \(H_{1}\) may not exist in general. Consider for example the testing problem \(H_{0}: X\sim \mathrm{Cauchy}(0,1)\) against \(H_{1}: X\sim \mathrm{Cauchy}(\theta,1)\) for \(\theta>0\). The graph of \(g(x,\theta)=f_{0}(x)/f_{\theta}(x)\) is given below:

For any alternative \(\theta_{1}>\theta_{0}\) there exist a separate MP test (depending on \(\theta_{1}\)). Thus, an UMP test does not exist.

In what follows, we will consider a special class of distributions for which a UMP test of one-sided hypotheses exists.

Definition: [Monotone Likelihood Ratio]

A family of distributions with pdf/pmf \(\{f_{\bf X}(; \theta),\theta\in\Theta\subseteq \mathbb{R}\}\) is said to have a monotone likelihood ratio (MLR) in a statistic \(T({\bf x})\) if for \(\theta_{1}<\theta_{2}\) the ratio \(f_{\bf X}({\bf x}; \theta_{1})/f_{\bf X}({\bf x}; \theta_{2})\) is a monotonically non-increasing (or non-decreasing) function of \(T({\bf x})\) for the set of values \({\bf x}\) for which at least one of \(f_{\bf X}({\bf x}; \theta_{1})\) and \(f_{\bf X}({\bf x}; \theta_{2})\) is positive.

Example 8: \(\mathtt{Uniform}(0,\theta), ~\theta>0\) distribution is MLR in \(T({\bf X})=X_{(n)}\).

The MLR family is large enough to include the exponential class of distributions. The following theorem states that result.

Theorem 2

The one-parameter exponential family \(f_{\bf X}({\bf x}; \theta)=\exp\left\{ Q(\theta) T({\bf x})+ S({\bf x}) +D(\theta)\right\}\) where \(Q(\theta)\) is non-decreasing has MLR in \(T({\bf x})\).

[Proof of Theorem 2]

Theorem 3

Let \(X_{1},\cdots, X_{n}\) be a random sample with some distribution with pdf/pmf \(\left\{f_{X}(\cdot ; \theta): \theta\in \Theta \right\}\), which has MLR in \(T({\bf X})\). For testing \(H_{0}:\theta\leq\theta_{0}\) against \(H_{1}:\theta>\theta_{0}\), \(\theta_{0}\in \Theta\), and test of the form \[ \phi({\bf x}) =\left\{\begin{array}{lll} 1 & \text{if}~~ T({\bf x}) >t_{0} \\ \gamma & \text{if}~~ T({\bf x}) =t_{0} \\ 0 & \text{if}~~ T({\bf x}) <t_{0}\end{array} \right. , \tag{***}\] has a non-decreasing power function and is UMP of its size \(E_{\theta_{0}} \left[\phi({\bf X}) \right]=\alpha\).

[Without Proof]

Remark: 1. If a family of pdfs/pmfs \(\{f_{X}(x;\theta), \theta\in \Theta\}\) has an MLR in \(T\), then the corresponding family of CDFs of \(T\) is stochastically increasing in \(\theta\).

2. Remark 1 implies that the power function of the test in Theorem 3 is monotone in \(\theta\). This property plays the key role in the proof of Theorem 3 (see Casella Berger).

3. If for \(\theta_{1}<\theta_{2}\), the ratio \(f_{\bf X}({\bf x};\theta_{1})/f_{\bf X}({\bf x};\theta_{2})\) is monotonically non-decreasing in \(T\), then a test of the form \[\phi^{\prime}({\bf x}) =\left\{\begin{array}{lll} 1 & \text{if}~~ T({\bf x}) <t_{0} \\ \gamma & \text{if}~~ T({\bf x}) =t_{0} \\ 0 & \text{if}~~ T({\bf x}) >t_{0}\end{array} \right. , \tag{\#\#\#}\] is an UMP for testing \(H_{0}:\theta\leq \theta_{0}\) against \(H_{1}:\theta > \theta_{0}\) with size size \(E_{\theta_{0}} \left[\phi({\bf X}) \right]=\alpha\).

   In general, there are four possible situations that may arise, w.r.t. the relation of \(T\) in \(f_{X}(\cdot; \theta)\) and the testing problem. The following table indicates the UMP test in all these situations.

   	\(\lambda({\bf x})\) is non-decreasing in \(T\)	\(\lambda({\bf x})\) is non-increasing in \(T\)
   \(H_{0}:\theta\leq \theta_{0}\) vs \(H_{1}:\theta>\theta_{0}\)	\(\phi^{\prime}\) as in (###) is an UMP	\(\phi\) as in (***) is an UMP
   \(H_{0}:\theta\geq \theta_{0}\) vs \(H_{1}:\theta<\theta_{0}\)	\(\phi\) as in (***) is an UMP	\(\phi^{\prime}\) as in (###) is an UMP

   Here \(\lambda({\bf x})=f_{\bf X}({\bf x}; \theta_{1})/f_{\bf X}({\bf x}; \theta_{2})\) where \(\theta_{1}<\theta_{2}\).

Example 8 (continue): Let \(X_{1},\ldots , X_{n}\) be a random sample from \(\mathtt{Uniform}(0,\theta)\). Find an UMP test for testing \(H_{0}: \theta\leq \theta_{0}\) against \(H_{1}: \theta>\theta_{0}\).

Example 9: Let \(X_{1},\ldots , X_{n}\) be a random sample from \(\mathrm{Location-scale~exponential}(\mu,\sigma_{0})\). Find an UMP test for testing \(H_{0}: \mu\geq \mu_{0},~\sigma=\sigma_{0}\) against \(H_{1}: \mu< \mu_{0},~\sigma=\sigma_{0}\).

Lecture 4: Likelihood Ratio Test:

In Neyman-Pearson lemma, we have seen that for simple vs simple testing problem, the most powerful test is based on the ratio \(\lambda({\bf x})=f_{1}({\bf x})/f_{0}({\bf x})\). We can interpret the ratio \(\lambda({\bf x})\) as the best possible explanation of occurrence of \({\bf x}\) under \(H_{1}\) and under \(H_{0}\). If occurrence of \({\bf x}\) is much more likely under \(H_{1}\) than that under \(H_{0}\), i.e., \(\lambda({\bf x})\) is large, then \(H_{0}\) must be rejected. We generalize this idea in this section.

Likelihood ratio test:

• Consider the problem of testing \(H_{0}:\theta \in \Theta_{0}\) against \(H_{1}:\theta \in \Theta_{1}\). Let \(\theta\in \Theta\subseteq \mathbb{R}^{k}\), and \({\bf X}\) be a random vector with joint pdf/pmf \(f_{\bf X}(\cdot ; \theta)\). Then given the realization \({\bf x}\) of \({\bf X}\) the likelihood function of \(\theta\) is \(L(\theta\mid {\bf x})\), and the generalized likelihood ratio (GLR) is \[\lambda({\bf x}) = \frac{\sup_{\theta\in\Theta_{0}} L(\theta\mid {\bf x})}{\sup_{\theta\in\Theta} L(\theta\mid {\bf x})}. \tag{\#} \]
• The numerator of (#) can be interpreted as the best possible explanation of the realization \({\bf x}\) under \(H_{0}\), and the denominator can be interpreted as the best possible explanation of \({\bf x}\) (without any constraint).
• The ratio \(\lambda({\bf x})\) lies in \([0,1]\), and a small value indicates that much better explanation of the realization is available when \(\theta\) lies outside \(\Theta_{0}\) (i.e., when \(H_{0}\) is not true). Therefore, \(H_{0}\) is rejected when \(\lambda({\bf x})\) is small, i.e., \(\lambda({\bf x})<c\).
• As usual, how small a value of \(\lambda({\bf x})\) is needed in order to reject \(H_{0}\) is determined by the level condition. Therefore, when the level of significance is \(\alpha\), we choose \(c\) such that \[ \sup_{\theta\in\Theta_{0}} E_{\theta} \left( \phi({\bf X}) \right) =\alpha. \tag{\#\#} \]

Example 10: Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Normal}(\mu,\sigma^{2})\) . Find the LRT for testing \(H_{0}:\mu = \mu_{0}\) against \(H_{1}:\mu\neq \mu_{0}\), where \(\sigma^{2}\) is unknown.

Example 11: Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Binomial}(m,p)\) . Find the LRT for testing \(H_{0}:p\leq p_{0}\) against \(H_{1}:p>p_{0}\), where \(p_{0}\in(0,1)\).

Properties of LRT:

Theorem 4

If for a given \(\alpha\in [0,1]\), non-randomized Neyman Pearson test and LRT exist for a simple vs simple hypothesis testing problem (i.e., \(\Theta_{0}=\{\theta_{0}\}\), \(\Theta_{1}=\{\theta_{0}\}\), and \(\Theta=\{\theta_{0},\theta_{1}\}\)), then both the tests are equivalent.

Theorem 5

For testing \(H_{0}:\theta\in\Theta_{0}\) against \(H_{1}:\theta\in\Theta_{1}\), the LR test statistic \(\lambda({\bf x})\) is a function of every sufficient statistic.

Theorem 6

Under some regularity conditions on the underlying class if distributions, the random variable \(T({\bf X})= -2 \log \lambda({\bf X})\) is asymptotically distributed as a chi-square distribution with degrees of freedom equal to the difference between the number of independent parameters in \(\Theta\) and that in \(\Theta_{0}\).

[Without Proof]

Some Important Testing Problems

1. One-sample/Two-sample \(t\)-tests:

   Example 10 (One-sample \(t\)-test): Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Normal}(\mu,\sigma^{2})\). Find the LRT for testing \(H_{0}:\mu=\mu_{0}\) against \(H_{1}:\mu\neq \mu_{0}\).

   Example 12 (Two-sample \(t\)-test): Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Normal}(\mu_{1},\sigma_{1}^{2})\) distribution, and \(Y_{1},\cdots, Y_{m}\) be a random sample from \(\mathtt{Normal}(\mu_{2},\sigma_{2}^{2})\) distribution. Consider the problem of testing \(H_{0}:\mu_{1}=\mu_{2}\) against \(H_{1}:\mu_{1}\neq \mu_{2}\), given \(\sigma_{1}^{2}=\sigma_{2}^{2}=\sigma^{2}\).

2. One-sample/Two-sample \(\chi^{2}\)-tests:

   Example 13 (One-sample \(\chi^{2}\)-test): Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Normal}(\mu,\sigma^{2})\). Find the LRT for testing \(H_{0}:\sigma^{2}=\sigma_{0}^{2}\) against \(H_{1}:\sigma^{2}\neq \sigma_{0}^{2}\).

   Example 14 (Two-sample \(F\)-test): Let \(X_{1},\cdots, X_{n}\) be a random sample from \(\mathtt{Normal}(\mu_{1},\sigma_{1}^{2})\) distribution, and \(Y_{1},\cdots, Y_{m}\) be a random sample from \(\mathtt{Normal}(\mu_{2},\sigma_{2}^{2})\) distribution. Consider the problem of testing \(H_{0}:\sigma_{1}^{2}=\sigma_{2}^{2}\) against \(H_{1}:\sigma_{1}^{2} \neq \sigma_{2}^{2}\).

3. LRT for regression coefficients:

   Example 15: Let \(Y_{1},\cdots, Y_{n}\) be a random sample from \(\mathtt{Normal}(\mu_{i},\sigma^{2})\) distribution, where \(\mu_{i} = \beta_{0}+\beta_{1}x_{1,i}+ \ldots + \beta_{p} x_{p,i}\), for \(i=1,\ldots,n\), \(p<n\). Find the LRT for testing \(H_{0}:\beta_{1}=\cdots=\beta_{p}=0\) against \(H_{1}: H_{0}\) is not true.