\[ % for commands \newcommand{\Pr}{\mathbf{Pr}} \] 9.15, 9.17, 9.18, 9.22, 9.25, 9.26, 9.28, 9.34, 9.41, 9.53

15.

Suppose that under \(H_0\) a measurement \(X\) is \(N(0, \sigma^2)\) and that under \(H_1\), \(X\) is \(N(1, \sigma^2)\) and that the prior porbability \(P(H_0) = P(H_1)\). For \(\sigma = 1\) and \(x\in [0,3]\), plot and compare (1) the \(p\)-value for the test \(H_0\) and (2) \(P(H_0 |\; x)\). Can the \(p\)-value be interpreted as the probability that \(H_0\) is true? Choose another value of \(\sigma\) and repeat.

Solution `

By definition, we have that the p-value is the smallest significance level at which the null is rejected, or better put the probability under the null of observing a value as or more extreme.

# set some global params ------------------------------------------------------
sigma <- 1
x <- seq(0,3, by = .1)
h0 <- data.frame(x, fx = dnorm(x, mean = 0, sd = 1))
h1 <- data.frame(x, fx = dnorm(x, mean = 1, sd = 1))

# plots -----------------------------------------------------------------------
# plot of the p-value for the test H_0
p1 <- ggplot(data = h0, aes(x, fx)) + geom_line() + geom_vline(xintercept = 2, color = "red")
p1
#grid.arrange(p1, p2, ncol=2)

17.

Let \(X \sim N(0, \sigma^2)\) and consider testing \(H_0: \sigma_1 = \sigma_0\) versus \(H_A: \sigma = \sigma_1\) where \(\sigma_1 > \sigma\). The values \(\sigma_0\) and \(\sigma_1\) are fixed.

  1. What is the likelihood ratio as a function of \(x?\) What values favor \(H_0?\) What is the rejection region of a level \(\alpha\) test?

  2. For a sample \(X_1, X_2, ..., X_n\) distributed as above repeat the previous question.

  3. Is the test in the previous question uniformly most powerful for testing \(H_0: \sigma = \sigma_0\) versus \(H_1: \sigma > \sigma_0?\)

Solutions

  1. The likelihood ratio is given by: \[T(x) = \frac{f_0(x)}{f_1(x)} = \frac{\frac{1}{\sigma_0\sqrt{2\pi}}\exp\Big[\frac{-x^2}{2\sigma_0^2}\Big]}{\frac{1}{\sigma_1\sqrt{2\pi}}\exp\Big[\frac{-x^2}{2\sigma_1^2}\Big]}\] A bit of simplication yields: \[T(x) = \frac{\sigma_0}{\sigma_1}\exp\Big[\frac{x^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)\Big]\] we have that both \(\sigma_0\) and \(\sigma_1\) are fixed so indeed this is the likelihood ratio as a function of \(x\). To find the values that would favor \(H_0\) we would be looking at the ratio for values that are larger than some constant \(c\). We can solve for this as follows: \[T(x) = \frac{\sigma_0}{\sigma_1}\exp\Big[\frac{x^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)\Big] > c\] we can take the \(\log\) to simplify \[\frac{x^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)>c^*\] note that terms being switched to the right hand side are just constant terms that get accumulated into what I call \(c^*\) \[x^2\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)>c^*\] \[x^2 < c^*\] the switch of the inequality occured from the fact that \(\sigma_1 > \sigma_0\). And so we have that small values of \(x^2\) would favor \(H_0\). In order to find the rejection region at a given \(\alpha\) level of significance we would solve for \(c\) such that \[\Pr_0(x^2 > c) = \alpha\]

  2. Let \(X_1, ..., X_n\) be distributed as above, we can find the likelihood ratio function in very much the same way, \[T(x) = \frac{f_0(\mathbf{X})}{f_1(\mathbf{X})} = \frac{\Big(\frac{1}{\sigma_0\sqrt{2\pi}}\Big)^n\exp\Big[\frac{-\sum x_i^2}{2\sigma_0^2}\Big]}{\Big(\frac{1}{\sigma_1\sqrt{2\pi}}\Big)^n\exp\Big[\frac{-\sum x_i^2}{2\sigma_1^2}\Big]}\] we can then eliminate terms to yield: \[T(\mathbf{X}) = \Big(\frac{\sigma_1}{\sigma_0}\Big)^n \exp\Big[\frac{\sum X_i^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)\Big]\] This is the likelihood ratio as a function of \(\mathbf{X}\). To find the values that would favor \(H_0\) we would look to the ratio and find what values of \(\mathbf{X}\) yield \(T(\mathbf{X}) > c\). We can solve for this as follows: \[T(\mathbf{X}) = \Big(\frac{\sigma_1}{\sigma_0}\Big)^n \exp\Big[\frac{\sum X_i^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)\Big] > c\] \[\exp\Big[\frac{\sum X_i^2}{2}\Big(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_0^2} \Big)\Big] > c^*\] \[\sum X_i^2 < c^*\] and so small values of \(\sum X_i^2\) would favor \(H_0\). And the rejection region would be \[\Pr_0\Big(\sum X_i^2 > c \Big) = \alpha\]

  3. The test is indeed UMP, since the likelihood ratio does not depend on \(\sigma_1 > \sigma_0\).

18.

Let \(X_1, X_2, ..., X_n\) be i.i.d random variables from a double exponential distribution with density \(f(x) = \frac{1}{2}\lambda \exp(-\lambda |x|)\). Derive a likelihood ratio test of the hypothesis \(H_0: \lambda = \lambda_0\) versus \(H_1: \lambda = \lambda_1\) where \(\lambda_0\) and \(\lambda_1 > \lambda_0\) are specified numbers. Is the test uniformly most powerful against the alternative \(H_1: \lambda > \lambda_0?\)

Solution

To develop the likelihood ratio test we first write out the likelihood ratio as follows: \[T(\mathbf{X}) = \frac{f_0(\mathbf{X})}{f_1(\mathbf{X})} = \frac{f_0(X_1)\cdots f_0(X_n)}{f_1(X_1)\cdots f_1(X_n)} = \frac{\frac{1}{2}\lambda_0 \exp[-\lambda_0\sum |X_i|]}{\frac{1}{2}\lambda_1 \exp[-\lambda_1\sum |X_i|]}=\] \[\Big(\frac{\lambda_0}{\lambda_1}\Big)^n \exp \Big[ -\lambda_0\sum |X_i| + \lambda_1\sum |X_i| \Big] = \] \[\Big(\frac{\lambda_0}{\lambda_1}\Big)^n\exp\Big[(\lambda_1 - \lambda_0) \sum |X_i|\Big]\] Now since we have that \(\lambda_0, \lambda_1\) and \(n\) are fixed numbers then the only variable is of course \(\sum |X_i|\), we also note that the function \(T(X)\) is monotonically decreasing for small values of \(\sum |X_i|\) so rather than looking at the entire ratio we can focus only on \(\sum |X_i|\), and reject for values smaller then some \(k\).

Now focusing on the sum of double exponentials since we are looking at the absolute value of these means that we are only using the positive half-line of the double exponential or rather, we have regular exponentials scaled by 1/2.

The test is UMP since the rejection region and the test do not depend on \(\lambda_1 > \lambda_0\).

22.

In Example A of Section 8.5.3 a confidence interval for the variance of a normal distribution was derived. Use Theorem B of Section 9.3 to derive an acceptance region for testing the hypothesis \(H_0: \sigma^2 = \sigma_0^2\) at the significance level \(\alpha\) based on a sample \(X_1, X_2, ..., X_n\). Precisely describe the rejection region if \(\sigma_0 = 1, n = 15, \alpha = .05\).

Solution

Using Theorem B from Section 9.3 we have that the acceptance region at a level \(\alpha\) for a parameter is the region defined by the values that are within the a \((1 - \alpha)\times 100 \%\) confidence interval of this parameter.

In the example this region is given to be \[\Big(\frac{n\sigma_0^2}{\chi_{n-1}^2(\alpha/2)}\Big),\frac{n\sigma_0^2}{\chi_{n-1}^2(1 - \alpha/2)}\] We can calculate for the given values using R

theta <- 1; n <- 15; a <- .05
l <- (n * theta^2)/(qchisq(p = (1 - a/2), df = n-1))
u <- (n * theta^2)/(qchisq(p = (a/2), df = n-1))
acceptance_R <- c(l,u)

And we have that the acceptance region is given by 0.5742957, 2.6649014

25.

Calculate the likelihood ratio for Example B of Section 9.5 and compare the results of a test based on the likelihood ratio to those of one based on Pearson’s chi-square statistic.

Solution

freq <- c(56,104, 80, 62, 42, 27, 9, 20)

# likelihood function just to make sure i can replicate in class example
like_f <- function(l) {
  sum_l <- dmultinom(x = freq, size = sum(freq), prob = c(dpois(0:6, lambda = l), 1 - ppois(6, l)))
  return (sum_l)
}
lam_o <- optimize(like_f, c(1,10), maximum = TRUE)
lam <- lam_o$maximum
# but just kidding we know that mle of lambda is 2.44 
lam <- 2.44
expect_this <- sum(freq) * c(dpois(0:6, lambda = lam), 1 - ppois(6, lam))
and_we_get_this <- 2 * sum(freq * log(freq/expect_this))

54.6187149 so it rejects just like the Pearson version.

26.

True of False.

  1. True

  2. False

  3. True

  4. False

  5. False

  6. False

28.

Suppose that a test statistic \(T\) has a standard normal null distribution.

  1. If the test rejects for large values of \(|T|\), what is the p-value corresponding to \(T = 1.50\)

  2. Answer the same question if the test rejects for large \(T\).

Solution

  1. By definition of the p-value it is the probability of observing a value as extreme or more. So we calculate \[\Pr(|T| > 1.50) = 2[1 - \Pr( T \leq 1.5)]\]
2 * (1 - pnorm(1.50))
## [1] 0.1336144

b . In this case we have \[\Pr(T > 1.50) = 1 - \Pr(T \leq 1.50)\]

1 - pnorm(1.5)
## [1] 0.0668072

34.

Test the goodness of fit of the data to the genetic model given in Problem 55 of Chapter 8.

Solution

The data is given from the problem along with the respective cell probabilties:

Type Count Cell Probability
Starchy Green (StG) 1997 \(.25(2+\theta)\)
Starchy White (StW) 906 \(.25(1-\theta)\)
Sugary Green (SgG) 904 \(.25(1-\theta)\)
Sugary White (SgW) 32 \(.25\theta\)

We can use R to find the mle of \(\theta\)

freq_34 <- c(1997, 906, 904, 32)

# likelihood function just to make sure i can replicate in class example
like_f_34 <- function(t) {
  sum_l_34 <- dmultinom(x = freq_34, size = sum(freq_34), prob = c((.25*(2+t)),(.25*(1-t)),(.25*(1-t)),(.25*t)))
  return (sum_l_34)
}
theta_o <- optimize(like_f_34, c(0, 1), maximum = TRUE); theta_o
## $maximum
## [1] 0.03571143
## 
## $objective
## [1] 6.249532e-06
t <- theta_o$maximum
# some calculations now 
this_time_we_expect <- sum(freq_34) * c((.25*(2+t)),(.25*(1-t)),(.25*(1-t)),(.25*t))
to_get <- 2 * sum(freq_34 * log(freq_34/this_time_we_expect)); to_get
## [1] 2.018721
whose_p_value_is <- 1 - pchisq(q=to_get, df=2)

And so the fit is not rejected.

41.

Let \(X \sim bin(n_i, p_i)\), for \(i = 1, ..., m\) be independent. Derive a likelihood ratio test for the hypothesis \[H_0: p_1 = p_2 = \cdots = p_m\] against the alternative hypothesis that the \(p_i\) are not all equal. What is the large sample distribution of the test statistic?

Solution

The mle of the unrestricted mle is given by \(\hat{p}_i = X_i/n_i\) and for the null the mle is given by \(\hat{p}_0 = \sum X_i / \sum n_i\) and so we can form the likelihood ratio to be \[\Lambda = \frac{\prod p_0^{\sum X_i}(1 - p_0)^{n_i - X_i}}{\prod p_i^{\sum X_i}(1 - p_i)^{n_i - X_i}}\]

53.

Given the number of points this is pretty staright in my opinion.

Really straight I think.