Worked with Brogan Pietrocini, Kaden Buckley, and Ryan Perez

Problem 1

You have a brand new coin and you want to estimate \(p\), the probability that it lands on heads. You flip the coin \(n\) times and observe \(X\) heads.

  1. Suggest an estimator of \(p\); it should be an expression involving \(X\) and \(n\). Describe in words what your estimator is. (Hint: what is the most obvious estimator you can think of?)

    An estimator of p would be p=X/n. It would just be the total heads flipped over the total amount of coin flips.

  2. Suppose you observe \(x = 6\) heads in \(n=10\) flips. Use your estimator to compute an estimate of \(p\) based on this sample.

    p=X/n

    p=6/10=0.6

  3. Consider the estimator (called the “plus four” estimator) \[ \frac{X + 2}{n + 4} = \left(\frac{4}{n+4}\right)\left(0.5\right) + \left(\frac{n}{n+4}\right)\left(\frac{X}{n}\right) \] Explain in words what this estimator does. (The two expressions in the equation above are just different ways of expressing the estimator.)

    When you use a plus four estimator, you modify the estimator of p. This is done by adding 4 to the total number of observations, and adding 2 hypothetical successes and failures. The end result is that we replace all instances of X/n with (X+2)/(n+4). A reason to do this would be a small sample size like the one we have in the previous question.

  4. Use the estimator from the previous part and compute an estimate of \(p\) based on a sample with 6 heads in 10 flips.

    p=(X+2)/(n+4)

    p=(6+2)/(10+4)

    p=8/14=0.571

  5. If you observe 6 heads in 10 flips, which of your two estimates is a better estimate of \(p\)? Explain.

    The plus four estimator would be a better estimate of p in this case, as we would expect to see 50% of the coins landing on heads, as that is the probability each time you flip. The plus four estimator of p (0.571) is closer to 50% than the initial estimate of p (0.6).

  6. Compute and interpret the probability of observing 6 heads in 10 flips if \(p = 0.6\).

    dbinom(6, 10, 0.6)
    ## [1] 0.2508227

    The probability that you observe 6 heads in 10 flips when p=0.6 is 0.251.

  7. Compute and interpret the probability of observing 6 heads in 10 flips if \(p = 0.571\).

    dbinom(6, 10, 0.571)
    ## [1] 0.2465274
    1. The probability that you observe 6 heads in 10 flips when p=0.571 is 0.247.

  8. Compare the values from the two previous parts. If you observed 6 heads in 10 flips, which of the values — 0.6 or 0.571 — would you choose as your estimate of \(p\)? Explain your reasoning.

    Although the estimator 0.6 has a higher probability than 0.571 for this specific case, I would still use the 0.571 estimator because it is closer to the long run probability of 0.5.

  9. Suppose you observe 6 heads in 10 flips. Carefully write the likelihood function.

    L(p) = p^6*(1-p)^(10-6)

  10. Continuing the previous part, use any software of your choice and plot the likelihood function. Based on your plot, what value appears to be the maximum likelihood estimate of \(p\) based on a sample with 6 heads in 10 flips? (You can do this just by zooming in on the plot.)

    p <- seq(0, 1, .001)
likelihood = p^6*(1-p)^(10-6)
plot(p,likelihood)

    It looks like the maximum likelihood function is around 0.6.

  11. Continuing the previous part, carefully write the log-likelihood function.

    L(p | x) = 6 * log(p) + 4 * log(1 - p)

  12. Use calculus to find the maximum likelihood estimate of \(p\) based on observing 6 heads in 10 flips.

    6 * (1/p) - 4 * (1/(1 - p)) = 0

    6 * (1 - p) - 4 * p = 0

    6 - 6p - 4p = 0

    -10p = -6

    p = 6 / 10

    p = 0.6

  13. Now suppose you observe \(x\) heads in 10 flips. Carefully write the likelihood function.

    L(p) = p^x*(1-p)^(10-x)

  14. Continuing the previous part, carefully write the log-likelihood function.

    L(p | x) = x * log(p) + (10-x) * log(1 - p)

  15. Use calculus to find a general expression for the maximum likelihood estimate of \(p\) based on observing \(x\) heads in 10 flips. (Your answer will be an expression involving \(x\).)

    p=X/n

Problem 2

Suppose that times (minutes) between earthquakes (of any magnitude in a certain region) follow an Exponential distribution with rate parameter \(\lambda\). Let \(X_1, \ldots, X_n\) be a random sample of times (minutes) between earthquakes for \(n\) earthquakes. It can be shown that the MLE of \(\lambda\) is \(1/\bar{X}\).

  1. Explain intuitively why \(1/\bar{X}\) is a reasonable estimator of \(\lambda\). Hint: it’s not just because it’s the MLE; think of properties of Exponential distributions.

    The interval is from 0 to infinity, so we need a rate parameter that is able to used to infinity. As X grows exponentially, 1/Xbar rapidly declines.

  2. Suppose the observed sample is: 20, 37, 13, 10, 25 minutes. Carefully write the likelihood function. (For a random sample from a continuous distribution, to find the likelihood function you plug each observed value into the pdf and then form the product.)

    L(u)=((e-uu20)/20!)) * ((e-u*u37)/(37!)) * ((e-u*u13)/(13!)) * ((e-u*u10)/(10!)) * ((e-u*u25)/(25!))

    L(u)=((e-5*u) * u(20+37+13+10+25)/(20! 37! 13! 10! 25!)

  3. Use any software you want to carefully plot the likelihood function from the previous part. Then use your plot to verify that the MLE of \(\lambda\) based on this sample is \(1/\bar{x} = 5/105 = 0.0476\). (You can just zoom in on the plot.)

    u <- seq(0, 100, .1)

#Likelihood Function
l = exp(-5*u) * u^(20 + 37 + 13 + 10 + 25) / (factorial(20)*factorial(37)*factorial(13)*factorial(10)*factorial(25))

#Plot function using the p sequence from 0 to 1
plot(u,l)

    The estimated mean was 21, therefore the estimate of \(\lambda\) is 1/21 which is equal to 5/105=0.0476.

  4. Write a clearly worded sentence reporting in context your estimate of \(\lambda\) from the previous part.

    Based on this exponential distribution, there’s 0.0476 minutes between each earthquake.