Question #1

Conditional probability: suppose that if θ = 1, then y has a normal distribution with mean 1 and standard deviation σ, and if θ = 2, then y has a normal distribution with mean 2 and standard deviation σ. Also, suppose Pr(θ = 1) = 0.5 and Pr(θ = 2) = 0.5.

(a) For σ = 2, write the formula for the marginal probability density for y and sketch it.

\[p(y)= \Sigma_ \theta p(y,\theta)=p(\theta=1)p(y|\theta=1)+p(\theta=2)p(y|\theta=2)=0.5 \times N(1,4) + 0.5 \times N(2,4)\]

sd=2
mean1=1
mean2=2
x <- seq(-4,4,length.out = 10000)*sd + 1.5
y <- 0.5*dnorm(x,mean1,sd) + 0.5*dnorm(x,mean2,sd)
plot(x,y)

(b) What is Pr(θ = 1|y = 1), again supposing σ = 2?

\[ Pr(\theta=1|y=1)= \frac{p(y=1|\theta=1)p(\theta=1)}{p(y=1)} \]

a <- dnorm(1,mean1,sd)
b <- 0.5*dnorm(1,mean1,sd) + 0.5*dnorm(1,mean2,sd)
c <- (a*0.5)/b
print (a)
## [1] 0.1994711
print (b)
## [1] 0.1877519
print (c)
## [1] 0.5312094

\[ Pr(\theta=1|y=1)=\frac{0.1994711 \times 0.5}{0.1877519}=0.5312094 \] (c) Describe how the posterior density of θ changes in shape as σ is increased and as it is decreased.

Using the code below, I am trying to examine effect of σ on the shape of posterior density of θ:

sds <- c(0.5,2,4)
theta1=1
theta2=2
labs <- c("sd=0.5","sd=2","sd=4")
cls <- c("black","red","blue")
x <- seq(-10,10,length.out = 10000)*sd + 1.5
y <- 0.5*dnorm(x,theta1,sds[1]) + 0.5*dnorm(x,theta2,sds[1])
th_cond_y <- (dnorm(x,theta1,sds[1])*0.5)/y
plot(x,th_cond_y,ylab = "density",type = "l",lty=1,lwd=2,main = "Comparison of different amounts of SD")
for (i in 2:3){
  y <- 0.5*dnorm(x,theta1,sds[i]) + 0.5*dnorm(x,theta2,sds[i])
  th_cond_y <- (dnorm(x,theta1,sds[i])*0.5)/y
  lines(x,th_cond_y,col=cls[i])
}

legend("topright",col=cls,lty=1,labs,title = "Standard Deviations")

As we can see in the plot above, increase in σ causes the density to be more and more similiar to uniform distribution. it is because when σ increases data has less and less information about the distribution.

Question #6

Conditional probability: approximately 1/125 of all births are fraternal twins and 1/300 of births are identical twins. Elvis Presley had a twin brother (who died at birth). What is the probability that Elvis was an identical twin? (You may approximate the probability of a boy or girl birth as 1/2)

\[P(t_B):\ probability\ of\ being\ twin\ brothers\\ P(i):\ probablity\ of\ being\ identical\ twins\\ P(ni):\ probablity\ of\ being\ non-identical\ twins\]

\[ P(i|t_B)=\frac{P(t_B|i)P(i)}{P(t_B)}=\frac{P(t_B|i)P(i)}{P(t_B|i)P(i) + P(t_B|ni)P(ni)}=\frac{\frac{1}{2}\times \frac{1}{300}}{\frac{1}{2}\times \frac{1}{300} + \frac{1}{4}\times \frac{1}{125}}=\frac{5}{11} \]