Here are some things to begin with:
\(E(\hat{\beta}_1) = \beta_1\) and
\(Var(\hat{\beta}_1) = \frac{\sigma^2}{\sum_{i=1}^n(X_i-\bar{X})^2}\)
The proofs in this class are rather rigorous, so I will not cover them all as they can require quite a bit of explanation. The proofs can be found here: https://onlinecourses.science.psu.edu/stat414/node/280/
We would like to do a confidence interval for \(\hat{\beta}_1\), and we can do this by testing the hypothesis:
\(H_0: \hat{\beta}_1=\beta_1\) vs \(H_1: \hat{\beta}_1 \neq \beta_1\)
The test statistic for the above hypothesis is:
\(\frac{\hat{\beta}_1-\beta_1}{s.e.(\hat{\beta}_1)} \sim t_{n-2}\)
You may be wondering why it follows a t-distribution, but the proof can be lengthy. Here is a link to a proof of why this is the case: https://onlinecourses.science.psu.edu/stat414/node/280/ (same as the previous link). Also, it is n-2 because we are estimating two parameters, \(\beta_0\) and \(\beta_1\).
The standard error is the square root of the estimate of the variance (2):
\(s.e.(\hat{\beta}_1)=\frac{\hat{\sigma}}{\sqrt{\sum_{i=1}^n(X_i - \bar{X})^2}}\)
So a confidence interval may appear like:
\(P(-t_{n-2,(1-\frac{\alpha}{2})} \leq \frac{\hat{\beta}_1-\beta_1}{s.e.(\hat{\beta}_1)} \leq t_{n-2,(1-\frac{\alpha}{2})}) = 1-\alpha\)
\(\Rightarrow P(\hat{\beta}_1-t_{n-2,(1-\frac{\alpha}{2})} s.e.(\hat{\beta}_1) \leq \beta_1 \leq \hat{\beta}_1+t_{n-2,(1-\frac{\alpha}{2})} s.e.(\hat{\beta}_1)) = 1-\alpha\)
The \(\beta_1\) in the center is a constant which represents the ‘true’ parameter for the entire population. In general, we don’t really know what it is, which is why we are estimating it with a confidence interval. The values on the left and right of \(\beta_1\) are considered random variables, and they can change depending on our sample of data.
An important bit of intuition going forward is that what the confidence interval is saying is that (given \(\alpha=0.05\)) if we repeated the experiment many times, 95% of intervals will contain the true value. This is different than saying for example \(P(20 \leq \beta \leq 30) = 0.95\). The statement \(P(20 \leq \beta \leq 30)\) is a true or false statement.