\(Y_{i}=\beta_{0}+\beta_{1}x_{i}+e\)

\[\frac{\partial S}{\partial \beta_{0}}= \sum(Y_{i}-\beta_{0}-\beta_{1}x_{i})(-2)=0 \rightarrow \sum Y_{i}=n \beta_{0}+\beta_{1}\sum x_{i}\]

\[\frac{\partial S}{\partial \beta_{1}}= \sum(Y_{i}-\beta_{0}-\beta_{1}x_{i})(-2x_{i})=0 \rightarrow \sum x_{i}Y_{i}=\beta_{0}\sum x_{i}+\beta_{1}\sum x^2_{i}\]

1.회귀분석의 정규방정식

\[\begin{vmatrix} \ n & \sum x_{i} \\ \sum x_{i} & \sum x^2_{i}\\ \end{vmatrix} \begin{vmatrix} \beta_{0}\\ \beta_{1}\\ \end{vmatrix}=\begin{vmatrix} \sum Y_{i}\\ \sum x_{i}Y_{i}\\ \end{vmatrix}\]

2. \(\beta_{1}\)구하기

정규방정식을 활용

\[ \sum x_{i}(n \beta_{0}+\sum x_{i}\beta_{1})=\sum x_{i}\sum Y_{i}\]

\[ n( \beta_{0}\sum x_{i}+\beta_{1}\sum x^2_{i})=n\sum x_iY_{i}\] 두식을 빼면

\[\rightarrow \beta_{1}((\sum x_{i})^2-n\sum x^2_i)=\sum x_{i} \sum Y_{i}-n\sum x_{i}Y_{i}\]

\[\beta_{1}=\frac{\sum x_{i}\sum Y_{i}-n\sum x_{i}Y_{i}}{(\sum x_{i})^2-n\sum x^2_i}\]

\(\sum x_{i}=n \bar{x}\) 이므로 분모는

\[n^2\bar{x}^2-n\sum x^2_{i}=n(n\bar{x}^2-\sum x^2_{i})=n(2n\bar{x}^2-n\bar{x}^2-\sum x^2_{i})\] \[=n(2\bar{x}\sum x_{i}-\sum \bar{x}^2-\sum x^2_{i})=n\sum(x_{i}-\bar{x})^2\]

분자는 \[\sum x_{i}\sum Y_{i}-n\sum x_{i}Y_{i}=(n\bar{x})(n\bar{Y})-n\sum x_{i}Y_{i}\]

\[=n(n\bar{x}\bar{y}-\sum x_{i}Y_{i})=n(2n\bar{x}\bar{y}-n\bar{x}\bar{y}-\sum x_{i}Y_{i})\]

\[=n((\sum x_{i}\bar{Y}+\sum Y_{i}\bar{x})-\sum \bar{x}\bar{Y}-\sum x_{i}Y_{i})\]

\[=n\sum(x_{i}-\bar{x})(Y_{i}-\bar{Y})\]

따라서

\[\beta_{1}=\frac{\sum(x_{i}-\bar{x})(Y_{i}-\bar{Y})}{\sum(x_{i}-\bar{x})^2}\]

\[\hat{\beta_{0}}=\bar{Y}-\beta_{1}\bar{x}\]

3.Gauss-Markov Theorem (BLUE 증명)

  1. \(E[e]=0\)

  2. \(cov[x,e]=0\)

  3. \(var[e]=E[e^2]=\sigma^2\)

  4. \(cov[e_{i},e_{j}]=0 for\ \ all \ \ i\ne j\)

BLUE란Best Linear Unbiased Estimator : 선형이고 치우쳐지지 않았으며 분산이 가장 작은 추정량을 뜻하며, 추가로 오차변수가 정규분포를 따른다면 MVUE(Minimum Variance Unbiased Estimator)라 부른다.

비편향성 증명

  1. \(E[\hat{\beta_{1}}]=\beta_{1}\\\)

\[\hat{\beta_{1}}=\frac{S_{xy}}{S_{xx}}=\frac{\sum(x_{i}-\bar{x})(Y_{i}-\bar{Y})}{S_{xx}}=\frac{\sum(x_{i}-\bar{x})((\beta_{0}+\beta_{1}x_{i}+e_{i})-(\beta_{0}+\beta_{1}\bar{x}+\bar{e}))}{S_{xx}}\]

\[= \frac{\beta_{1} \sum(x_{i}-\bar{x})^2+ \sum e_{i}(x_{i}-\bar{x}) - \bar{e} \sum(x_{i}-\bar{x})}{S_{xx}}=\beta_{1}+\frac{\sum e_{i} (x_{i}-\bar{x})}{S_{xx}}-\bar{e}\frac{\sum(x_{i}-\bar{x})}{S_{xx}}\]

\[\sum (x_{i}-\bar{x})=0 이므로\beta_{1}+\frac{\sum e_{i}(x_{i}-\bar{x})}{S_{xx}}=\beta_{1}+\sum k_{i}e_{i}\]

\[\beta_{1}+\sum k_{i} e_{i}\ 에서\ E(e_{i}) =0 \ 이므로\ E(\hat{\beta_{1}})=\beta_{1}\]

  1. \(E[\hat{\beta_{0}}]=\beta_{0}\\\)

\[\hat{\beta_{0}}=\bar{Y}-\hat{\beta_{1}\bar{x}}=\frac{1}{n}\sum Y_{i}-\hat{\beta_{1}}\bar{x}=\frac{1}{n}\sum[\beta_{0}+\beta_{1}x_{i}+e_{i}]-(\beta_{1}+\sum k_{i}e_{i})\bar{x}\]

\[=\beta_{0}+\frac{\beta_{1}}{n}\sum x_{i}+\frac{1}{n}\sum e_{i}-\beta_{1}\bar{x}-\bar{x}\sum k_{i} e_{i}=\beta_{0}+\frac{1}{n}\sum e_{i}-\bar{x}\sum k_{i}e_{i}=\]

\[\beta_{0}+\sum(\frac{1}{n}-\bar{x}k_{i})e_{i}=\beta_{0}+\sum m_{i}e_{i},\ \ \beta_{1}에서와 \ 같은 \ 방법으로 \ E(\hat{\beta_{0}})=\beta_{0}\]

분산 증명

  1. \(E[\hat{\beta_{1}}]=\beta_{1}\\\)

\[\hat{\beta_{1}}=\frac{S_{xy}}{S_{xx}}=\frac{\sum(x_{i}-\bar{x})(Y_{i}-\bar{Y})}{S_{xx}}=\frac{\sum(x_{i}-\bar{x})((\beta_{0}+\beta_{1}x_{i}+e_{i})-(\beta_{0}+\beta_{1}\bar{x}+\bar{e}))}{S_{xx}}\]

\[= \frac{\beta_{1} \sum(x_{i}-\bar{x})^2+ \sum e_{i}(x_{i}-\bar{x}) - \bar{e} \sum(x_{i}-\bar{x})}{S_{xx}}=\beta_{1}+\frac{\sum e_{i} (x_{i}-\bar{x})}{S_{xx}}-\bar{e}\frac{\sum(x_{i}-\bar{x})}{S_{xx}}\]

\[\sum (x_{i}-\bar{x})=0 이므로\beta_{1}+\frac{\sum e_{i}(x_{i}-\bar{x})}{S_{xx}}=\beta_{1}+\sum k_{i}e_{i}\]

\[\beta_{1}+\sum k_{i} e_{i}\ 에서\ E(e_{i}) =0 \ 이므로\ E(\hat{\beta_{1}})=\beta_{1}\]