4-1 Please derive the expressions \(\frac{\partial D_{WLS}(\beta)}{\partial\beta_0}\) and \(\frac{\partial D_{WLS}(\beta)}{\partial\beta_j}\).

原式 \(D_{WLS}(\beta)=\frac{1}{N}\sum_{n=1}^{N}\omega_n(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np})^2\)，對 \(\beta_0\) 做偏微分，得

\(\frac{\partial D_{WLS}(\beta)}{\partial\beta_0}=-\frac{2}{N}\sum_{n=1}^{N}\omega_n(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np})\)

原式對 \(\beta_j\) 做偏微分(假設 \(1\leq j \leq P\))，得

\(\frac{\partial D_{WLS}(\beta)}{\partial\beta_j}=-\frac{2}{N}\sum_{n=1}^{N}[\omega_n(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np})\sum_{p=1}^{P}x_{np}]\)

4-2 Please show that the WLS criterion can be written as \(D_{WLS}(\beta)=\frac{1}{N}(y-X\beta)^TW(y-X\beta)\)

原式 \(D_{WLS}(\beta)=\frac{1}{N}\sum_{n=1}^{N}\omega_n(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np})^2\)，其中 \(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np}\) 可表示為 \(y-\beta_0-X\beta\)，

其中 \(y=(y_1,y_2,...,y_n)\), \(X=[(x_{1,1},x_{1,2},...,x_{1,P}),(x_{2,1},x_{2,2},...,x_{2,P}),...,(x_{n,1},x_{n,2},...,x_{N,P})]\), \(\beta=(\beta_1,\beta_2,...,\beta_P)\),

又可簡化為 \(y-\beta X\)，其中 \(y=(y_1,y_2,...,y_N)\), \(X=[(1,x_{1,1},x_{1,2},...,x_{1,P}),(1,x_{2,1},x_{2,2},...,x_{2,P}),...,(1,x_{n,1},x_{n,2},...,x_{N,P})]\), \(\beta=(\beta_0,\beta_1,\beta_2,...,\beta_P)\),

將式子再新增 \(\frac{1}{N}, \omega_n\) 項得 \(D_{WLS}(\beta)=\frac{1}{N}(y-X\beta)^TW(y-X\beta)\)，其中 \(W\) 為對角線 \(N*N\) 矩陣，其對角線為\({\omega_1,\omega_2,...,\omega_N}\)，其餘為 0

4-3 Please show that the first derivative of the WLS criterion is \(\frac{\partial D_{WLS}(\beta)}{\partial \beta} = -\frac{2}{N}X^TW(y-X\beta)\)

使用 4-1 的結果，當 \(1\leq j \leq P\) 時

\(\frac{\partial D_{WLS}(\beta)}{\partial\beta_j}=-\frac{2}{N}\sum_{n=1}^{N}[\omega_n(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np})\sum_{p=1}^{P}x_{np}]\)

其中 \(y_n-\beta_0-\sum_{p=1}^{P}\beta_px_{np}\) 可表示為 \(y-\beta X\)，將式子再新增 \(-\frac{2}{N}, \omega_n, x_{np}\) 項得

\(\frac{\partial D_{WLS}(\beta)}{\partial\beta_j}=-\frac{2}{N}X^TW(y-X\beta)\)，其中 \(y=(y_1,y_2,...,y_n)\), \(X=[(x_{1,1},x_{1,2},...,x_{1,p}),(x_{2,1},x_{2,2},...,x_{2,p}),...,(x_{n,1},x_{n,2},...,x_{n,p})]\), \(\beta=(\beta_1,\beta_2,...,\beta_p)\), \(W=(\omega_1,\omega_2,...,\omega_n)\)

再考慮 \(\frac{\partial D_{WLS}(\beta)}{\partial\beta_0}\) 的結果得

\(\frac{\partial D_{WLS}(\beta)}{\partial \beta} = -\frac{2}{N}X^TW(y-X\beta)\)，其中 \(y=(y_1,y_2,...,y_n)\), \(X=[(1,x_{1,1},x_{1,2},...,x_{1,p}),(1,x_{2,1},x_{2,2},...,x_{2,p}),...,(1,x_{n,1},x_{n,2},...,x_{n,p})]\), \(\beta=(\beta_0,\beta_1,\beta_2,...,\beta_p)\), \(W=(\omega_1,\omega_2,...,\omega_n)\)

4-4 Please find the expression of WLS estimator \(\hat{\beta}\).

有了 4-3 的結果，可以直接令 \(\frac{\partial D_{WLS}(\hat{\beta})}{\partial \beta} = -\frac{2}{N}X^TW(y-X\hat{\beta})=0\)，來求 \(\hat{\beta}\)，使得 \(D_{WLS}(\hat{\beta})\) 最小

經由移項得 \((X^TW)y=(X^TW)(X\hat{\beta})\), \(\hat{\beta}=(X^TW)y(X^TW)^{-1}X^{-1}=X^{-1}y\)