Example 5.5. Find the maximum likelihood estimator of the standard deviation \(\sigma\) for the normal distribution with parameters \(\mu\) and \(\sigma^2\).
sol:
Let \(X_1,\ldots,X_n\) be i.i.d. with \(X_i\sim\mathcal N(\mu,\sigma^2)\), \(\sigma>0\). The individual density is \[ f(x_i;\mu,\sigma)=\frac{1}{\sqrt{2\pi}\,\sigma}\exp\!\left(-\frac{(x_i-\mu)^2}{2\sigma^2}\right). \]
By independence, the likelihood is \[ L(\mu,\sigma^2) =\prod_{i=1}^n \frac{1}{\sqrt{2\pi}\,\sigma}\exp\!\left(-\frac{(x_i-\mu)^2}{2\sigma^2}\right). \]
Taking \(\ell(\mu,\sigma^2)=\ln L(\mu,\sigma^2)\), we obtain: \[ \begin{aligned} L(\mu,\sigma^2) &=\left(\frac{1}{\sqrt{2\pi}\,\sigma}\right)^{\!n} \prod_{i=1}^n \exp\!\left(-\frac{(x_i-\mu)^2}{2\sigma^2}\right) \\[2mm] &=\left(\frac{1}{\sqrt{2\pi}\,\sigma}\right)^{\!n} \exp\!\left(\sum_{i=1}^n -\frac{(x_i-\mu)^2}{2\sigma^2}\right) \\[2mm] &=\left(\frac{1}{\sqrt{2\pi}\,\sigma}\right)^{\!n} \exp\!\left(-\frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2\right). \end{aligned} \] \[ \begin{aligned} \ell(\mu,\sigma^2) &=\ln L(\mu,\sigma^2) \\ &=\ln\!\left(\left(\tfrac{1}{\sqrt{2\pi}\,\sigma}\right)^{\!n}\right) +\ln\!\left[\exp\!\left(-\tfrac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2\right)\right] \\[2mm] &=n\,\ln\!\left(\tfrac{1}{\sqrt{2\pi}\,\sigma}\right) -\tfrac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2 \\[2mm] &=-n\,\ln(\sqrt{2\pi}\,\sigma) -\tfrac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2 \\[2mm] &=-n\big[\ln(\sqrt{2\pi})+\ln(\sigma)\big] -\tfrac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2 \\[2mm] &=-\tfrac{n}{2}\ln(2\pi)-\tfrac{n}{2}\ln(\sigma^2) -\tfrac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2. \end{aligned} \]
Differentiate with respect to \(\mu\). For each \(i\), \(\frac{d}{d\mu}(x_i-\mu)^2=-2(x_i-\mu)\). Thus, \[ \frac{\partial \ell}{\partial \mu} =\frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu)=0 \ \Longrightarrow\ \sum_{i=1}^n x_i-n\mu=0 \ \Longrightarrow\ \boxed{\ \widehat\mu=\bar X=\frac{1}{n}\sum_{i=1}^n x_i\ }. \]
Substituting \(\mu=\widehat\mu=\bar X\) into \(\ell\) and differentiating with respect to \(\sigma^2\): \[ \frac{\partial \ell}{\partial(\sigma^2)} =-\frac{n}{2}\cdot\frac{1}{\sigma^2} +\frac{1}{2(\sigma^2)^2}\sum_{i=1}^n (x_i-\bar X)^2=0 \] \[ \Longrightarrow\ -n\,\sigma^2+\sum_{i=1}^n (x_i-\bar X)^2=0 \Longrightarrow\ \boxed{\ \widehat{\sigma}^{\,2}=\frac{1}{n}\sum_{i=1}^n (x_i-\bar X)^2\ } \ \Longrightarrow\ \boxed{\ \widehat{\sigma}=\sqrt{\frac{1}{n}\sum_{i=1}^n (x_i-\bar X)^2}\ }. \]