Taylor series is quite important in statistics, such as it can be used to derive the variance of a function of asymptotically normal random variables, derive the likelihood ratio test, the Wald-type test and the score-type test, et al.
A Taylor series is a series expansion of a function about a point. A one-dimensional Taylor series is an expansion of a real function \(f(x)\) about a point \(a\) is given by
\[\begin{align*}f\left( x \right) & = \sum\limits_{n = 0}^\infty {\frac{{{f^{\left( n \right)}}\left( a \right)}}{{n!}}{{\left( {x - a} \right)}^n}} \\ & = f\left( a \right) + f'\left( a \right)\left( {x - a} \right) + \frac{{f''\left( a \right)}}{{2!}}{\left( {x - a} \right)^2} + \frac{{f'''\left( a \right)}}{{3!}}{\left( {x - a} \right)^3} + \cdots \tag{1}\end{align*}\]
If \(a=0\), the expansion is known as a Maclaurin series.\(^1\)
Here, I am reviewing the meaning of the Taylor series “about a point \(a\)” or “centered at a point \(a\)”.
Let us expand the \(f(x)=e^x\) at an arbitrary point \(a\) then we have
\[e^x=e^a+e^a(x-a)+\frac{e^a}{2!}(x-a)^2+\frac{e^a}{3!}(x-a)^3+\frac{e^a}{4!}(x-a)^4+\frac{e^a}{5!}(x-a)^5+...\tag{2}\]
Note, the \(n^{th}\) derivative of the function \(e^x\) is still \(e^x\)
We set \(a=0\) for \((2)\), then
\[e^x=1+x+\frac{1}{2!}x^2+\frac{1}{3!}x^3+\frac{1}{4!}x^4+\frac{1}{5!}x^5+...\tag{3}\]
We set arbitrarily \(a=2\) for \((2)\), then
\[e^x=e^2+e^2(x-2)+\frac{e^2}{2!}(x-2)^2+\frac{e^2}{3!}(x-2)^3+\frac{e^2}{4!}(x-2)^4+\frac{e^2}{5!}(x-2)^5+...\tag{4}\] We can continue to set \(a\) equal to different numbers and get different equations which means the function \(e^x\) can be represented by different Taylor series.
Now let us calculate the value of \(e^{1.5}\) by the equations \((3)\) and \((4)\)
The value of \(e^{1.5}\) can be calculated by R exp() function which is \(4.481689\), we think this is the real value of the \(e^{1.5}\)
exp(1.5)## [1] 4.481689Let us calculate \(e^{1.5}\) by the equation \((3)\) use the six terms and we get \(4.461719\)
1+1.5+1/factorial(2)*1.5^2+1/factorial(3)*1.5^3+1/factorial(4)*1.5^4+1/factorial(5)*1.5^5## [1] 4.461719Let us calculate \(e^{1.5}\) by the equation \((4)\) use the six terms,we plug in \(x=1.5\) and get \(4.481539\)
exp(2)+exp(2)*(1.5-2)+exp(2)/factorial(2)*(1.5-2)^2+exp(2)/factorial(3)*(1.5-2)^3+exp(2)/factorial(4)*(1.5-2)^4+exp(2)/factorial(5)*(1.5-2)^5## [1] 4.481539We can see when we use equation \((4)\) we get a closer value to the real value of \(e^{1.5}\) than equation \((3)\). Why?
Because it is related to how we choose a “center” of a Taylor series,
When we choose center =0 i.e \(a=0\) the distance between \(0\) and \(1.5\) (equation \((3)\))is greater than the distance between \(1.5\) and \(2\) (equation \((4)\)),therefore equation \((4)\) will get an accurate approximate value for the function \(e^{1.5}\).
In fact, choose a center value of a Taylor series is related to the accuracy of approximation for a given number of terms of the series of a function. As showed above, when we choose a center of \(a=0\) and the Taylor series to evaluate a function when \(x=1.5\) will be less accurate than choose a center at \(a=2\), and we will need to take more terms to obtain a similar accuracy. If we choose a center at \(a=1.6\) then we need less number of terms to obtain an even more accurate approximation.
When use Taylor series to approximate, we should keep the center close to the values of \(x\) that we want to evaluate.
Application of Taylor series also can be shown in the proof the asymptotic normality of MLE, i.e
\[\sqrt{n}(\hat{\theta_n}-\theta_0)\overset{D}\rightarrow N(0,\frac{1}{I(\theta_0)})\] where we need to expand the derivative of log-likelihood function \(l'(\hat{\theta_n})\)into a Taylor series about the real parameter \(\theta_0\), note \(\hat{\theta_n}\) and the \(\theta_0\) should be quite close. We can expand the derivative of log-likelihood function as:
\[l'(\hat{\theta_n})=l'(\theta_0)+l''(\theta_0)(\hat{\theta_n}-\theta_0)+\frac{1}{2}l'''(\theta_n^*)(\hat{\theta_n}-\theta_0)^2\] \(\square\)
1.Weisstein, Eric W. “Taylor Series.” From MathWorld–A Wolfram Web Resource. https://mathworld.wolfram.com/TaylorSeries.html