Hunter Liu's Website

6. Differentiating Power Series, Taylor Series

≪ 5. Comments on Homework 2, Problem 3 | Table of Contents | 7. Comments on Homework 3 ≫

This week, we started talking about power series and some of their convergence properties; in the upcoming lectures, we will be discussing Taylor series and its approximate forms.

Perhaps you don’t need to be convinced of why Taylor series are useful. On one hand, the series itself is not particularly useful in theory or in practise; anything one could say about a Taylor series, one could probably say about a smooth function. Moreover, most functions that one encounters on the street or at the grocery store aren’t even smooth; they are usually only \(k\) times continuously differentiable.

Taylor polynomials (or Taylor expansions, or Taylor approximations, or truncated Taylor series) are, in contrast, genuinely useful in both theory and in practise, in the following two settings:

Both of these techniques involving Taylor series are common both in applications and in theory, perhaps nowhere more clearly than in numerical analysis.

Today, we’ll be laying some foundations by looking at some ways to differentiate power series, and we’ll actually see that power series are smooth. We’ll also give a reminder of some different forms of Taylor’s formula, as well as different ways to use them.

Terminology 1.

Throughout these notes, and in many sources, you’ll see authors use the term “formal power series”. A formal power series refers to the symbols \[\sum _{n=1}^{\infty} a_nx^n,\] for instance. It is often helpful to write down power series without knowing if they converge, then perform some manipulations (e.g. differentiate it term-by-term, integrate it term-by-term, etc.) before justifying that these are valid and well-defined operations. In this context, the “formal derivative” of the above would be \[\sum _{n=1}^{\infty} na_n x ^{n-1}.\]

Power Series are Smooth

Last week, we have discussed some scenarios under which we could move derivatives underneath integrals. Today, we’ll be looking at how we could move derivatives under an infinite sum; an important corollary will be that power series are smooth where they converge.

Theorem 2.

Suppose \(\left\lbrace f_n(x) \right\rbrace\) is a sequence of continuously differentiable functions on \(\left[ a, b \right]\), and suppose \(f(x) = \sum _{n=1}^{\infty}f_n’(x)\) converges uniformly on \(\left[ a,b \right]\). If \(F(x) = \sum _{n=1}^{\infty}f_n(x)\) converges at one point, then \(F(x)\) converges on \(\left[ a,b \right]\) to a continuously differentiable function, and \(F’(x) =f(x)\).

In other words, what this theorem is saying is that \[\frac{d}{dx} \sum _{n=1}^{\infty}f_n(x) = \sum _{n=1}^{\infty} \frac{d}{dx}f_n(x),\] provided the sum on the right converges uniformly.

This theorem looks quite difficult to prove at first glance. Attempting to use the definition of differentiability quickly falls apart, and no amount of managing \(\epsilon \)’s and \(N\)’s will lead anywhere. This is not the way to go!

If you take away just one thing from this class, I hope it is the fact that derivatives are awful and integrals are amazing. In fact, integrals are so amazing that they can be used to prove things about derivatives, which are terrible.

Proof
Suppose \(F\) converges at a point \(x_0\in \left[ a, b \right]\), and let \(x\in \left[ a, b \right]\) be arbitrary. Since \(f\) is the uniform limit of continuous functions (its partial sums are continuous), it is continuous and hence integrable on \(\left[ a,b \right]\). Thus, we write by the fundamental theorem of calculus \[ \sum _{n=1}^{\infty} f_n(x) = \sum _{n=1}^{\infty} \left[ f_n\left( x_0 \right) + \int _{x_0}^{x} f_n’(t) dt \right]. \] We have that \(\sum _{n=1}^{\infty}f_n\left( x_0 \right) = F\left( x_0 \right)\) by assumption. Moreover, since the convergence of \(\sum _{n=1}^{\infty}f_n’(t)\) is uniform on \(\left[ a,b \right]\), we have \[\sum _{n=1}^{\infty}\int _{x_0}^{x} f_n’(t) dt = \int _{x_0}^{x} f_n’(t) dt = \int _{x_0}^{x}f(t) dt.\] Together, we get that \[F(x) = \sum _{n=1}^{\infty} f_n\left( x_0 \right) + \int _{x_0}^{x} \sum _{n=1}^{\infty}f_n’(t) dt = F\left( x_0 \right)+ \int _{x_0}^{x} f(t) dt\] for all \(x\in \left[ a, b \right]\). In particular, by the fundamental theorem of calculus again, \(F(x)\) is continuously differentiable, and \(F’(x) = f(x)\). \(\square\)

A big, big reason this theorem is helpful is that it lets us “formally” differentiate a power series, then a posteriori justify this strictly formal manipulation.

Exercise 3.

Show that the function \[f(x) = \sum _{n=1}^{\infty} \frac{\sin(nx)}{n!}\] is smooth on \(\mathbb{R}\).

Solution (Sketch)

We compute that the \(k\)-th derivative is formally given by \[f ^{(k)}(x) = \sum _{n=1}^{\infty} \frac{n^k}{n!} \sin(nx).\] We claim this converges uniformly on \(\mathbb{R}\). Since \[\sup _{x\in \mathbb{R}}\left\lvert \frac{n^k}{n!}\sin (nx) \right\rvert = \frac{n^k}{n!},\] by the Weierstrass M-test, it suffices to show that \(\sum _{n=1}^{\infty} \frac{n^k}{n!}\) converges for all \(k\). This can be proven using the ratio test (exercise from AP calculus!).

Thus, the formal power series for \(f ^{(k)}(x)\) actually converges uniformly on \(\mathbb{R}\). We can then apply induction to show that all derivatives of \(f\) exist. \(\square\)

Corollary 4.

Suppose \(f(x) = \sum _{n=1}^{\infty} a_n \left( x - x_0 \right)^n\) is a power series with a radius of convergence \(R>0\). Then \(f(x)\) is smooth on \(\left( x_0-R, x_0+R \right)\).

The proof is nearly identical to the previous exercise, though this time you need to use the root test to say that the power series’ formal derivatives converge uniformly on compact subsets of \(\left( x_0-R, x_0+R \right)\). This allows us to apply the theorem above and use induction. I’ll leave the details as an exercise!

Taylor Series

You have probably seen Taylor series before in the setting of high school or undergraduate calculus, so perhaps this section will not be very surprising. For the sake of completeness, we’ll be presenting Taylor’s theorem from the ground up, and we’ll also point out some unintuitive facts along the way.

Theorem 5. Taylor's Theorem

Let \(f:\mathbb{R}\to \mathbb{R}\) be continuous. Let \(x_0, x\in \mathbb{R}\), and suppose \(f\) is \(k+1\) times differentiable on the open interval from \(x_0\) to \(x \) (in particular, \(f\) will be \(k\) times continuously differentiable). Then, there exists some \(\xi\) between \(x_0\) and \(x\) such that \[\begin{align*} f(x) = f\left( x_0 \right)&+ f’\left( x_0 \right) \left( x-x_0 \right) + \frac{f’’\left( x_0 \right)}{2!} \left( x-x_0 \right)^2 + \\ &\cdots + \frac{f ^{(k)}\left( x_0 \right)}{k!}\left( x-x_0 \right)^k + \frac{f ^{(k+1)}\left( \xi \right)}{ (k+1)!} \left( x - x_0 \right) ^{k+1}. \end{align*}\]

The remainder term at the end (together with the condition on \(\xi \)) is hopefully vaguely reminiscent of the mean value theorem. In fact, this version of Taylor’s theorem can be proved by iterating the mean value theorem. In the case that \(f\) is only one time differentiable, Taylor’s theorem just says that there exists a \(\xi \) between \(x_0\) and \(x\) such that \[f(x) = f\left( x_0 \right) + f’\left( \xi \right)\left( x - x_0 \right).\] This is nothing more than the mean value theorem itself, and from this perspective, we can think of Taylor’s theorem as a “higher order variant” of the mean value theorem.

Often, this last term is called the “error term”; the leading terms are thus interpreted as a polynomial that (hopefully, closely) approximates the function \(f\).

There is another way to prove Taylor’s theorem by spamming integration by parts ad nauseum, and this is the proof that will be provided in lecture. This produces a slightly different form of the error term:

Theorem 6. Taylor's Theorem

Let \(f:\mathbb{R}\to \mathbb{R}\) be continuous. Let \(x_0, x\in \mathbb{R}\), and suppose \(f\) is \(k+1\) times differentiable on the open interval from \(x_0\) to \(x\). Suppose additionally that \(f ^{(k+1)}\) is integrable on this interval. Then, \[\begin{align*}f(x) = f\left( x_0 \right)&+ f’\left( x_0 \right)\left( x-x_0 \right)+\frac{f’’\left( x_0 \right)}{2!}\left( x-x_0 \right)^2 + \\ &\cdots + \frac{f ^{(k)}\left( x_0 \right)}{k!} \left( x-x_0 \right) ^{k} + \int _{x_0}^{x} \frac{f ^{(k+1)}\left( t \right)}{(k+1)!} \left( t-x_0 \right) ^{k} dt.\end{align*}\]

This second form of Taylor’s theorem actually implies the first form. The remainder term here is a differentiable function of \(x\), and one can apply the mean value theorem together with the fundamental theorem of calculus to recover the previous form of the error term. In practise, one usually has the freedom to choose which form of the error term to use.

Are Smooth Functions Analytic?

Suppose \(f:\mathbb{R}\to \mathbb{R}\) is a smooth function. One is inclined to believe that \(f\) and its Taylor series ought to coincide. After all, \(f\) is arbitrarily well-approximated by its Taylor series’ partial sums, and the Taylor series itself is smooth. Why wouldn’t they be the same? That is to say, one fully expects \[f(x) = \sum _{n=0}^{\infty} \frac{f ^{(n)}\left( x_0 \right)}{n!}\left( x-x_0 \right)^n\] for a suitable choice of \(x_0\).

Indeed, one needs to worry about convergence; last week we saw that \(\frac{1}{1+x^2}\) was equal to some power series on \(\left( -1, 1 \right)\). With some effort, one can show that this power series is its Taylor series. Moreover, \(\frac{1}{1+x^2}\) is analytic — by changing our choice of \(x_0\), this function will always locally resemble a power series.

However, smooth functions on (\mathbb{R}) are a strictly larger class of functions than power series. That is to say, not every smooth function can be represented by a power series.

Exercise 7.

Define the function \(f(x) = \exp \left( -\frac{1}{x^2} \right)\). Show that \(f ^{(n)}(0) = 0\) for all \(n\). Deduce that \(f\) is a bounded smooth function, and that its Taylor series \[\sum _{n=0}^{\infty} \frac{f ^{(n)}(0)}{n!}x^n\] converges on \(\mathbb{R}\), yet simultaneously does not converge to \(f(x)\) for all \(x\neq 0\).

One (extremely loose) way to interpret this is that this function \(\exp \left( -\frac{1}{x^2} \right)\) approaches \(0\) faster than any polynomial as \(x\to 0\). As such, one should not expect any nonzero terms in the Taylor series; using our Taylor polynomial estimates, the remainder term cannot “catch up” to this extremely quick decay.

As with last week’s discussion, we’re actually seeing the shadow of complex analysis in this phenomenon. We saw that power series converging to a function are limited by that functions poles; for functions like \(\frac{1}{1+x^2}\), there are hidden poles on the complex plane that still cause problems on the real line.

If you continue to believe the snake oil that complex-differentiable functions are analytic (i.e., they always agree with their Taylor series at every point), the problem with this function becomes clearer. Whereas \(\frac{1}{1+z^2}\) is complex differentiable at \(z=0\), the function \(\exp \left( -\frac{1}{z^2} \right)\) is not. In fact, if you approach \(z=0\) from the right direction, this function blows up. This behaviour is completely invisible to us from the real line!