Hunter Liu's Website

4. Integration Review

≪ 3. Weierstrass' Approximation Theorem and Convolutions | Table of Contents | 5. Comments on Homework 2, Problem 3 ≫

Welcome to week 4! We have a handful of miscellaneous topics to go over today. On one hand, we’ll be refreshing some important ideas and statements about Riemann integration; on the other, we’ll be describing some foundational ideas for power series.

These seem like concepts that diametrically oppose each other: on one hand, power series can be thought of as containing information about all of the derivatives of a function (à la Taylor series); on the other, Riemann integrals literally “undo” derivatives.

Sometimes, the two concepts can be used in conjunction: for instance, the sum \(\sum _{n=1}^{\infty} n 2 ^{-n}\) can be evaluated by integrating the power series \(\sum _{n=0}^{\infty} x^n = \frac{1}{1-x}\) term-by-term and “plugging in” \(x=\frac{1}{2}\). However, there are several key steps along the way that require justification. Specifically, swapping an infinite sum with an integral is secretly moving two limits past each other, and we need to know simultaneously about the convergence properties of the power series and how Riemann integrals interact with them to perform these manipulations.

Remark 1.

In the real world, there are an abundance of smooth functions on \(\mathbb{R}\) that don’t admit a global power series expansion (e.g., \(\frac{1}{1+x^2}\)). However, in the complex plane, every holomorphic function admits a power series expansion! This somewhat surprising fact is born out of an interaction between integrals and power series in the complex plane, and it’s much more consequential than being able to evaluate funny power series.

Riemann Integrals

Last quarter, we saw a very brief overview of the theory of Riemann integration. Following some basic properties (e.g. linearity and monotonicity), we saw the following key fact:

Lemma 2.

Let \(f_n:\left[ a,b \right]\to \mathbb{R}\) be a sequence of Riemann integrable functions that converge uniformly to some \(f:\left[ a, b \right]\to \mathbb{R}\). Then \(f\) is also Riemann integrable, and \[\int _{a}^{b} f(x) dx = \lim _{n\to\infty}\int _{a}^{b} f_n(x) dx.\]

In other words, Riemann integrals “respect” uniform limits on a compact domain. Both the uniform convergence and the compactness of the domain of integration are strictly necessary for this to be true!

Exercise 3.

  1. Give an example of a sequence of Riemann integrable functions \(f_n: \left[ 0,\infty \right)\to \mathbb{R}\) converging uniformly to some \(f:\left[ 0,\infty \right)\to \mathbb{R}\) such that \[\int _{0}^{\infty} f(x) dx \neq \int _{0}^{\infty} f_n(x) dx.\]

  2. Give an example of a sequence of Riemann integrable functions \(f_n:\left[ 0,1 \right]\to \mathbb{R}\) converging pointwise to a function \(f:\left[ 0,1 \right]\to \mathbb{R}\) that is not even Riemann integrable.

Hint
For the first part, imagine a gliding hump. This construction is a frequently cited counterexample to many claims in analysis, and it is generally known as the “method of the gliding hump”.

One of the most important facets of Riemann integration is its relationship to derivatives, which you will probably recognise (unwillingly) as the fundamental theorem of calculus. You have hopefully seen this theorem before, but let’s give a proof of the theorem.

Theorem 4. The Fundamental Theorem of Calculus

Let \(f:\left[ a,b \right]\to \mathbb{R}\) be a continuous function. Then, the function \(F:\left[ a,b \right]\to \mathbb{R}\) given by \[F(x) = \int _{a}^{x}f(t) dt\] is continuously differentiable, and \(F’(x) = f(x)\) for all \(x\in \left( a,b \right)\).

The proof is actually extremely simple: we know that \[F’(x) = \lim _{h\to 0} \frac{F(x+h) - F(x)}{h} = \lim _{h\to 0} \frac{1}{h}\int _{x}^{x+h} f(t) dt.\] The idea is now that \(\int _{x}^{x+h}f(t) dt \approx hf(x)\) by the continuity of \(f\). Draw a picture!

Specifically, for any fixed \(x\) and \(\epsilon >0\), by the continuity of \(f\) at \(x\), there is some \(\delta >0\) such that \(\left\lvert x-t \right\rvert<\delta \) implies \(\left\lvert f(x) - f(t) \right\rvert < \epsilon \). Hence, \[\left\lvert \int _{x}^{x+h}f(t) dt - hf(x) \right\rvert= \left\lvert \int _{x}^{x+h} \left( f(t)-f(x) \right) dx \right\rvert \leq \int _{x}^{x+h}\left\lvert f(t) - f(x) \right\rvert dx < h \epsilon .\] This technically only works for positive \(h\), but it’s not hard to imagine that the same argument works for negative \(h\).

Dividing by \(h\), we see that for all \(h \in \left( -\delta , \delta \right)\), \[\left\lvert \frac{1}{h}\int _{x}^{x+h}f(t) dt - f(x) \right\rvert < \epsilon.\] We conclude by definition that \(F’(x) = f(x)\). \(\square\)

Note that we did not need to use the compactness of the domain at all in this proof; thus the theorem’s statement can actually be extended to the entire real line.

Corollary 5. The Fundamental Theorem of Calculus

Let \(f: \mathbb{R} \to \mathbb{R}\) be continuously differentiable. Then, for all \(a, b\in \mathbb{R}\), \[\int _{a}^{b} f’(x) dx = f(b) - f(a).\]

I’ll leave the proof of this as an exercise!

The last thing to comment on is when multiple variables are involved under an integral sign. We saw this last week when we discussed convolutions and how convolving with any (integrable) function with a (suitable) smooth function produced another smooth function. Specifically, at some point we had imagined a limit travelling under the integral sign, which thereby allowed us to say “differentiating a convolution is convolving with a derivative”.

Whenever you see a limit switching places with an integral, you should always think about uniform convergence; after all, this is the only tool available to us for performing this switch.

We can formulate this a lot more precisely: let \(f:\left[ a,b \right]\times \left[ c,d \right]\to \mathbb{R}\) be a function in two variables \(x\in \left[ a,b \right]\) and \(t\in \mathbb{R}\), satisfying the following conditions:

  1. \(f\) is continuous in both variables.
  2. For any fixed \(t\), the function \(x\mapsto f\left( x,t \right)\) is continuously differentiable.

Then, define the function \[F(x) = \int _{c}^{d} f(x, t) dt.\] \(F\) is continuous, differentiable, and \[F’(x) = \int _{c}^{d} \frac{\partial}{\partial x} f(x, t) dt.\] I’ll leave the proof as an exercise as well. There are several things to highlight: first, we are assuming continuous differentiability in the \(x\) variable. Since the domain is compact, there will be some uniform continuity of the \(x\)-partial. Using the mean value theorem, we can then establish some notion of “uniform convergence” of the \(x\)-partials, which ultimately lets us move the derivative into the integral. At this step, the compactness of the domain of integration becomes important!

Preliminaries for Power Series

As with integration, you’ve probably seen power series and Taylor series at some point in your life, be it in high school calculus or lower division calculus. And again, as with integration, a lot of facts and theorems were probably stated without proof and taken on blind faith. It’s now time for us to rigorously develop these elements, more or less from the ground up!

We’ll be looking at functions of the form \(f(x) = \sum _{n=0}^{\infty}a_n x^n\) for some real numbers \(a_n\). Secretly \(f\) is a limit, and we’re interested in knowing

To understand the latter two questions, we’ll need more than just pointwise convergence. While power series usually do not usually converge uniformly over their entire domain, they very often do converge uniformly on compact subsets of their domain. From tools like the Weierstrass M-test, we immediately know that such power series will be continuous.

So, our goal is to introduce some terminology and elementary tools that help us understand when and where these power series do or don’t converge.

Definition 6.

Let \(\left\lbrace a_n \right\rbrace\) be a sequence of real numbers. The limit superior is defined as the quantity \[\limsup _{n\to\infty} a_n := \lim _{n\to\infty}\left( \sup \left\lbrace a_m : m \geq n \right\rbrace \right).\] The limit inferior is defined as the quantity \[\liminf _{n\to\infty}a_n := \lim _{n\to\infty} \left( \inf \left\lbrace a_m : m \geq n \right\rbrace \right).\]

To make our lives easier, we allow these two to take the values of \(\pm\infty\). Importantly, when the sequence \(\left\lbrace a_n \right\rbrace\) is bounded from both directions, then the \(\limsup\) is guaranteed to exist! This is because \(\sup \left\lbrace a_m : m \geq n \right\rbrace\) is a nonincreasing sequence of bounded real numbers, hence a limit is guaranteed to exist. Similarly, the \(\liminf\) is guaranteed to exist as well.

The definition given above is somewhat clunky, but there’s an alternative characterisation of these quantities that’s helpful:

Exercise 7.

Let \(\left\lbrace a_n \right\rbrace\) be a bounded sequence of real numbers. Let \(I=\liminf _{n\to\infty}a_n\) and \(S=\limsup _{n\to\infty}a_n\).

  1. Show that for any \(\epsilon > 0\), there exists some \(N_ \epsilon \) such that \(n > N_ \epsilon \) implies \(a_n > I - \epsilon \). Show that \(I\) is the unique real number with this property.
  2. Show that for any \(\epsilon > 0\), there exists some \(N_ \epsilon \) such that \(n > N_ \epsilon \) implies \(a_n < S + \epsilon \). Show that \(S\) is the unique real number with this property.
  3. Show that \(\lim _{n\to\infty}a_n\) exists if and only if \(I = S\), and that the limit is equal to this common value.
  4. What happens when \(I=-\infty\) or \(S=+\infty\)?

You will most likely recognise the following theorem as the “root test”, but it’s also known as the Cauchy-Hadamard theorem.

Theorem 8.

Let \(f(x) = \sum _{n=0}^{\infty}a_nx^n\) be a formal power series with \(a_n\) a sequence of real numbers. Define \[R = \left(\limsup _{n\to\infty} \left\lvert a _{n}\right\rvert ^{\frac{1}{n}} \right) ^{-1}.\] \(R\) is taken to be \(\infty\) is the \(\limsup\) is zero. Then \(f\) converges uniformly on compact subsets of \(\left( -R, R \right)\), and \(f\) diverges outside of \(\left[ -R, R \right]\).

The \(R\) in the theorem is called the “radius of convergence”. I hope this does not dredge up bad memories.

The intuition behind this theorem is that eventually, \(a_n\) will resemble the geometric sequence \(R ^{-n}\); thus, as long as \(\left\lvert x \right\rvert < R\), the power series will resemble a converging geometric series. Likewise, if \(\left\lvert x \right\rvert> R\), the power series will instead resemble a diverging geometric series.

It’s important to recognise that this theorem cannot predict the behaviour of a power series on the boundary of the interval, where \(x=\pm R\). In fact, it’s possible for one endpoint to converge while the other diverges, or for both the diverge, or even for both to converge! Consider the following examples:

  1. The power series \(\sum _{n=0}^{\infty} x^n\) has a radius of convergence \(R=1\), and it does not converge at either \(x=1\) (where it just diverges) or \(x=-1\) (where the partial sums oscillate).
  2. The power series \(\sum _{n=1}^{\infty} \frac{(-1)^n}{n} x^n\) has a radius of convergence \(R=1\). It converges at \(x=1\) but diverges at \(x=-1\).
  3. The power series \(\sum _{n=0}^{\infty} \left( -1 \right)^n x ^{2n}\) has a radius of convergence of \(R=1\), but it diverges at both \(x=-1\) and \(x=1\).

You might recognise these three power series as \(\frac{1}{1-x}\), \(\ln \left( 1+x \right)\), and \(\frac{1}{1+x^2}\), respectively. For the first series, you might recognise that \(\frac{1}{1-x}\) has a “pole” at \(x=1\), hence there’s no way for its power series to converge at \(x=1\). Likewise, the \(\ln \left( 1+x \right)\) has a pole at \(x=-1\), and this prevents the power series from converging there, too. But without a pole at the other endpoint, the behaivour there is rather unpredictable, and it may or may not fail to converge to a value.

The third power series is somewhat strange; \(\frac{1}{1+x^2}\) is smooth on the entire real line, so there is no visible pole obstructing the power series’ convergence. What’s actually happening is that it does have poles, they’re just hidden from the real line! They lie at \(x=\pm i\). (I will one day repent for the sin of using \(x\) as a complex variable.)