5  Completeness. Metrics for Function Spaces

5.1 Completeness and Compact Metric Spaces

Definition 5.1  

  • A sequence \(\{x_n\}\) in a space \(S\) with a (pseudo)metric \(d\) is called a Cauchy sequence if \(\lim_{n \to \infty} \sup_{m \ge n} d(x_m, x_n) = 0\).
  • The pseudometric space \((S, d)\) is called complete iff every Cauchy sequence in it converges.
  • A point \(x\) in a topological space is called a limit point of a set \(E\) iff every neighborhood of \(x\) contains points of \(E\) other than \(x\).

Example 5.1 As an example of a compact metric space, first consider the interval \([0,1]\). Every number \(x \in [0, 1]\) has a decimal expansion \(x = 0.d_1d_2d_3... = \sum_{j \ge 1} d_j / 10^j\). Here each \(d_j = d_j(x)\) is an integer, \(0 \le d_j \le 9\) for all \(j\).

If a number \(x\) has an expansion \(d_j = 9\) for all \(j > m\), \(d_m < 9\) for some \(m\), then the numbers \(d_j\) are not uniquely determined (e.g. \(0.1999... = 0.2000...\)). In all other cases, the digits \(d_j\) are unique given \(x\).

Another property of numbers in \([0, 1]\): given any prescribed accuracy (any \(\varepsilon > 0\)), there is a finite set \(F\) of numbers in \([0, 1]\) s.t. every number \(x \in [0, 1]\) can be represented by a number \(y \in F\) to the desired accuracy (\(|x - y| < \varepsilon\)).

This property extends to metric spaces as follows:

Definition 5.2 A metric space \((S, d)\) is called totally bounded iff for every \(\varepsilon > 0\), there is a finite set \(F \subset S\) s.t. for every \(x \in S\), there is some \(y \in F\) with \(d(x, y) < \varepsilon\).

Theorem 5.1 For any metric space \((S, d)\), the following properties are equivalent:

  1. \((S, d)\) is compact: every open cover has a finite subcover
  2. \((S, d)\) is both complete and totally bounded
  3. Every infinite subset of \(S\) has a limit point
  4. Every sequence of points of \(S\) has a convergent subsequence

Definition 5.3 For any metric space \((S, d)\) and \(A \subset S\), the diameter of \(A\) is defined as \(\text{diam}(A) := \sup\{d(x, y): x \in A, y \in A\}\). Then \(A\) is called bounded iff its diameter is finite.

Example 5.2 Let \(S\) be any infinite set. For \(x\ne y\) in \(S\), let \(d(x, y)=1\) and \(d(x,x)=0\). Then \(S\) is complete and bounded, but not totally bounded. The characterization of compact sets in Euclidean spaces as closed bounded sets thus does not extend to general complete metric spaces.

Totally bounded metric spaces can be compared to how totally bounded they are in terms of the following quantities.

Let \((S, d)\) be a totally bounded metric space; given \(\varepsilon > 0\), let \(N(\varepsilon, S)\) be the smallest \(n\) s.t. \(S = \bigcup_{1\le i \le n}A_i\) for some sets \(A_i\) with \(\text{diam}(A_i) \le 2\varepsilon\) for \(i = 1,...,n\). let \(D(\varepsilon, S)\) be the largest number \(m\) of points \(x_i\), \(i = 1,...,m\) such that \(d(x_i, x_j) > \varepsilon\) when \(i \ne j\).

Proposition 5.1  

  • For any metric space \((S, d)\), if \(\{x_n\}\) is a Cauchy sequence, then it is bounded (i.e. the range is bounded).
  • If it has a convergent subsequence \(x_{n(k)}\to x\), then \(x_n \to x\).
  • Any closed subset of a complete metric space is complete.

If \(d(x_m, x_n) < 1\) for \(m > n\), then for all \(m\), \(d(x_m, x_n) < 1 + \max \{d(x_j, x_n, j < n\} < \infty\), so the sequence is bounded.

If \(x_{n(k)} \to x\), then given \(\varepsilon > 0\),

  • take \(m\) such that if \(n > m\) then \(d(x_n, x_m) < \varepsilon / 3\),
  • take \(k\) such that \(n(k) > m\) and \(d(x_{n(k)}, x) < \varepsilon / 3\)

Then \(d(x_n, x) < d(x_n, x_m) + d(x_m, x_{n(k)}) + d(x_{n(k)}, x) < \varepsilon / 3 + \varepsilon / 3 + \varepsilon / 3 = \varepsilon\), so \(x_n \to x\).

Thus, a closed subset of a complete space is complete, since a set \(F \subset S\) is closed if for every sequence \(x_i \to x\) in \(S\) with \(x_i \in F\) for all \(i\) we have \(x \in F\).

Example 5.3 \(\mathbb R\) with its usual metric is complete.
Indeed, let \(\{x_n\}\) be a Cauchy sequence. By the above proposition, it is bounded and thus included in some finite interval \([-M, M]\) (which is compact). Thus \(\{x_n\}\) has a convergent subsequence, so \(\{x_n\}\) converges by the above proposition.

5.2 Metrics for Function Spaces

Let \((S, d)\) and \((T, e)\) be any two metric spaces. It can be seen that

  • \(f: S\to T\) is continuous iff \(\forall x \in S, \varepsilon > 0\), \(\exists \delta > 0\) s.t. \(d(x, y) < \delta\) we have \(e(f(x), f(y)) < \varepsilon\).
  • \(f\) is continuous at \(x\) if the above holds for a fixed \(x\).

Definition 5.4 \(f\) is uniformly continuous from \((S, d)\) to \((T, e)\) if \(\forall \varepsilon > 0\), \(\exists \delta > 0\) s.t. \(d(x, y) < \delta \implies e(f(x), f(y)) < \varepsilon\) for all \(x, y \in S\).

Example 5.4 \(f(x) = x^2\) is continuous but not uniformly continuous. For a given \(\varepsilon\), as \(|x|\) gets larger, \(\delta\) must get smaller.

Before taking countable Cartesian products it is useful to make metrics bounded, which can be done as follows.

Proposition 5.2 Let \(f: [0, \infty) \to [0, \infty)\) be any continuous function, such that:

  1. \(f(x) \le f(y)\) whenever \(x \le y\)
  2. \(f(x+y) \ge f(x)+f(y)\) for all \(x, y \ge 0\)
  3. \(f(x) = 0 \iff x = 0\)

Then for any metric space \((S, d)\), \(f\circ d\) is a metric, and the identity function \(g(s)\equiv s\) from \(S\) to itself is uniformly continuous from \((S, d)\) to \((S, f\circ d)\) and from \((S, f\circ d)\) to \((S, d)\).

\(0 \le f(d(x, y)) = f(d(y, x))\), which is 0 iff \(d(x, y) = 0\), for all \(x, y \in S\).
For the triangle inequality, \(f(d(x,z)) \le f(d(x,y) + d(y, z)) \le f(d(x,y)) + f(d(y, z))\), do \(f \circ d\) is a metric.
Since \(f(t) > 0\) for all \(t > 0\), and \(f\) is continuous and nondecreasing, we have \(\forall \varepsilon > 0\), \(\delta > 0\) such that \(f(t) < \varepsilon\) if \(t < \delta\), and \(t < \varepsilon\) if \(f(t) < \lambda := f(\varepsilon)\).
Thus we have uniform continuity in both directions.

Suppose \(f''(x) < 0\) for \(x > 0\). Then \(f'\) is decreasing, so for any \(x, y > 0\), \[ f(x+y) - f(x) = \int_0^y f'(x+t)dt < \int_0^y f'(t)dt = f(y) - f(0) \] Thus if \(f(0)=0\), \(f\) is subadditive.

Example 5.5 Bounded functions \(f\) satisfying the conditions above:

  • \(f(x) := \frac{x}{1+x}\)
  • \(f(x) := \arctan x\)

Proposition 5.3 For any sequence \((S_n, d_n)\) of metric spaces, \(n = 1, 2, ...\), the product \(S := \prod_n S_n\) with product topology is metrizable by the metric \(d(\{x_n\}, \{y_n\}) := \sum_n f(d_n(x_n, y_n))/ 2^n\), where \(f(t) := t/(1+t)\)

A product of uncountably many metric spaces (each with more than one point) is not metrizable.

Example 5.6 A product of copies of \(\{0, 1\}\) over an uncountable index set \(I\), i.e. the set of all indicators of subsets of \(I\). Let the finite subsets \(F\) of \(I\) be directed by inclusion. Then the net \(1_F\) for all finite \(F\) converges to \(1\) for the product topology, but not sequence \(1_{F(n)}\) of indicator functions of finite sets can converge to \(1\), since the union of \(F(n)\), being countable, is not all of \(I\).

To get metrizable spaces of real functions on possibly uncountable sets, one needs to restrict the space of functions and/or consider a topology other than the product topology.

Example 5.7 For any compact topological space \(K\), let \(C(K)\) be the space of all continuous real-valued functions on \(K\). For \(f \in C(K)\), we have \(\sup |f| := \sup \{|f(x)| : x\in K\} < \infty\) since \(f[K]\) is compact in \(\mathbb R\). It is easily seen that \(d_{\sup}(f, g) := \sup |f-g|\) is a metric on \(C(K)\).

Roughly speaking, a family of functions is equicontinuous if all the functions are continuous and they have equal variation over a given neighbourhood, in a manner precisely described below.

Definition 5.5  

  • A collection \(\mathcal F\) of continuous functions from a topological space \(S\) into \(X\), where \((X, d)\) is a metric space, is called equicontinuous at \(x \in S\) iff \(\forall \varepsilon > 0\), there is a neighborhood \(U\) of \(x\) s.t. \(d(f(x),f(y))<\varepsilon\) \(\forall y\in U\) and \(g \in \mathcal F\).
  • \(\mathcal F\) is called equicontinuous iff it is equicontinuous at every \(x \in S\).
  • If \((S, e)\) is a metric space, and \(\forall \varepsilon > 0\), \(\exists \delta > 0\) s.t. \(e(x,y) < \delta \implies d(f(x),f(y)) < \varepsilon\) for all \(x,y\in S\) and all \(f\in \mathcal F\), then \(\mathcal F\) is called uniformly equicontinuous.

Theorem 5.2 If \((K, d)\) is a compact metric space, \((Y, e)\) is a metric space, then any equicontinuous family of functions from \(K\) into \(Y\) is uniformly equicontinuous.

Corollary 5.1 A continuous function from a compact metric space to any metric space is uniformly continuous.

Definition 5.6 A collection \(\mathcal F\) of functions on a set \(X\) into \(\mathbb R\) is called uniformly bounded iff \(\sup \{|f(x)|: f \in \mathcal F, x \in X\} < \infty\).

Example 5.8  

  • On any collection of bounded real functions, e.g. \(C(K)\) for \(K\) compact, let \(d_{\sup}(f, g):= \sup|f-g|\). Then \(d_\sup\) is a metric.
  • The sequence of functions \(f_n(t):= t^n\) on \([0, 1]\) consists of continuous functions, and the sequence is uniformly bounded: \(\sup_n\sup_t |f_n(t)| = 1\). Then \(\{f_n\}\) is not equicontinuous at \(1\), so not totally bounded for \(d_\sup\) by the following characterization:

Theorem 5.3 (Arzelà-Ascoli) Let \((K, e)\) be a compact metric space, \(\mathcal F\subset C(K)\). Then \(\mathcal F\) is totally bounded for \(d_\sup\) iff it is uniformly bounded and equicontinuous, thus uniformly equicontinuous.

For any topological space \((S, \mathcal T)\), let \(C_b(S) := C_b(S, \mathcal T)\) be the set of all bounded, real-valued, continuous functions on \(S\). The metric \(d_\sup\) is defined on \(C_b(S)\). Any sequence \(f_n\) that converges for \(d_\sup\) is said to converge uniformly. Uniform convergence preserves boundedness and continuity.

Theorem 5.4 For any topological space \((S, \mathcal T)\), if \(f_n \in C_b(S, \mathcal T)\) and \(f_n \to f\) uniformly as \(n \to \infty\), then \(f \in C_b(S, \mathcal T)\).

Theorem 5.5 For any topological space \((S, \mathcal T)\), the metric space \((C_b(S, \mathcal T), d_\sup)\) is complete.

We write \(c_n \downarrow c\) for real numbers \(c_n\) iff \(c_n \ge c_{n+1}\) for all \(n\) and \(c_n \to c\) as \(n \to \infty\). If \(f_n\) are real-valued on a set \(X\), then \(f_n \downarrow f\) means \(f_n(x) \downarrow f(x)\) for all \(x \in X\).

Theorem 5.6 (Dini’s Theorem) If \((K, \mathcal T)\) is a compact topological space, \(f_n\) are continuous real-valued functions on \(K\) for all \(n \in \mathbb N\); \(f_n \downarrow f_0\), then \(f_n \to f_0\) uniformly on \(K\).

Example 5.9 These counterexample shows why some of the hypotheses in Dini’s theorem are needed.

  • The function \(x^n \downarrow 0\) on \([0, 1)\) but not uniformly since the half-interval is not compact.
  • On \([0,1]\), which is compact, \(x^n \downarrow 1_{\{1\}}\) not uniformly since the limit is not continuous.

Definition 5.7  

  • A collection \(\mathcal F\) of real-valued functions on a set \(X\) forms a vector space iff for any \(f, g \in \mathcal F\), \(c \in \mathcal R\), we have \(cf + g \in \mathcal F\).
  • If \(fg \in \mathcal F\), then \(\mathcal F\) is called an algebra.
  • \(\mathcal F\) is said to separate points of \(X\) if for all \(x \ne y\) in \(X\), we have \(f(x) \ne f(y)\) for some \(f \in \mathcal F\).

Theorem 5.7 (Stone-Weierstrass) Let \(K\) be any compact Hausdorff space, \(\mathcal F\) be an algebra included in \(C(K)\) such that \(\mathcal F\) separates points and contains the constants. Then \(\mathcal F\) is dense in \(C(K)\) for \(d_\sup\).

Corollary 5.2 (Weierstrass) On any compact set \(K \subset \mathbb R^d\), \(d < \infty\), the sets of all polynomials in \(d\) variables is dense in \(C(K)\) for \(d_\sup\).

Corollary 5.3 (generalization in complex case) Let \((K, \mathcal T)\) be a compact Hausdorff space. Let \(\mathcal A\) be an algebra of continuous functions: \(K \mapsto \mathbb C\), separating the points and containing the constants. Suppose also \(\mathcal A\) is self-adjoint, i.e. \(\bar f = g - ih \in \mathcal A\) whenever \(f = g+ih \in \mathcal A\), where \(g\) and \(h\) are real-valued functions. Then \(\mathcal A\) is dense in the space of all continuous complex-valued functions on \(K\) for \(d_\sup\).

Example 5.10 The hypothesis that \(\mathcal A\) is self-adjoint cannot be omitted. Let \(T^1 := \{z\in \mathbb C: |z| = 1\}\), the unit circle, which is compact. Let \(\mathcal A\) be the set of all polynomials \(z \mapsto \sum_{j=0}^n a_j z^j\), \(a_j \in \mathbb C\), \(n = 0,1,...\). Then \(\mathcal A\) is an algebra satisfying all conditions above except for self-adjointness. The function \(f(z) := \bar z = 1/z\) on \(T^1\) cannot be uniformly approximated by a polynomial \(P_n \in \mathcal A\) as follows. For any such \(P_n\), \[ \frac{1}{2\pi} \int_0^{2\pi} |f(e^{i\theta}) - P_n(e^{i\theta})|^2 d\theta \ge 1 \] because the “cross terms” \(\int_0^{2\pi}e^{-i(k+1)\theta}d\theta = 0\) for \(k = 0,1,...\) and likewise if \(-i\) is replaced by \(i\).

5.3 Isometry

Definition 5.8 Let \((S, d)\) and \((T, e)\) be two metric spaces. A function \(f: S \to T\) is called an isometry iff \(e(f(x), f(y)) = d(x,y)\) for all \(x, y \in S\)

Example 5.11 \(S = T = \mathbb R^2\) with the usual Euclidean distance metric. Isometries are found by:

  • Taking \(f(u) = u + v\) for a vector \(v\) (translation)
  • Rotations around any center
  • Reflection in any line
  • Composition of the above operations

It will be shown that any metric space \(S\) is isometric to a dense subset of a complete one, \(T\). In a classic example, \(S\) is the space \(\mathbb Q\) of rational numbers, \(T = \mathbb R\).

Theorem 5.8 Let \((S, d)\) be any metric space. Then there is a complete metric space \((T, e)\) and an isometry \(f\) from \(S\) onto a dense subset of \(T\).

Remark 5.1. Since \(f\) preserves the only given structure of \(S\) (the metric), we can consider \(S\) as a subset of \(T\). \(T\) is the completion of \(S\).

Let \((T', e')\) and \(f'\) also satisfy the conclusion of the above theorem in place of \((T, e)\) and \(f\). Then on the range of \(f\), \(f' \circ f^{-1}\) is an isometry of a dense subset of \(T\) onto a dense subset of \(T'\). This isometry extends naturally to an isometry of \(T\) onto \(T'\) since both \((T, e)\) and \((T', e')\) are complete. Thus, \((T, e)\) is uniquely determined up to isometry.