Introduction
On previous pages, we analyzed specific behaviors of metric spaces such as
convergence,
boundedness,
continuity,
completeness,
connectedness, and
compactness. However,
we have not yet defined when two different spaces are "essentially" the same.
When data passes through the layers of a neural network, we often want the "intrinsic structure" of the data manifold
to be preserved. While isometry (preserving exact distances) is often too rigid for modern data analysis, we need a
more flexible concept of equivalence known as topological equivalence. It describes a scenario where one space can be
continuously deformed into another without tearing or gluing. This concept ensures that even if a latent space is stretched or twisted,
the fundamental connectivity of the data remains intact.
Topologically Equivalent Metrics
Before comparing entirely different spaces, we first examine how different distance functions on the
same underlying set compare to one another. In data analysis, the exact distance values might be
distorted by transformations or scaling. Two metrics may yield the same collection of open sets, and
therefore preserve the same convergent sequences and continuous functions. Understanding when this happens is
needed for selecting an appropriate metric representation for a dataset without altering its intrinsic topological structure.
Definition: Comparability of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). Then,
-
\(d\) is topologically stronger than \(e\) and \(e\) is topologically weaker than
\(d\) if and only if every open subset of \((X, e)\) is open in \((X, d)\).
-
\(d\) and \(e\) are topologically equivalent if and only if \(d\) is both weaker and
stronger than \(e\).
-
\(d\) and \(e\) are not comparable if and only if \(d\) is neither topologically stronger
nor topologically weaker than \(e\).
There are multiple ways to describe the comparability of metrics.
Theorem: Criteria for Topological Strength of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). The following statements are logically
equivalent.
-
Every open ball of \((X, e)\) includes an open ball of \((X, d)\) with the
same center. (The "same center" requirement is essential. Topological strength is a
local property at each point, so what must be compared are neighborhoods of the same point in both metrics.)
-
Every open subset of \((X, e)\) is open in \((X, d)\).
-
Every closed subset of \((X, e)\) is closed in \((X, d)\).
-
The identity function from \((X, d)\) to \((X, e)\) is continuous.
-
Every sequence that converges in \((X, d)\) converges in \((X, e)\) to the same limit.
- For every metric space \(Z\) and every function \(g: X \to Z\), if \(g\) is continuous with
respect to \(e\), then \(g\) is continuous with respect to \(d\) (where the metric on \(Z\)
is the same in both statements).
-
For every metric space \(Z\) and every function \(g: Z \to X\), if \(g\) is continuous with respect to
\(d\), then \(g\) is continuous with respect to \(e\) (where the metric on \(Z\) is the same in both statements).
Proof of (2) \(\Longleftrightarrow\) (4): Continuity of the Identity Function
Let us show the fundamental equivalence between the topological strength of metrics and the continuity of
the identity function (Criteria 2 and 4). We claim that \(d\) is topologically stronger than \(e\) if and
only if the identity function \(f:(X, d) \to (X, e)\) is continuous.
Suppose \(d\) is topologically stronger than \(e\). By definition, every open subset \(U\) in \((X, e)\) is also
open in \((X, d)\). For the identity function \(f\), the preimage of \(U\) is \(f^{-1}(U) = U\).
Since \(U\) is open in \((X, d)\), the
topological characterization of continuity
(preimage of any open set in the codomain is open in the domain) gives that \(f\) is continuous.
Suppose \(f: (X, d) \to (X, e)\) is continuous. Let \(U\) be an open subset in \((X, e)\). By the
topological characterization of continuity,
the preimage \(f^{-1}(U)\) must be open in \((X, d)\). Since \(f^{-1}(U) = U\), \(U\) is open in \((X, d)\).
Therefore, \(d\) is topologically stronger than \(e\).
This straightforward proof highlights why "comparing topologies" is fundamentally identical to analyzing the continuity of
mappings between them.
Proof Sketch: Remaining Equivalences
(1) \(\Longleftrightarrow\) (2).
(1) is the "ball version" of (2). Assume (1). By the
open-ball criterion for openness,
any open set \(U\) in \((X, e)\) and any \(x \in U\) admit an \(e\)-ball \(B_e(x; r) \subseteq U\). By
(1), that ball contains a \(d\)-ball \(B_d(x; s)\), so every point of \(U\) has a \(d\)-ball inside \(U\),
and the same criterion makes \(U\) \(d\)-open. Conversely (2) \(\Rightarrow\) (1): every \(e\)-ball \(B_e(x; r)\) is
an open subset of
\((X, e)\), hence \(d\)-open, hence contains a \(d\)-ball centered at \(x\).
(2) \(\Longleftrightarrow\) (3).
Open and closed subsets are set-theoretic
complements. A set \(F\) is closed if and only if \(X \setminus F\) is open. Hence the family of open sets
in a topology determines, and is determined by, the family of closed sets. The equivalence follows directly.
(4) \(\Longleftrightarrow\) (5).
Identity-map continuity is equivalent to its
sequential version,
so (4) says precisely that \(d\)-convergent sequences are \(e\)-convergent to the same limit.
(4) \(\Longleftrightarrow\) (6), (4) \(\Longleftrightarrow\) (7).
Assume (4), so that
\(\operatorname{id}_{d \to e}\) is continuous. If \(g: X \to Z\) is \(e\)-continuous, then
\(g \circ \operatorname{id}_{d \to e}\) is a
composition of continuous functions,
hence continuous on \((X, d)\). As a function it is \(g\) itself, which gives (6). The dual argument
composes on the other side and handles maps \(g: Z \to X\), which gives (7). Conversely, (6) applied to
\(Z = (X, e)\) and \(g = \operatorname{id}\) returns (4), and so does (7) applied to \(Z = (X, d)\) and
\(g = \operatorname{id}\). So (4) is what lets the identity translate continuity hypotheses between the two metrics on \(X\).
This establishes the criteria for topological strength. Applying the result symmetrically to both \(d\) and
\(e\) gives the criteria for topological equivalence stated in the following corollary.
Corollary: Criteria for Topological Equivalence
Two metrics \(d\) and \(e\) on a set \(X\) are topologically equivalent if and only if:
-
The collection of open (closed) subsets of \((X, d)\) is the same as that of \((X, e)\).
-
The identity functions from \((X, d)\) to \((X, e)\) and from \((X, e)\) to \((X, d)\) are both continuous.
-
Every convergent sequence of \((X, d)\) is convergent in \((X, e)\) with the same limit, and vice versa.
-
Every function from \(X\) into a metric space is continuous with respect to \(d\) if and only if
it is continuous with respect to \(e\) assuming the metric on the codomain is fixed.
-
Every function from a metric space into \(X\) is continuous with respect to \(d\) if and only if
it is continuous with respect to \(e\) assuming the metric on the domain is fixed.
Proof:
Each criterion in the corollary is obtained by conjoining the corresponding criterion of the preceding theorem
with its reversal (swap the roles of \(d\) and \(e\)). Since topological equivalence means \(d\) is both stronger
and weaker than \(e\), each item reduces to a symmetric pair of already-proved statements.
In summary, when \(d\) and \(e\) are topologically equivalent metrics on \(X\), \((X, d)\) and \((X, e)\) have the same
open (closed) subsets, dense subsets,
compact subsets, locally compact subsets,
connected subsets,
convergent sequences, and
continuous functions with \(X\) as domain (codomain).
Uniformly Equivalent Metrics
While topological equivalence preserves basic continuity and limit points, it fails to preserve "global" metric properties such
as Cauchy sequences and completeness. To guarantee that a space remains complete, or that an algorithm's convergence properties
are not destroyed when swapping metrics, we need a condition stronger than topological equivalence. The standard one, which
bounds the variation globally, is uniform equivalence.
When \(d\) and \(e\) are uniformly equivalent metrics on \(X\), then in addition to everything preserved by
topological equivalence, \((X, d)\) and \((X, e)\) have the same
Cauchy sequences,
totally bounded subsets, and
uniformly continuous functions with \(X\) as domain (codomain).
Lipschitz Equivalent Metrics
The strongest of the three forms of metric equivalence on this page requires that the distances under one metric are bounded by
a constant multiple of the distances under another. This condition guarantees that the geometric distortion between the two
metrics is strictly linearly limited. In machine learning, Lipschitz continuity of a single map underlies much of
optimization theory and gradient bounding, while two-sided Lipschitz bounds on a map and its inverse, the map-level version of
this condition defined later on this page, are used to keep invertible networks stable.
Definition: Lipschitz Equivalence of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). We say that \(d\) is Lipschitz stronger
than \(e\) and that \(e\) is Lipschitz weaker than \(d\) if and only if the identity function
from \((X, d)\) to \((X, e)\) is a Lipschitz function. Moreover, \(d\) and \(e\) are Lipschitz equivalent
if and only if each of \(d\) and \(e\) is Lipschitz stronger than the other.
When \(d\) and \(e\) are Lipschitz equivalent metrics on \(X\), then in addition to everything preserved by
uniform equivalence, \((X, d)\) and \((X, e)\) have the same
bounded subsets,
subsets with the nearest-point property, and
Lipschitz functions with \(X\) as domain (codomain).
Equivalent Metric Spaces
Having established how to compare different metrics on a single set, we now elevate these concepts to compare entirely distinct
metric spaces. This brings us to the formal definition of a homeomorphism, a mapping that establishes a
topological correspondence between two spaces.
Definition: Homeomorphism (Topologically Equivalent Metric Spaces)
Suppose \((X, d)\) and \((Y, e)\) are metric spaces.
Then \(X\) and \(Y\) are said to be homeomorphic or topologically equivalent if and
only if there exists a bijection \(f: X \to Y\) that is continuous and has continuous inverse. Such a function \(f\)
is called a homeomorphism.
Definition: Lipschitz Equivalent Metric Spaces
Suppose \((X, d)\) and \((Y, e)\) are metric spaces.
Then \(X\) and \(Y\) are said to be Lipschitz equivalent if and only if there exists a bijection
\(f: X \to Y\) that is Lipschitz continuous and whose inverse is also Lipschitz continuous.
The word equivalent now carries two readings, and they are not the same. For two metrics on a single
set \(X\), topological equivalence is the stronger requirement that the identity map be a
homeomorphism. Calling two metric spaces topologically equivalent asks only that some bijection between them be a homeomorphism.
These three notions form a strict hierarchy. The chain
Lipschitz implies uniformly continuous implies continuous,
applied to a bijection and to its inverse, makes every Lipschitz equivalence uniform and every uniform
equivalence a homeomorphism. Neither converse holds. Example 2 below exhibits a homeomorphism that is not a uniform equivalence.
Finally, we consider two fundamental examples that bridge theoretical analysis and practical applications and lead us toward the study of manifolds.
Example 1: Equivalence of Norms in \(\mathbb{R}^n\) (Lipschitz Equivalence)
In \(\mathbb{R}^n\), consider the \(L_1\) (Manhattan), \(L_2\) (Euclidean), and \(L_\infty\) (Chebyshev)
norms. All norms on a finite-dimensional vector space are
Lipschitz equivalent,
a fundamental theorem that we return to when we take up normed spaces. For any \(x \in \mathbb{R}^n\), we have the standard inequalities:
\[
\|x\|_\infty \le \|x\|_2 \le \|x\|_1 \le n\|x\|_\infty.
\]
Because these metrics bound each other up to a constant factor, the identity mappings between them are Lipschitz continuous.
For instance, \(\|x - y\|_\infty \le \|x - y\|_1\) shows that the identity
\((\mathbb{R}^n, \|\cdot\|_1) \to (\mathbb{R}^n, \|\cdot\|_\infty)\) is Lipschitz with constant \(1\), while
\(\|x - y\|_1 \le n \|x - y\|_\infty\) gives the reverse direction with constant \(n\). In machine learning, this guarantees
that whether we measure distances between parameter vectors with the \(L_1\) or the \(L_2\) norm, the underlying topological
properties (such as convergence of sequences) remain identical, even though the geometric shape of the unit ball changes.
Example 2: \((-1, 1)\) and \(\mathbb{R}\) (Topological Equivalence)
Consider the open interval \(X = (-1, 1)\) and \(Y = \mathbb{R}\) equipped with the standard Euclidean metric.
Let \(f: (-1, 1) \to \mathbb{R}\) be defined by:
\[
f(x) = \tan\left(\frac{\pi}{2}x\right).
\]
This function is a bijection, and both \(f\) and its inverse \(f^{-1}(y) = \frac{2}{\pi}\arctan(y)\) are continuous.
Thus, \(f\) is a homeomorphism, meaning \((-1, 1)\) and \(\mathbb{R}\) are topologically equivalent.
Notice what this example shows. The space \(X\) is bounded, but \(Y\) is unbounded. Furthermore, \(Y\) is complete,
but \(X\) is not (for example, the sequence \(x_n = 1 - 1/n\) is Cauchy in \(X\) but does not converge in
\(X\)). This demonstrates that topological equivalence preserves the "connectivity" and "openness"
of a space, but does not necessarily preserve metric properties like boundedness or completeness.
Why Uniform Equivalence Matters: Completeness Preservation
Uniform equivalence, which may look like a "niche" middle ground between topological and Lipschitz equivalence, is motivated
by the failure of completeness in Example 2 above.
A uniformly continuous map sends Cauchy sequences to Cauchy sequences.
Since a metric space is complete exactly when every Cauchy sequence in it converges,
a uniform equivalence preserves the completeness of a space. This
makes uniform equivalence a sufficient condition under which the Cauchy structure, and therefore
completeness, is preserved.
Properties of a space that are preserved under homeomorphisms, such as
compactness and
connectedness, are called
topological invariants. For manifolds, connectedness and local compactness (a local form of compactness) are
taken up on the page topological properties of manifolds.
Homeomorphisms also enter the definition of a manifold. An \(n\)-dimensional
topological manifold is a
topological space that is locally homeomorphic to \(\mathbb{R}^n\) and satisfies two further conditions, the Hausdorff property
and second countability. Manifolds, equipped with further smooth and Riemannian structure, form one of the domains, alongside
graphs and groups, on which Geometric Deep Learning formulates equivariance and invariance.