Introduction
In previous chapters, we analyzed specific behaviors of metric spaces such as convergence, boundedness,
continuity, completeness, connectedness, and compactness. However, we have not yet defined when two different
spaces are "essentially" the same.
When data passes through the layers of a neural network, we often want the "intrinsic structure" of the data manifold
to be preserved. While isometry (preserving exact distances) is often too rigid for modern data analysis, we need a
more flexible concept of equivalence known as topological equivalence. It describes a scenario where one space can be
continuously deformed into another without tearing or gluing. This concept ensures that even if a latent space is stretched or twisted,
the fundamental connectivity of the data remains intact.
Topologically Equivalent Metrics
Before comparing entirely different spaces, we first examine how different distance functions on the same underlying
set compare to one another. In data analysis, the exact distance values might be distorted by transformations or scaling.
Understanding when two metrics yield the exact same collection of open sets - and therefore preserve the exact same convergent
sequences and continuous functions - is crucial when selecting an appropriate metric representation for a dataset without
altering its intrinsic topological structure.
Definition: Comparability of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). Then,
-
\(d\) is topologically stronger than \(e\) and \(e\) is topologically weaker than
\(d\) if and only if every open subset of \((X, e)\) is open in \((X, d)\).
-
\(d\) and \(e\) are topologically equivalent if and only if \(d\) is both weaker and
stronger than \(e\).
-
\(d\) and \(e\) are not comparable if and only if \(d\) is neither topologically stronger
nor topologically weaker than \(e\).
There are multiple ways to describe the Comparability of metrics.
Theorem: Criteria for Topological Strength of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). The following statements are logically
equivalent.
-
Every open ball of \((X, e)\) includes an open ball of \((X, d)\) with the same center.
-
Every open subset of \((X, e)\) is open in \((X, d)\).
-
Every closed subset of \((X, e)\) is closed in \((X, d)\).
-
The identity function from \((X, d)\) to \((X, e)\) is continuous.
-
Every sequence that converges in \((X, d)\) converges in \((X, e)\) to the same limit.
- Every function from \(X\) into a metric space that is continuous with respect to \(e\) is
continuous with respect to \(d\), assuming the metric on the codomain is fixed.
-
Every function from a metric space into \(X\) that is continuous with respect to \(d\) is
continuous with respect to \(e\), assuming the metric on the domain is fixed.
Proof: Continuity of the Identity Function
Let us show the fundamental equivalence between the topological strength of metrics and the
continuity of the identity function (Criterion 2 and 4): \(d\) is topologically stronger than \(e\) if and only
if the identity function \(f:(X, d) \to (X, e)\) is continuous.
Suppose \(d\) is topologically stronger than \(e\). By definition, every open subset \(U\) in \((X, e)\) is also
open in \((X, d)\). For the identity function \(f\), the preimage of \(U\) is \(f^{-1}(U) = U\).
Since \(U\) is open in \((X, d)\), the preimage of any open set in the codomain is open in the domain.
Thus, \(f\) is continuous.
Suppose \(f: (X, d) \to (X, e)\) is continuous. Let \(U\) be an open subset in \((X, e)\). By the topological definition
of continuity, the preimage \(f^{-1}(U)\) must be open in \((X, d)\). Since \(f^{-1}(U) = U\), \(U\) is open in \((X, d)\).
Therefore, \(d\) is topologically stronger than \(e\).
This straightforward proof highlights why "comparing topologies" is fundamentally identical to analyzing the continuity of
mappings between them.
This establishes the criteria for topological strength; by applying this result symmetrically to both \(d\) and \(e\), we
obtain the criteria for topological equivalence as stated in the following corollary.
Corollary: Criteria for Topological Equivalence
Two metrics \(d\) and \(e\) on a set \(X\) are topologically equivalent if and only if:
-
The collection of open (closed) subsets of \((X, d)\) is the same as that of \((X, e)\).
-
The identity functions from \((X, d)\) to \((X, e)\) and from \((X, e)\) to \((X, d)\) are both continuous.
-
Every convergent sequence of \((X, d)\) is convergent in \((X, e)\) with the same limit, and vice versa.
-
Every function from \(X\) into a metric space is continuous with respect to \(d\) if and only if
it is continuous with respect to \(e\) assuming the metric on the codomain is fixed.
-
Every function from a metric space into \(X\) is continuous with respect to \(d\) if and only if
it is continuous with respect to \(e\) assuming the metric on the domain is fixed.
In Summary, when we can say \(d\) and \(e\) are equivalent metrics on \(X\), \((X, d)\) and \((X, e)\) hold the same open (closed) subsets,
dense subsets, compact subsets, locally compact subsets, connected subsets, convergent sequences, and continuous functions
with \(X\) as domain (codomain).
Uniformly Equivalent Metrics
While topological equivalence preserves basic continuity and limit points, it fails to preserve "global" metric properties such as
Cauchy sequences and completeness. To guarantee that a space remains complete, or that an algorithm's convergence properties are not
destroyed when swapping metrics, we must introduce a stricter condition that bounds the variation globally: uniform equivalence.
Definition: Uniform Equivalence of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). We say that \(d\) is uniformly stronger
than \(e\) and that \(e\) is uniformly weaker than \(d\) if and only if the identity function
from \((X, d)\) to \((X, e)\) is uniformly continuous. Moreover, \(d\) and \(e\) are uniformly equivalent
if and only if each of \(d\) and \(e\) is uniformly stronger than the other.
When we can say \(d\) and \(e\) are uniformly equivalent metrics on \(X\), besides topologically equivalent metrics condition,
\((X,d)\) and \((X, e)\) hold the same Cauchy sequences, totally bounded subsets, and uniformly continuous functions with \(X\)
as domain (codomain).
Lipschitz Equivalent Metrics
The strongest form of metric equivalence requires that the distances under one metric are bounded by a constant multiple of the distances under another.
This condition guarantees that the geometric distortion between the two metrics is strictly linearly limited. In machine learning, Lipschitz equivalence
is heavily relied upon in optimization theories, bounding gradients, and ensuring the theoretical stability of neural networks.
Definition: Lipschitz Equivalence of Metrics
Suppose \(X\) is a set and \(d\) and \(e\) are metrics on \(X\). We say that \(d\) is Lipschitz stronger
than \(e\) and that \(e\) is Lipschitz weaker than \(d\) if and only if the identity function
from \((X, d)\) to \((X, e)\) is a Lipschitz function. Moreover, \(d\) and \(e\) are Lipschitz equivalent
if and only if each of \(d\) and \(e\) is Lipschitz stronger than the other.
When we can say \(d\) and \(e\) are Lipschitz equivalent metrics on \(X\), besides uniformly equivalent metrics condition, \((X,d)\) and \((X, e)\)
hold the same bounded subsets, subsets with the nearest-point property, and Lipschitz functions with \(X\) as domain (codomain).
Equivalent Metric Spaces
Having established how to compare different metrics on a single set, we now elevate these concepts to compare entirely distinct
metric spaces. This brings us to the formal definition of a homeomorphism - a mapping that establishes a perfect
topological correspondence between two spaces. This continuous, invertible correspondence between spaces forms the foundational base
for defining manifolds, which is the core mathematical structure behind Geometric Deep Learning.
Definition: Equivalent Metric Spaces
Suppose \((X, d)\) and \((Y, e)\) are metric spaces.
-
\(X\) and \(Y\) are said to be homeomorphic or topologically equivalent if and
only if there exists a bijection \(f: X \to Y\) that is continuous and has continuous inverse. Such a function \(f\)
is called a homeomorphism.
-
\(X\) and \(Y\) are said to be uniformly equivalent if and only if there exists a bijection
function \(f: X \to Y\) that is uniformly continuous and has uniformly continuous inverse.
-
\(X\) and \(Y\) are said to be Lipschitz equivalent if and only if there exists a bijection
function \(f: X \to Y\) that is Lipschitz continuous and whose inverse is also Lipschitz continuous.
Finally, let us observe two fundamental examples that bridge theoretical analysis and practical applications,
leading us toward the study of manifolds.
Example 1: Equivalence of Norms in \(\mathbb{R}^n\) (Lipschitz Equivalence)
In \(\mathbb{R}^n\), consider the \(L_1\) (Manhattan), \(L_2\) (Euclidean), and \(L_\infty\) (Chebyshev) norms.
It is a fundamental theorem that all norms on a finite-dimensional vector space are equivalent. Specifically,
they are Lipschitz equivalent. For any \(x \in \mathbb{R}^n\), we have the standard inequalities:
\[
\|x\|_\infty \le \|x\|_2 \le \|x\|_1 \le n\|x\|_\infty.
\]
Because these metrics bound each other up to a constant factor, the identity mappings between them are Lipschitz continuous.
In machine learning, this guarantees that whether you use \(L_1\) or \(L_2\) regularization, the underlying topological properties
(such as convergence of sequences) remain strictly identical, even though the geometric shape of the unit ball changes.
Example 2: \((-1, 1)\) and \(\mathbb{R}\) (Topological Equivalence)
Consider the open interval \(X = (-1, 1)\) and \(Y = \mathbb{R}\) equipped with the standard Euclidean metric.
Let \(f: (-1, 1) \to \mathbb{R}\) be defined by:
\[
f(x) = \tan\left(\frac{\pi}{2}x\right).
\]
This function is a bijection, and both \(f\) and its inverse \(f^{-1}(y) = \frac{2}{\pi}\arctan(y)\) are continuous.
Thus, \(f\) is a homeomorphism, meaning \((-1, 1)\) and \(\mathbb{R}\) are topologically equivalent.
Notice a profound consequence: \(X\) is bounded, but \(Y\) is unbounded. Furthermore, \(Y\) is complete, but \(X\) is
not (e.g., the sequence \(x_n = 1 - 1/n\) is Cauchy on \(X\) but does not converge on \(X\)). This demonstrates that
topological equivalence preserves the "connectivity" and "openness" of a space, but does not necessarily
preserve metric properties like boundedness or completeness.
Note on Uniform Equivalence:
While uniform equivalence might seem like a 'niche' middle ground between topological and Lipschitz equivalence,
it plays a vital role in analysis: it is the minimal condition required to preserve the completeness
of a metric space. While a homeomorphism (topological) can map a complete space to an incomplete one (as seen in this example),
a uniform homeomorphism ensures that Cauchy sequences remain Cauchy, thus preserving the convergence structure essential
for the completion of spaces.
Properties of a space that are preserved under homeomorphisms - such as compactness and connectedness - are called topological invariants.
As we move toward Geometric Deep Learning, we will study Manifolds. A topological space is considered an \(n\)-dimensional manifold if it is
locally homeomorphic to \(\mathbb{R}^n\) with additional technical conditions. Understanding homeomorphism is the absolute prerequisite for formalizing how
complex data structures (like graphs or 3D surfaces) can be analyzed mathematically.