Symmetry & Representation Theory in ML

Symmetry as an Inductive Bias

On the page introducing geometric deep learning we saw a single design principle behind convolutional networks, Transformers, and graph neural networks. Each architecture can be built to respect a symmetry of its input domain, so the network does not have to learn that symmetry from data. There the symmetries were discrete, such as translations of a pixel grid and permutations of a set of nodes. We closed by pointing past that case, toward the continuous symmetries of three-dimensional space. This page takes up that thread and asks what mathematics is actually required to build a layer that respects a continuous symmetry, and the answer turns out to be the representation theory of Lie groups.

The motivating situation is concrete. Consider a model that predicts the energy of a molecule from the positions of its atoms, or the force on each atom, or a segmentation of a spherical panoramic image. In each case the data lives in physical three-dimensional space, and there is a transformation that ought to leave the problem unchanged.

If we rotate a molecule in space, its total energy does not change, so the predicted energy should be invariant under rotation. If instead we predict a force vector on each atom, then rotating the molecule should rotate every predicted force in exactly the same way. Such a prediction should be equivariant under rotation. A network that does not know this must discover it from data. It must see the same molecule in many orientations before it learns that orientation does not matter. By contrast, a network whose every layer respects rotation by construction has the regularity built in from the start.

Let us make the two words precise, since the whole development rests on the distinction. Let \(G\) be a group acting on an input space \(\mathcal{X}\) and on an output space \(\mathcal{Y}\), and write \(g \cdot x\) for the action of \(g \in G\) on \(x\). A map \(\Phi : \mathcal{X} \to \mathcal{Y}\) is \(G\)-equivariant if it commutes with the action, \[ \Phi(g \cdot x) = g \cdot \Phi(x) \quad \text{for all } g \in G,\ x \in \mathcal{X}, \] so that transforming the input and then applying \(\Phi\) gives the same result as applying \(\Phi\) and then transforming the output. One says that \(\Phi\) intertwines the two actions.

Invariance is the special case in which the group acts trivially on the output, \(\Phi(g \cdot x) = \Phi(x)\). Equivariance is the more fundamental notion. In the architectures this page builds, the hidden layers are equivariant so that they carry the symmetry forward intact. When the target is invariant, invariance is imposed only at the top, where a single symmetric quantity is finally read off.

The promise of building symmetry into the architecture is improved sample efficiency. The network does not spend capacity, or training data, learning a regularity that the designer already knows. This is the same bargain that makes a convolutional layer more data-efficient than a fully connected one on images. There, weight sharing across spatial positions makes the layer translation-equivariant (exactly so for stride one on an unbounded or periodic grid). The question this page answers is what the analogous construction is when the symmetry is the continuous rotation group rather than discrete translation. The answer is considerably richer, because rotations mix the components of a feature, which integer translations do not.

It is worth flagging at the outset that whether one should hard-code a symmetry into the architecture, rather than let the model learn it from data and augmentation, is an empirical question. We return to it in the section on symmetry in practice, once the representation theory needed even to state it precisely is in place.

Features as Group Representations

To build a rotation-equivariant layer we must first decide how the data it processes transforms under rotation, and this question is the one representation theory was built to answer. A feature attached to a point in space is not just a list of numbers but a list of numbers together with a rule for how those numbers change when space is rotated.

A scalar quantity, such as the mass of an atom or a predicted energy density, does not change at all when the molecule is rotated. A vector quantity, whether a velocity, a force, or a dipole, rotates rigidly with the molecule. A symmetric-matrix quantity, such as a moment of inertia or a stress, transforms by conjugation. These are three different transformation rules, and a representation of \(SO(3)\) is precisely the data of such a rule: a way of assigning to each rotation \(R \in SO(3)\) a linear map \(\rho(R)\) on the feature space, smoothly in \(R\) and compatibly with composition, \(\rho(R_1 R_2) = \rho(R_1)\rho(R_2)\).

Once features are viewed this way, the central structural fact of the theory does the organizing work for us. Since \(SO(3)\) is compact, each of its finite-dimensional representations is completely reducible. Such a representation decomposes into a direct sum of irreducible representations, the indivisible building blocks out of which every transformation rule is assembled. For \(SO(3)\) the irreducible complex representations are completely classified, and the classification rests on the representation theory of \(\mathfrak{sl}(2,\mathbb{C})\). For each non-negative integer \(\ell\) there is exactly one irreducible complex representation of dimension \(2\ell+1\), and there are no others.

We call \(\ell\) the type (or degree). In the notation of our representation-theory pages this irreducible is \(\Sigma_m\) with \(m = 2\ell\). The half-integer values, corresponding to odd \(m\), are exactly the ones that fail to descend from \(SU(2)\) to \(SO(3)\). The low types are familiar in their real forms: \(\ell = 0\) is the scalars, \(\ell = 1\) the vectors \(SO(3)\) acts on in \(\mathbb{R}^3\), \(\ell = 2\) the five-dimensional traceless symmetric matrices, and the ladder continues one irreducible per \(\ell\).

A feature in an equivariant network is therefore organized not as a flat vector but as a collection of type-\(\ell\) pieces, each transforming under the corresponding irreducible \(\rho_\ell\): so many scalars, so many vectors, so many type-2 fragments, and so on. This is the continuous-symmetry analogue of the channels of a convolutional network, but now each channel carries a definite transformation law rather than being an inert number. The classification matters so directly because it is forced upon us. Whatever representation the raw data happens to carry, decomposing it into irreducibles tells us exactly which types are present and with what multiplicity. The network can then be built to act on each type appropriately.

Each type also has a concrete realization as functions on the sphere. We call a polynomial \(f\) on \(\mathbb{R}^3\) with complex coefficients harmonic if \(\partial_1^2 f + \partial_2^2 f + \partial_3^2 f = 0\), where \(\partial_j\) is the partial derivative in the \(j\)-th coordinate. For each integer \(\ell \geq 0\), the harmonic polynomials that are homogeneous of degree \(\ell\), restricted to the unit sphere \(S^2\), form a \((2\ell+1)\)-dimensional space whose elements are the spherical harmonics of degree \(\ell\).

Rotations carry the space of degree-\(\ell\) spherical harmonics into itself by \((R \cdot f)(x) = f(R^{-1} x)\), and the resulting representation of \(SO(3)\) is irreducible. By the classification above it is therefore isomorphic to \(\rho_\ell\). We use the dimension count, the invariance under rotations, and the irreducibility without proof, and we do not develop the theory of spherical harmonics on this page. The Schur and Clebsch-Gordan arguments below do not depend on these facts.

The spherical-harmonic realization is how the types meet the data a network actually sees. A signal on the sphere, such as a panoramic image, an angular distribution of mass, or a directional field of measurements, is expanded in spherical harmonics much as a periodic signal is expanded in a Fourier series.

Once a basis \(Y^\ell_{-\ell}, \ldots, Y^\ell_{\ell}\) of each degree-\(\ell\) space is fixed, the coefficients of a signal at degree \(\ell\) form a \((2\ell+1)\)-dimensional vector. A rotation \(R\) acts on it by the matrix of \(f \mapsto f(R^{-1}\,\cdot\,)\) in that basis, so the vector is a type-\(\ell\) feature in the sense above. That the spaces of different degrees are mutually orthogonal in \(L^2(S^2)\) for the rotation-invariant surface measure, which makes these coefficients well defined, and that the expansion captures every square-integrable function on \(S^2\) are further facts we take without proof.

In the standard basis of spherical harmonics, an orthonormal basis of eigenvectors of the rotations about the third axis with conventionally chosen phases, the matrices of this action are the Wigner \(D\)-matrices. Conventions differ on whether these matrices, their transposes, or their complex conjugates carry the name. By the classification of the irreducibles, they are, up to a change of basis, the matrices of the \(SO(3)\) irreducible representations constructed earlier by descending the polynomial representations of \(SU(2)\) to \(SO(3)\). The spherical harmonics are where that abstract construction becomes a concrete, computable basis.

Schur's Lemma as a Weight Constraint

We now have a language for the data. A feature is a direct sum of type-\(\ell\) pieces, and the piece of type \(\ell\) transforms by the irreducible \(\rho_\ell\). The next question is what a layer may look like. A linear layer is a linear map \(W\) from the input feature space to the output feature space, and the requirement that the layer be equivariant is the requirement that \(W\) be an intertwining map. That is, \(W \rho_{\text{in}}(R) = \rho_{\text{out}}(R)\, W\) must hold for every rotation \(R\). Equivariant linear layers and intertwiners are the same thing. The entire content of an equivariant linear layer is therefore the question: which linear maps intertwine two given representations? And this is the question Schur's lemma answers completely.

Suppose the input and output features are decomposed into irreducibles, \[ \rho_{\text{in}} = \bigoplus_{\ell} m_\ell\, \rho_\ell, \quad \rho_{\text{out}} = \bigoplus_{\ell} n_\ell\, \rho_\ell, \] where \(m_\ell\, \rho_\ell\) denotes the direct sum of \(m_\ell\) copies of \(\rho_\ell\), so that \(m_\ell\) is the number of type-\(\ell\) channels in the input and \(n_\ell\) the number in the output. Such decompositions exist by complete reducibility. An intertwiner \(W\) respects this block structure, because the inclusions of the summands and the projections onto them are themselves intertwiners, and we can read off its possible form block by block. Restricted to a single input copy of \(\rho_\ell\) and a single output copy of \(\rho_{\ell'}\), the corresponding block of \(W\) is itself an intertwiner between the irreducibles \(\rho_\ell\) and \(\rho_{\ell'}\).

Schur's lemma now forces two things. When \(\ell \neq \ell'\) the two irreducibles have different dimensions and are therefore inequivalent, so the only intertwiner between them is zero. An equivariant linear layer cannot move information between features of different type. When \(\ell = \ell'\), the block is a self-intertwiner of the complex irreducible \(\rho_\ell\), and over \(\mathbb{C}\) Schur's lemma says every such map is a scalar multiple of the identity. Each pairing of an input copy with an output copy of the same type therefore contributes exactly one free scalar.

The consequence for the layer's parameter count is dramatic, and it is the mathematical content behind the sample-efficiency argument. An unconstrained linear map between the same two feature spaces would have a number of free parameters equal to the product of their dimensions, \(\bigl(\sum_\ell m_\ell (2\ell+1)\bigr)\bigl(\sum_\ell n_\ell (2\ell+1)\bigr)\). The equivariant layer, by contrast, has exactly \(\sum_\ell m_\ell n_\ell\) free scalars, one per matched pair of same-type channels. Every block between two different types is zero, and every block between two copies of one type is a multiple of the identity.

The \((2\ell+1)\times(2\ell+1)\) machinery inside each type is not learned but dictated by the representation. What the layer learns is only how much of each input channel flows into each output channel of the same type, not the orientation-dependent details, because those details are precisely what the symmetry fixes. The reduction from a product of dimensions to \(\sum_\ell m_\ell n_\ell\) is not a heuristic compression. That count is exactly the dimension of the space of intertwiners, so for the linear layer there is provably nothing more to learn.

One subtlety is worth a word, and it is where the real-versus-complex distinction in Schur's lemma earns its keep. The clean statement "every self-intertwiner is a single scalar" holds over \(\mathbb{C}\). Over \(\mathbb{R}\) an irreducible may in general have a larger endomorphism algebra, so that the count per matched pair exceeds one.

The rotation group is better behaved than the general case. In practice a type-\(\ell\) feature is often stored in a real representation of \(SO(3)\) whose extension to \(\mathbb{C}\), the same matrices acting on complex vectors, is isomorphic to \(\rho_\ell\). A real representation with an irreducible extension in this sense is called absolutely irreducible. Such a representation is irreducible over \(\mathbb{R}\) as well, since a proper nonzero real invariant subspace would extend to a proper nonzero invariant subspace over \(\mathbb{C}\). That every \(\rho_\ell\) arises in this way from a real representation, as the low types listed earlier illustrate, is a fact we take for granted.

For an absolutely irreducible real representation the self-intertwiners are exactly the real scalar multiples of the identity, just as over \(\mathbb{C}\). The reason is that the intertwining equations are linear with real coefficients, so their real solution space has the same dimension as their complex solution space, and the latter is one-dimensional by Schur's lemma applied to the irreducible extension. This is what makes the one-scalar-per-pair count hold in the real-valued arithmetic that many practical networks use, where features are stored as real vectors rather than complex amplitudes. The general bookkeeping, valid for any compact matrix Lie group and either field, is governed by complete reducibility together with Schur's lemma.

Coupling Representations: Clebsch-Gordan and Wigner-Eckart

Schur's lemma settles the linear layers, but a network of only linear layers is just one big linear map. Expressivity requires combining features nonlinearly, and here a new question arises: if we multiply a type-\(\ell_1\) feature by a type-\(\ell_2\) feature, what type is the result? The product of two features lives in the tensor product representation \(\rho_{\ell_1} \otimes \rho_{\ell_2}\), and the tensor product of two irreducibles is in general no longer irreducible. To restore the type-indexed bookkeeping we must decompose it back into irreducibles, and that decomposition is the Clebsch-Gordan theorem.

The Clebsch-Gordan theorem is stated for the Lie algebra \(\mathfrak{sl}(2,\mathbb{C})\), the complexification of \(\mathfrak{so}(3)\), and it transfers to \(SO(3)\) in two steps. Because \(SO(3)\) is connected, the group-algebra correspondence shows that passing to the Lie algebra representation preserves irreducibility and isomorphism classes. The further passage to the complex-linear extension on \(\mathfrak{sl}(2,\mathbb{C})\) preserves them as well, as shown in the transfer from \(SO(3)\) to \(\mathfrak{sl}(2,\mathbb{C})\) that accompanies the \(SO(3)\) classification.

Tensor products are compatible with this passage. The representation of the algebra associated to \(\rho_{\ell_1} \otimes \rho_{\ell_2}\) is the tensor product of the algebra representations, by the formula giving the Lie algebra of an outer tensor product evaluated on the diagonal. A decomposition of \(\rho_{\ell_1} \otimes \rho_{\ell_2}\) into group irreducibles, which exists by complete reducibility, is therefore also a decomposition into algebra irreducibles, and the multiplicities of the latter are fixed by the weights.

In the type indexing, with \(\rho_\ell\) corresponding to \(V_{2\ell}\) in the notation of the Clebsch-Gordan theorem, the decomposition reads \[ \rho_{\ell_1} \otimes \rho_{\ell_2} \cong \bigoplus_{\ell = |\ell_1 - \ell_2|}^{\ell_1 + \ell_2} \rho_\ell, \] a single copy of each type from \(|\ell_1 - \ell_2|\) up to \(\ell_1 + \ell_2\), the decomposition being multiplicity-free. The change of basis effecting this isomorphism is carried out by the Clebsch-Gordan coefficients, the entries of the linear map \(C_{\ell_1 \ell_2 \ell}\) that projects the tensor product onto its type-\(\ell\) summand.

Concretely, a nonlinear coupling layer takes a type-\(\ell_1\) feature and a type-\(\ell_2\) feature, forms their tensor product, and reads off the type-\(\ell\) component \[ f^{\ell} = C_{\ell_1 \ell_2 \ell}\bigl(f^{\ell_1} \otimes f^{\ell_2}\bigr), \] which is again a clean type-\(\ell\) feature, transforming by \(\rho_\ell\). Because the tensor product of two equivariant inputs transforms by \(\rho_{\ell_1} \otimes \rho_{\ell_2}\) and \(C_{\ell_1 \ell_2 \ell}\) is by construction the intertwiner onto \(\rho_\ell\), the output is equivariant. The coupling is thus a genuine nonlinearity that nonetheless preserves the type structure. Clebsch-Gordan is the multiplication table of the type-\(\ell\) features.

A further theorem tightens this picture into its most concise form. Once the types of the input, the output, and the operator being applied are fixed, how much freedom remains in an equivariant map between them? The Wigner-Eckart theorem answers: essentially none. In its simplest case it states that any two vector operators on a fixed irreducible representation are proportional. Given one nonzero such operator, a single complex constant, the reduced matrix element, determines the entire operator, with all of its orientation-dependent structure fixed by the representation theory. The general form, for tensor operators of any type \(k\), lies beyond what our pages prove, and we state it without proof. An equivariant map carrying type-\(\ell_1\) features through a type-\(k\) operator to type-\(\ell_2\) features is determined, up to one scalar per allowed type channel, by Clebsch-Gordan coefficients alone.

Symmetry in Practice

Assembling the pieces gives the shape of an equivariant network for three-dimensional data. Inputs are organized into type-\(\ell\) features, and linear layers are Schur-constrained intertwiners. Nonlinear couplings tensor features together and re-sort the result by Clebsch-Gordan. Where a single rotation-invariant prediction such as a total energy or a class label is finally wanted, the network keeps only the type-\(\ell = 0\) part. The same representation theory that classifies the irreducibles of \(SO(3)\) is, layer by layer, the design manual for the architecture.

Hard-coding the symmetry is not the only way to handle symmetric data. The alternative is to use a more generic architecture and let it learn the symmetry from data, typically by augmenting the training set with transformed copies of each example. Which of the two is preferable is an empirical question whose answer appears to depend on the regime. Both approaches are principled.

There is also a thread that runs forward from here rather than back. The expansion of a spherical signal into type-\(\ell\) components is a Fourier analysis on the sphere. The same idea extends far beyond \(SO(3)\) to general compact groups, where one again decomposes functions on a space with symmetry into pieces indexed by the irreducible representations of the acting group.

The theorem behind this, for functions on the group itself, is the Peter-Weyl theorem, which identifies the suitably normalized matrix entries of the irreducible unitary representations, one from each isomorphism class and written in an orthonormal basis, as a complete orthonormal system for square-integrable functions on the group. It turns the representation theory of this page into a full harmonic analysis, and it is the bridge by which the Fourier analysis met earlier in the curriculum reappears in the setting of groups.

References

• C. Esteves, "Theoretical Aspects of Group Equivariant Neural Networks," arXiv:2004.05154, 2020.
• J. E. Gerken, J. Aronsson, O. Carlsson, H. Linander, F. Ohlsson, C. Petersson, and D. Persson, "Geometric Deep Learning and Equivariant Neural Networks," Artificial Intelligence Review, vol. 56, no. 12, pp. 14605-14662, 2023. arXiv:2105.13926.
• J. Brehmer, S. Behrends, P. de Haan, and T. Cohen, "Does Equivariance Matter at Scale?," Transactions on Machine Learning Research, 2025. arXiv:2410.23179.