Simplicial Neural Networks

From Edges to Simplices

A graph neural network processes data that lives on the vertices of a graph by repeatedly exchanging information between neighbors. Each layer is a message passing layer, in which a vertex collects a permutation-invariant aggregate of its neighbors' features and updates its own state accordingly. The information available to a message passing layer is therefore routed through a single relation, the edge. An edge encodes a relationship between exactly two vertices. A graph, by its very definition, is a model of pairwise interaction.

For a great many systems this is the right abstraction, but for many others it quietly discards structure. A chemical reaction may involve three molecules at once in a way that is not reducible to its three pairwise sub-interactions. A co-authorship record in which four researchers wrote a single paper together is not faithfully represented by the six edges of the pairwise graph on those four authors. That graph cannot distinguish "four people co-wrote one paper" from "these same four people happen to share six separate two-person papers." A triangle of vertices that bounds a filled face is a genuinely different object from a triangle that merely encloses a hole. In each case the missing information is a higher-order relation: an interaction supported on a set of three, four, or more vertices that is irreducibly present as a unit, not as a collection of its pairwise shadows.

The mathematical structure that records relations of every order at once is the abstract simplicial complex. A graph is a simplicial complex truncated at dimension one, so that only vertices (\(0\)-simplices) and edges (\(1\)-simplices) remain. Admitting \(2\)-simplices (triangles), \(3\)-simplices (tetrahedra), and higher faces restores the higher-order relations a graph cannot see, and it does so in an internally consistent way. Whenever a \(k\)-simplex belongs to the complex, all of its faces do too. Data now lives not only on vertices but on simplices of every dimension, with a feature vector attached to each triangle, each tetrahedron, and so on.

Higher-order data raises the question that the rest of the page answers. For a graph neural network the neighborhood of a vertex is unambiguous: the vertices joined to it by an edge. But what is the neighborhood of a \(2\)-simplex? Two triangles can be adjacent because they share a common edge, or because they are both faces of a common tetrahedron. These two adjacencies point in opposite dimensional directions, and a message passing scheme must account for both. The operator that organizes them is already in hand: the discrete Hodge Laplacian, built from the same boundary maps that define the complex's homology.

The Hodge Laplacian as a Propagation Operator

The two notions of adjacency identified at the end of the previous section are not ad hoc. They are read directly off the maps that build the complex. Recall that the boundary operator \(\partial_k\) sends each oriented \(k\)-simplex to the alternating sum of its \((k-1)\)-dimensional faces. It acts on the chain group \(C_k(K)\), the space of real feature vectors indexed by the \(k\)-simplices. We equip each \(C_k(K)\) with the inner product for which the oriented simplices form an orthonormal basis, so that the adjoint \(\partial_k^\top\) is the matrix transpose and maps a \((k-1)\)-chain back up to the \(k\)-chains whose boundary it touches.

With a down-map \(\partial_k\) and an up-map \(\partial_{k+1}^\top\) available at each level, the operator that combines both is the Hodge Laplacian \[ \boldsymbol{L}_k = \underbrace{\partial_{k+1}\partial_{k+1}^\top}_{\boldsymbol{L}_k^{\mathrm{up}}} + \underbrace{\partial_k^\top \partial_k}_{\boldsymbol{L}_k^{\mathrm{down}}}. \]

The two summands are exactly the two adjacencies. The upper Laplacian \(\boldsymbol{L}_k^{\mathrm{up}}\) couples two \(k\)-simplices when they are faces of a common \((k+1)\)-simplex. The lower Laplacian \(\boldsymbol{L}_k^{\mathrm{down}}\) couples them when they share a common \((k-1)\)-face. For a feature vector \(\mathbf{h} \in C_k(K)\), the products \(\boldsymbol{L}_k^{\mathrm{up}} \mathbf{h}\) and \(\boldsymbol{L}_k^{\mathrm{down}} \mathbf{h}\) aggregate each simplex's feature with those of its coface neighbors and of its shared-face neighbors, respectively. Each product is thus a message passing step in which the complex's own combinatorics supplies the neighborhood.

In the sum \(\boldsymbol{L}_k \mathbf{h}\), the upper and lower couplings cancel for simplices that are adjacent in both ways. This cancellation is a consequence of the identity \(\partial_k \partial_{k+1} = 0\), which states that the boundary of a boundary is zero. For the three edges of a single filled triangle, \(\boldsymbol{L}_1\) is three times the identity, so the edges exchange nothing through it.

The case \(k = 0\) shows that the construction genuinely extends the graph case. The boundary map \(\partial_1\) from edges to vertices is, up to an overall sign, the oriented incidence matrix \(\boldsymbol{B}\), and with the convention \(\partial_0 = 0\) the lower part vanishes. What remains is \[ \boldsymbol{L}_0 = \partial_1 \partial_1^\top = \boldsymbol{B}\boldsymbol{B}^\top, \] which is exactly the graph Laplacian on vertices.

The diffusion \(\mathbf{h} \mapsto \mathbf{h} - \alpha \boldsymbol{L}_0 \mathbf{h}\) is a symmetric Laplacian smoothing step. It is the unnormalized analogue of the propagation used in spectral graph convolutions, which typically replace \(\boldsymbol{L}_0\) by a normalized variant. The Hodge Laplacian thus contains the graph Laplacian as its \(0\)-dimensional level, and the higher levels \(\boldsymbol{L}_1, \boldsymbol{L}_2, \ldots\) bring in, through the simplices of dimension two and above, the higher-order interactions a graph could not express. The resulting propagation layer is the following.

Separating \(\boldsymbol{L}_k^{\mathrm{up}}\) from \(\boldsymbol{L}_k^{\mathrm{down}}\) with independent weights lets the network weigh the two directions differently rather than collapse them into \(\boldsymbol{L}_k\). One subtlety remains. The orientation chosen for each simplex enters through \(\partial_k\), so a simplex's feature carries a sign that flips with reorientation, and a well-posed architecture must respect that sign. The next section takes up this orientation question.

Harmonic Features and What the Network Sees

For suitable weights, a message passing layer driven by \(\boldsymbol{L}_k\) is, before its nonlinearity, a diffusion. It repeatedly damps a feature vector in the directions \(\boldsymbol{L}_k\) sees. To understand what such a network can stably represent, we ask what survives this diffusion. Because \(\boldsymbol{L}_k = \partial_{k+1}\partial_{k+1}^\top + \partial_k^\top\partial_k\) is a sum of two operators of the form \(MM^\top\), it is symmetric positive semi-definite, and \[ \langle \boldsymbol{L}_k \mathbf{h}, \mathbf{h} \rangle = \lVert \partial_{k+1}^\top \mathbf{h} \rVert^2 + \lVert \partial_k \mathbf{h} \rVert^2 . \]

A feature vector lies in \(\ker \boldsymbol{L}_k\) precisely when both terms vanish, and these are the features a Laplacian smoothing \(\mathbf{h} \mapsto \mathbf{h} - \alpha \boldsymbol{L}_k \mathbf{h}\) leaves fixed: the harmonic \(k\)-chains. Every other eigendirection has a strictly positive eigenvalue and, for a suitably small step size \(\alpha\), is attenuated under repeated smoothing. The harmonic space is thus the fixed-point subspace of this underlying linear diffusion. A trained network, which interleaves learnable weights and a nonlinearity, is not constrained to converge there.

What that fixed-point subspace contains is the content of the discrete Hodge theorem, which states that the harmonic \(k\)-chains are isomorphic to the \(k\)-th homology of the complex, \[ \ker \boldsymbol{L}_k \cong H_k(K; \mathbb{R}), \quad \dim \ker \boldsymbol{L}_k = \beta_k . \]

The stable features of a simplicial network are therefore not arbitrary. The dimension of the space the diffusion preserves equals the \(k\)-th Betti number, and a harmonic \(k\)-chain is the canonical representative of a homology class. At \(k = 0\) this counts connected components, at \(k = 1\) independent loops, and at \(k = 2\) enclosed voids. The linear diffusion underlying a network built on \(\boldsymbol{L}_k\) reads the topology of its domain directly, in a subspace the propagation itself preserves. These architectures are topological in precisely this sense, since the invariants they most readily express are the homological ones.

The harmonic viewpoint also answers the orientation question raised at the close of the previous section. Reorienting a \(k\)-simplex flips the sign of its column in \(\partial_k\) and of its row in \(\partial_{k+1}\), and hence the sign of the corresponding coordinate of every feature vector. If \(\boldsymbol{S}\) is the diagonal matrix of these signs, the Laplacian changes to \(\boldsymbol{S}\boldsymbol{L}_k\boldsymbol{S}\), so \(\boldsymbol{S}\) carries each eigenspace of the old \(\boldsymbol{L}_k\), and in particular \(\ker \boldsymbol{L}_k\), onto the corresponding eigenspace of the new one. A feature read off the harmonic space therefore reflects genuine topological content rather than the bookkeeping of a chosen orientation.

The Hodge-Laplacian layer itself commutes with a reorientation when \(\sigma\) is odd, as \(\tanh\) is, since a reorientation multiplies the argument of \(\sigma\) by \(\boldsymbol{S}\) and \(\sigma(-z) = -\sigma(z)\). We can now assemble the architecture.

The definition describes the minimal model, which updates each level independently, so different dimensions exchange no information within the layers and have in common only the boundary maps from which their Laplacians are built. Richer variants, which go beyond this definition, also pass messages between a simplex and its faces and cofaces explicitly. What every such model shares is the organizing role of the Hodge Laplacian and the homological meaning of the features it preserves.

Simplicial Networks in the Geometric Deep Learning Framework

The geometric deep learning framework organizes architectures by the symmetry of their domain. Its graph neural network instance is equivariant to permutations of a vertex set and exchanges information across edges. Its equivariant instance is equivariant to a continuous group such as the rotations \(SO(3)\) and decomposes features into irreducible representation types.

The simplicial neural network developed here is a third instance with a distinct place. Its domain is a discrete combinatorial object, but one carrying structure of every dimension. The symmetries it respects, all of which leave the complex intact, are the relabelings of the simplices and, when \(\sigma\) is odd, their reorientations. Where the graph network is the pairwise, one-dimensional case, the simplicial network is a higher-dimensional discrete model. It is the discrete counterpart to the continuous, manifold-based architectures for data on curved domains.

The Hodge-Laplacian layer of the preceding sections is one concrete realization of a more general scheme. Its propagation made the specific choice that two \(k\)-simplices are neighbors when they share a coface or a face, but nothing forces that choice. Replacing it with an abstract neighborhood function, a rule that assigns to each cell a set of related cells, and allowing a family of such functions yields the general message passing scheme on a higher-order domain.

Recovering the concrete layer is now a matter of choosing \(\mathcal{N}\). On a simplicial complex, we take two neighborhoods of \(x\), \(\mathcal{N}_1(x)\) the cells \(y\) with a nonzero entry \((\boldsymbol{L}_k^{\mathrm{up}})_{xy}\) and \(\mathcal{N}_2(x)\) those with a nonzero entry \((\boldsymbol{L}_k^{\mathrm{down}})_{xy}\). The messages are \(m_{x,y}^1 = (\boldsymbol{L}_k^{\mathrm{up}})_{xy}\,\mathbf{h}_y \boldsymbol{W}_k^{\mathrm{up}}\) and \(m_{x,y}^2 = (\boldsymbol{L}_k^{\mathrm{down}})_{xy}\,\mathbf{h}_y \boldsymbol{W}_k^{\mathrm{down}}\), with \(\bigoplus\) the sum and \(\bigotimes\) addition, and \(\psi\) adds the self term \(\mathbf{h}_x \boldsymbol{W}_k^{\mathrm{self}}\) and applies \(\sigma\). Other choices recover message passing on cell complexes and hypergraphs, so one set of update rules subsumes all these cases. Graph neural networks correspond to the case where \(\mathcal{X}\) is a graph and \(\mathcal{N}\) consists of its single neighborhood of adjacent vertices.

The same Hodge Laplacian points beyond the discrete setting. Its kernel-equals-homology statement is the discrete shadow of a classical theorem for the Laplace-de Rham operator on a compact Riemannian manifold, where harmonic differential forms represent de Rham cohomology. On a smooth triangulation of such a manifold, when it is oriented, with an inner product on the chains induced by the metric in place of the one fixed above, the correspondence can be made precise as a convergence result under repeated subdivision, which we state without proof.

A separate strand leads instead toward topological data analysis. Tracking how the homology of a complex changes as it is built up through a filtration yields persistent homology, a complementary way of turning topology into a feature, independent of the message passing developed here.