Complete Reducibility and Schur's Lemma

Proof:

That \(\Pi\) is a representation is the matrix identity \[ \Pi(s)\,\Pi(x) = \begin{pmatrix} 1 & s \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & x \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & s + x \\ 0 & 1 \end{pmatrix} = \Pi(s + x), \] so \(\Pi\) carries addition in \(\mathbb{R}\) to multiplication in \(GL(2, \mathbb{C})\). The additive group \(\mathbb{R}\) is a matrix Lie group, since \(x \mapsto \Pi(x)\) identifies it with the closed subgroup of \(GL(2, \mathbb{C})\) of upper unitriangular matrices with real upper-right entry, and \(\Pi\) is smooth, its entries being polynomial in \(x\).

Let \(\{e_1, e_2\}\) be the standard basis of \(\mathbb{C}^2\). Since \(\Pi(x)\,e_1 = e_1\) for every \(x\), the line \(\langle e_1 \rangle\) is an invariant subspace. We claim it is the only nontrivial invariant subspace. Suppose \(V\) is a nonzero invariant subspace containing a vector outside \(\langle e_1 \rangle\), say \(v = a e_1 + b e_2\) with \(b \neq 0\). Invariance gives \(\Pi(1) v \in V\), hence also \[ \Pi(1) v - v = b\, e_1 \in V. \] Because \(b \neq 0\), this forces \(e_1 \in V\), and then \(e_2 = (v - a e_1)/b \in V\) as well. Thus \(V = \mathbb{C}^2\). Every invariant subspace is therefore one of \(\{0\}\), \(\langle e_1 \rangle\), or \(\mathbb{C}^2\).

Suppose, for contradiction, that \(\mathbb{C}^2\) were a direct sum of irreducible invariant subspaces. Each summand has dimension \(1\) or \(2\). A two-dimensional summand would be all of \(\mathbb{C}^2\), making \(\mathbb{C}^2\) itself irreducible. But \(\langle e_1 \rangle\) is a nontrivial invariant subspace, so \(\mathbb{C}^2\) is not irreducible. The decomposition would therefore have to be a sum of two one-dimensional invariant subspaces. Yet we showed \(\langle e_1 \rangle\) is the only one-dimensional invariant subspace, and a space cannot be the direct sum of one line with itself. No such decomposition exists, so \(\Pi\) is not completely reducible.

Proof of Point 1:

By complete reducibility, write \[ V = U_1 \oplus U_2 \oplus \cdots \oplus U_k, \] where each \(U_j\) is an irreducible invariant subspace, and let \(U\) be any invariant subspace of \(V\). If \(U = V\), take \(W = \{0\}\) and there is nothing to prove. If \(U \neq V\), then some summand is not contained in \(U\). Choose \(j_1\) with \(U_{j_1} \not\subseteq U\). Because \(U_{j_1}\) is irreducible and \(U_{j_1} \cap U\) is an invariant subspace of \(U_{j_1}\) that is not all of \(U_{j_1}\), we must have \(U_{j_1} \cap U = \{0\}\). Consequently the sum \(U + U_{j_1}\) is direct.

If \(U + U_{j_1} = V\) we are finished, with \(W = U_{j_1}\). Otherwise some summand is not contained in \(U + U_{j_1}\). Choose \(j_2\) with \(U_{j_2} \not\subseteq U + U_{j_1}\). The same irreducibility argument applied to \(U_{j_2}\) gives \((U + U_{j_1}) \cap U_{j_2} = \{0\}\), so \(U + U_{j_1} + U_{j_2}\) is direct. Proceeding in this way, and using that \(V\) is finite-dimensional so the process terminates, we obtain indices \(j_1, j_2, \dots, j_l\) with \[ U + U_{j_1} + \cdots + U_{j_l} = V \] and the sum direct. Setting \(W := U_{j_1} + \cdots + U_{j_l}\), which is an invariant subspace as a sum of invariant subspaces, gives \(V = U \oplus W\), as required.

Proof of Point 2:

Let \(U\) be an invariant subspace of \(V\). We first show that \(U\) inherits the invariant-complement property of Point 1 internally. If \(X \subseteq U\) is an invariant subspace, then \(X\) has an invariant complement within \(U\). By Point 1 applied in \(V\), there is an invariant subspace \(Y\) with \(V = X \oplus Y\). Put \(Z := Y \cap U\), an invariant subspace contained in \(U\). We claim \(U = X \oplus Z\).

For any \(u \in U\), write \(u = x + y\) with \(x \in X\) and \(y \in Y\). Since \(X \subseteq U\), we have \(x \in U\), and therefore \(y = u - x \in U\), so \(y \in Y \cap U = Z\). This shows \(U = X + Z\). The sum is direct because \(X \cap Z \subseteq X \cap Y = \{0\}\). Hence \(U = X \oplus Z\), establishing the internal invariant-complement property.

We may now decompose \(U\) into irreducibles. If \(U\) is irreducible, it is already a (one-term) direct sum of irreducibles. If not, \(U\) has a nontrivial invariant subspace \(X\), and by the property just proved \(U = X \oplus Z\) for some invariant \(Z\). If \(X\) and \(Z\) are irreducible we are done. If not, we apply the same splitting to whichever factor is reducible. Since \(U\) is finite-dimensional, each split strictly lowers the dimension of the factor being decomposed, so the process terminates with \(U\) written as a direct sum of irreducible invariant subspaces. Thus \(U\) is completely reducible.

Proof:

Let \(V\) denote the inner product space on which \(\Pi\) acts, with inner product \(\langle \cdot, \cdot \rangle\). The crux is the claim that the orthogonal complement of an invariant subspace is again invariant. Let \(W \subseteq V\) be an invariant subspace and \(W^{\perp}\) its orthogonal complement, so that \(V = W \oplus W^{\perp}\) as inner product spaces by the projection theorem, which applies because \(V\), being finite-dimensional, is complete, and \(W\) is closed. Because \(\Pi(A)\) preserves the inner product and \(\Pi(A)\Pi(A^{-1}) = I\), for any \(w \in W\) and any \(v \in W^{\perp}\), \[ \begin{align*} \langle \Pi(A) v, w \rangle &= \langle \Pi(A) v, \Pi(A)\Pi(A^{-1}) w \rangle = \langle v, \Pi(A^{-1}) w \rangle \\\\ &= \langle v, w' \rangle = 0, \end{align*} \] where \(w' := \Pi(A^{-1}) w\) lies in \(W\) by invariance, so the pairing vanishes because \(v \perp W\). Thus \(\Pi(A) v \perp W\), that is, \(\Pi(A) v \in W^{\perp}\), and \(W^{\perp}\) is invariant.

The Lie algebra statement follows from the same computation, with skew-adjointness in place of unitarity, where \(T^\ast\) denotes the adjoint of an operator \(T\) with respect to the inner product, characterized by \(\langle Tv, w \rangle = \langle v, T^\ast w \rangle\). Let \(V\) now denote the inner product space on which \(\pi\) acts, and let \(W \subseteq V\) be invariant under \(\pi\), so that \(\pi(X) w \in W\) for every \(X \in \mathfrak{g}\) and every \(w \in W\). Then for any \(X \in \mathfrak{g}\), any \(w \in W\), and any \(v \in W^{\perp}\), \[ \begin{align*} \langle \pi(X) v, w \rangle &= \langle v, \pi(X)^\ast w \rangle \\\\ &= \langle v, -\pi(X) w \rangle = 0, \end{align*} \] where \(-\pi(X) w\) lies in \(W\), by invariance of \(W\) together with its closure under scalar multiplication, so the pairing vanishes because \(v \perp W\). Hence \(\pi(X) v \in W^{\perp}\), and \(W^{\perp}\) is invariant under \(\pi\) as well.

With the orthogonal complement of an invariant subspace known to be invariant, the decomposition is immediate. If \(V\) is irreducible we are done. Otherwise choose an invariant subspace \(W\) with \(\{0\} \neq W \neq V\). Then \(V = W \oplus W^{\perp}\) with both summands invariant, hence representations in their own right. Each of \(W\) and \(W^{\perp}\) is either irreducible or splits further as an orthogonal direct sum of invariant subspaces. Since \(V\) is finite-dimensional this cannot continue indefinitely, and we arrive at a decomposition of \(V\) into irreducible invariant subspaces. Hence \(\Pi\) is completely reducible, and the same induction applied to \(\pi\) gives the Lie algebra statement.

Construction of the invariant integral:

We require a notion of integration over \(G\) that is invariant under the right action of the group. We obtain it from a right-invariant differential form. If \(G \subseteq M_n(\mathbb{C})\) is a matrix Lie group, the tangent space at the identity is its Lie algebra \(\mathfrak{g}\), and the tangent space \(T_A G\) at any point \(A \in G\) is the space of matrices \(\{ X A : X \in \mathfrak{g} \}\). A smooth curve \(\gamma\) in \(G\) through \(A\) is \(\gamma = \delta A\) for the curve \(\delta = \gamma A^{-1}\) through \(I\), so its velocity is \(\delta'(0) A\) with \(\delta'(0) \in \mathfrak{g}\), and conversely \(t \mapsto \exp(tX) A\) has velocity \(XA\). Let \(k\) be the dimension of \(\mathfrak{g}\) as a real vector space, and choose a nonzero \(k\)-linear alternating form \(\alpha_I : \mathfrak{g}^k \to \mathbb{R}\). Such a form exists. Fix a basis of \(\mathfrak{g}\) and let \(\alpha_I(Y_1, \dots, Y_k)\) be the determinant of the matrix whose columns are the coordinates of \(Y_1, \dots, Y_k\). We transport \(\alpha_I\) to every point of \(G\) by the right action, defining a \(k\)-linear alternating form \(\alpha_A\) on \(T_A G\) by \[ \alpha_A(Y_1, \dots, Y_k) = \alpha_I\bigl(Y_1 A^{-1}, \dots, Y_k A^{-1}\bigr), \quad Y_1, \dots, Y_k \in T_A G. \] The assignment \(A \mapsto \alpha_A\) is a \(k\)-form \(\alpha\) on \(G\), smooth because \(A^{-1}\) depends smoothly on \(A\), and right-invariant by construction.

Such a top-degree form is exactly what is needed to integrate. For a smooth function \(f : G \to \mathbb{R}\), the product \(f\alpha\) is again a \(k\)-form. On the compact manifold \(G\) it has a well-defined integral, which we write as \[ \int_G f(A)\, \alpha(A). \] That this integral does not depend on the choice of local coordinates is the algebraic content of how a top-degree form pulls back. Under an orientation-preserving change of coordinates the form acquires precisely the Jacobian determinant, which is then positive and so equals the factor that the classical change-of-variables theorem inserts, and the two cancel. The full construction of the integral of a differential form belongs to integration theory on manifolds, together with orientation and the partition-of-unity assembly over charts, and we take it as given here. We use only three of its properties. The integral is linear in the form, it is positive when \(f\) is a positive continuous function, and it is unchanged under pulling back by an orientation-preserving diffeomorphism of \(G\). The group enters only through the third, in the form of the right-invariance recorded next.

A word on orientation belongs here. Integration of a top-degree form on \(G\) requires \(G\) to be oriented, and \(\alpha\) supplies the orientation itself. Each \(\alpha_A\) is the transport of the nonzero form \(\alpha_I\) by the linear isomorphism \(Y \mapsto YA^{-1}\) of \(T_A G\) onto \(\mathfrak{g}\), so \(\alpha\) vanishes nowhere, and a nowhere-vanishing top-degree form determines an orientation of \(G\), the one in which \(\alpha\) is positive. With respect to this orientation every right translation \(R_B : A \mapsto AB\) is orientation-preserving, since \(R_B^*\alpha = \alpha\) by construction. No connectedness assumption is needed, and disconnected groups such as \(O(n)\) are covered without change.

Because \(\alpha\) was built from the right action, the resulting integral is invariant under that action. For \(B \in G\) the right translation \(R_B\) satisfies \(R_B^*\alpha = \alpha\), since \((R_B^*\alpha)_A(Y_1, \dots, Y_k) = \alpha_{AB}(Y_1 B, \dots, Y_k B) = \alpha_I(Y_1 A^{-1}, \dots, Y_k A^{-1})\). Hence \(R_B^*(f\alpha) = (f \circ R_B)\,\alpha\), and invariance of the integral under \(R_B\) gives, for every \(B \in G\), \[ \int_G f(AB)\, \alpha(A) = \int_G f(A)\, \alpha(A). \] Compactness of \(G\) is what guarantees that the integral is finite for every smooth function, the averaging integrand below among them.

Proof of the theorem:

Let \(\Pi\) be a finite-dimensional representation of \(G\) on \(V\). Choose any inner product \(\langle \cdot, \cdot \rangle\) on \(V\) and average it over the group, defining \(\langle \cdot, \cdot \rangle_G\) by \[ \langle v, w \rangle_G = \int_G \langle \Pi(A) v, \Pi(A) w \rangle\, \alpha(A). \] The integrand is complex-valued, and we integrate its real and imaginary parts. The result is again an inner product. Linearity and conjugate symmetry pass through the integral, and positive definiteness holds because the integrand \(\langle \Pi(A) v, \Pi(A) v \rangle\) is a positive continuous function of \(A\) when \(v \neq 0\), so its integral against \(\alpha\) is positive.

We claim \(\Pi(B)\) is unitary with respect to \(\langle \cdot, \cdot \rangle_G\) for every \(B \in G\). Using that \(\Pi\) is a homomorphism and then the right-invariance of the integral, \[ \begin{align*} \langle \Pi(B) v, \Pi(B) w \rangle_G &= \int_G \langle \Pi(A)\Pi(B) v, \Pi(A)\Pi(B) w \rangle\, \alpha(A) \\\\ &= \int_G \langle \Pi(AB) v, \Pi(AB) w \rangle\, \alpha(A) \\\\ &= \int_G \langle \Pi(A) v, \Pi(A) w \rangle\, \alpha(A) \\\\ &= \langle v, w \rangle_G, \end{align*} \] where the third equality is the right-invariance applied to the function \(A \mapsto \langle \Pi(A) v, \Pi(A) w \rangle\). Thus every \(\Pi(B)\) preserves \(\langle \cdot, \cdot \rangle_G\), so \(\Pi\) is a unitary representation with respect to this averaged inner product. By the preceding proposition, \(\Pi\) is completely reducible.

Proof of Point 1:

We argue the group case. The Lie algebra case requires only the obvious notational changes. Write \(\Pi\) for the action on \(V\) and \(\Sigma\) for the action on \(W\), so that \(\phi(\Pi(A) v) = \Sigma(A) \phi(v)\) for all \(A\). Then \(\ker\phi\) is an invariant subspace of \(V\). If \(v \in \ker\phi\), then \[ \phi(\Pi(A) v) = \Sigma(A) \phi(v) = \Sigma(A) \cdot 0 = 0, \] so \(\Pi(A) v \in \ker\phi\). Since \(V\) is irreducible, \(\ker\phi = \{0\}\) or \(\ker\phi = V\), so \(\phi\) is either one-to-one or zero.

Suppose \(\phi\) is one-to-one. Then \(\operatorname{im}\phi\) is a nonzero subspace of \(W\), and it is invariant. If \(w = \phi(v)\) lies in the image, then \[ \Sigma(A) w = \Sigma(A) \phi(v) = \phi(\Pi(A) v) \in \operatorname{im}\phi. \] Since \(W\) is irreducible and \(\operatorname{im}\phi\) is nonzero and invariant, we must have \(\operatorname{im}\phi = W\). Thus \(\phi\) is either zero or one-to-one and onto, that is, an isomorphism.

Proof of Point 2:

Suppose \(V\) is an irreducible complex representation and \(\phi : V \to V\) intertwines, so \(\phi\,\Pi(A) = \Pi(A)\,\phi\) for all \(A\). Since we are working over the algebraically closed field \(\mathbb{C}\), the operator \(\phi\) has at least one eigenvalue \(\lambda \in \mathbb{C}\). Let \(U\) be the corresponding eigenspace. Then each \(\Pi(A)\) maps \(U\) into itself. If \(\phi u = \lambda u\), then \[ \phi\bigl(\Pi(A) u\bigr) = \Pi(A)\,\phi(u) = \Pi(A)(\lambda u) = \lambda\,\Pi(A) u, \] so \(\Pi(A) u\) is again a \(\lambda\)-eigenvector, that is, \(\Pi(A) u \in U\). Thus \(U\) is an invariant subspace. Since \(\lambda\) is an eigenvalue, \(U \neq \{0\}\), and irreducibility forces \(U = V\). But \(U = V\) means \(\phi\) acts as \(\lambda\) on all of \(V\), that is, \(\phi = \lambda I\).

The eigenvalue step is exactly where complexity is essential. The existence of an eigenvalue used the algebraic closure of \(\mathbb{C}\). Over \(\mathbb{R}\) the conclusion fails. The rotation representation of \(SO(2)\) on \(\mathbb{R}^2\) is irreducible, since no real line is fixed by all rotations, yet the rotation by a right angle commutes with every element of the representation while being no real scalar multiple of the identity.

Proof of Point 3:

Let \(\phi_1, \phi_2 : V \to W\) be nonzero intertwining maps between irreducible complex representations. By Point 1, \(\phi_2\) is an isomorphism, so \(\phi_2^{-1}\) exists and \(\phi_1 \circ \phi_2^{-1} : W \to W\) is an intertwining map of \(W\) with itself. By Point 2 it equals \(\lambda I\) for some \(\lambda \in \mathbb{C}\), whence \(\phi_1 = \lambda\, \phi_2\).

Proof:

If \(A\) lies in the center of \(G\), then \(\Pi(A)\) commutes with \(\Pi(B)\) for every \(B \in G\), since \(\Pi(A)\Pi(B) = \Pi(AB) = \Pi(BA) = \Pi(B)\Pi(A)\). Thus \(\Pi(A)\) is an intertwining map of the irreducible complex representation with itself, and Point 2 of Schur's lemma gives \(\Pi(A) = \lambda I\). The Lie algebra case is identical, with the bracket relation \(\pi(X)\pi(Y) - \pi(Y)\pi(X) = \pi([X, Y]) = 0\) for central \(X\) showing \(\pi(X)\) commutes with all \(\pi(Y)\).

Proof:

We argue the group case. If \(G\) is commutative, then the center of \(G\) is all of \(G\), so by the preceding corollary \(\Pi(A) = \lambda I\) for each \(A \in G\), with the scalar depending on \(A\). But if every \(\Pi(A)\) is a multiple of the identity, then every subspace of \(V\) is invariant. The only way \(V\) can avoid having a nontrivial invariant subspace, and so the only way it can be irreducible, is for \(V\) to be one-dimensional. The Lie algebra case is identical, the center of a commutative Lie algebra being the whole algebra.