Complete Reducibility and Schur's Lemma

Proof:

That \(\Pi\) is a representation is the matrix identity \[ \Pi(s)\,\Pi(x) = \begin{pmatrix} 1 & s \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & x \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & s + x \\ 0 & 1 \end{pmatrix} = \Pi(s + x), \] so \(\Pi\) carries addition in \(\mathbb{R}\) to multiplication in \(GL(2; \mathbb{C})\).

Let \(\{e_1, e_2\}\) be the standard basis of \(\mathbb{C}^2\). Since \(\Pi(x)\,e_1 = e_1\) for every \(x\), the line \(\langle e_1 \rangle\) is an invariant subspace. We claim it is the only nontrivial invariant subspace. Suppose \(V\) is a nonzero invariant subspace containing a vector outside \(\langle e_1 \rangle\), say \(v = a e_1 + b e_2\) with \(b \neq 0\). Invariance gives \(\Pi(1) v \in V\), hence also \[ \Pi(1) v - v = b\, e_1 \in V. \] Because \(b \neq 0\), this forces \(e_1 \in V\), and then \(e_2 = (v - a e_1)/b \in V\) as well. Thus \(V = \mathbb{C}^2\). Every invariant subspace is therefore one of \(\{0\}\), \(\langle e_1 \rangle\), or \(\mathbb{C}^2\).

Suppose, for contradiction, that \(\mathbb{C}^2\) were a direct sum of irreducible invariant subspaces. Each summand has dimension \(1\) or \(2\). A two-dimensional summand would be all of \(\mathbb{C}^2\), making \(\mathbb{C}^2\) itself irreducible. But \(\langle e_1 \rangle\) is a nontrivial invariant subspace, so \(\mathbb{C}^2\) is not irreducible. The decomposition would therefore have to be a sum of two one-dimensional invariant subspaces. Yet we showed \(\langle e_1 \rangle\) is the only one-dimensional invariant subspace, and a space cannot be the direct sum of one line with itself. No such decomposition exists, so \(\Pi\) is not completely reducible.

Proof of Point 1:

By complete reducibility, write \[ V = U_1 \oplus U_2 \oplus \cdots \oplus U_k, \] where each \(U_j\) is an irreducible invariant subspace, and let \(U\) be any invariant subspace of \(V\). If \(U = V\), take \(W = \{0\}\) and there is nothing to prove. If \(U \neq V\), then some summand is not contained in \(U\). Choose \(j_1\) with \(U_{j_1} \not\subseteq U\). Because \(U_{j_1}\) is irreducible and \(U_{j_1} \cap U\) is an invariant subspace of \(U_{j_1}\) that is not all of \(U_{j_1}\), we must have \(U_{j_1} \cap U = \{0\}\). Consequently the sum \(U + U_{j_1}\) is direct.

If \(U + U_{j_1} = V\) we are finished, with \(W = U_{j_1}\). Otherwise some summand is not contained in \(U + U_{j_1}\). Choose \(j_2\) with \(U_{j_2} \not\subseteq U + U_{j_1}\). The same irreducibility argument applied to \(U_{j_2}\) gives \((U + U_{j_1}) \cap U_{j_2} = \{0\}\), so \(U + U_{j_1} + U_{j_2}\) is direct. Proceeding in this way, and using that \(V\) is finite-dimensional so the process terminates, we obtain indices \(j_1, j_2, \dots, j_l\) with \[ U + U_{j_1} + \cdots + U_{j_l} = V \] and the sum direct. Setting \(W := U_{j_1} + \cdots + U_{j_l}\), which is an invariant subspace as a sum of invariant subspaces, gives \(V = U \oplus W\), as required.

Proof of Point 2:

Let \(U\) be an invariant subspace of \(V\). We first show that \(U\) inherits the invariant-complement property of Point 1 internally. If \(X \subseteq U\) is an invariant subspace, then \(X\) has an invariant complement within \(U\). By Point 1 applied in \(V\), there is an invariant subspace \(Y\) with \(V = X \oplus Y\). Put \(Z := Y \cap U\), an invariant subspace contained in \(U\). We claim \(U = X \oplus Z\).

For any \(u \in U\), write \(u = x + y\) with \(x \in X\) and \(y \in Y\). Since \(X \subseteq U\), we have \(x \in U\), and therefore \(y = u - x \in U\), so \(y \in Y \cap U = Z\). This shows \(U = X + Z\). The sum is direct because \(X \cap Z \subseteq X \cap Y = \{0\}\). Hence \(U = X \oplus Z\), establishing the internal invariant-complement property.

We may now decompose \(U\) into irreducibles. If \(U\) is irreducible, it is already a (one-term) direct sum of irreducibles. If not, \(U\) has a nontrivial invariant subspace \(X\), and by the property just proved \(U = X \oplus Z\) for some invariant \(Z\). If \(X\) and \(Z\) are irreducible we are done. If not, we apply the same splitting to whichever factor is reducible. Since \(U\) is finite-dimensional, each split strictly lowers the dimension of the factor being decomposed, so the process terminates with \(U\) written as a direct sum of irreducible invariant subspaces. Thus \(U\) is completely reducible.

Proof:

Let \(V\) denote the inner product space on which \(\Pi\) acts, with inner product \(\langle \cdot, \cdot \rangle\). The crux is the claim that the orthogonal complement of an invariant subspace is again invariant. Let \(W \subseteq V\) be an invariant subspace and \(W^{\perp}\) its orthogonal complement, so that \(V = W \oplus W^{\perp}\) as inner product spaces. Because \(\Pi\) is unitary, \(\Pi(A)^\ast = \Pi(A)^{-1} = \Pi(A^{-1})\) for every \(A \in G\). Then for any \(w \in W\) and any \(v \in W^{\perp}\), \[ \begin{align*} \langle \Pi(A) v, w \rangle &= \langle v, \Pi(A)^\ast w \rangle = \langle v, \Pi(A^{-1}) w \rangle \\\\ &= \langle v, w' \rangle = 0, \end{align*} \] where \(w' := \Pi(A^{-1}) w\) lies in \(W\) by invariance, so the pairing vanishes because \(v \perp W\). Thus \(\Pi(A) v \perp W\), that is, \(\Pi(A) v \in W^{\perp}\), and \(W^{\perp}\) is invariant.

The Lie algebra statement follows from the same computation, with skew-adjointness in place of unitarity. Let \(V\) now denote the inner product space on which \(\pi\) acts, and let \(W \subseteq V\) be invariant under \(\pi\), so that \(\pi(X) w \in W\) for every \(X \in \mathfrak{g}\) and every \(w \in W\). Then for any \(X \in \mathfrak{g}\), any \(w \in W\), and any \(v \in W^{\perp}\), \[ \begin{align*} \langle \pi(X) v, w \rangle &= \langle v, \pi(X)^\ast w \rangle \\\\ &= \langle v, -\pi(X) w \rangle = 0, \end{align*} \] where \(-\pi(X) w\) lies in \(W\), by invariance of \(W\) together with its closure under scalar multiplication, so the pairing vanishes because \(v \perp W\). Hence \(\pi(X) v \in W^{\perp}\), and \(W^{\perp}\) is invariant under \(\pi\) as well.

With the orthogonal complement of an invariant subspace known to be invariant, the decomposition is immediate. If \(V\) is irreducible we are done. Otherwise choose an invariant subspace \(W\) with \(\{0\} \neq W \neq V\). Then \(V = W \oplus W^{\perp}\) with both summands invariant, hence representations in their own right. Each of \(W\) and \(W^{\perp}\) is either irreducible or splits further as an orthogonal direct sum of invariant subspaces. Since \(V\) is finite-dimensional this cannot continue indefinitely, and we arrive at a decomposition of \(V\) into irreducible invariant subspaces. Hence \(\Pi\) is completely reducible, and the same induction applied to \(\pi\) gives the Lie algebra statement.

Construction of the invariant integral:

We require a notion of integration over \(G\) that is invariant under the right action of the group. We obtain it from a right-invariant differential form. If \(G \subseteq M_n(\mathbb{C})\) is a matrix Lie group, the tangent space at the identity is its Lie algebra \(\mathfrak{g}\), and the tangent space \(T_A G\) at any point \(A \in G\) is the space of matrices \(\{ X A : X \in \mathfrak{g} \}\). Let \(k\) be the dimension of \(\mathfrak{g}\) as a real vector space, and choose a nonzero \(k\)-linear alternating form \(\alpha_I : \mathfrak{g}^k \to \mathbb{R}\). Such a form exists and is unique up to a scalar multiple, since the alternating \(k\)-forms on a \(k\)-dimensional space form a one-dimensional space. We transport \(\alpha_I\) to every point of \(G\) by the right action, defining a \(k\)-linear alternating form \(\alpha_A\) on \(T_A G\) by \[ \alpha_A(Y_1, \dots, Y_k) = \alpha_I\bigl(Y_1 A^{-1}, \dots, Y_k A^{-1}\bigr), \quad Y_1, \dots, Y_k \in T_A G. \] The assignment \(A \mapsto \alpha_A\) is a \(k\)-form \(\alpha\) on \(G\), right-invariant by construction.

Such a top-degree form is exactly what is needed to integrate. For a smooth function \(f : G \to \mathbb{R}\), the product \(f\alpha\) is again a \(k\)-form, and on the compact manifold \(G\) it has a well-defined integral, which we write as \[ \int_G f(A)\, \alpha(A). \] That this integral does not depend on the choice of local coordinates is the algebraic content of how a top-degree form pulls back. Under a change of coordinates the form acquires precisely the Jacobian determinant that the classical change-of-variables theorem inserts, and the two cancel. The full construction of the integral of a differential form belongs to integration theory on manifolds, together with orientation and the partition-of-unity assembly over charts, and we take it as given here. Only the right-invariance, recorded next, enters the argument.

One caveat belongs here. Integration of a top-degree form is sensitive to orientation, and on a disconnected group such as \(O(n)\) a right translation by an element of determinant \(-1\) reverses the orientation, which would flip the sign of the integral. To keep the construction valid for every compact group, one integrates the associated density \(|\alpha|\) rather than the oriented form \(\alpha\). The density is insensitive to orientation, so the right-invariance and the positivity used below hold without a connectedness assumption. For a connected group the distinction is immaterial and the oriented form suffices. We continue to write \(\alpha\), understanding the density when the group is not connected.

Because \(\alpha\) was built from the right action, the resulting integral is invariant under that action: for every \(B \in G\), \[ \int_G f(AB)\, \alpha(A) = \int_G f(A)\, \alpha(A). \] Compactness of \(G\) is what guarantees the integral is finite, so this right-invariant integral exists precisely in the setting we care about.

Proof of the theorem:

Let \(\Pi\) be a finite-dimensional representation of \(G\) on \(V\). Choose any inner product \(\langle \cdot, \cdot \rangle\) on \(V\) and average it over the group, defining \(\langle \cdot, \cdot \rangle_G : V \times V \to \mathbb{C}\) by \[ \langle v, w \rangle_G = \int_G \langle \Pi(A) v, \Pi(A) w \rangle\, \alpha(A). \] This is again an inner product. It is linear and conjugate-symmetric as an integral of such, and it is positive definite because the integrand \(\langle \Pi(A) v, \Pi(A) v \rangle\) is positive for every \(A\) when \(v \neq 0\), so its integral against \(\alpha\) is positive.

We claim \(\Pi(B)\) is unitary with respect to \(\langle \cdot, \cdot \rangle_G\) for every \(B \in G\). Using that \(\Pi\) is a homomorphism and then the right-invariance of the integral, \[ \begin{align*} \langle \Pi(B) v, \Pi(B) w \rangle_G &= \int_G \langle \Pi(A)\Pi(B) v, \Pi(A)\Pi(B) w \rangle\, \alpha(A) \\\\ &= \int_G \langle \Pi(AB) v, \Pi(AB) w \rangle\, \alpha(A) \\\\ &= \int_G \langle \Pi(A) v, \Pi(A) w \rangle\, \alpha(A) \\\\ &= \langle v, w \rangle_G, \end{align*} \] where the third equality is the right-invariance applied to the function \(A \mapsto \langle \Pi(A) v, \Pi(A) w \rangle\). Thus every \(\Pi(B)\) preserves \(\langle \cdot, \cdot \rangle_G\), so \(\Pi\) is a unitary representation with respect to this averaged inner product. By the preceding proposition, \(\Pi\) is completely reducible.

Proof of Point 1:

We argue the group case. The Lie algebra case requires only the obvious notational changes. Write \(\Pi\) for the action on \(V\) and \(\Sigma\) for the action on \(W\), so that \(\phi(\Pi(A) v) = \Sigma(A) \phi(v)\) for all \(A\). Then \(\ker\phi\) is an invariant subspace of \(V\). If \(v \in \ker\phi\), then \[ \phi(\Pi(A) v) = \Sigma(A) \phi(v) = \Sigma(A) \cdot 0 = 0, \] so \(\Pi(A) v \in \ker\phi\). Since \(V\) is irreducible, \(\ker\phi = \{0\}\) or \(\ker\phi = V\), so \(\phi\) is either one-to-one or zero.

Suppose \(\phi\) is one-to-one. Then \(\operatorname{im}\phi\) is a nonzero subspace of \(W\), and it is invariant. If \(w = \phi(v)\) lies in the image, then \[ \Sigma(A) w = \Sigma(A) \phi(v) = \phi(\Pi(A) v) \in \operatorname{im}\phi. \] Since \(W\) is irreducible and \(\operatorname{im}\phi\) is nonzero and invariant, we must have \(\operatorname{im}\phi = W\). Thus \(\phi\) is either zero or one-to-one and onto, that is, an isomorphism.

Proof of Point 2:

Suppose \(V\) is an irreducible complex representation and \(\phi : V \to V\) intertwines, so \(\phi\,\Pi(A) = \Pi(A)\,\phi\) for all \(A\). Since we are working over the algebraically closed field \(\mathbb{C}\), the operator \(\phi\) has at least one eigenvalue \(\lambda \in \mathbb{C}\). Let \(U\) be the corresponding eigenspace. Then each \(\Pi(A)\) maps \(U\) into itself. If \(\phi u = \lambda u\), then \[ \phi\bigl(\Pi(A) u\bigr) = \Pi(A)\,\phi(u) = \Pi(A)(\lambda u) = \lambda\,\Pi(A) u, \] so \(\Pi(A) u\) is again a \(\lambda\)-eigenvector, that is, \(\Pi(A) u \in U\). Thus \(U\) is an invariant subspace. Since \(\lambda\) is an eigenvalue, \(U \neq \{0\}\), and irreducibility forces \(U = V\). But \(U = V\) means \(\phi\) acts as \(\lambda\) on all of \(V\), that is, \(\phi = \lambda I\).

This is exactly where complexity is essential. The existence of an eigenvalue used the algebraic closure of \(\mathbb{C}\). Over \(\mathbb{R}\) the conclusion fails. The rotation representation of \(SO(2)\) on \(\mathbb{R}^2\) is irreducible, since no real line is fixed by all rotations, yet the rotation by a right angle commutes with every element of the representation while being no real scalar multiple of the identity.

Proof of Point 3:

Let \(\phi_1, \phi_2 : V \to W\) be nonzero intertwining maps between irreducible complex representations. By Point 1, \(\phi_2\) is an isomorphism, so \(\phi_2^{-1}\) exists and \(\phi_1 \circ \phi_2^{-1} : W \to W\) is an intertwining map of \(W\) with itself. By Point 2 it equals \(\lambda I\) for some \(\lambda \in \mathbb{C}\), whence \(\phi_1 = \lambda\, \phi_2\).

Proof:

If \(A\) is central, then \(\Pi(A)\) commutes with \(\Pi(B)\) for every \(B \in G\), since \(\Pi(A)\Pi(B) = \Pi(AB) = \Pi(BA) = \Pi(B)\Pi(A)\). Thus \(\Pi(A)\) is an intertwining map of the irreducible complex representation with itself, and Point 2 of Schur's lemma gives \(\Pi(A) = \lambda I\). The Lie algebra case is identical, with the bracket relation \(\pi(X)\pi(Y) - \pi(Y)\pi(X) = \pi([X, Y]) = 0\) for central \(X\) showing \(\pi(X)\) commutes with all \(\pi(Y)\).

Proof:

We argue the group case. If \(G\) is commutative, then the center of \(G\) is all of \(G\), so by the preceding corollary \(\Pi(A) = \lambda I\) for each \(A \in G\), with the scalar depending on \(A\). But if every \(\Pi(A)\) is a multiple of the identity, then every subspace of \(V\) is invariant. The only way \(V\) can avoid having a nontrivial invariant subspace, and so the only way it can be irreducible, is for \(V\) to be one-dimensional.