Linear Transformations

Linear Transformations

On the previous page, we viewed the matrix equation \(A\mathbf{x} = \mathbf{b}\) as a way to find a vector. Now, we shift our perspective to view the matrix \(A\) as an object that acts on vectors. This dynamic view where a matrix transforms one space into another is the core of "Linear Transformation," a concept that underpins computer graphics, signal processing, and the layers of deep neural networks.

Two consequences follow immediately from the definition. First, setting \(c = 0\) in the second property gives \[ T(\mathbf{0}) = T(0 \cdot \mathbf{u}) = 0 \cdot T(\mathbf{u}) = \mathbf{0}. \] Second, combining both properties yields linearity over arbitrary linear combinations: for any scalars \(a, b\) and vectors \(\mathbf{u}, \mathbf{v}\) in the domain, \[ T(a\mathbf{u} + b\mathbf{v}) = a\, T(\mathbf{u}) + b\, T(\mathbf{v}). \]

Surjectivity and injectivity may already be familiar from single-variable functions \(f: \mathbb{R} \to \mathbb{R}\). The same definitions apply to linear transformations between higher-dimensional spaces, where they reveal deep structural information about the underlying matrix.

Intuitively, surjectivity means \(T\) reaches every point of the target space \(\mathbb{R}^m\). For a linear transformation \(T(\mathbf{x}) = A\mathbf{x}\), the image of \(T\) is exactly the set of all linear combinations of the columns of \(A\), that is, the span of the columns. Surjectivity therefore reduces to a question about the columns.

Intuitively, injectivity means \(T\) loses no information. Distinct inputs produce distinct outputs. For linear transformations, this property is rigidly tied to the linear independence of the columns of \(A\). Equivalently, \(A\mathbf{x} = \mathbf{0}\) must admit only the trivial solution.

Linearity in Mathematics

Linearity reaches well beyond vectors and matrices. It is one of the most pervasive structural ideas in mathematics, and the same pattern recurs in several other contexts. We collect a few familiar examples.

A real-valued function \(f(x) = m x\) (with no constant term) is a linear transformation \(f: \mathbb{R} \to \mathbb{R}\): \[ \begin{align*} f(ax + by) &= m(ax + by) \\\\ &= a(mx) + b(my) \\\\ &= a\, f(x) + b\, f(y). \end{align*} \] Note that a general line \(g(x) = mx + c\) with nonzero intercept is not a linear transformation in this sense. Such maps are called affine. The graph of every linear transformation must pass through the origin, since \(T(\mathbf{0}) = \mathbf{0}\).

In calculus, differentiation distributes over sums and constants: \[ \frac{d}{dx}\bigl(a\, X(x) + b\, Y(x)\bigr) = a\, \frac{d}{dx} X(x) + b\, \frac{d}{dx} Y(x). \] For example, \[ \begin{align*} \frac{d}{dx}(5x^3 + 4x^2) &= 15x^2 + 8x \\\\ &= 5\, \frac{d}{dx}(x^3) + 4\, \frac{d}{dx}(x^2), \end{align*} \] so differentiation is a linear operator on functions. (Integration is too.)

In statistics, the expected value is linear. For any random variables \(X, Y\) with finite expectations and constants \(a, b\), \[ \mathbb{E}[a X + b Y] = a\, \mathbb{E}[X] + b\, \mathbb{E}[Y]. \]

This ubiquity is what makes "linear" algebra so powerful. Any time we recognize a linear structure, much of the toolkit developed here transfers.

Matrix Multiplication

Each column of \(AB\) is therefore a linear combination of the columns of \(A\) with weights from the corresponding column of \(B\). The entrywise formula \[ (AB)_{ij} = \sum_{k=1}^n a_{ik} b_{kj} \] follows by reading off the \(i\)-th coordinate.

Note that \(BA\) is not defined here, because the number of columns of \(B\) (namely \(2\)) does not match the number of rows of \(A\) (namely \(3\)). Even when both \(AB\) and \(BA\) are defined, in general \(AB \neq BA\). Matrix multiplication is not commutative. Two further pitfalls come from scalar intuition: the equation \(AB = AC\) does not guarantee \(B = C\) (there is no cancellation in general), and \(AB = 0\) does not imply that \(A = 0\) or \(B = 0\).

It is reasonable to ask why matrix multiplication is not simply entrywise, given that addition and scalar multiplication are. The reason is structural. Matrix multiplication is composition of linear transformations.

Concretely, multiplying a vector \(\mathbf{x}\) by \(B\) yields \(B\mathbf{x}\), and multiplying the result by \(A\) yields \(A(B\mathbf{x})\). A direct computation from the column-wise definition of \(AB\) confirms that \[ (AB)\mathbf{x} = A(B\mathbf{x}). \] Writing \(\mathbf{x} = x_1 \mathbf{e}_1 + \cdots + x_p \mathbf{e}_p\), both sides expand to \(x_1 (A\mathbf{b}_1) + \cdots + x_p (A\mathbf{b}_p)\). Thus the column-wise product \(AB\) is precisely the matrix of the composition \(\mathbf{x} \mapsto A(B\mathbf{x})\). This identity has a powerful computational consequence. Instead of forming the matrix product \(AB\) first, we may compute the cheaper matrix-vector product \(B\mathbf{x}\) and then multiply by \(A\).