Fourier Transform

Continuous Fourier Transform Properties of Fourier Transform Convolution Theorem Discrete Fourier Transform Fast Fourier Transform Applications in Machine Learning Interactive Demos

Continuous Fourier Transform

Fourier series decompose periodic functions into discrete frequency components. However, most signals in practice are not periodic. Examples include a spoken sentence, a transient pulse, and a neural network's activation. The Fourier transform extends the Fourier series from periodic functions to suitably integrable functions on \(\mathbb{R}\), replacing the discrete spectrum with a continuous one. This generalization is the mathematical backbone of signal processing, spectral analysis, and frequency-domain methods throughout machine learning.

To see how this arises, consider a function \(f_L\) periodic with period \(2L\). Its complex Fourier series is: \[ f_L(x) = \sum_{n=-\infty}^{\infty} c_n e^{-i\frac{n\pi x}{L}}, \quad c_n = \frac{1}{2L}\int_{-L}^{L} f_L(x)e^{i\frac{n\pi x}{L}} \, dx. \] As \(L \to \infty\), the discrete frequencies \(\omega_n = \frac{n\pi}{L}\) become densely packed with spacing \(\Delta\omega = \frac{\pi}{L} \to 0\), and the sum approaches an integral. This limiting process yields the Fourier transform.

Definition: Fourier Transform

For a function \(f: \mathbb{R} \to \mathbb{C}\), we define: \[ \hat{f}(\xi) = \mathcal{F}\{f\}(\xi) = \int_{-\infty}^{\infty} f(x)e^{i x\xi} \, dx \] where \(\xi \in \mathbb{R}\) is the frequency variable.

The inverse Fourier transform recovers \(f\) from \(\hat{f}\): \[ f(x) = \mathcal{F}^{-1}\{\hat{f}\}(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat{f}(\xi)e^{-i x\xi} \, d\xi. \]

Note on Conventions:
As discussed in the Fourier series conventions, we use the convention with the positive sign in the forward transform and the negative sign in the inverse. Engineering texts and most analysis texts use the opposite sign, for example \(\hat{f}(\omega) = \int_{-\infty}^{\infty} f(t)e^{-i\omega t} \, dt\). Both are mathematically equivalent. Just be consistent within your work.

The Fourier transform is well-defined for different function classes:

Example: Gaussian Function

Consider the Gaussian function: \[ f(x) = e^{-\frac{x^2}{2\sigma^2}}. \] Its Fourier transform (using the convention of this site) is: \[ \begin{align*} \hat{f}(\xi) &= \int_{-\infty}^{\infty} e^{-\frac{x^2}{2\sigma^2}}e^{i\xi x} \, dx \\\\ &= \int_{-\infty}^{\infty} e^{-\frac{x^2}{2\sigma^2} + i\xi x} \, dx. \end{align*} \] Complete the square in the exponent: \[ \begin{align*} -\frac{x^2}{2\sigma^2} + i\xi x &= -\frac{1}{2\sigma^2}(x^2 - 2i\sigma^2\xi x) \\\\ &= -\frac{1}{2\sigma^2}((x - i\sigma^2\xi)^2 + \sigma^4\xi^2). \end{align*} \] Therefore: \[ \hat{f}(\xi) = e^{-\frac{\sigma^2\xi^2}{2}} \int_{-\infty}^{\infty} e^{-\frac{(x-i\sigma^2\xi)^2}{2\sigma^2}} \, dx. \] The integrand is entire and decays exponentially as \(|\mathrm{Re}\, z| \to \infty\), uniformly on horizontal strips of bounded imaginary part. Cauchy's theorem applied to a rectangular contour therefore justifies shifting the contour from \(\mathbb{R}\) to \(\mathbb{R} + i\sigma^2\xi\) without changing the value, since the vertical-side contributions vanish in the limit: \[ \begin{align*} \int_{-\infty}^{\infty} e^{-\frac{(x-i\sigma^2\xi)^2}{2\sigma^2}} \, dx &= \int_{-\infty}^{\infty} e^{-\frac{u^2}{2\sigma^2}} \, du \\\\ &= \sigma\sqrt{2\pi}. \end{align*} \] Thus: \[ \hat{f}(\xi) = \sigma\sqrt{2\pi} \cdot e^{-\frac{\sigma^2\xi^2}{2}}. \]

The transform of a Gaussian is again a Gaussian, with the width inverted. A function narrow in space has a wide transform, and conversely. The whole family is therefore preserved by the Fourier transform, and the single choice \(\sigma = 1\) gives a genuine eigenfunction, since \(e^{-x^2/2}\) transforms to \(\sqrt{2\pi}\, e^{-\xi^2/2}\). Normalizing by \(\sqrt{2\pi}\) therefore fixes \(e^{-x^2/2}\) outright, and the same unitary rescaling reappears in the discrete setting, where dividing the Fourier matrix by the square root of its size puts its eigenvalues on the unit circle. This behavior makes Gaussians fundamental in uncertainty principles, quantum mechanics, and signal processing.

Definition: The Schwartz Class

The Schwartz class \(\mathcal{S}(\mathbb{R})\) is the set of infinitely differentiable functions \(f : \mathbb{R} \to \mathbb{C}\) satisfying \[ \sup_{x \in \mathbb{R}} \bigl| x^m f^{(n)}(x) \bigr| \lt \infty \quad \text{for all integers } m, n \geq 0. \] Its members are the rapidly decreasing functions, so named because every derivative decays faster than any power of \(1/|x|\).

Two consequences of the definition are used repeatedly below. Let \(f \in \mathcal{S}(\mathbb{R})\). The case \(m = 0\) says that \(f\) and every one of its derivatives are bounded. Taking \((m, n) = (0, 0)\) and \((m, n) = (2, 0)\) together bounds \((1 + x^2)|f(x)|\), so \(|f(x)|\) is dominated by a constant multiple of \((1 + x^2)^{-1}\), and therefore \(\mathcal{S}(\mathbb{R}) \subset L^1(\mathbb{R}) \cap L^2(\mathbb{R})\).

The Gaussian of the example above is the canonical member of \(\mathcal{S}(\mathbb{R})\), and every smooth function with compact support belongs to it as well. Membership is preserved by differentiation, by multiplication by polynomials, and by the reflection \(f(x) \mapsto f(-x)\). The first and third of these are immediate from the defining bounds, and the second follows from them by the Leibniz rule.

Properties of Fourier Transform

The Fourier transform is far more than a definition. Its power lies in a collection of algebraic properties that convert difficult operations (differentiation, convolution) into simple ones (multiplication, pointwise products). These properties, stated below using the convention of this site \(\hat{f}(\xi) = \int_{-\infty}^{\infty} f(x)e^{ix\xi} \, dx\), form the foundation of spectral methods in both analysis and computation.

Unless indicated otherwise, the algebraic properties below are stated for \(f\) and \(g\) in the Schwartz class, where every integral converges absolutely and differentiation under the integral sign is legitimate. Each property extends to larger classes under its own hypotheses. The theorems of this section, by contrast, carry the hypotheses given with them.

Property: Linearity

\[ \mathcal{F}\{\alpha f + \beta g\} = \alpha \mathcal{F}\{f\} + \beta \mathcal{F}\{g\} \] for constants \(\alpha, \beta \in \mathbb{C}\).

Property: Translation (Shift)

\[ \mathcal{F}\{f(x - a)\}(\xi) = e^{ia\xi}\hat{f}(\xi) \] A shift in space corresponds to a phase shift in frequency.

Property: Scaling

\[ \mathcal{F}\{f(ax)\}(\xi) = \frac{1}{|a|}\hat{f}\left(\frac{\xi}{a}\right), \quad a \neq 0 \] This embodies the uncertainty principle. Compressing a signal in time stretches its frequency spectrum, and vice versa. One cannot localize a signal arbitrarily well in both time and frequency simultaneously.

Property: Differentiation

For \(f\) in the Schwartz class, \[ \mathcal{F}\{f'(x)\}(\xi) = -i\xi \hat{f}(\xi) \] More generally, for the \(n\)-th derivative: \[ \mathcal{F}\{f^{(n)}(x)\}(\xi) = (-i\xi)^n \hat{f}(\xi) \] This transforms differentiation into algebraic multiplication, which is why Fourier methods are powerful for solving differential equations. Note the sign difference from the engineering convention.

Proof (the four properties):

Linearity is the linearity of the integral. The translation property follows from the substitution \(u = x - a\): \[ \int_{-\infty}^{\infty} f(x - a)\, e^{ix\xi}\, dx = \int_{-\infty}^{\infty} f(u)\, e^{i(u + a)\xi}\, du = e^{ia\xi} \hat{f}(\xi). \] Scaling follows from the substitution \(u = ax\). When \(a \lt 0\) this reverses the orientation of the line, and the reversal combines with the Jacobian into the single factor \(1/|a|\): \[ \int_{-\infty}^{\infty} f(ax)\, e^{ix\xi}\, dx = \frac{1}{|a|} \int_{-\infty}^{\infty} f(u)\, e^{iu\xi/a}\, du = \frac{1}{|a|} \hat{f}\left(\frac{\xi}{a}\right). \] Differentiation follows from integration by parts. The boundary term vanishes because \(f(x) \to 0\) as \(|x| \to \infty\), which leaves \[ \int_{-\infty}^{\infty} f'(x)\, e^{ix\xi}\, dx = -\int_{-\infty}^{\infty} f(x)\, i\xi\, e^{ix\xi}\, dx = -i\xi\, \hat{f}(\xi). \] Every derivative of a Schwartz function is again a Schwartz function, so the same step applies with \(f'\) in place of \(f\), and induction on \(n\) gives \(\mathcal{F}\{f^{(n)}\}(\xi) = (-i\xi)^n \hat{f}(\xi)\).

Theorem: Invariance of the Schwartz Class

If \(f \in \mathcal{S}(\mathbb{R})\), then \(\hat{f} \in \mathcal{S}(\mathbb{R})\).

Proof:

Fix integers \(m, n \geq 0\). Because \(x \mapsto x^k f(x)\) lies in \(\mathcal{S}(\mathbb{R})\) and hence in \(L^1(\mathbb{R})\) for every integer \(k \geq 0\), the integral defining \(\hat{f}\) may be differentiated under the integral sign any number of times. This makes \(\hat{f}\) smooth and gives \[ \frac{d^n}{d\xi^n} \hat{f}(\xi) = \widehat{(ix)^n f}(\xi). \] Write \(h(x) = (ix)^n f(x)\), so that \(h \in \mathcal{S}(\mathbb{R})\) and \(\frac{d^n}{d\xi^n} \hat{f} = \hat{h}\). Applying the differentiation property to \(h\) gives \(\widehat{h^{(m)}}(\xi) = (-i\xi)^m \hat{h}(\xi)\), and therefore \[ \left| \xi^m \frac{d^n}{d\xi^n} \hat{f}(\xi) \right| = \bigl| \xi^m \hat{h}(\xi) \bigr| = \bigl| \widehat{h^{(m)}}(\xi) \bigr| \leq \| h^{(m)} \|_{L^1}. \] The right-hand side is finite and independent of \(\xi\), because \(h^{(m)}\) again lies in \(\mathcal{S}(\mathbb{R}) \subset L^1(\mathbb{R})\). Since \(m\) and \(n\) were arbitrary, \(\hat{f}\) satisfies every bound in the definition of the Schwartz class.

Theorem: Fourier Inversion

For suitable functions (for example, \(f \in L^1(\mathbb{R})\) with \(\hat{f} \in L^1(\mathbb{R})\)), the original function can be recovered for almost every \(x\), and for every \(x\) when \(f\) is continuous, in particular on \(\mathcal{S}(\mathbb{R})\): \[ f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat{f}(\xi)e^{-ix\xi} \, d\xi. \] A corollary, in the same sense, is \(\mathcal{F}\{\mathcal{F}\{f\}\}(x) = 2\pi f(-x)\), a deep symmetry between the spatial and frequency domains.

Proof (Schwartz class):

The theorem statement gives the inversion formula for the class \(\{f \in L^1(\mathbb{R}) : \hat f \in L^1(\mathbb{R})\}\). The proof we present here is restricted to the smaller Schwartz class \(\mathcal{S}(\mathbb{R}) \subset \{f \in L^1 : \hat f \in L^1\}\). The extension from \(\mathcal{S}(\mathbb{R})\) to the full \(L^1\) class indicated in the statement is discussed after the proof. The Schwartz-class technique is Gaussian regularization. We insert a Gaussian damping factor that makes Fubini applicable, then remove it in the limit.

Step 1: regularized inverse.
For \(\epsilon \gt 0\), define \[ I_\epsilon(x) = \frac{1}{2\pi} \int_{-\infty}^{\infty} \hat{f}(\xi)\, e^{-\epsilon \xi^2/2}\, e^{-ix\xi}\, d\xi. \] Since \(\hat f \in \mathcal{S}(\mathbb{R})\) by the invariance of the Schwartz class, the integrand is absolutely integrable for every \(\epsilon \geq 0\). In particular the integral converges and \(I_\epsilon(x)\) is well-defined.

Step 2: Fubini interchange.
We substitute \(\hat f(\xi) = \int f(y) e^{iy\xi}\, dy\). The resulting double integrand satisfies \[ \bigl|f(y)\, e^{iy\xi}\, e^{-\epsilon\xi^2/2}\, e^{-ix\xi}\bigr| = |f(y)|\, e^{-\epsilon\xi^2/2} \in L^1(\mathbb{R} \times \mathbb{R}), \] the product of an \(L^1\) function in \(y\) with an \(L^1\) function in \(\xi\). Therefore Fubini's theorem, applied to the real and imaginary parts of the integrand separately, permits the order swap: \[ \begin{align*} I_\epsilon(x) &= \int_{-\infty}^{\infty} f(y)\, K_\epsilon(y - x)\, dy, \\\\ K_\epsilon(z) &= \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{iz\xi}\, e^{-\epsilon\xi^2/2}\, d\xi. \end{align*} \] The kernel \(K_\epsilon\) is the forward Fourier transform of the Gaussian \(\xi \mapsto e^{-\epsilon\xi^2/2}\), evaluated at \(z\), divided by \(2\pi\). We reuse the Gaussian-transform calculation given earlier, with the roles of the space and frequency variables exchanged. Apply the formula \[ \widehat{e^{-x^2/(2\sigma^2)}}(\xi) = \sigma\sqrt{2\pi}\, e^{-\sigma^2 \xi^2/2} \] to the variable \(\xi\) in the role of \(x\), with \(\sigma^2 = 1/\epsilon\). This gives \[ \int e^{iz\xi}\, e^{-\epsilon\xi^2/2}\, d\xi = \sqrt{2\pi/\epsilon}\, e^{-z^2/(2\epsilon)}. \] Thus \[ K_\epsilon(z) = \frac{1}{\sqrt{2\pi\epsilon}}\, e^{-z^2/(2\epsilon)}, \] a Gaussian of unit mass: \(\int K_\epsilon(z)\, dz = 1\).

Step 3: convolution limit.
By the substitution \(z = y - x\), we have \(I_\epsilon(x) = \int f(x + z)\, K_\epsilon(z)\, dz\). Using \(\int K_\epsilon = 1\) to write \(f(x) = \int f(x)\, K_\epsilon(z)\, dz\), we obtain \[ I_\epsilon(x) - f(x) = \int_{-\infty}^{\infty} \bigl[ f(x + z) - f(x) \bigr]\, K_\epsilon(z)\, dz. \] Fix \(\eta \gt 0\). Every derivative of a Schwartz function is bounded, so \(|f(x + z) - f(x)| \leq \|f'\|_\infty\, |z|\) for all \(x\) and \(z\). Taking \(\delta = \eta / (1 + \|f'\|_\infty)\) therefore gives \(|f(x + z) - f(x)| \lt \eta\) whenever \(|z| \leq \delta\), uniformly in \(x\). Split the integral accordingly. On the inner region \(|z| \leq \delta\), positivity of \(K_\epsilon\) and unit mass give \[ \left| \int_{|z| \leq \delta} [f(x+z) - f(x)]\, K_\epsilon(z)\, dz \right| \leq \eta. \] On the outer region \(|z| \gt \delta\), the substitution \(u = z/\sqrt{\epsilon}\) shows \[ \int_{|z| \gt \delta} K_\epsilon(z)\, dz = \int_{|u| \gt \delta/\sqrt{\epsilon}} \frac{1}{\sqrt{2\pi}}\, e^{-u^2/2}\, du \longrightarrow 0 \quad \text{as } \epsilon \to 0, \] since the lower bound \(\delta/\sqrt{\epsilon} \to \infty\) and the integrand is the standard-normal density. Combined with the trivial bound \(|f(x+z) - f(x)| \leq 2 \sup_{\mathbb{R}} |f|\) on the outer region, \[ \limsup_{\epsilon \to 0}\, |I_\epsilon(x) - f(x)| \leq \eta. \] Since \(\eta \gt 0\) was arbitrary, \(\lim_{\epsilon \to 0} I_\epsilon(x) = f(x)\).

Step 4: spectral-side limit.
We return to the definition of \(I_\epsilon\). The integrand \[ \hat f(\xi)\, e^{-\epsilon \xi^2/2}\, e^{-ix\xi} \] is dominated by \(|\hat f(\xi)| \in L^1(\mathbb{R})\) uniformly in \(\epsilon\) and converges pointwise to \(\hat f(\xi)\, e^{-ix\xi}\) as \(\epsilon \to 0\). The dominated convergence theorem applies along any sequence \(\epsilon_j \to 0\), and the resulting value does not depend on the sequence chosen, so \[ \lim_{\epsilon \to 0} I_\epsilon(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat f(\xi)\, e^{-ix\xi}\, d\xi. \] Equating the two limits established in Steps 3 and 4 yields the inversion formula for every \(f \in \mathcal{S}(\mathbb{R})\).

On the \(L^1\) extension.
The Schwartz proof above does not directly extend to the larger class \[ \{f \in L^1(\mathbb{R}) : \hat f \in L^1(\mathbb{R})\} \] by a one-line density argument. Schwartz functions are dense in \(L^1\), by the density lemma below, but the resulting approximation \(\hat{f_n} \to \hat f\) is only uniform, not \(L^1\). The right-hand side of the inversion formula therefore does not pass to the limit for free. The \(L^1\) case can nevertheless be reached by the same Gaussian regularization. The pointwise limit of Step 3 is replaced by convergence of the Gaussian smoothing in \(L^1\), and an almost everywhere convergent subsequence then gives the formula for almost every \(x\). We do not carry this out here. If in addition \(f\) is continuous, equality holds for every \(x\): the right-hand side is continuous by dominated convergence, since \(\hat f \in L^1(\mathbb{R})\), and two continuous functions that agree almost everywhere agree everywhere.

Inversion shows that the Fourier transform is invertible on the Schwartz class. The formula \(f = \mathcal{F}^{-1}\mathcal{F} f\) holds for every \(f \in \mathcal{S}(\mathbb{R})\), and the corollary \(\mathcal{F}\mathcal{F} f(x) = 2\pi f(-x)\) ensures invertibility from the other side as well.

Stage 2 of the proof of Plancherel's theorem below needs one approximation fact. An arbitrary \(f \in L^1 \cap L^2(\mathbb{R})\) must be approached by Schwartz functions in both norms at once. We prove it now. The smoothing device is the Gaussian kernel of the inversion proof, applied this time to \(f\) itself rather than to its transform.

In the lemma below, integrals over \(\mathbb{R}\) are taken against the Lebesgue measure on the Borel sets, and all functions are Borel measurable. We write \(|E|\) for the measure of a Borel set \(E\). Two properties of this measure are used without proof. It is translation invariant, \(|E + y| = |E|\) for every Borel set \(E\) and every \(y \in \mathbb{R}\), and hence \(\int h(x + y)\, dx = \int h(x)\, dx\) for every non-negative or integrable Borel function \(h\), the identity passing from indicators to simple functions to monotone limits. It is outer regular: for every Borel set \(E\) of finite measure and every \(\eta \gt 0\), there is an open set \(U \supseteq E\) with \(|U \setminus E| \lt \eta\). The first reflects the fact that \(B \mapsto |B + y|\) is again a measure giving each interval \((a, b]\) its length, and the second that the construction measures a set by covering it with countably many intervals.

Lemma: Density of the Schwartz Class

Let \(p \in \{1, 2\}\). The Schwartz class \(\mathcal{S}(\mathbb{R})\) is dense in \(L^p(\mathbb{R})\): for every \(f \in L^p(\mathbb{R})\) there are \(f_n \in \mathcal{S}(\mathbb{R})\) with \(\|f_n - f\|_p \to 0\). When \(f \in L^1 \cap L^2(\mathbb{R})\), a single sequence \(\{f_n\}\) converges to \(f\) in \(L^1\) and in \(L^2\) simultaneously.

Proof:

For a function \(h\) on \(\mathbb{R}\) and \(y \in \mathbb{R}\), write \((\tau_y h)(x) = h(x + y)\) for its translate. Translation invariance gives \(\|\tau_y h\|_p = \|h\|_p\). A step function is a finite linear combination of indicators of bounded intervals.

Step 1: step functions are dense in \(L^p\).
Let \(g \in L^p(\mathbb{R})\). Writing \(g = (\Re g)^+ - (\Re g)^- + i\,(\Im g)^+ - i\,(\Im g)^-\), each of the four parts is non-negative, Borel, and dominated by \(|g|\), so it suffices to treat \(0 \leq g \in L^p\). First, \(g\, \mathbf{1}_{[-M, M]} \to g\) in \(L^p\) as \(M \to \infty\) through the integers, by the dominated convergence theorem with dominator \(|g|^p\), so we may assume that \(g\) vanishes outside \([-M, M]\). Next, the staircase functions \[ s_n = \min\bigl( 2^{-n} \lfloor 2^n g \rfloor,\ n \bigr) \] are simple, satisfy \(0 \leq s_n \leq g\), and obey \(|s_n - g| \leq 2^{-n}\) wherever \(g \lt n\). Since \(g\) is finite almost everywhere, \(s_n \to g\) almost everywhere, and the dominated convergence theorem with dominator \(g^p\) gives \(\|s_n - g\|_p \to 0\). Each \(s_n\) is a finite combination \(\sum_j \alpha_j \mathbf{1}_{E_j}\) of indicators of Borel sets \(E_j \subseteq [-M, M]\).

It remains to approximate \(\mathbf{1}_E\) for a Borel set \(E \subseteq [-M, M]\). Fix \(\eta \gt 0\). Outer regularity gives an open \(U \supseteq E\) with \(|U \setminus E| \lt \eta\), and \(W = U \cap (-M - 1, M + 1)\) is open, contains \(E\), and has \(|W \setminus E| \lt \eta\) and \(|W| \leq 2M + 2\). Every open subset of \(\mathbb{R}\) is the disjoint union of its connected components, which are open intervals, and there are at most countably many of them because each contains a rational number. Write \(W = \bigsqcup_k I_k\) accordingly. By countable additivity \(\sum_k |I_k| = |W| \lt \infty\), so there is \(K\) with \(\sum_{k \gt K} |I_k| \lt \eta\). The set \(V = I_1 \sqcup \cdots \sqcup I_K\) satisfies \(V \triangle E \subseteq (W \setminus E) \cup (W \setminus V)\), hence \(|V \triangle E| \lt 2\eta\), and \(\mathbf{1}_V = \sum_{k \leq K} \mathbf{1}_{I_k}\) is a step function with \[ \|\mathbf{1}_V - \mathbf{1}_E\|_p = |V \triangle E|^{1/p} \lt (2\eta)^{1/p}. \] Replacing each \(\mathbf{1}_{E_j}\) in \(s_n\) this way and chaining the approximations proves Step 1.

Step 2: translation is continuous in \(L^p\).
We claim \(\|\tau_y g - g\|_p \to 0\) as \(y \to 0\) for every \(g \in L^p(\mathbb{R})\). For the indicator of a bounded interval \(I\) with endpoints \(a \lt b\), the function \(\tau_y \mathbf{1}_I - \mathbf{1}_I\) takes the values \(0\) and \(\pm 1\), the latter exactly on \((I - y) \triangle I\), a set of measure \(2 \min(|y|, b - a) \leq 2|y|\). Hence \(\|\tau_y \mathbf{1}_I - \mathbf{1}_I\|_p \leq (2|y|)^{1/p} \to 0\), and by the triangle inequality the claim holds for every step function. For general \(g\), fix \(\eta \gt 0\) and a step function \(s\) with \(\|g - s\|_p \lt \eta\) by Step 1. Then \[ \begin{align*} \|\tau_y g - g\|_p &\leq \|\tau_y (g - s)\|_p + \|\tau_y s - s\|_p + \|s - g\|_p \\\\ &\lt 2\eta + \|\tau_y s - s\|_p, \end{align*} \] so \(\limsup_{y \to 0} \|\tau_y g - g\|_p \leq 2\eta\) for every \(\eta \gt 0\).

Step 3: Gaussian smoothing converges in \(L^p\).
For \(\epsilon \gt 0\), let \[ K_\epsilon(z) = \frac{1}{\sqrt{2\pi\epsilon}}\, e^{-z^2/(2\epsilon)} \] be the kernel of the inversion proof, positive, even, and of unit mass. Let \(g \in L^p\) vanish outside \([-M, M]\), and assume in addition \(g \in L^1\). Define \[ \begin{align*} g_\epsilon(x) &= \int_{-\infty}^{\infty} g(y)\, K_\epsilon(x - y)\, dy \\\\ &= \int_{-\infty}^{\infty} g(x + z)\, K_\epsilon(z)\, dz, \end{align*} \] the integral converging absolutely for every \(x\) because \(g \in L^1\) and \(K_\epsilon \leq (2\pi\epsilon)^{-1/2}\). The second form comes from the translation \(y = x + z\) and the evenness of \(K_\epsilon\). Using unit mass to write \(g(x) = \int g(x)\, K_\epsilon(z)\, dz\), \[ g_\epsilon(x) - g(x) = \int_{-\infty}^{\infty} \bigl[ g(x + z) - g(x) \bigr]\, K_\epsilon(z)\, dz. \] For \(p = 1\) the triangle inequality for integrals, and for \(p = 2\) the Cauchy-Schwarz inequality applied to the factors \([g(x + z) - g(x)] \sqrt{K_\epsilon(z)}\) and \(\sqrt{K_\epsilon(z)}\), give in both cases \[ |g_\epsilon(x) - g(x)|^p \leq \int_{-\infty}^{\infty} |g(x + z) - g(x)|^p\, K_\epsilon(z)\, dz. \]

The function \((x, z) \mapsto |g(x + z) - g(x)|^p K_\epsilon(z)\) is non-negative and Borel on \(\mathbb{R}^2\), being built from \(g\) and continuous maps. The Borel sets of \(\mathbb{R}^2\) lie in the product \(\sigma\)-algebra of the Borel sets of \(\mathbb{R}\), since every open subset of \(\mathbb{R}^2\) is a countable union of open rectangles with rational corners. Integrating in \(x\) and applying, Lebesgue measure being \(\sigma\)-finite, Tonelli's theorem, \[ \begin{align*} \|g_\epsilon - g\|_p^p &\leq \int_{-\infty}^{\infty} \omega(z)\, K_\epsilon(z)\, dz, \\\\ \omega(z) &= \|\tau_z g - g\|_p^p. \end{align*} \]

Here \(\omega(z) \leq (\|\tau_z g\|_p + \|g\|_p)^p = 2^p \|g\|_p^p\) for every \(z\), and \(\omega(z) \to 0\) as \(z \to 0\) by Step 2. Fix \(\eta \gt 0\) and choose \(\delta \gt 0\) with \(\omega(z) \lt \eta\) for \(|z| \leq \delta\). Splitting the integral as in the inversion proof, \[ \|g_\epsilon - g\|_p^p \leq \eta + 2^p \|g\|_p^p \int_{|z| \gt \delta} K_\epsilon(z)\, dz, \] and the tail integral tends to \(0\) as \(\epsilon \to 0\), as computed in Step 3 of that proof. Hence \(\limsup_{\epsilon \to 0} \|g_\epsilon - g\|_p^p \leq \eta\) for every \(\eta \gt 0\), that is, \(g_\epsilon \to g\) in \(L^p\).

Step 4: the smoothed function is a Schwartz function.
Keep \(g \in L^1\) vanishing outside \([-M, M]\). The kernel \(K_\epsilon\) is a constant multiple of the Gaussian \(e^{-z^2/(2\sigma^2)}\) of the Gaussian example above with \(\sigma^2 = \epsilon\), hence a member of \(\mathcal{S}(\mathbb{R})\). Every derivative \(K_\epsilon^{(n)}\) is therefore bounded, and every quantity \(C_{m,n} = \sup_{u} |u|^m |K_\epsilon^{(n)}(u)|\) is finite. We claim \[ g_\epsilon^{(n)}(x) = \int_{-M}^{M} g(y)\, K_\epsilon^{(n)}(x - y)\, dy \quad \text{for all } n \geq 0. \] Assuming it for \(n\), the difference quotients of the integrand in \(x\) are bounded, by the mean value theorem, by \(|g(y)|\, \|K_\epsilon^{(n+1)}\|_\infty\), an integrable function of \(y\) independent of \(x\). The dominated convergence theorem, applied along any sequence of increments tending to zero, then differentiates under the integral sign and gives the case \(n + 1\). So \(g_\epsilon\) is smooth.

For the decay, let \(|x| \geq 2M\). Every \(y\) in the range of integration has \(|y| \leq M\), so \(|x - y| \geq |x| - |y| \geq |x|/2\), and \[ \begin{align*} |x^m g_\epsilon^{(n)}(x)| &\leq \int_{-M}^{M} |g(y)|\, 2^m |x - y|^m\, |K_\epsilon^{(n)}(x - y)|\, dy \\\\ &\leq 2^m C_{m,n}\, \|g\|_1. \end{align*} \] For \(|x| \lt 2M\) the same integral is at most \((2M)^m \|K_\epsilon^{(n)}\|_\infty \|g\|_1\). Thus \(\sup_x |x^m g_\epsilon^{(n)}(x)| \lt \infty\) for all \(m, n \geq 0\), and \(g_\epsilon \in \mathcal{S}(\mathbb{R})\).

Step 5: assembly.
Let \(f \in L^p(\mathbb{R})\) and set \(f_M = f\, \mathbf{1}_{[-M, M]}\). As in Step 1, \(\|f - f_M\|_p \to 0\) as \(M \to \infty\). Each \(f_M\) lies in \(L^1\). For \(p = 2\) this follows from the Cauchy-Schwarz inequality, which gives \(\int |f_M| \leq (2M)^{1/2} \|f\|_2\). Steps 3 and 4 therefore apply to \(g = f_M\), and \((f_M)_\epsilon \in \mathcal{S}(\mathbb{R})\) converges to \(f_M\) in \(L^p\) as \(\epsilon \to 0\). Choose \(M_n\) with \(\|f - f_{M_n}\|_p \lt 1/n\), then \(\epsilon_n\) with \(\|(f_{M_n})_{\epsilon_n} - f_{M_n}\|_p \lt 1/n\), and set \(f_n = (f_{M_n})_{\epsilon_n}\). Then \(\|f_n - f\|_p \lt 2/n\).

The construction \(f \mapsto (f_M)_\epsilon\) does not depend on \(p\). If \(f \in L^1 \cap L^2(\mathbb{R})\), both requirements on \(M_n\) hold for all large \(M\), and both requirements on \(\epsilon_n\) hold for all small \(\epsilon\), so \(M_n\) and \(\epsilon_n\) can be chosen to satisfy the \(L^1\) and \(L^2\) conditions at once. The resulting sequence converges to \(f\) in both norms simultaneously.

The next theorem extracts from inversion the deeper structural fact: on \(L^1 \cap L^2(\mathbb{R})\), the Fourier transform also preserves the \(L^2\) inner product, up to the normalization constant \(\tfrac{1}{2\pi}\). This is Plancherel's identity, the continuous-frequency analogue of the Parseval identity for Fourier series. Its proof uses inversion (Stage 1 below), then extends from \(\mathcal{S}(\mathbb{R})\) to \(L^1 \cap L^2(\mathbb{R})\) by a density argument resting on the lemma just proved (Stage 2).

Theorem: Plancherel's Theorem (on \(L^1 \cap L^2\))

For \(f, g \in L^1 \cap L^2(\mathbb{R})\), the classical Fourier transform defined by the absolutely convergent integral \(\hat f(\xi) = \int f(x) e^{ix\xi}\, dx\) satisfies \[ \int_{-\infty}^{\infty} f(x)\overline{g(x)} \, dx = \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat{f}(\xi)\overline{\hat{g}(\xi)} \, d\xi. \] In particular, setting \(f = g\): \[ \|f\|_2^2 = \int_{-\infty}^{\infty} |f(x)|^2 \, dx = \frac{1}{2\pi}\int_{-\infty}^{\infty} |\hat{f}(\xi)|^2 \, d\xi. \]

Proof:

We prove the identity in two stages: first on the Schwartz class \(\mathcal{S}(\mathbb{R})\), then by density on all of \(L^1 \cap L^2(\mathbb{R})\). It suffices to prove the polarized form. The special case \(f = g\) follows by setting them equal.

Stage 1: identity on the Schwartz class.
Fix \(f, g \in \mathcal{S}(\mathbb{R})\). By the invariance of the Schwartz class, \(\hat f, \hat g \in \mathcal{S}(\mathbb{R})\) as well. Apply the Fourier inversion theorem (proved above) to \(g\): \[ g(x) = \frac{1}{2\pi}\int \hat g(\xi)\, e^{-ix\xi}\, d\xi. \] Taking the complex conjugate of both sides, and using \(\overline{e^{-ix\xi}} = e^{+ix\xi}\), \[ \overline{g(x)} = \frac{1}{2\pi}\int \overline{\hat g(\xi)}\, e^{ix\xi}\, d\xi. \] Multiplying by \(f(x)\) and integrating in \(x\), \[ \int f(x)\overline{g(x)}\, dx = \frac{1}{2\pi}\iint f(x)\, e^{ix\xi}\, \overline{\hat g(\xi)}\, dx\, d\xi. \] To justify the order swap in the double integral, we check the integrability hypothesis of Fubini's theorem. The absolute value of the integrand is \(|f(x)| \cdot |\hat g(\xi)|\), and Tonelli's theorem applied to this non-negative measurable function gives \[ \iint |f(x)| \cdot |\hat g(\xi)|\, dx\, d\xi = \|f\|_1 \cdot \|\hat g\|_1 \lt \infty, \] since both \(f\) and \(\hat g\) are Schwartz, hence in \(L^1(\mathbb{R})\). Therefore the integrand is in \(L^1(\mathbb{R} \times \mathbb{R})\), and Fubini's theorem, again applied to the real and imaginary parts separately, permits the order swap: \[ \begin{align*} \frac{1}{2\pi}\iint f(x)\, e^{ix\xi}\, \overline{\hat g(\xi)}\, dx\, d\xi &= \frac{1}{2\pi}\int \overline{\hat g(\xi)} \left( \int f(x)\, e^{ix\xi}\, dx \right) d\xi \\\\ &= \frac{1}{2\pi}\int \hat f(\xi)\, \overline{\hat g(\xi)}\, d\xi, \end{align*} \] where the inner integral is \(\hat f(\xi)\) by definition. This establishes the polarized identity on \(\mathcal{S}(\mathbb{R})\).

Stage 2: extension to \(L^1 \cap L^2(\mathbb{R})\).
Equip \(L^1 \cap L^2(\mathbb{R})\) with the \(L^2\) norm. Then \(\mathcal{S}(\mathbb{R})\) is a dense subspace, by the density lemma proved above. The lemma gives more, and the argument uses it. For a fixed \(f \in L^1 \cap L^2\), it produces an approximating sequence \(\{f_n\} \subset \mathcal{S}(\mathbb{R})\) converging to \(f\) simultaneously in both \(L^1\) and \(L^2\). We fix such a sequence for the remainder of the proof.

Stage 1 specialized to \(f = g\) gives \[ \|f\|_2^2 = \frac{1}{2\pi}\|\hat f\|_2^2 \] for every \(f \in \mathcal{S}(\mathbb{R})\). Equivalently, the linear map \(T : \mathcal{S}(\mathbb{R}) \to L^2(\mathbb{R})\), \(Tf = \hat f / \sqrt{2\pi}\), is an isometry from \((\mathcal{S}(\mathbb{R}), \|\cdot\|_2)\) into \(L^2(\mathbb{R})\). From this, two observations about our sequence \(\{f_n\}\) follow:

  • The sequence \(\hat{f_n}\) is Cauchy in \(L^2\).
    By Stage 1 applied to the differences \(f_n - f_m \in \mathcal{S}(\mathbb{R})\), \(\|\hat{f_n} - \hat{f_m}\|_2 = \sqrt{2\pi}\, \|f_n - f_m\|_2 \to 0\). Hence by completeness of \(L^2\), \(\hat{f_n}\) converges to some \(F \in L^2(\mathbb{R})\).
  • The limit \(F\) coincides with the classical Fourier transform \(\hat f\).
    Since \(f \in L^1\), the integral \(\hat f(\xi) = \int f(x)\, e^{ix\xi}\, dx\) is absolutely convergent and defines \(\hat f \in L^\infty(\mathbb{R})\). The \(L^1\)-convergence \(f_n \to f\) (which our sequence provides) gives \(\hat{f_n} \to \hat f\) uniformly on \(\mathbb{R}\), by the elementary estimate \[ \begin{align*} \|\hat{f_n} - \hat f\|_\infty &= \sup_\xi \left|\int (f_n - f)(x)\, e^{ix\xi}\, dx\right| \\\\ &\leq \|f_n - f\|_1 \to 0. \end{align*} \] Combined with \(\hat{f_n} \to F\) in \(L^2\), passing to a subsequence (using subsequential a.e. convergence for \(L^2\) convergence) yields pointwise a.e. convergence to both \(F\) and \(\hat f\), so \(F = \hat f\) a.e.

The \(L^2\) norm is continuous, so we may take the limit \(n \to \infty\) in the Stage 1 identity \(\|f_n\|_2^2 = \tfrac{1}{2\pi}\|\hat{f_n}\|_2^2\). This yields \[ \|f\|_2^2 = \tfrac{1}{2\pi}\|\hat f\|_2^2. \] The polarized form follows by the same argument applied to a second sequence \(\{g_n\} \subset \mathcal{S}(\mathbb{R})\) approximating \(g \in L^1 \cap L^2\) simultaneously in \(L^1\) and \(L^2\). The bilinear identity \[ \int f_n \overline{g_n}\, dx = \tfrac{1}{2\pi}\int \hat{f_n}\, \overline{\hat{g_n}}\, d\xi \] passes to the limit on both sides by continuity of the \(L^2\) inner product. Here \(f_n \to f\) and \(g_n \to g\) in \(L^2\), and the same argument used above for \(\hat f\) gives \(\hat{f_n} \to \hat f\) and \(\hat{g_n} \to \hat g\) in \(L^2\).

Looking ahead: both theorems on \(L^2(\mathbb{R})\).
The two main results of this section are Fourier inversion and Plancherel's identity. Both are stated above on classes that require \(f \in L^1\): inversion on \(\{f \in L^1 : \hat f \in L^1\}\) (proved here for the smaller Schwartz class), and Plancherel on \(L^1 \cap L^2\). For functions \(f \in L^2 \setminus L^1\) (such as \((1+|x|)^{-1}\)), the defining integral \(\hat f(\xi) = \int f(x)\, e^{ix\xi}\, dx\) fails to converge absolutely, so neither result applies directly. The classes \(L^1\) and \(L^2\) are mutually non-contained on \(\mathbb{R}\) (for instance \(|x|^{-1/2}\mathbf{1}_{[0,1]} \in L^1 \setminus L^2\)), so neither setting subsumes the other.

The remedy is to redefine the Fourier transform on \(L^2(\mathbb{R})\) as a bounded linear operator, obtained as the unique continuous extension of its restriction to the dense subspace \(L^1 \cap L^2\). Thanks to the completeness of \(L^2\) as a Hilbert space, this approach is structurally cleaner than the \(L^1\) inversion theory. The normalized map \(f \mapsto \hat f / \sqrt{2\pi}\) becomes a unitary operator on \(L^2(\mathbb{R})\), whose inverse coincides with its Hilbert-space adjoint. That single fact delivers an \(L^2\) inversion formula and a Plancherel identity on all of \(L^2\) at once. This construction is carried out on Fourier analysis in Hilbert spaces.

Convolution Theorem

The differentiation property above shows that the Fourier transform simplifies differentiation. The convolution theorem reveals an even more powerful algebraic simplification. It converts the integral operation of convolution, which costs \(O(n^2)\) to compute directly, into pointwise multiplication in the frequency domain. This single result is the reason FFT-based convolution outperforms direct convolution for sufficiently long filters and large kernels, and it underlies frequency-domain filtering throughout signal processing.

The convolution of two functions \(f, g: \mathbb{R} \to \mathbb{C}\) is defined as: \[ (f * g)(x) = \int_{-\infty}^{\infty} f(y)g(x-y) \, dy = \int_{-\infty}^{\infty} f(x-y)g(y) \, dy \] (The second equality shows convolution is commutative.)

Theorem: Convolution Theorem

For \(f, g \in L^1(\mathbb{R})\), the Fourier transform converts convolution into pointwise multiplication: \[ \mathcal{F}\{f * g\}(\xi) = \hat{f}(\xi) \cdot \hat{g}(\xi) \] and conversely, whenever \(f, g \in L^1(\mathbb{R})\) with \(\hat f \in L^1(\mathbb{R})\) (in particular for \(f, g\) in the Schwartz class), the Fourier transform of a product is a (scaled) convolution: \[ \mathcal{F}\{f \cdot g\}(\xi) = \frac{1}{2\pi}(\hat{f} * \hat{g})(\xi). \]

This is one of the most important results in applied mathematics, and it allows us to:

Insight: Convolution Theorem in CS and ML

  • Signal filtering. Low-pass, high-pass, and band-pass filters are convolutions in the time domain but simple multiplications in the frequency domain
  • Image processing. Gaussian blur, Sobel edge filters, and sharpening are convolution operations, efficiently computed via FFT for large kernels. Canny edge detection begins with such convolutions.
  • Deep learning. While CNNs typically use small kernels \((3 \times 3, 5 \times 5)\) where direct convolution is efficient, FFT-based convolution is useful for:
    • Large kernels in certain architectures
    • Global convolutions in some vision transformers
  • Probability. The PDF of \(X + Y\) for independent random variables is \(f_{X+Y} = f_X * f_Y\), computed efficiently via FFT
Proof (convolution to product):

Using the convention of this site with \(\hat{f}(\xi) = \int f(x)e^{ix\xi} \, dx\): \[ \begin{align*} \mathcal{F}\{f * g\}(\xi) &= \int_{-\infty}^{\infty} (f * g)(x)e^{ix\xi} \, dx \\\\ &= \int_{-\infty}^{\infty} \left(\int_{-\infty}^{\infty} f(y)g(x-y) \, dy\right) e^{ix\xi} \, dx \\\\ &= \int_{-\infty}^{\infty} f(y) \left(\int_{-\infty}^{\infty} g(x-y)e^{ix\xi} \, dx\right) dy. \end{align*} \] Substituting \(u = x - y\) (so \(x = u + y\) and \(dx = du\)): \[ \begin{align*} &= \int_{-\infty}^{\infty} f(y) \left(\int_{-\infty}^{\infty} g(u)e^{i(u+y)\xi} \, du\right) dy \\\\ &= \int_{-\infty}^{\infty} f(y)e^{iy\xi} \left(\int_{-\infty}^{\infty} g(u)e^{iu\xi} \, du\right) dy \\\\ &= \left(\int_{-\infty}^{\infty} f(y)e^{iy\xi} \, dy\right) \cdot \left(\int_{-\infty}^{\infty} g(u)e^{iu\xi} \, du\right) \\\\ &= \hat{f}(\xi) \cdot \hat{g}(\xi). \end{align*} \] The interchange of integration order is justified by Fubini's theorem when \(f, g \in L^1(\mathbb{R})\), applied to the real and imaginary parts of the integrand separately.

Proof (product to convolution):

Under sufficient regularity (for instance, \(f, g \in L^1\) with \(\hat f \in L^1\)), the inversion formula gives \[ f(x) = \frac{1}{2\pi}\int \hat f(\eta) e^{-ix\eta}\, d\eta. \] Substituting into the Fourier transform of \(fg\) and applying Fubini's theorem to the real and imaginary parts separately: \[ \begin{align*} \mathcal{F}\{f \cdot g\}(\xi) &= \int_{-\infty}^{\infty} \left[\frac{1}{2\pi}\int_{-\infty}^{\infty} \hat f(\eta)\, e^{-ix\eta}\, d\eta\right] g(x)\, e^{ix\xi}\, dx \\\\ &= \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat f(\eta)\left[\int_{-\infty}^{\infty} g(x)\, e^{ix(\xi - \eta)}\, dx\right] d\eta \\\\ &= \frac{1}{2\pi}\int_{-\infty}^{\infty} \hat f(\eta)\, \hat g(\xi - \eta)\, d\eta = \frac{1}{2\pi}(\hat f * \hat g)(\xi). \end{align*} \]

Discrete Fourier Transform (DFT)

The continuous Fourier transform operates on functions defined on all of \(\mathbb{R}\), but computers work with finite, sampled data. The Discrete Fourier Transform (DFT) bridges this gap. It is the exact discrete analogue of the continuous transform, operating on sequences of \(N\) complex numbers and producing \(N\) frequency coefficients. The DFT inherits all the key properties of its continuous counterpart in discrete form. They include linearity, shift, and convolution.

Note on Sign Convention for DFT

In the continuous section above, we used the convention of this site (\(e^{+ix\xi}\)), chosen to match the characteristic functions of probability theory. However, in the discrete domain (DFT/FFT), the Engineering/Data Science convention (negative sign in the forward transform) is the convention of the standard FFT libraries such as NumPy, PyTorch, and MATLAB.

To ensure that the formulas below match the code you will write in Python, we switch to the standard computational convention for the remainder of this page: \[ \text{DFT: } e^{-i \dots}, \quad \text{IDFT: } e^{+i \dots}. \]

Definition: DFT & IDFT

For a sequence \(\{x_0, x_1, \ldots, x_{N-1}\}\) of \(N\) complex numbers, the DFT is: \[ X_k = \sum_{n=0}^{N-1} x_n e^{-\frac{2\pi i}{N}kn}, \quad k = 0, 1, \ldots, N-1. \] The inverse DFT (IDFT) is: \[ x_n = \frac{1}{N}\sum_{k=0}^{N-1} X_k e^{\frac{2\pi i}{N}kn}, \quad n = 0, 1, \ldots, N-1. \]

Connection to Continuous Transform. The DFT can be viewed as:

Matrix Formulation:
Define \(\omega = e^{-\frac{2\pi i}{N}}\), a primitive \(N\)-th root of unity (so \(\omega^N = 1\)). The DFT matrix is: \[ W = \begin{bmatrix} 1 & 1 & 1 & \cdots & 1 \\\\ 1 & \omega & \omega^2 & \cdots & \omega^{N-1} \\\\ 1 & \omega^2 & \omega^4 & \cdots & \omega^{2(N-1)} \\\\ \vdots & \vdots & \vdots & \ddots & \vdots \\\\ 1 & \omega^{N-1} & \omega^{2(N-1)} & \cdots & \omega^{(N-1)^2} \end{bmatrix} \] where \(W_{kn} = \omega^{kn}\). Then: \[ \mathbf{X} = W\mathbf{x}, \quad \mathbf{x} = \frac{1}{N}W^*\mathbf{X} \] where \(W^*\) is the conjugate transpose.

Properties of the DFT Matrix:

Normalization conventions across domains

The choice between the unnormalized \(W\) and the normalized \(F = \tfrac{1}{\sqrt{N}} W\) is not a mere cosmetic rescaling. It reflects which property of the transform the user cares about most. Three regimes coexist in practice:

  • Signal processing / scientific computing (unnormalized \(W\)):
    NumPy, SciPy, MATLAB, PyTorch (torch.fft.fft), and JAX all default to forward DFT without normalization, placing the full \(1/N\) factor in the inverse. FFTW leaves both directions unnormalized. This is the long-standing numerical-computation convention, reinforced by Cooley-Tukey 1965 and its descendants. It has two practical advantages: the convolution theorem \(\mathcal{F}(f * g) = \hat f \cdot \hat g\) holds with no stray constants, and only the inverse transform pays the \(1/N\) cost. The price is that the forward transform is not an isometry. Norms are scaled by \(\sqrt{N}\).
  • Harmonic analysis / functional analysis (normalized \(F\)):
    The discrete Plancherel identity \(\|f\|_2 = \|Ff\|_2\) requires the normalized version. The map \(F : \mathbb{C}^N \to \mathbb{C}^N\) is then a unitary operator (\(F^*F = I\)), and all of \(L^2\)-based Fourier theory transfers to the discrete setting without rescaling. The clean eigenvalue structure \(\{\pm 1, \pm i\}\) derived above is a symptom of the fourth-power identity \(F^4 = I\), with unitarity forcing the roots onto the unit circle.
  • Quantum computation (normalized \(F\) is mandatory):
    The Quantum Fourier Transform, the key subroutine behind Shor's algorithm and phase estimation, is, in its standard definition, the matrix \(\overline{F} = F^{-1}\), the normalized DFT with the opposite sign in the exponent. It must be unitary, because every gate operation on qubits is a unitary transformation by the postulates of quantum mechanics. There is no choice. The normalized version is the only one that corresponds to a physically realizable quantum gate.

Most numerical libraries expose both options via a norm= argument. NumPy's np.fft.fft(x, norm='ortho') returns the normalized transform, and norm='backward' (the default) returns the unnormalized one. When porting a formula from a textbook to code (or vice versa), the first question to ask is always: which convention is this written in?

Computational Complexity:

Practical Considerations for CS/ML:

Fast Fourier Transform (FFT)

The DFT requires \(O(N^2)\) operations when computed directly from its definition. That cost is prohibitive for large signals. The Fast Fourier Transform (FFT) is not a different transform but an efficient algorithm that reduces the complexity to \(O(N \log N)\). This reduction turns hours into seconds for large \(N\), which is what makes real-time audio processing, medical imaging, and spectral methods in ML computationally feasible.

Cooley-Tukey Algorithm (Radix-2 FFT):
The key insight is to exploit the structure of the DFT matrix using divide-and-conquer. For \(N = 2^m\), split the input into even and odd indices: \[ \begin{align*} X_k &= \sum_{n=0}^{N-1} x_n \omega^{kn} \\\\ &= \sum_{j=0}^{N/2-1} x_{2j} \omega^{k(2j)} + \sum_{j=0}^{N/2-1} x_{2j+1} \omega^{k(2j+1)} \\\\ &= \sum_{j=0}^{N/2-1} x_{2j} (\omega^2)^{kj} + \omega^k \sum_{j=0}^{N/2-1} x_{2j+1} (\omega^2)^{kj}. \end{align*} \] Note that \(\omega^2 = e^{-\frac{4\pi i}{N}} = e^{-\frac{2\pi i}{N/2}}\) is a primitive \((N/2)\)-th root of unity.

Let \(E_k\) be the \((N/2)\)-point DFT of the even-indexed subsequence and \(O_k\) the \((N/2)\)-point DFT of the odd-indexed subsequence. Since \(E_k\) and \(O_k\) are themselves \((N/2)\)-periodic in \(k\) (\(E_{k+N/2} = E_k\), \(O_{k+N/2} = O_k\)), and since \(\omega^{k+N/2} = -\omega^k\) (because \(\omega^{N/2} = e^{-i\pi} = -1\)), we obtain, for \(k = 0, 1, \ldots, N/2-1\): \[ X_k = E_k + \omega^k O_k, \] \[ X_{k+N/2} = E_k - \omega^k O_k. \]

This gives the recurrence \(T(N) = 2T(N/2) + O(N)\), yielding \(T(N) = O(N \log N)\) by the Master theorem.

Other FFT Algorithms:

Implementation Considerations:

Applications in Machine Learning

Fourier methods are fundamental to modern machine learning, particularly in areas requiring efficient computation, signal processing, and function approximation. Here we highlight three significant applications.

1. Audio and Speech Processing

Speech recognition systems such as OpenAI's Whisper (2022) and its open-source successors convert raw audio into log-Mel spectrograms via the Short-Time Fourier Transform (STFT). The audio is windowed into overlapping segments, each transformed via FFT to obtain frequency content, then mapped to the Mel scale (which approximates human auditory perception). This frequency-domain representation serves as input to the neural network, typically a Transformer encoder. The result is robust transcription across languages and noisy environments.

Reference: Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision" (2022)

2. Fourier Neural Operators (FNO) for Scientific Computing

Introduced by Li et al. (ICLR 2021), Fourier Neural Operators (FNOs) learn solution operators of parametric PDEs directly in Fourier space. They achieve up to three orders of magnitude speedup over traditional PDE solvers on benchmark problems such as the Burgers and Navier-Stokes equations. The key idea is to parameterize the kernel \(\hat{k}(\xi)\) in the frequency domain, then multiply with the Fourier transform of the input. The approach has since inspired a rich family of spectral neural operators.

FNO Architecture Sketch

An FNO performs the following steps, repeating steps 2 to 4 in each Fourier layer:

  1. Lift input to higher-dimensional channel space
  2. Apply spectral convolution: \(\mathcal{F}^{-1}(R \cdot \mathcal{F}(v))\) where \(R\) is a learnable weight matrix
  3. Add the pointwise linear term \(Wv\)
  4. Apply non-linearity (activation function), for example GELU
  5. Project to output

The spectral convolution step is where the convolution theorem provides the key efficiency gain.

Reference: Li et al., "Fourier Neural Operator for Parametric PDEs"

3. Fourier Features as Positional Encoding

Neural networks exhibit a spectral bias. They tend to learn low-frequency functions more easily than high-frequency ones. Fourier feature mappings overcome this limitation by lifting low-dimensional inputs into higher-dimensional spaces via sinusoidal functions. The Transformer architecture (Vaswani et al., 2017) introduced sinusoidal positional encoding for sequence positions. This idea was later adapted to continuous spatial coordinates, with Neural Radiance Fields (NeRF, 2020) mapping a scalar coordinate \(p\) to \[ \gamma(p) = \left[\sin(2^0 \pi p), \cos(2^0 \pi p), \ldots, \sin(2^{L-1} \pi p), \cos(2^{L-1} \pi p)\right] \] so that neural networks can capture fine geometric details. Tancik et al. (2020) gave a unified Neural Tangent Kernel analysis explaining why such encodings tame spectral bias.

The same Fourier-feature principle reappears in modern Large Language Models as Rotary Position Embedding (RoPE), which encodes token positions through rotations in 2D subspaces of the query/key vectors and has become a de facto standard across recent LLM families.

References:
Mildenhall et al., "NeRF: Representing Scenes as Neural Radiance Fields" (2020)
Tancik et al., "Fourier Features Let Networks Learn High Frequency Functions" (NeurIPS 2020)
Su et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding" (2021)

Interactive Demos

Now that we've covered the theory, let's see the Fourier Transform in action. We will start with a simple 1D signal to understand the core principles, then move to a 2D image to see a practical application (and a great example of the Convolution Theorem).

1D Signal & Frequency Spectrum

This demo sums two sine waves and seeded Gaussian noise in the Time Domain and displays the single-sided amplitude spectrum (every bin from DC up to and including the Nyquist frequency \(f_s/2\)) in the Frequency Domain. Move the sliders to see how the two domains are linked in real time. Notice how the spectral peaks remain clearly visible above the noise floor. This is a basic illustration of how frequency-domain analysis separates deterministic structure from broadband noise. The noise is drawn from a seeded stream, so the picture is reproducible. Regenerate noise advances the stream to a fresh realization.

The frequency sliders move in 0.5 Hz steps for a reason. The record is exactly one second long, so integer frequencies land exactly on DFT bins, while half-integer frequencies land between bins. At an off-bin frequency the peak's energy leaks across the whole axis and the nearest-bin readout under-reports the amplitude (scalloping). This is the finite-length effect that the windowing remark in the DFT section is about. Switch the window from Rectangular to Hann and watch the leakage skirt collapse. The demo divides windowed spectra by the window's coherent gain, so an on-bin peak still reads its true amplitude.

2D Image & Frequency Domain Filtering

An image is a 2D signal. This demo converts the image to grayscale luminance and computes the 2D FFT to reveal its magnitude spectrum. The center represents low frequencies (broad regions, gradients), while points farther from the center represent high frequencies (edges, fine textures). As established in the Convolution Theorem section, multiplication in the frequency domain corresponds to convolution in the spatial domain, so reshaping the spectrum reshapes the image.

The demo exposes one frequency-domain operation at a time:

  • Low-pass / High-pass / Band-pass filters attenuate frequencies outside a chosen range. The results are blurring, edge extraction, and texture isolation respectively.
  • Notch (auto-remove periodic interference) detects magnitude spikes above the 99.7th percentile of the spectrum (with the DC region protected) and suppresses each one with an inverted-Gaussian notch that is deepest at the spike itself. Try it on the Photo + interference image, where the sinusoidal banding collapses to a handful of off-center spikes. Then try it on the Pattern image for an honest failure. A genuinely periodic image produces exactly the same kind of spectral spikes as interference does, and the notch cannot tell the two apart. It will eat your checkerboard as if it were interference.
  • Compression retains only the top-\(k\%\) of Fourier coefficients by magnitude and zeroes the rest, illustrating the principle behind lossy transform compression. The readout shows the fraction of spectral energy the kept coefficients carry. By Parseval's identity, the discarded energy is the squared reconstruction error, a fact the demo's self-tests verify numerically.
  • Sharpening amplifies high frequencies via a high-frequency-emphasis filter \(1 + \alpha\, H_{\text{high}}(\xi)\), enhancing edges and fine detail.
  • Phase-only and magnitude-only reconstruction put the classic claim that "phase carries the structure" to a direct test. Phase-only replaces every magnitude by 1 and inverts. Magnitude-only replaces every phase by 0 and inverts. On the Text image the contrast is stark. The phase-only image still shows the letter outlines, while the magnitude-only image is an unreadable centrosymmetric blob. Setting all phases to zero forces the output to be symmetric under \((x, y) \mapsto (-x, -y)\), and since the magnitudes do not change when the input is translated circularly, the output cannot record where anything is.
  • Custom Mask lets you draw on the magnitude spectrum to suppress chosen frequencies and observe the result. Since the demo keeps only the real part of the inverse transform, a one-sided mask acts as its centro-symmetrized average, a small but instructive consequence of Hermitian symmetry.

Note on filter shape. The low/high/band-pass filters use a smooth Butterworth-style roll-off rather than an ideal sharp cutoff. Ideal cutoffs introduce ringing artifacts (Gibbs phenomenon) in the spatial domain. Smooth filters trade a slight frequency-domain transition band for cleaner spatial output.