The Ornstein-Uhlenbeck Process

The Equation and Its Solution

This page is devoted to a single stochastic differential equation, the one among the solvable examples with the deepest reach into statistics and machine learning. The concept of an SDE, the conventions, and the working notion of solution are all inherited unchanged from the previous page: \(w_t\) is standard one-dimensional Brownian motion with Brownian filtration \(\{\mathcal{F}_t\}_{t \geq 0}\) containing all null sets, integrals denote continuous versions, every equation is read in the Itô sense, and a solution means an explicitly exhibited Itô process verified against the integral equation, with uniqueness deferred to the general theory. Two placements of the unknown are already solved there, both with the noise proportional to the state. The remaining basic configuration places the unknown in the drift and leaves the noise additive,

\[ dX_t = \mu X_t\, dt + \sigma\, dw_t , \quad X_0 \in \mathbb{R} \text{ a given constant} , \]

with \(\mu, \sigma \in \mathbb{R}\). This is the Ornstein-Uhlenbeck equation, also called the Langevin equation, and its original physical content makes the parameters concrete. Take \(X_t\) to be the velocity of a heavy particle suspended in a fluid. The drift \(\mu X_t\) with \(\mu \lt 0\) is friction, always opposing the current velocity in proportion to it, while \(\sigma\, dw_t\) is the relentless bombardment by fluid molecules, whose strength does not depend on how fast the particle moves.

Because the noise term no longer involves the unknown, the integrating factor sheds its randomness. Multiply by \(e^{-\mu t}\) and run the now familiar informal product rule. The cross term vanishes outright, since \(d(e^{-\mu t})\) carries only \(dt\), and

\[ d \bigl( e^{-\mu t} X_t \bigr) = e^{-\mu t}\, \sigma\, dw_t , \]

so that integrating and multiplying back by \(e^{\mu t}\) produces the candidate \(X_t = e^{\mu t} X_0 + \sigma \int_0^t e^{\mu (t - s)}\, dw_s\). The stochastic term is now an integral of a deterministic integrand, exactly the kind of Wiener integral whose Gaussian law was established for the representation theorem, and this is what gives the Ornstein-Uhlenbeck family its complete statistical transparency.

The friction regime \(\mu \lt 0\) is where the family does its real work. The mean \(e^{\mu t} X_0\) forgets the initial condition exponentially fast, while the variance rises from zero and saturates at \(\sigma^2 / (2 |\mu|)\), the level at which dissipation and noise injection balance. The process neither settles nor explodes. It equilibrates, and this makes it the basic model of a quantity that fluctuates persistently around a rest state. For \(\mu \gt 0\) the same formula describes exponential instability seeded by noise, and for \(\mu = 0\) the equation degenerates to \(dX_t = \sigma\, dw_t\), whose solution is scaled Brownian motion itself.

Mean Reversion and Diffusion Models

Applications rarely relax toward zero. The mean-reverting form of the equation pulls the state toward a level \(m \in \mathbb{R}\) at rate \(\theta \gt 0\),

\[ dX_t = \theta \bigl( m - X_t \bigr)\, dt + \sigma\, dw_t , \quad X_0 \in \mathbb{R} \text{ a given constant} , \]

and reduces to the theorem by recentering. If \(\widetilde{X}\) denotes the solution of the theorem with \(\mu = - \theta\) and initial value \(X_0 - m\), then \(X_t = m + \widetilde{X}_t\) differs from \(\widetilde{X}_t\) by a constant, so it is an Itô process with the same coefficients, and \(dX_t = - \theta \widetilde{X}_t\, dt + \sigma\, dw_t = \theta ( m - X_t )\, dt + \sigma\, dw_t\). Explicitly,

\[ X_t = m + e^{- \theta t} \bigl( X_0 - m \bigr) + \sigma \int_0^t e^{- \theta (t - s)}\, dw_s , \]

Gaussian at each time with

\[ \begin{align*} \mathbb{E}[X_t] &= m + e^{- \theta t} \bigl( X_0 - m \bigr) , \\\\ \operatorname{Var}[X_t] &= \frac{\sigma^2}{2 \theta} \bigl( 1 - e^{- 2 \theta t} \bigr) . \end{align*} \]

The mean glides from \(X_0\) to \(m\) and the variance saturates at \(\sigma^2 / (2 \theta)\), so the process forgets where it started and fluctuates around the level it is pulled toward. The unit-rate case \(\theta = 1\) is the form in which the equation is most often written; carrying \(\theta\) explicitly costs nothing and is repaid immediately below by machine learning, and again by the covariance of the next section.

The Gaussian Process Reading

Statement (ii) of the theorem describes one time point in isolation. In fact the whole process is Gaussian in the sense of the Gaussian process definition, meaning that every finite collection \((X_{t_1}, \ldots, X_{t_k})\) is jointly Gaussian. The reduction is short. For constants \(c_1, \ldots, c_k\) and times \(t_1 \lt \cdots \lt t_k\), the additivity and linearity of the integral assemble the combination \(\sum_i c_i X_{t_i}\) into a constant plus a single Wiener integral \(\int_0^{t_k} h\, dw_s\), where \(h\) is deterministic and piecewise continuous, equal on each interval \([t_{j-1}, t_j)\) to the sum \(\sum_{i \geq j} c_i \sigma e^{\mu ( t_i - s )}\) of the integrands still active there.

Every affine combination of the family is therefore Gaussian, and we take on faith the standard characterization that a family with this property has jointly Gaussian finite-dimensional laws; its verification is a characteristic-function argument belonging to the multivariate Gaussian toolkit rather than to this track.

A Gaussian process is determined by its mean and covariance functions, and the mean is already known, so only the covariance remains. Two earlier results assemble it. For deterministic \(f, g \in L^2([0, T])\), linearity and the Itô isometry combine into a polarization identity,

\[ \begin{align*} \mathbb{E} \Bigl[ \int_0^T f\, dw \int_0^T g\, dw \Bigr] &= \tfrac{1}{4}\, \mathbb{E} \Bigl[ \Bigl( \int_0^T ( f + g )\, dw \Bigr)^2 \Bigr] - \tfrac{1}{4}\, \mathbb{E} \Bigl[ \Bigl( \int_0^T ( f - g )\, dw \Bigr)^2 \Bigr] \\\\ &= \tfrac{1}{4} \int_0^T ( f + g )^2\, ds - \tfrac{1}{4} \int_0^T ( f - g )^2\, ds \\\\ &= \int_0^T f g\, ds . \end{align*} \]

Apply this to the mean-reverting solution with rate \(\theta \gt 0\), whose fluctuation at time \(t\) is the Wiener integral of \(\sigma e^{- \theta ( t - s )}\, \mathbf{1}_{[0, t]}(s)\). For \(s \leq t\),

\[ \begin{align*} \operatorname{Cov} ( X_s, X_t ) &= \sigma^2 \int_0^s e^{- \theta ( s - u )}\, e^{- \theta ( t - u )}\, du \\\\ &= \sigma^2 e^{- \theta ( s + t )} \, \frac{ e^{2 \theta s} - 1 }{ 2 \theta } \\\\ &= \frac{ \sigma^2 }{ 2 \theta } \Bigl( e^{- \theta ( t - s )} - e^{- \theta ( t + s )} \Bigr) . \end{align*} \]

For times far from the start, with the lag \(t - s\) held fixed, the second term dies and the covariance approaches the stationary covariance

\[ \operatorname{Cov} ( X_s, X_t ) \longrightarrow \frac{ \sigma^2 }{ 2 \theta }\, e^{- \theta \lvert t - s \rvert } , \]

a function of the lag alone. Up to the variance scale \(\sigma^2 / ( 2 \theta )\), this is the exponential kernel \(\exp ( - \lvert t - s \rvert / \ell )\) with length-scale \(\ell = 1 / \theta\), the roughest member of the Matérn family surveyed on the Gaussian process page. The equilibrated Ornstein-Uhlenbeck process is thus a dynamical construction of a kernel that regression practice usually postulates directly. The reversion rate is the reciprocal length-scale, and the exponential decay of memory in the kernel is the exponential forgetting of the friction term.

Beyond Closed Forms

Three equations across two pages, three placements of the unknown, three closed forms. What the family shares is linearity. In every case the coefficients were affine in the state, an integrating factor built from the homogeneous part absorbed the difficult term, and the solution emerged as an explicit functional of the driving path. This closing section asks how far that reach extends, and what must replace it where it ends.

One further solved equation has already appeared on this track, and no integrating factor produced it. The radial part of Brownian motion in dimension \(n \geq 2\), started away from the origin, was shown by the multidimensional Itô formula to satisfy

\[ dR_t = \sum_{i=1}^{n} \frac{w_t^{(i)}}{R_t}\, dw_t^{(i)} + \frac{n - 1}{2 R_t}\, dt , \]

an equation whose drift is genuinely nonlinear in the state and singular at the edge of its domain. Read from the present perspective, the logic ran backwards. None of the methods of these two pages could have produced \(R_t\) from its equation. The process came first, as a transformation of one already understood, and the Itô formula then disclosed which equation it solves. That reversed route is a solution method in its own right, and our heuristics were miniature instances of it, each guessing a transformation, a logarithm here, an integrating factor there, under which the unknown becomes a process already in hand.

When neither route applies, one needs theorems, and the deterministic special case \(\sigma = 0\) already shows what any general theory must rule out. The innocent-looking quadratic equation \(dX_t / dt = X_t^2\) with \(X_0 = 1\) is solved on \([0, 1)\) by \(X_t = 1 / (1 - t)\), as differentiation checks, and this solution escapes to infinity as \(t\) approaches \(1\). Growth faster than linear can drive a solution to infinity in finite time, so global existence is not free.

The equation \(dX_t / dt = 3 X_t^{2/3}\) with \(X_0 = 0\) is solved by the zero function, and equally, for every \(a \gt 0\), by the function that stays at zero up to time \(a\) and continues as \((t - a)^3\) afterwards; the check, including the smooth fit at \(t = a\), is again one line. Infinitely many solutions sprout from a single initial value. A drift that is continuous but not Lipschitz at one point can fail to pin the solution down, so uniqueness is not free either.

The general theory answers both threats at once, and it is the destination of the next stretch of this track. For coefficients Lipschitz in the state and growing at most linearly, an iteration of Picard type produces a solution for every square-integrable initial condition independent of the driving noise, and a comparison inequality of Gronwall type shows there is only one. The two counterexamples above mark exactly where those hypotheses bite. Linear growth forbids finite-time escape, and the Lipschitz condition forbids branching. Once existence and uniqueness are in hand, the closed forms of these two pages stop being isolated trophies and become what they were always meant to be, the exactly solvable landmarks of a theory that covers the terrain between them.