Stopping the Integral
The probabilistic half of the strong Markov property is
already assembled. After a
stopping time
\(\tau\) that is finite almost surely, the driving noise
restarts
as a fresh standard Brownian motion
\(\tilde{B}_v = w_{\tau+v} - w_\tau\), independent of the past
\(\mathcal{F}_\tau\) and carrying the two properties of the
enlargement lemma over the shifted filtration
\(\mathcal{G}_v = \mathcal{F}_{\tau+v}\). The half that
remains is analytic. The diffusion after time \(\tau\) is
built from integrals against \(w\) with random limits, and
these must be rewritten as integrals against the restarted
motion with deterministic limits before the solution theory
can recognize them.
This page proves the two rewriting tools and then harvests
the strong Markov property itself. The first tool stops an
Itô integral at a stopping time and is the identity
that later powers the expectation computations of the
generator theory. The second restarts the diffusion at a
stopping time with finitely many values. Both proofs run on
the same engine. We freeze the value of the stopping time,
apply the deterministic
clock-shift lemma
on the frozen event, and paste the events back together with
the take-out lemma.
General stopping times are then reached by the dyadic
squeeze, carried out for the integral here and for the
diffusion in the final section.
For an integrand \(f\) on a horizon \([0, T]\), the continuous
version \(J\) of the integral process
\(t \mapsto \int_0^t f\, dw\) is secured on the
properties page. The
value \(J_{\tau \wedge T}\) reads the integral at the moment
the clock is stopped. The theorem identifies it as an ordinary
Itô integral whose integrand is switched off at \(\tau\).
Theorem: The Stopped Itô Integral
Let \(T \gt 0\), let \(f \in \mathcal{V}(0, T)\), the
class of admissible integrands,
let \(J\) be the continuous version of
\(t \mapsto \int_0^t f\, dw\) on \([0, T]\), and let \(\tau\)
be a stopping time with respect to \(\{\mathcal{F}_t\}\).
Then \(\mathbf{1}_{[0, \tau)} f \in \mathcal{V}(0, T)\), and
\[
J_{\tau \wedge T}
= \int_0^T \mathbf{1}_{[0, \tau)}(s)\, f(s, \omega)\, d w_s
\quad \text{almost surely.}
\]
Consequently,
\[
\begin{align*}
\mathbb{E} \bigl[ J_{\tau \wedge T} \bigr] &= 0 , \\\\
\mathbb{E} \bigl[ J_{\tau \wedge T}^2 \bigr]
&= \mathbb{E} \Bigl[ \int_0^{\tau \wedge T}
f(s, \omega)^2 \, ds \Bigr] .
\end{align*}
\]
Proof.
The switched-off integrand is admissible.
The set \(\{ (s, \omega) : s \lt \tau(\omega) \}\),
intersected with \([0, t] \times \Omega\), can be written as
\[
\Bigl(
\bigcup_{q \in \mathbb{Q},\, 0 \lt q \leq t}
\bigl( [0, q) \cap [0, t] \bigr) \times \{ \tau \geq q \}
\Bigr)
\cup
\bigl( [0, t] \times \{ \tau \gt t \} \bigr) ,
\]
because \(s \lt \tau\) holds exactly when some rational
\(q\) satisfies \(s \lt q \leq \tau\), and any such \(q\)
exceeding \(t\) forces \(\tau \gt t\) while \(s \leq t\)
already gives \(s \lt q\). Each set
\(\{ \tau \geq q \} = \Omega \setminus \{\tau \lt q\}\)
lies in \(\mathcal{F}_q \subseteq \mathcal{F}_t\), the set
\(\{\tau \lt q\} = \bigcup_m \{\tau \leq q - 1/m\}\) being
a countable union of sets in \(\mathcal{F}_q\), and
\(\{\tau \gt t\} \in \mathcal{F}_t\), so the displayed set
lies in \(\mathcal{B}([0, t]) \otimes \mathcal{F}_t\).
Hence the indicator \(\mathbf{1}_{[0, \tau)}\) is
progressively measurable.
The product \(\mathbf{1}_{[0, \tau)} f\) is therefore
adapted, and measurable on \([0, \infty) \times \Omega\)
as the union over \(t \in \mathbb{N}\) of its
restrictions, so (V1) and (V2) hold. Condition (V3) holds
because the product is dominated by \(|f|\). So
\(\mathbf{1}_{[0, \tau)} f\) lies in the
class \(\mathcal{V}(0, T)\),
and the right-hand integral is defined.
A stopping time with finitely many values.
Let \(\rho\) be a stopping time taking finitely many values
\(0 \leq a_1 \lt \cdots \lt a_N \leq T\). Pointwise on
\([0, T] \times \Omega\),
\[
\mathbf{1}_{[0, \rho)}(s)
= 1 - \sum_{i=1}^{N}
\mathbf{1}_{\{\rho = a_i\}}\,
\mathbf{1}_{[a_i, T]}(s)
\quad \text{for } s \in [0, T) ,
\]
since exactly one of the events \(\{\rho = a_i\}\) occurs
and, on it, \(s \geq \rho\) reads \(s \geq a_i\). Each
summand lies in \(\mathcal{V}(0, T)\) when multiplied by
\(f\), because the level \(\mathbf{1}_{\{\rho = a_i\}}\)
is \(\mathcal{F}_{a_i}\)-measurable, the factor vanishes
before \(a_i\), and the product is dominated by \(|f|\).
By linearity,
\[
\int_0^T \mathbf{1}_{[0, \rho)} f \, d w
= J_T - \sum_{i=1}^{N}
\int_0^T \mathbf{1}_{\{\rho = a_i\}}\,
\mathbf{1}_{[a_i, T]}\, f \, d w .
\]
Fix \(i\) with \(a_i \lt T\). The integrand of the
\(i\)-th term vanishes on \([0, a_i)\), so additivity over
intervals reduces its integral to \([a_i, T]\). There the
clock-shift lemma converts it to an integral from \(0\) to
\(T - a_i\) against the shifted motion
\(w_{a_i + v} - w_{a_i}\), the same lemma placing the
shifted integrand in \(\mathcal{V}(0, T - a_i)\) over the
shifted filtration \(\{\mathcal{F}_{a_i + v}\}\). Over
that filtration the factor \(\mathbf{1}_{\{\rho = a_i\}}\)
is measurable at time zero, and its product with the
shifted \(f\), itself admissible by the same lemma applied
to \(f\) alone, is that same integrand. The take-out
lemma, available over the shifted pair like the rest of
the integral theory, therefore extracts the factor, and
shifting the clock back, we obtain
\[
\begin{align*}
\int_0^T \mathbf{1}_{\{\rho = a_i\}}\, \mathbf{1}_{[a_i, T]}\, f \, d w
&= \mathbf{1}_{\{\rho = a_i\}} \int_{a_i}^T f \, d w \\\\
&= \mathbf{1}_{\{\rho = a_i\}} \bigl( J_T - J_{a_i} \bigr) ,
\end{align*}
\]
the last step by additivity again. If \(a_i = T\), the
integrand is null for the time integral and both sides
vanish, so the display holds for every \(i\). Summing and
using \(\sum_i \mathbf{1}_{\{\rho = a_i\}} = 1\),
\[
\begin{align*}
\int_0^T \mathbf{1}_{[0, \rho)} f \, d w
&= J_T - J_T + \sum_{i=1}^{N} \mathbf{1}_{\{\rho = a_i\}}\, J_{a_i} \\\\
&= J_\rho \quad \text{almost surely.}
\end{align*}
\]
The dyadic squeeze.
For a general stopping time \(\tau\), the time
\(\rho_n = \tau_n \wedge T\) is a stopping time with
finitely many values in \([0, T]\). The dyadic
discretization \(\tau_n\) of the
calculus of stopping times
takes grid values, the minimum with the constant \(T\) is
a stopping time by the same calculus, and only the grid
values below \(T\), together with \(T\) itself, survive.
The previous step gives
\(J_{\rho_n} = \int_0^T \mathbf{1}_{[0, \rho_n)} f\, dw\)
for every \(n\). On the left,
\(\rho_n \downarrow \tau \wedge T\) and \(J\) has
continuous paths, so \(J_{\rho_n} \to J_{\tau \wedge T}\)
almost surely. On the right, the
Itô isometry
gives
\[
\mathbb{E} \Bigl[ \Bigl(
\int_0^T \bigl( \mathbf{1}_{[0, \rho_n)}
- \mathbf{1}_{[0, \tau \wedge T)} \bigr) f \, d w
\Bigr)^2 \Bigr]
= \mathbb{E} \Bigl[
\int_{\tau \wedge T}^{\rho_n}
f(s, \omega)^2 \, ds \Bigr] ,
\]
the two indicators differing only for
\(s \in [\tau \wedge T, \rho_n)\). For each
\(\omega\) the integration windows decrease to a
set of at most one point, which has Lebesgue
measure zero, while \(f(\cdot, \omega)^2\) is
integrable on \([0, T]\) almost surely, so the
inner integral tends to zero pathwise, and it is
dominated by \(\int_0^T f^2\, ds\), which is
integrable because \(f\) belongs to the class.
Dominated convergence
sends the right side to zero. The integrals on the
right of the discrete identity therefore converge
in \(L^2(\mathbb{P})\) to
\(X := \int_0^T \mathbf{1}_{[0, \tau \wedge T)} f\, dw\).
Since \(J_{\rho_n} \to J_{\tau \wedge T}\) almost surely,
Fatou's lemma
applied to the nonnegative variables
\(( J_{\rho_n} - X )^2\) gives
\[
\begin{align*}
\mathbb{E} \bigl[ ( J_{\tau \wedge T} - X )^2 \bigr]
&\leq \liminf_n \mathbb{E} \bigl[ ( J_{\rho_n} - X )^2 \bigr] \\\\
&= 0 ,
\end{align*}
\]
so \(J_{\tau \wedge T} = X\) almost surely. Since
\(\mathbf{1}_{[0, \tau \wedge T)}\) and \(\mathbf{1}_{[0, \tau)}\)
agree at every \(s \lt T\), the two integrands define the
same integral, and the identity of the theorem follows.
Mean and variance.
The display represents \(J_{\tau \wedge T}\) as an
Itô integral of a member of \(\mathcal{V}(0, T)\), so the
zero-mean property
applies, and the isometry gives
\[
\begin{align*}
\mathbb{E} \bigl[ J_{\tau \wedge T}^2 \bigr]
&= \mathbb{E} \Bigl[ \int_0^T \mathbf{1}_{[0, \tau)}(s)\, f(s, \omega)^2 \, ds \Bigr] \\\\
&= \mathbb{E} \Bigl[ \int_0^{\tau \wedge T} f(s, \omega)^2 \, ds \Bigr] .
\end{align*}
\]
The theorem says that stopping the clock cannot manufacture
drift. However elaborate the stopping rule, the stopped
integral still averages to zero, and its second moment still
accumulates \(f^2\) only over the time actually elapsed. Both
halves return in force when
the integral is read at a stopping time
that no horizon dominates, the zero mean carrying the identity
and the isometry carrying the passage to the limit.
Restarting the Diffusion at a Discrete Stopping Time
The second tool restarts the diffusion itself. Here the
stopping time is assumed to take finitely many values. The
passage to general stopping times is the business of the final
section, where it is carried out at the level of expectations
rather than of integrals. Throughout, \(X^x\) is the Itô
diffusion, its defining identity
\[
X^x_t = x + \int_0^t b( X^x_s )\, ds
+ \int_0^t \sigma( X^x_s )\, d w_s
\]
holding almost surely, simultaneously for every \(t \geq 0\),
with both integral processes continuous in their upper limit.
Theorem: The Diffusion Restarted at a Discrete Stopping Time
Let \(\rho\) be a stopping time with respect to
\(\{\mathcal{F}_t\}\) taking finitely many values
\(0 \leq a_1 \lt \cdots \lt a_N\), and let
\(\tilde{B}_v = w_{\rho + v} - w_\rho\) and
\(\mathcal{G}_v = \mathcal{F}_{\rho + v}\) be the
restarted data of the
strong Markov property of Brownian motion.
Set \(Z_v = X^x_{\rho + v}\), where \(X^x\) is the
Itô diffusion
started at \(x\). Then:
(i)
\(Z\) is \(\{\mathcal{G}_v\}\)-adapted with continuous
paths, \(\mathbb{E}[ Z_0^2 ] \lt \infty\), and the process
\(u \mapsto \sigma( Z_u )\) lies in \(\mathcal{V}(0, h)\)
over
\(\{\mathcal{G}_v\}\) for every \(h \gt 0\).
(ii)
Almost surely, simultaneously for every \(v \geq 0\),
\[
Z_v = X^x_\rho + \int_0^v b( Z_u )\, du
+ \int_0^v \sigma( Z_u )\, d \tilde{B}_u ,
\]
the stochastic integral being the Itô integral over
the filtration \(\{\mathcal{G}_v\}\) with respect to the
standard motion \(\tilde{B}\).
Proof.
Write \(C_i = \{\rho = a_i\}\), a finite partition of
\(\Omega\), with \(C_i \in \mathcal{F}_{a_i}\) and
\(C_i \in \mathcal{F}_\rho\), the latter because \(\rho\) is
\(\mathcal{F}_\rho\)-measurable. Write
\(\tilde{w}^{(i)}_u = w_{a_i + u} - w_{a_i}\) for the motion
restarted at the deterministic time \(a_i\). On \(C_i\) the
processes \(\tilde{B}\) and \(\tilde{w}^{(i)}\) coincide.
All integrals against \(\tilde{B}\) below are Itô
integrals over \(( \tilde{B}, \{\mathcal{G}_v\} )\),
legitimate because the
strong Markov property of Brownian motion,
applied to the stopping time \(\rho\), delivers the two
properties of the enlargement lemma for this pair, and the
integral theory runs verbatim over any data with these
properties. The same applies to
\(( \tilde{w}^{(i)}, \{\mathcal{F}_{a_i + u}\} )\), the
case the clock-shift lemma
already ran on.
Admissibility.
\(Z\) has continuous paths because \(X^x\) does. The
diffusion is progressively measurable, being continuous
and adapted, by
the criteria for progressive measurability,
so item (v) of the
calculus of stopping times,
applied at the stopping time \(\rho + v\), makes \(Z_v\)
measurable with respect to \(\mathcal{G}_v\).
For the moments,
\(\mathbb{E}[ Z_0^2 ] = \sum_i
\mathbb{E}[ \mathbf{1}_{C_i} ( X^x_{a_i} )^2 ]
\leq \sum_i \mathbb{E}[ ( X^x_{a_i} )^2 ] \lt \infty\) by
the second-moment bound,
and for \(u \leq h\) the same decomposition gives
\(\mathbb{E}[ Z_u^2 ] \leq \sum_i
\mathbb{E}[ ( X^x_{a_i + u} )^2 ]\), bounded uniformly
over \(u \in [0, h]\) by the moment bound on the horizon
\(a_N + h\). Since \(\sigma\) is Lipschitz, it grows at
most linearly, so
\(\mathbb{E} \int_0^h \sigma( Z_u )^2\, du \lt \infty\),
and the process \(u \mapsto \sigma( Z_u )\) is continuous
and adapted, hence progressively measurable by the same
criteria. This proves (i), and the stochastic integral in
(ii) is defined.
A trace of measurability.
One bookkeeping fact is used twice below. If
\(A \in \mathcal{F}_{a_i + u}\), then
\(A \cap C_i \in \mathcal{G}_u = \mathcal{F}_{\rho + u}\).
Indeed, for \(t \geq 0\),
\[
( A \cap C_i ) \cap \{ \rho + u \leq t \}
= \begin{cases}
A \cap C_i , & a_i + u \leq t , \\
\emptyset , & a_i + u \gt t ,
\end{cases}
\]
because \(\rho = a_i\) on \(C_i\). In the first case,
\(A \cap C_i \in \mathcal{F}_{a_i + u} \subseteq
\mathcal{F}_t\), the factor \(C_i\) lying already in
\(\mathcal{F}_{a_i}\). Taking \(A = \Omega\) puts \(C_i\)
itself in \(\mathcal{G}_u\), so \(\mathbf{1}_{C_i}\) is
\(\mathcal{G}_u\)-measurable, and a process of the form
\(\mathbf{1}_{C_i} p\) with \(p\) adapted to
\(\{\mathcal{F}_{a_i + u}\}\) is adapted to
\(\{\mathcal{G}_u\}\) as well.
Two integrators, one integral.
Fix \(v \gt 0\) and \(i\), and abbreviate
\(q_u = \mathbf{1}_{C_i}\, \sigma( X^x_{a_i + u} )\), a
member of \(\mathcal{V}(0, v)\) over
\(\{\mathcal{F}_{a_i + u}\}\) by the argument of the
previous step, and over \(\{\mathcal{G}_u\}\) by the trace
fact, the two readings being the same function of
\((u, \omega)\). We claim
\[
\int_0^v q_u \, d \tilde{B}_u
= \int_0^v q_u \, d \tilde{w}^{(i)}_u
\quad \text{almost surely.}
\]
By the definition of the
Itô integral,
the left side is the \(L^2\) limit of the elementary
integrals of any sequence of bounded elementary processes
converging to \(q\) in the mean-square sense, and likewise
the right side over its own filtration.
Choose bounded elementary \(p_j\) over
\(\{\mathcal{F}_{a_i + u}\}\) with \(p_j \to q\), and set
\(q_j = \mathbf{1}_{C_i} p_j\). Then \(q_j \to \mathbf{1}_{C_i} q = q\)
in the same sense, each \(q_j\) is bounded and elementary
over \(\{\mathcal{F}_{a_i + u}\}\), and by the trace fact
its levels are measurable with respect to the
corresponding \(\mathcal{G}_u\) as well, so \(q_j\) is
elementary over \(\{\mathcal{G}_u\}\) with the same
partition and levels. The elementary integral of \(q_j\)
against \(\tilde{B}\) and against \(\tilde{w}^{(i)}\) is
the same random variable, because off \(C_i\)
every level vanishes and both sums are zero, while on
\(C_i\) the increments of the two integrators coincide.
The two sides of the claim are therefore \(L^2\) limits of
one and the same sequence, and they agree almost surely.
Splicing on one cell.
Fix \(v \gt 0\) and \(i\). We take out the initially known
factor \(\mathbf{1}_{C_i}\) twice, first over
\(( \tilde{B}, \{\mathcal{G}_u\} )\), where
\(C_i \in \mathcal{F}_\rho = \mathcal{G}_0\), then over
\(( \tilde{w}^{(i)}, \{\mathcal{F}_{a_i+u}\} )\), where
\(C_i \in \mathcal{F}_{a_i}\), and use the claim in between.
The products below lie in the relevant class because they
are dominated by admissible integrands and the factor is
measurable at time zero. Together these steps give
\[
\begin{align*}
\mathbf{1}_{C_i} \int_0^v \sigma( Z_u )\, d \tilde{B}_u
&= \int_0^v \mathbf{1}_{C_i}\, \sigma( Z_u )\,
d \tilde{B}_u \\\\
&= \int_0^v \mathbf{1}_{C_i}\,
\sigma( X^x_{a_i + u} )\, d \tilde{w}^{(i)}_u \\\\
&= \mathbf{1}_{C_i} \int_0^v
\sigma( X^x_{a_i + u} )\, d \tilde{w}^{(i)}_u ,
\end{align*}
\]
where the middle equality also used
\(\mathbf{1}_{C_i} \sigma( Z_u )
= \mathbf{1}_{C_i} \sigma( X^x_{a_i + u} )\), the two
processes agreeing on \(C_i\). The clock-shift lemma,
read from the shifted side back to the original clock,
turns the last integral into
\(\int_{a_i}^{a_i + v} \sigma( X^x_s )\, d w_s\), and the
defining identity of the diffusion, evaluated at
\(a_i + v\) and at \(a_i\) and subtracted, computes it
pathwise:
\[
\int_{a_i}^{a_i + v} \sigma( X^x_s )\, d w_s
= X^x_{a_i + v} - X^x_{a_i}
- \int_{a_i}^{a_i + v} b( X^x_s )\, ds .
\]
On \(C_i\) the right side reads
\(Z_v - Z_0 - \int_0^v b( Z_u )\, du\), the time integral
rewritten by the deterministic substitution
\(s = a_i + u\), path by path. Combining,
\[
\mathbf{1}_{C_i} \int_0^v \sigma( Z_u )\, d \tilde{B}_u
= \mathbf{1}_{C_i}
\Bigl( Z_v - Z_0 - \int_0^v b( Z_u )\, du \Bigr) .
\]
Pasting the cells.
Summing the last display over the finitely many \(i\) and
using \(\sum_i \mathbf{1}_{C_i} = 1\) gives, for each
fixed \(v\),
\[
Z_v = Z_0 + \int_0^v b( Z_u )\, du
+ \int_0^v \sigma( Z_u )\, d \tilde{B}_u
\quad \text{almost surely,}
\]
and \(Z_0 = X^x_\rho\). Every term is continuous in \(v\):
the left side and the time integral pathwise, the
stochastic integral by taking its continuous version,
which the theory over \(( \tilde{B}, \{\mathcal{G}_v\} )\)
provides. Each fixed \(v\) excludes a null event, and
countably many rational \(v\) exclude their union. Adding
the null event where the continuous version fails to be
continuous leaves one full-probability event on which
agreement holds at every rational \(v\) and extends by
continuity to every \(v \geq 0\). This proves (ii).
The theorem has the exact shape of the input that the solution
theory requires: a continuous adapted process with a
square-integrable start, satisfying the equation over driving
data with the two properties of the enlargement lemma. The
final section feeds it to the uniqueness theory, replaces the
discrete stopping time by a general one, and concludes with
the strong Markov property.
The Strong Markov Property
Everything is now in place. The discrete restart theorem hands
the solution theory a diffusion relaunched at a stopping time
with finitely many values. The machinery of the
Markov property page
converts such a relaunch into a conditional expectation
identity. Two squeezes, one dyadic and one by bounded
truncation, then carry the identity from discrete stopping
times to arbitrary almost surely finite ones. Throughout,
\(u( y ) = E^y[ f( X_h ) ]\) is the function of the
Markov property,
bounded and Borel whenever \(f\) is.
Theorem: The Strong Markov Property of Itô Diffusions
Let \(f : \mathbb{R} \to \mathbb{R}\) be bounded and
Borel, let \(X^x\) be the Itô diffusion started at
\(x \in \mathbb{R}\), let \(h \geq 0\), and let \(\tau\)
be a stopping time with respect to \(\{\mathcal{F}_t\}\)
with \(\tau \lt \infty\) almost surely. Then
\[
\mathbb{E} \bigl[ f( X^x_{\tau + h} )
\bigm| \mathcal{F}_\tau \bigr]
= u( X^x_\tau )
\quad \text{almost surely,}
\]
where \(u( y ) = E^y[ f( X_h ) ]\).
Proof.
The discrete case.
Let \(\rho\) be a stopping time with finitely many values,
and let \(\tilde{B}\), \(\{\mathcal{G}_v\}\), and
\(Z_v = X^x_{\rho+v}\) be as in the restart theorem, which
exhibits \(Z\) as a solution over
\(( \tilde{B}, \{\mathcal{G}_v\} )\) with the
\(\mathcal{G}_0\)-measurable, square-integrable initial
value \(X^x_\rho\). The assembly that the Markov property
page ran at a deterministic time now runs again at \(\rho\),
each of its three inputs being available in the restarted
setting. First, every \(\mathcal{F}_\rho\)-measurable start
is \(\mathcal{G}_0\)-measurable, since
\(\mathcal{G}_0 = \mathcal{F}_\rho\), and the existence
proof and the uniqueness half, which rest only on the two
properties of the carrying pair and on a square-integrable
initial value known at time zero, run over
\(( \tilde{B}, \{\mathcal{G}_v\} )\) for every such start.
Second, we run the dyadic-snapping construction of the
measurable version
over the restarted noise, on solutions built over the
smaller filtration generated by \(\tilde{B}\) and the
null sets. It produces a map \(\tilde{\Phi}_h\)
measurable with respect to
\(\mathcal{B}(\mathbb{R}) \otimes \mathcal{H}\), where
\(\mathcal{H} = \sigma( \tilde{B}_v : v \geq 0 ) \vee \mathcal{N}\)
is the restarted noise augmented by the null sets, with
\(\tilde{\Phi}_h( y, \cdot )\) a version of the solution
from the constant start \(y\) at time \(h\). The
\(\sigma\)-algebra \(\mathcal{H}\) is independent of
\(\mathcal{F}_\rho\). Clause (ii) of the
strong Markov property of Brownian motion
gives independence of the restarted noise itself, and
adjoining the null sets changes no probabilities, since
any set of the join differs from a set of
\(\sigma( \tilde{B}_v : v \geq 0 )\) by a null symmetric
difference, which no probability sees.
Third, pathwise uniqueness over the restarted data
identifies \(Z\) with the solution from \(X^x_\rho\) of
the substitution lemma,
and the lemma, applied verbatim there with independence
and the measurable version in hand, gives
\(Z_h = \tilde{\Phi}_h( X^x_\rho, \cdot )\) almost surely.
The
freezing lemma,
applied with \(\mathcal{G} = \mathcal{F}_\rho\), the
independent \(\mathcal{H}\) above, the
\(\mathcal{F}_\rho\)-measurable variable \(X^x_\rho\), and
the bounded
\(\mathcal{B}(\mathbb{R}) \otimes \mathcal{H}\)-measurable
function \(\varphi( y, \omega )
= f( \tilde{\Phi}_h( y, \omega ) )\), gives
\[
\begin{align*}
\mathbb{E} \bigl[ f( X^x_{\rho + h} ) \bigm| \mathcal{F}_\rho \bigr]
&= g( X^x_\rho ) , \\\\
g( y ) &= \mathbb{E} \bigl[ f( \tilde{\Phi}_h( y, \cdot ) ) \bigr] .
\end{align*}
\]
It remains to identify \(g\) with \(u\). For fixed \(y\),
the variable \(\tilde{\Phi}_h( y, \cdot )\) is almost
surely the time-\(h\) value of the solution from the
constant start \(y\) over
\(( \tilde{B}, \{\mathcal{G}_v\} )\). That solution and
the diffusion \(X^y\) are both solutions of the same
equation from the same deterministic start, each over
driving data with the two properties of the enlargement
lemma, so each qualifies as a
weak solution
with initial law \(\delta_y\), and
uniqueness in law
makes their time-\(h\) values equal in distribution. Hence
\(g( y ) = \mathbb{E}[ f( X^y_h ) ] = u( y )\), and the
discrete case is proved.
Continuity of \(u\) for continuous \(f\).
Let \(f\) be bounded and continuous and let
\(y_j \to y\). The mean-square stability bound,
its constant \(A\) read on a horizon \(T \geq h\)
with \(T \gt 0\), gives
\[
\mathbb{E} \bigl[ | X^{y_j}_h - X^y_h |^2 \bigr]
\leq 3\, | y_j - y |^2\, e^{A h} \to 0 .
\]
From any subsequence, select a further subsequence along
which the bound is summable. Along it the series
\(\sum_j | X^{y_j}_h - X^y_h |^2\) has finite expectation,
the sum and the expectation exchanging by
monotone convergence,
so the series is finite almost surely and its terms tend to
zero. Thus \(X^{y_j}_h \to X^y_h\) almost surely along the
subsequence, and continuity of \(f\), boundedness, and
dominated convergence
give \(u( y_j ) \to u( y )\) along it. Every subsequence of
the bounded real sequence \(u( y_j )\) therefore has a
further subsequence converging to \(u( y )\), which forces
convergence of the full sequence. Hence \(u\) is continuous.
The dyadic squeeze, for bounded \(\tau\) and
continuous \(f\).
Suppose \(\tau \leq m\) for a constant \(m\), and let
\(\tau_n\) be its dyadic discretization, a stopping time
with finitely many values below \(m + 1\). Fix
\(A \in \mathcal{F}_\tau\). Since \(\tau \leq \tau_n\),
the monotonicity clause of the
calculus
places \(A\) in \(\mathcal{F}_{\tau_n}\). Integrating the
discrete case at \(\tau_n\) over \(A\),
\[
\mathbb{E} \bigl[ \mathbf{1}_A\,
f( X^x_{\tau_n + h} ) \bigr]
= \mathbb{E} \bigl[ \mathbf{1}_A\,
u( X^x_{\tau_n} ) \bigr] .
\]
As \(n \to \infty\), \(\tau_n \downarrow \tau\) and the
paths of the diffusion are continuous, so
\(X^x_{\tau_n + h} \to X^x_{\tau + h}\) and
\(X^x_{\tau_n} \to X^x_\tau\) almost surely. Both \(f\)
and \(u\) are bounded and continuous, so dominated
convergence passes the display to the limit:
\[
\mathbb{E} \bigl[ \mathbf{1}_A\,
f( X^x_{\tau + h} ) \bigr]
= \mathbb{E} \bigl[ \mathbf{1}_A\,
u( X^x_\tau ) \bigr]
\quad \text{for every } A \in \mathcal{F}_\tau .
\]
The truncation squeeze, for general \(\tau\) and
continuous \(f\).
For general \(\tau \lt \infty\) almost surely,
the truncation \(\tau \wedge m\) is a bounded
stopping time. If \(A \in \mathcal{F}_\tau\),
then \(A \cap \{ \tau \leq m \} \in \mathcal{F}_{\tau \wedge m}\).
For \(t \lt m\), its intersection with
\(\{ \tau \wedge m \leq t \}\) is \(A \cap \{ \tau \leq t \} \in \mathcal{F}_t\),
and for \(t \geq m\) it is
\(A \cap \{ \tau \leq m \}\) itself, which lies
in \(\mathcal{F}_m \subseteq \mathcal{F}_t\) by
the definition of \(\mathcal{F}_\tau\). On
\(\{ \tau \leq m \}\) the times \(\tau\) and
\(\tau \wedge m\) coincide, so the previous
step, applied at \(\tau \wedge m\) with the
event \(A \cap \{ \tau \leq m \}\), reads
\[
\mathbb{E} \bigl[ \mathbf{1}_{A \cap \{\tau \leq m\}}\,
f( X^x_{\tau + h} ) \bigr]
= \mathbb{E} \bigl[ \mathbf{1}_{A \cap \{\tau \leq m\}}\,
u( X^x_\tau ) \bigr] .
\]
Since \(\tau \lt \infty\) almost surely, the indicators
increase to \(\mathbf{1}_A\) as \(m \to \infty\), and both
integrands are bounded, so dominated convergence applied along
that increasing family removes the truncation. The display of
the previous step therefore holds for every almost surely
finite stopping time and every bounded continuous \(f\).
From continuous to Borel \(f\).
Fix \(A \in \mathcal{F}_\tau\) and consider the two set
functions
\[
\begin{align*}
\mu_1( B )
&= \mathbb{E} \bigl[ \mathbf{1}_A\,
\mathbf{1}_{\{ X^x_{\tau + h} \in B \}} \bigr] , \\\\
\mu_2( B )
&= \mathbb{E} \bigl[ \mathbf{1}_A\,
u_{\mathbf{1}_B}( X^x_\tau ) \bigr] ,
\end{align*}
\]
for Borel \(B \subseteq \mathbb{R}\), where
\(u_{\mathbf{1}_B}( y ) = \mathbb{P}[ X^y_h \in B ]\).
Both are finite measures: \(\mu_1\) visibly, and \(\mu_2\)
because each \(u_{\mathbf{1}_B}\) is bounded Borel,
countable additivity holding pointwise in \(y\) and
passing through the outer expectation by
monotone convergence.
For a closed set \(F\) with distance function \(d_F\), the
continuous approximants \(\varphi_j( z ) = \max( 0,\, 1 - j\, d_F( z ) )\)
decrease to \(\mathbf{1}_F\), the device already used to
extend the restart of the noise
beyond continuous tests. Dominated convergence on both
sides of the display, and inside \(u\), shows
\(\mu_1( F ) = \mu_2( F )\). Closed sets are stable under
intersection and generate the Borel \(\sigma\)-algebra, so
the two measures coincide, by the monotone-class step this
track takes on faith. The display therefore holds for
\(f = \mathbf{1}_B\), hence for simple \(f\) by linearity,
hence for every bounded Borel \(f\) by uniform
approximation with simple functions, the error on both
sides being at most \(\| f - s \|_\infty\).
The conditional form.
For bounded Borel \(f\), the function \(u\) is bounded and
Borel, and \(\{\tau \lt \infty\}\) is an event of full
probability lying in \(\mathcal{F}_\tau\), on which
\(X^x_\tau\) is measurable by the calculus, the
diffusion being progressively measurable. Setting the
variable to zero off that event changes nothing almost
surely, so the variable
\(u( X^x_\tau )\) is
\(\mathcal{F}_\tau\)-measurable, bounded, and satisfies
the defining integral identity of the
conditional expectation
of \(f( X^x_{\tau + h} )\) given \(\mathcal{F}_\tau\)
against every \(A \in \mathcal{F}_\tau\). This is the
theorem.
Following the convention of
the Markov property page,
later pages write the conclusion in the compact form
\[
E^x \bigl[ f( X_{\tau + h} ) \bigm| \mathcal{F}_\tau \bigr]
= E^{X_\tau} \bigl[ f( X_h ) \bigr] ,
\]
the right side being the function
\(y \mapsto E^y[ f( X_h ) ]\) evaluated at the random position
\(X^x_\tau\). The deterministic-time Markov property is the
special case \(\tau \equiv t\), so the strong form genuinely
subsumes it. The summary that closed the Markov property page
survives the upgrade. What the diffusion does after a stopping
time depends on the past only through where the diffusion
stands when the clock stops.
Finitely Many Times After the Stop
The single-time identity extends to any finite collection of
times after the stop, and the extension costs only the tower
property and the theorem itself, applied at the later stopping
times \(\tau + h_i\).
Theorem: The Strong Markov Property at Finitely Many Times
Let \(f_1, \ldots, f_k : \mathbb{R} \to \mathbb{R}\) be
bounded Borel functions, let
\(0 \leq h_1 \leq \cdots \leq h_k\), and let \(\tau\) be a
stopping time with \(\tau \lt \infty\) almost surely.
Then
\[
E^x \Bigl[ \prod_{i=1}^{k} f_i( X_{\tau + h_i} )
\Bigm| \mathcal{F}_\tau \Bigr]
= E^{X_\tau} \Bigl[ \prod_{i=1}^{k} f_i( X_{h_i} )
\Bigr]
\quad \text{almost surely,}
\]
the right side being the function
\(y \mapsto \mathbb{E} \bigl[ \prod_i f_i( X^y_{h_i} )
\bigr]\), bounded and Borel, evaluated at
\(X^x_\tau\).
Proof.
Induction on \(k\), the case \(k = 1\) being the theorem
above. Let \(k \geq 2\), abbreviate
\(P = \prod_{i=1}^{k-1} f_i( X^x_{\tau + h_i} )\), and set
\(u_k( y ) = E^y[ f_k( X_{h_k - h_{k-1}} ) ]\), bounded
and Borel. Each \(\tau + h_i\) is a stopping time by the
calculus, and \(X^x_{\tau + h_i}\) is measurable with
respect to
\(\mathcal{F}_{\tau + h_i} \subseteq
\mathcal{F}_{\tau + h_{k-1}}\) on the full-probability
event \(\{\tau \lt \infty\}\) as before, so \(P\) is
\(\mathcal{F}_{\tau + h_{k-1}}\)-measurable and bounded.
The
tower property
over
\(\mathcal{F}_\tau \subseteq \mathcal{F}_{\tau + h_{k-1}}\),
then
taking out what is known,
then the single-time theorem at the almost surely finite
stopping time \(\tau + h_{k-1}\) with the function
\(f_k\) and the gap \(h_k - h_{k-1}\), give
\[
\begin{align*}
\mathbb{E} \Bigl[ P\, f_k( X^x_{\tau + h_k} )
\Bigm| \mathcal{F}_\tau \Bigr]
&= \mathbb{E} \Bigl[ P\,
\mathbb{E} \bigl[ f_k( X^x_{\tau + h_k} )
\bigm| \mathcal{F}_{\tau + h_{k-1}} \bigr]
\Bigm| \mathcal{F}_\tau \Bigr] \\\\
&= \mathbb{E} \Bigl[ P\,
u_k( X^x_{\tau + h_{k-1}} )
\Bigm| \mathcal{F}_\tau \Bigr] .
\end{align*}
\]
The last conditional expectation involves \(k - 1\)
bounded Borel functions of the positions at
\(\tau + h_1, \ldots, \tau + h_{k-1}\), the final one
being the product \(f_{k-1}\, u_k\). The induction
hypothesis evaluates it as \(w( X^x_\tau )\), where the
product over \(i \leq k-2\) is empty when \(k = 2\),
\[
w( y ) = \mathbb{E} \Bigl[
\prod_{i=1}^{k-2} f_i( X^y_{h_i} ) \cdot
( f_{k-1}\, u_k )( X^y_{h_{k-1}} ) \Bigr] .
\]
Finally, the same two moves at the deterministic clock
identify \(w\) with the right side of the theorem. For each
\(y\), the Markov property
at time \(h_{k-1}\) with gap \(h_k - h_{k-1}\) gives
\(\mathbb{E}[ f_k( X^y_{h_k} ) \mid
\mathcal{F}_{h_{k-1}} ]
= u_k( X^y_{h_{k-1}} )\), and the tower property together
with taking out the \(\mathcal{F}_{h_{k-1}}\)-measurable
factor \(\prod_{i \leq k-1} f_i( X^y_{h_i} )\) turn
\(w( y )\) into
\(\mathbb{E} \bigl[ \prod_{i=1}^{k} f_i( X^y_{h_i} )
\bigr]\). Measurability of the right side in \(y\) comes from
the same reduction, which exhibits the product expectation as
finitely many nested applications of maps of the form
\(f \mapsto u\), each preserving bounded Borel functions.
Episodes, Exit Times, and Optimal Stopping
An episode in
reinforcement learning
ends at a random time, when the agent reaches a terminal
state and the trajectory stops. That time is computed from
the trajectory alone, never from its future: the
discrete-time counterpart of a stopping time, and
typically a first exit time from the set of non-terminal
states. Random times of the same kind also occur inside an
episode, most visibly when an option in hierarchical
reinforcement learning runs until its termination
condition fires, and methods built on options treat what
follows as a fresh start from the state the trajectory
then occupies. In discrete time this restart is cheap. The
Markov decision process
builds the one-step restart into its definition, and
splitting on the values of the random time, as the
discrete case above does, delivers the rest. When the
model is a diffusion, the restart is no longer cheap. It
is exactly the strong Markov property, and this page is
what it costs to prove.
A second family of problems inverts the perspective and
treats the stopping time itself as the decision variable,
asking when to sell an asset, exercise an option, or halt
an experiment so as to maximize an expected reward. Such
optimal stopping problems are posed in precisely the
vocabulary built here: a class of admissible stopping
times, the information \(\mathcal{F}_\tau\) each one
commands, and the restart identity that lets the value of
continuing be read from the current state alone.
The strong Markov property completes the probabilistic core of
the diffusion theory on this track, and what follows it turns
analytic. The integral is next
read at a stopping time,
which converts Itô's formula into a statement about
averages. That statement then attaches to each diffusion
a second-order differential
operator, its generator, computed from the coefficients
\(b\) and \(\sigma\). The two combine into
a formula evaluating expectations at
first exit times, the point where the stopped integral of
this page and the
first exit times
certified before it meet.