Lagrange's Mean Value Theorem
Until now we have focused on the instantaneous rate of change at a single point.
Lagrange's Mean Value Theorem (MVT) bridges the gap between this local derivative and
the average change over an interval.
Theorem: Lagrange's Mean Value Theorem
Suppose \(f\) is continuous on \([a, b]\) and differentiable on \((a, b)\). Then there exists
\(c \in (a, b)\) such that
\[
f'(c) = \frac{f(b) - f(a)}{b - a}. \tag{1}
\]
The theorem says that at some point \(c\), the instantaneous rate of change equals the average rate of
change over the entire interval. The special case \(f(a) = f(b)\) is known as Rolle's
Theorem. Under the same regularity hypotheses, if \(f(a) = f(b)\) then there exists
\(c \in (a,b)\) with \(f'(c) = 0\).
Sketch of the proof of Rolle's Theorem.
Since \(f\) is continuous on the compact interval \([a,b]\), the Extreme Value Theorem
(a continuous function on a compact set attains its maximum and minimum, a fact we use
without proof on this page) guarantees that \(f\) attains its maximum \(M\) and minimum \(m\) on
\([a,b]\). Since
\(m \leq f(a) = f(b) \leq M\), at least one of the following three cases holds:
- If \(M = m\), then \(f\) is constant and \(f'(c) = 0\) for every \(c \in (a,b)\).
- If \(M \gt f(a) = f(b)\), the maximum is attained at some interior point \(c \in (a,b)\).
Since \(f\) is differentiable there and \(c\) is a local maximum, \(f'(c) = 0\), because the
difference quotients at an interior maximum are nonnegative from the left and nonpositive
from the right.
- If \(m \lt f(a) = f(b)\), the minimum is attained at an interior point \(c\), and the
same argument yields \(f'(c) = 0\).
These three cases are exhaustive. If \(M = m\) fails then \(M \gt m\), so at least one of the
values \(M, m\) differs from \(f(a) = f(b)\), which triggers case 2 or case 3.
Now we use Rolle's Theorem to prove Lagrange's MVT.
Proof of Lagrange's MVT.
Let the secant line joining \((a, f(a))\) and \((b, f(b))\) be
\[
y = f(a) + \frac{f(b)-f(a)}{b - a}\,(x - a),
\]
and define \(g(x)\) as the vertical gap between the graph of \(f\) and this secant line:
\[
g(x) = f(x) - f(a) - \frac{f(b)-f(a)}{b - a}\,(x - a). \tag{2}
\]
Since \(g\) is the sum of \(f\) and a polynomial of degree at most one, and sums of continuous
(resp. differentiable) functions are continuous (resp. differentiable), \(g\) is continuous on
\([a, b]\) and differentiable on \((a, b)\). Differentiating (2),
\[
g'(x) = f'(x) - \frac{f(b)-f(a)}{b - a}.
\]
A direct substitution shows \(g(a) = g(b) = 0\), so by Rolle's Theorem there exists
\(c \in (a,b)\) with \(g'(c) = 0\), that is,
\[
f'(c) - \frac{f(b)-f(a)}{b - a} = 0,
\]
which rearranges to the desired identity.
Taylor's Theorem
By rearranging Equation (1) as \(f(b) = f(a) + f'(c)(b-a)\), we see that the MVT has the shape
of a zeroth-order Taylor expansion with an exact remainder. The polynomial
part is the constant \(f(a)\), and the remainder \(f'(c)(b-a)\) carries the derivative at an
unknown point. Taylor's Theorem, stated below for \(n \geq 1\), extends the polynomial part to
degree at most \(n\).
We state the theorem in two forms that differ in their hypotheses and in the
information they provide. The Peano form captures the local
asymptotic behavior of \(f\) near a point \(a\). Its hypothesis asks for \(f^{(n)}\)
only at the single point \(a\). The Lagrange form instead provides
a pointwise remainder expression. Its hypothesis asks for \(f^{(n+1)}\) at every
point of an interval that contains \(a\).
We first introduce the \(n\)th-order Taylor polynomial of \(f\) at \(a\), defined whenever
\(f^{(n)}(a)\) exists. It is the polynomial part common to both forms:
\[
\begin{align*}
T_n(x)
&= \sum_{k = 0}^{n} \frac{f^{(k)}(a)}{k!}\,(x - a)^k \\\\
&= f(a) + \frac{f'(a)}{1!}(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \cdots + \frac{f^{(n)}(a)}{n!}(x-a)^n.
\end{align*}
\]
The first-order Taylor polynomial
\[
T_1(x) = f(a) + f'(a)(x-a)
\]
is precisely the
linearization
of \(f\) at \(a\). Taylor's Theorem is its systematic higher-order generalization.
Theorem: Taylor's Theorem, Peano Form
Let \(n \geq 1\) be an integer, let \(U \subseteq \mathbb{R}\) be an open neighborhood of
\(a\), and suppose \(f: U \to \mathbb{R}\) is \((n-1)\) times differentiable on \(U\) with
\(f^{(n-1)}\) differentiable at \(a\) (so that \(f^{(n)}(a)\) exists). Then
\[
f(x) = T_n(x) + o\bigl(|x - a|^n\bigr) \quad \text{as } x \to a,
\]
meaning that the remainder \(R_n(x) := f(x) - T_n(x)\) satisfies
\(\lim_{x \to a} R_n(x) / |x-a|^n = 0\).
The little-\(o\) symbol,
introduced on the Linear Approximations page, means that
\(R_n(x)\) vanishes strictly faster than \(|x-a|^n\) as \(x \to a\). The Peano form is the natural
statement for asymptotic analysis. It characterizes how well \(T_n\) approximates
\(f\) in the limit \(x \to a\), but it gives no explicit formula or bound for \(R_n\) at any fixed
\(x \neq a\). For that, we need a stronger hypothesis.
Theorem: Taylor's Theorem, Lagrange Form
Let \(n \geq 1\) be an integer and let \(I \subseteq \mathbb{R}\) be an interval
containing both \(a\) and \(x\), with \(x \neq a\). Suppose \(f: I \to \mathbb{R}\) is \((n+1)\) times
differentiable on \(I\). Then there exists
\(c\) strictly between \(a\) and \(x\) such that
\[
R_n(x) = \frac{f^{(n+1)}(c)}{(n+1)!}\,(x - a)^{n+1},
\]
where \(R_n(x) = f(x) - T_n(x)\) is the remainder.
The Lagrange form is useful when a concrete pointwise bound on \(R_n\) is needed. Any bound
on \(|f^{(n+1)}|\) over \(I\) yields a bound on \(|R_n(x)|\) via
\(|R_n(x)| \leq \frac{\sup_{c \in I} |f^{(n+1)}(c)|}{(n+1)!}\,|x-a|^{n+1}\). This is the form invoked in error
analysis for numerical methods and in convergence proofs for optimization algorithms where an explicit
remainder estimate is required.
Proof sketch of both forms.
Both reduce to the mean value machinery of this page. Every derivative of \(R_n\) up to order
\(n\) vanishes at \(a\), because \(T_n\) matches \(f\) to that order there. For the Lagrange
form, apply Cauchy's Mean Value Theorem below to the pair \(R_n\) and \((x-a)^{n+1}\), then to
their derivatives, and so on \(n+1\) times, each time moving the evaluation point strictly
closer to \(a\). Since \(T_n^{(n+1)} = 0\), the last quotient is \(f^{(n+1)}(c)/(n+1)!\).
For the Peano form the same descent stops after \(n-1\) steps, leaving
\(R_n(x)/(x-a)^n = R_n^{(n-1)}(\xi)/\bigl(n!\,(\xi - a)\bigr)\) with \(\xi\) between \(a\)
and \(x\). Writing out
\(R_n^{(n-1)}(\xi) = f^{(n-1)}(\xi) - f^{(n-1)}(a) - f^{(n)}(a)(\xi - a)\), the quotient tends
to \(0\) as \(x \to a\) precisely because \(f^{(n-1)}\) is differentiable at \(a\).
When \(f\) is infinitely differentiable on \(I\) and the Lagrange remainder satisfies
\(R_n(x) \to 0\) as \(n \to \infty\), the partial sums \(T_n(x)\) converge to \(f(x)\), and
the limit is the Taylor series of \(f\) centered at \(a\):
\[
f(x) = \sum_{k = 0}^{\infty} \frac{f^{(k)}(a)}{k!}\,(x - a)^k.
\]
Approximation vs. Reality
We often use the first-order approximation \(T_1(x) = f(a) + f'(a)(x-a)\). In its Lagrange form,
Taylor's Theorem bounds the error of this approximation. If \(f\) is twice differentiable on an
interval containing \(a\) and \(x\), a bound on \(|f''|\) over that interval becomes a bound on
\(|f(x) - T_1(x)|\). Bounds of this kind enter convergence analyses of optimization algorithms,
where higher-order remainders determine how large a step a method can safely take.
A subtle point deserves emphasis. Being infinitely differentiable is not enough for the Taylor
series to recover the function. The standard counter-example is
\[
f(x) = \begin{cases} e^{-1/x^2} & x \neq 0, \\\\ 0 & x = 0, \end{cases}
\]
which is \(C^\infty\) on all of \(\mathbb{R}\) with \(f^{(k)}(0) = 0\) for every \(k\). Its Taylor
series at \(0\) is therefore identically zero, yet \(f(x) \gt 0\) for every \(x \neq 0\). The
series converges everywhere but fails to equal \(f\) anywhere except at the origin.
Functions whose Taylor series converges to \(f\) on some neighborhood of every point are called
real-analytic, a condition strictly stronger than \(C^\infty\). This distinction
matters in information geometry, where the exponential and other parametric families are typically
real-analytic, and in the high-accuracy convergence theory of spectral methods (Chebyshev
expansions, and Fourier expansions of periodic functions), where analyticity yields exponential
convergence, \(C^\infty\) regularity alone gives convergence faster than any power of the
truncation order, and finite differentiability only an algebraic rate.
Cauchy's Mean Value Theorem
Theorem: Cauchy's Mean Value Theorem
Let \(f\) and \(g\) be continuous on \([a,b]\) and differentiable on \((a,b)\). If
\(g'(x) \neq 0\) for every \(x \in (a, b)\), then \(g(b) \neq g(a)\) (else Rolle's Theorem
would give a point where \(g'\) vanishes), and there exists \(c \in (a, b)\) such that
\[
\frac{f(b)-f(a)}{g(b)-g(a)} = \frac{f'(c)}{g'(c)}.
\]
Proof sketch.
The proof is the same auxiliary-function reduction used for Lagrange's MVT above. Apply
Rolle's Theorem to
\[
h(x) = f(x) - f(a) - \tfrac{f(b)-f(a)}{g(b)-g(a)}\bigl(g(x) - g(a)\bigr),
\]
which satisfies \(h(a) = h(b) = 0\). The resulting point \(c\) with \(h'(c) = 0\) gives the stated
identity after rearranging
\[
h'(c) = f'(c) - \tfrac{f(b)-f(a)}{g(b)-g(a)}\,g'(c).
\]
Cauchy's Mean Value Theorem generalizes Lagrange's MVT (\(g(x) = x\)). It is used to prove
L'Hôpital's Rule. Convergence analyses of optimization methods use it less
directly than Lagrange's MVT or Taylor's Theorem.
Why it matters: From Limits to Information Geometry
While Lagrange's MVT deals with the change of a single function, Cauchy's MVT considers the
ratio of changes between two functions. This ratio form is what the proof of
L'Hôpital's Rule uses.
In Information Geometry one again compares two rates of change at once. The
negative of the
natural gradient
is the direction along which a loss decreases fastest, to first order, per unit
Fisher-Rao length of the parameter step, the length measured by the Fisher information
matrix. That rate is a ratio of two derivatives, the rate of change of the loss over
the rate of change of length, which has the form of the right-hand side of Cauchy's
MVT. The natural gradient demo
derives this property.
Higher-dimensional MVT
Before moving to optimization, we must extend the MVT to multidimensional spaces. While gradient descent relies on local
linear approximations in practice, this theorem is one of the ingredients in proving the convergence of such
algorithms. It guarantees that the discrete change in a scalar-valued loss function can be related to the continuous field
of its gradients.
Theorem: Higher-Dimensional MVT
Let \(f: X \to \mathbb{R}\) be differentiable on an open set \(X \subseteq \mathbb{R}^n\) (in the
sense of the
vector-valued definition
with \(m = 1\), that is, \(f\) is Fréchet differentiable, a condition
stronger than mere existence of all partial derivatives). Let \(a, b \in X\) be distinct points with
the closed segment \([a,b] \subset X\). Then there exists \(c \in (a,b)\) such that
\[
f(b) - f(a) = \nabla f(c) \cdot (b - a).
\]
Here \([a,b] := \{(1-t)a + tb : t \in [0,1]\}\) and \((a,b) := \{(1-t)a + tb : t \in (0,1)\}\)
denote the closed and open segments joining \(a\) and \(b\), and \(\nabla f\) is the
gradient in the convention fixed on the Linear
Approximations page.
Proof.
Define \(h : [0,1] \to \mathbb{R}\) by \(h(t) = f\bigl((1-t)a + tb\bigr) = f(a + t(b-a))\). Since
\([a,b] \subset X\) and \(f\) is differentiable on \(X\), \(h\) is continuous on \([0,1]\) and
differentiable on \((0,1)\). By the chain rule,
\[
h'(t) = \nabla f(a + t(b-a)) \cdot (b - a).
\]
Applying Lagrange's MVT to \(h\) on \([0,1]\), there exists
\(\theta \in (0,1)\) such that
\[
h(1) - h(0) = h'(\theta)(1 - 0) = h'(\theta).
\]
Since \(h(1) = f(b)\) and \(h(0) = f(a)\), setting \(c := a + \theta(b - a) \in (a,b)\) gives
\[
f(b) - f(a) = \nabla f(c) \cdot (b - a).
\]
For a vector-valued map \(F = (F^1, \dots, F^m)\) the exact equality fails. Each
scalar component \(F^j\) supplies its own mean value point \(c_j\), and there is in general no
single \(c\) serving all components at once. What survives of the mean value form is an
inequality, and it survives with no loss in the constant. The standard device is to test the
vector increment against a fixed direction, reducing the vector statement to a single application
of the scalar theorem.
Theorem: Mean Value Inequality
Let \(F : X \to \mathbb{R}^m\) be
differentiable
on an open set
\(X \subseteq \mathbb{R}^n\), and let \(a, b \in X\) with the closed segment
\([a,b] \subset X\). Then
\[
\|F(b) - F(a)\| \leq \Big(\sup_{c \in [a,b]} \|DF(c)\|\Big)\,\|b - a\|,
\]
where \(\|DF(c)\|\) denotes the
operator norm
of the total derivative
\(DF(c) : \mathbb{R}^n \to \mathbb{R}^m\), with Euclidean norms on \(\mathbb{R}^n\) and
\(\mathbb{R}^m\) throughout. The right-hand side may be \(+\infty\), in which case the inequality
holds trivially. In particular, if \(F\) is of class \(C^1\)
and \(K \subseteq X\) is compact and convex, then \(F\) is
Lipschitz continuous
on \(K\) with Lipschitz constant \(\sup_{x \in K} \|DF(x)\|\).
Proof.
If \(F(b) = F(a)\) the inequality is trivial, so assume \(F(b) \neq F(a)\) and set
\(\mathbf{u} := F(b) - F(a) \in \mathbb{R}^m\). Define the scalar function
\(\varphi : [0,1] \to \mathbb{R}\) by
\[
\varphi(t) = \big\langle \mathbf{u},\, F(a + t(b-a)) \big\rangle.
\]
Since \([a,b] \subset X\) and \(F\) is differentiable on \(X\), the composition with the segment
\(t \mapsto a + t(b-a)\) is differentiable on \((0,1)\) and continuous on \([0,1]\). By the chain
rule,
\[
\varphi'(t) = \big\langle \mathbf{u},\, DF\big(a + t(b-a)\big)(b - a) \big\rangle.
\]
Applying Lagrange's MVT to \(\varphi\) on \([0,1]\) yields
\(\theta \in (0,1)\) with \(\varphi(1) - \varphi(0) = \varphi'(\theta)\). The left-hand side is
\(\langle \mathbf{u}, F(b)\rangle - \langle \mathbf{u}, F(a)\rangle
= \langle \mathbf{u}, \mathbf{u}\rangle = \|\mathbf{u}\|^2\).
Writing \(c := a + \theta(b-a)\) and applying the Cauchy-Schwarz inequality
\(|\langle \mathbf{u}, \mathbf{v}\rangle| \leq \|\mathbf{u}\|\,\|\mathbf{v}\|\) on \(\mathbb{R}^m\),
which we take as known, to the right-hand side \(\varphi'(\theta)\),
\[
\begin{align*}
\|\mathbf{u}\|^2
&= \big\langle \mathbf{u},\, DF(c)(b-a) \big\rangle \\\\
&\leq \|\mathbf{u}\|\,\|DF(c)(b-a)\| \\\\
&\leq \|\mathbf{u}\|\,\|DF(c)\|\,\|b-a\|,
\end{align*}
\]
the last step by the definition of the operator norm. Dividing by \(\|\mathbf{u}\| \gt 0\) gives
\(\|F(b) - F(a)\| \leq \|DF(c)\|\,\|b-a\| \leq \big(\sup_{z \in [a,b]}\|DF(z)\|\big)\|b-a\|\).
For the \(C^1\) statement, fix a compact convex \(K \subseteq X\). The operator norm
\(x \mapsto \|DF(x)\|\) is continuous, since
\(\bigl|\|A\| - \|B\|\bigr| \leq \|A - B\| \leq \sum_{i,j} |A_{ij} - B_{ij}|\) and the entries of
\(DF\) are continuous for \(F\) of class \(C^1\), hence attains a finite maximum
\(M = \sup_{x \in K}\|DF(x)\|\) on \(K\). For any \(a, b \in K\), convexity gives
\([a,b] \subseteq K\), so the inequality above yields
\(\|F(b) - F(a)\| \leq M\,\|b-a\|\), which is precisely the Lipschitz condition with
constant \(M\).
Important Constraint: Scalar-valued Outputs
The exact equality of the Higher-Dimensional MVT is guaranteed only for
scalar-valued functions \(f : \mathbb{R}^n \to \mathbb{R}\). For
vector-valued maps no single mean value point exists in general, and the Mean Value Inequality
above takes the place of the exact equality. First-order convergence proofs for gradient
descent work with the scalar loss rather than the vector-valued forward map, and it is for the
scalar loss that the exact identity above is available to them. The inequality form, by
contrast, is the tool used to control how far a smooth map can spread points apart, and it is
exactly the estimate that converts a bound on a derivative into the contraction property
underlying the inverse function theorem.
Multivariate Taylor's Theorem
The Higher-Dimensional MVT controlled the first-order behavior of a scalar field along a
segment. To control approximation to arbitrary order, and to obtain an exact
remainder rather than an existential mean value point, we extend Taylor's Theorem to functions of
several variables.
The single-variable Lagrange form above located the remainder at an unknown point \(c\). The
multivariate version below instead expresses the remainder as an integral, a form that can be
manipulated algebraically rather than merely bounded. The foundations of differential geometry use
this exact integral remainder to identify tangent vectors with directional-derivative operators.
The mechanism is identical to the one used in the proof of the Higher-Dimensional MVT. We restrict
\(f\) to the segment from \(a\) to \(x\) by composing with the curve \(t \mapsto a + t(x-a)\), which
reduces a multivariable problem to a single-variable one, and then invoke the single-variable theory.
What changes is only the bookkeeping for higher-order derivatives, which we organize with multi-index
notation. Throughout, \(f \in C^{k+1}(X)\) means that all partial derivatives of order at most
\(k+1\) exist and are continuous on \(X\). Such a function is differentiable in the vector-valued
sense, a fact recorded without proof on the Jacobian page, so the chain rule applies to it.
Multi-index notation
Writing higher-order partial derivatives and the matching monomials explicitly becomes unwieldy
past first order. A multi-index packages the bookkeeping. We use the
ordered convention throughout (each slot independently ranges over
\(1,\dots,n\)), which is what makes the compressed sums below coincide with fully written-out
sums with no extra combinatorial factors.
Definition: Multi-index (ordered convention)
Fix \(n \geq 1\). A multi-index of length \(m\) is an ordered tuple
\(I = (i_1, \dots, i_m)\) with each \(i_j \in \{1, \dots, n\}\). We write \(|I| = m\) for its
order. For a point \(x = (x^1,\dots,x^n) \in \mathbb{R}^n\) and a base point
\(a = (a^1,\dots,a^n) \in \mathbb{R}^n\), the associated scalar monomial is
\[
(x - a)^I := (x^{i_1} - a^{i_1})\,(x^{i_2} - a^{i_2}) \cdots (x^{i_m} - a^{i_m}) \in \mathbb{R},
\]
and for an \(m\)-times differentiable scalar function \(f\) the associated partial derivative is
\[
\partial_I f := \frac{\partial^m f}{\partial x^{i_1} \partial x^{i_2} \cdots \partial x^{i_m}} \in \mathbb{R}.
\]
Both \((x-a)^I\) and \(\partial_I f\) are scalars, even though \(x\) and \(a\)
are vectors. Because the convention is ordered, summing a quantity over all multi-indices of a
fixed order \(|I| = m\) is the same as summing independently over each slot:
\[
\sum_{I : |I| = m} = \sum_{i_1 = 1}^{n} \sum_{i_2 = 1}^{n} \cdots \sum_{i_m = 1}^{n}.
\]
Before stating the general theorem, it is worth writing out the lowest orders explicitly, since these
are the cases that arise most often. At first order (\(k = 1\)), the expansion reads
\[
\begin{align*}
f(x)
&= f(a) + \sum_{i=1}^{n} \frac{\partial f}{\partial x^i}(a)\,(x^i - a^i) \\\\
&\quad + \underbrace{\sum_{i,j=1}^{n} (x^i - a^i)(x^j - a^j) \int_0^1 (1 - t)\,
\frac{\partial^2 f}{\partial x^i \partial x^j}\bigl(a + t(x-a)\bigr)\,dt}_{\text{remainder } R_1(x)} .
\end{align*}
\]
Every term here is a scalar. Both \(f(a)\) and each \(\partial f/\partial x^i(a)\)
are scalars, \((x^i - a^i)\) is a scalar component of the displacement vector \(x - a\), and the
integral is a scalar (the integrand is evaluated at the vector \(a + t(x-a)\) but returns a real
number). The linear term \(\sum_i \partial_i f(a)(x^i - a^i)\) is the directional term
\(\nabla f(a) \cdot (x - a)\). The Higher-Dimensional MVT has a term of the same form, with the
gradient evaluated at an unknown point \(c\) instead of at \(a\). Here the gradient sits at \(a\),
and an explicit second-order remainder accounts for the difference instead of a mean value point.
At second order (\(k = 2\)), the new term is the quadratic
\[
\frac{1}{2} \sum_{i,j=1}^{n} \frac{\partial^2 f}{\partial x^i \partial x^j}(a)\,(x^i - a^i)(x^j - a^j),
\]
which is precisely one-half the quadratic form of the Hessian, the matrix
of second partial derivatives \(\partial^2 f / \partial x^i \partial x^j(a)\) (symmetric here, since
continuous mixed partials commute, a fact we do not prove on this page), evaluated on the
displacement vector \(x - a\). This quadratic form is itself a scalar. The \(k = 2\) expansion is what
underlies second-order optimization and Laplace-type approximations.
The general statement compresses these patterns using the multi-index notation above.
Theorem: Multivariate Taylor's Theorem (integral remainder)
Let \(X \subseteq \mathbb{R}^n\) be open and let \(a \in X\) be fixed. Suppose
\(f \in C^{k+1}(X)\) for some integer \(k \geq 0\). If \(W \subseteq X\) is any convex set
containing \(a\), then for all \(x \in W\),
\[
f(x) = P_k(x) + R_k(x),
\]
where the \(k\)th-order Taylor polynomial \(P_k\) is the scalar-valued
function
\[
P_k(x) = f(a) + \sum_{m=1}^{k} \frac{1}{m!} \sum_{I : |I| = m} \partial_I f(a)\,(x - a)^I,
\]
and the \(k\)th remainder term \(R_k\) is
\[
R_k(x) = \frac{1}{k!} \sum_{I : |I| = k+1} (x - a)^I \int_0^1 (1 - t)^k\,
\partial_I f\bigl(a + t(x-a)\bigr)\,dt .
\]
Proof.
Fix \(x \in W\). Since \(W\) is convex and contains both \(a\) and \(x\), the segment
\(\{a + t(x-a) : t \in [0,1]\}\) lies in \(W \subseteq X\), so the scalar function
\[
u(t) := f\bigl(a + t(x-a)\bigr), \quad t \in [0,1],
\]
is well defined. The set \(J := \{t \in \mathbb{R} : a + t(x-a) \in X\}\) is open, as the
preimage of the open set \(X\) under a continuous map, and it contains \([0,1]\). The same
formula defines \(u\) on \(J\), and \(u\) is of class \(C^{k+1}\) there. We prove the claim
by induction on \(k\), tracking derivatives of \(u\) via the
chain rule.
Base case \(k = 0\). Here \(P_0(x) = f(a)\). By the chain rule,
\(u'(t) = \sum_{i=1}^n \partial_i f(a + t(x-a))\,(x^i - a^i)
= \sum_{I : |I| = 1} (x-a)^I\, \partial_I f(a + t(x-a))\).
The Fundamental Theorem of Calculus (for a \(C^1\) function \(\int_0^1 u' = u(1) - u(0)\), used
without proof here, as is the integration by parts it yields from the product rule) applied to
\(u\) on \([0,1]\) gives
\[
\begin{align*}
f(x) - f(a)
&= u(1) - u(0) \\\\
&= \int_0^1 u'(t)\,dt \\\\
&= \sum_{I : |I| = 1} (x-a)^I \int_0^1 \partial_I f\bigl(a + t(x-a)\bigr)\,dt,
\end{align*}
\]
which is exactly \(f(x) = P_0(x) + R_0(x)\), since \((1-t)^0 = 1\) and \(0! = 1\).
Inductive step. Suppose the formula holds for some \(k\) and every \(C^{k+1}\)
function. We promote it to \(k + 1\), so now \(f \in C^{k+2}(X)\) and each \(\partial_I f\) with
\(|I| = k+1\) is \(C^1\).
Each remainder integral has the form \(\int_0^1 (1-t)^k\, g(t)\,dt\) with
\(g(t) = \partial_I f(a + t(x-a))\). Integrating by parts, using
\(\frac{d}{dt}\!\left[-\frac{(1-t)^{k+1}}{k+1}\right] = (1-t)^k\) and again the chain rule for
\(g'(t) = \sum_{j=1}^n \partial_j \partial_I f(a + t(x-a))\,(x^j - a^j)\), yields
\[
\begin{align*}
\int_0^1 (1-t)^k\, g(t)\,dt
&= \frac{1}{k+1}\, g(0) \\\\
&\quad + \frac{1}{k+1} \sum_{j=1}^n (x^j - a^j) \int_0^1 (1-t)^{k+1}\,
\partial_j \partial_I f\bigl(a + t(x-a)\bigr)\,dt .
\end{align*}
\]
The boundary term contributes \(\frac{1}{k+1}\,\partial_I f(a)\), which (summed with the prefactor
\(\frac{1}{k!}\) over \(|I| = k+1\)) supplies exactly the order-\((k+1)\) term of \(P_{k+1}\),
since \(\frac{1}{k!}\cdot\frac{1}{k+1} = \frac{1}{(k+1)!}\). The surviving integral, with one
extra differentiation slot \(j\) prepended, reassembles into the sum over multi-indices of order
\(k + 2\), giving \(R_{k+1}\) with weight \((1-t)^{k+1}\) and prefactor \(\frac{1}{(k+1)!}\). This
is precisely the asserted formula with \(k\) replaced by \(k + 1\), completing the induction.
When only a bound on the error is needed rather than the exact integral, the integral remainder
reduces to the following estimate.
Theorem: Taylor Error Bound
Let \(X \subseteq \mathbb{R}^n\) be open, \(a \in X\), and \(f \in C^{k+1}(X)\) for some
\(k \geq 0\). If \(W \subseteq X\) is convex, contains \(a\), and every partial derivative of
\(f\) of order \(k + 1\) is bounded in absolute value by a constant \(M\) on \(W\), then for
all \(x \in W\),
\[
\bigl| f(x) - P_k(x) \bigr| \leq \frac{n^{k+1}\,M}{(k+1)!}\,\lVert x - a\rVert^{\,k+1},
\]
where \(P_k\) is the \(k\)th Taylor polynomial of \(f\) at \(a\) and \(\lVert\cdot\rVert\)
is the Euclidean norm.
Proof.
The remainder \(R_k(x) = f(x) - P_k(x)\) is a sum over the multi-indices \(I\) with \(|I| = k+1\).
By the ordered convention there are \(n^{k+1}\) such terms. For each, the scalar factor satisfies
\(\lvert (x-a)^I \rvert \leq \lVert x - a\rVert^{\,k+1}\), since each of the \(k+1\) scalar
factors \(\lvert x^{i_j} - a^{i_j}\rvert\) is at most the Euclidean norm \(\lVert x - a\rVert\).
The integrand obeys \(\lvert \partial_I f(a + t(x-a))\rvert \leq M\), and
\(\int_0^1 (1-t)^k\,dt = \tfrac{1}{k+1}\). Combining with the prefactor \(\tfrac{1}{k!}\) and
\(\tfrac{1}{k!}\cdot\tfrac{1}{k+1} = \tfrac{1}{(k+1)!}\), the triangle inequality over the
\(n^{k+1}\) terms gives the stated bound.
Why the integral remainder matters
The Lagrange form earlier in this page is suited to bounding error, and the Taylor
error bound above recovers exactly that role in the multivariate setting. But the integral form
of the remainder does something the Lagrange form cannot. It exhibits the remainder as an
explicit sum in which each term carries the product \((x - a)^I\) of \(k+1\) factors
\((x^{i_j} - a^{i_j})\), each vanishing at \(x = a\), multiplied by a continuous
coefficient (smooth whenever \(f\) is).
This algebraic structure, a remainder built from products that die at the base point, is exactly
what is needed when one must apply a linear operator obeying the product rule at \(a\) to the
remainder and have it annihilate the leftover terms. For \(k \geq 1\) every term has at least two
vanishing factors, and the product rule sends such a term to zero. This annihilation is the step
used in the smooth-manifold theory of tangent vectors, where
the first-order (\(k = 1\)) case of this theorem identifies abstract tangent vectors with
classical directional derivatives.