Derivative (multivariable calculus)

In mathematics, the derivative of a function at a point is the linear part of the best affine approximation to the function near the point. In one-variable calculus, this is the tangent line approximation. In multivariable calculus, the same property is generalized to define the derivative of a vector-valued function or function of a vector argument. Sometimes called the total derivative, in contrast with partial derivatives, the derivative approximates the function with respect to all of its arguments, not just a single one.Template:Sfn In many situations, this is the same as considering all partial derivatives simultaneously.

In functional analysis, particularly in infinite dimensions, the derivative in this sense is called the Fréchet derivative.Template:SfnTemplate:Sfn

Derivative as a linear map

Let $U\subseteq {ℝ}^n$ be an open subset. Then a function $f:U\to {ℝ}^m$ is said to be differentiable at a point $a\in U$ if there exists a linear transformation $D{f}_a:{ℝ}^n\to {ℝ}^m$ such thatTemplate:SfnTemplate:Sfn

$\underset{x\to a}{lim}\frac{f(x)-f(a)-D{f}_a(x-a)}{\Vert x-a\Vert }=0$

where $\Vert \dots \Vert$ denotes the norm of $\dots$. The linear map $D{f}_a$ is called the derivative or differential of $f$ at $a$.Template:SfnTemplate:Sfn Here $D{f}_a(x-a)$ refers to applying the linear transformation $D{f}_a$ to the vector $(x-a)$; in coordinates, this is a matrix-vector product. Other notations for the derivative include $D_{a}f$ and $Df(a)$. A function is differentiable if its derivative exists at every point in its domain.

Conceptually, the definition of the derivative expresses the idea that $D{f}_a$ is the best linear approximation to $f(a+h)-f(a)$ for small $h$. This can be made precise by quantifying the error in the linear approximation $D{f}_a$. To do so, write

$f(a+h)=f(a)+D{f}_a(h)+\epsilon (h),$

where $\epsilon (h)$ equals the error in the approximation. To say that the derivative of $f$ at $a$ is $D{f}_a$ is equivalent to the statement

$\epsilon (h)=o(h),$

where $o$ is little-o notation and means that $\epsilon (h)/\Vert h\Vert$ tends to zero as $h\to 0$. The derivative $D{f}_a$ is the unique linear transformation for which the error term is this small, and this is the sense in which it is the best linear approximation to $f(a+h)-f(a)$.

Differentiability

Plot of $\frac{x^{2}y}{x^4+y^2}\sqrt{x^2+y^2}$, a function such that the directional derivative ${\mathrm{\nabla }}_{u}f(0,0)=0$, a linear functional of $u$, but which is not differentiable

The function $f$ is differentiable if and only if each of its components $f_i:U\to ℝ$ is differentiable, so when studying derivatives, it is often possible to work one coordinate at a time in the codomain. However, the same is not true of the coordinates in the domain. It is true that if $f$ is differentiable at $a$, then each partial derivative $\partial f/\partial x_i$ exists at $a$.Template:Sfn

The converse does not hold: it can happen that all of the partial derivatives of $f$ at $a$ exist, but $f$ is not differentiable at $a$.Template:Sfn An example is the following function, which is continuous and has both partial derivatives zero at the origin, but is not differentiable there: $$f(x,y)=\{\begin{array}{cc}\frac{xy}{\sqrt{x^2+y^2}}& (x,y)\ne (0,0)\\ 0& (x,y)=(0,0)\end{array}.$$ (In polar coordinates, this function is $f=r\cos \theta \sin \theta$.)

Even the existence and linearity of all directional derivatives at a point is not sufficient for differentiability; the essential additional requirement is that the linear approximation hold uniformly as the increment tends to zero from all directions. An example is $$f(x,y)=\{\begin{array}{cc}\frac{x^{2}y}{x^4+y^2}\sqrt{x^2+y^2}& (x,y)\ne (0,0)\\ 0& (x,y)=(0,0)\end{array}$$ whose directional derivatives are all 0 at (0,0), but which fails to be differentiable there.

However, if all the partial derivatives of $f$ at $a$ exist in a neighborhood of $a$ and are continuous at $a$, then $f$ is differentiable at $a$.Template:Sfn If $f$ is differentiable at a point, then the derivative of $f$ is the linear transformation corresponding to the Jacobian matrix of partial derivatives at the point.Template:Sfn

Differentials

In some advanced calculus texts, the derivative is also called the differential.Template:SfnTemplate:Sfn However, this term has several different, but closely connected meanings, in mathematics and the sciences.

When a differentiable function $f:{ℝ}^n\to ℝ$ is scalar valued, the derivative of $f$ at $a$ may be written as the Jacobian matrix, which in this instance is a row matrix (a matrix consisting of elements in a single row, i.e., a row vector):

$D{f}_a=\left[\begin{array}{ccc}\frac{\partial f}{\partial x_1}(a)& \cdots & \frac{\partial f}{\partial x_n}(a)\end{array}\right].$

The linear approximation property of the derivative implies that if

$\mathrm{\Delta }x={\left[\begin{array}{ccc}\mathrm{\Delta }{x}_1& \cdots & \mathrm{\Delta }{x}_n\end{array}\right]}^{𝖳}$

is a small vector (where the $𝖳$ denotes transpose, so that this vector is a column vector), then

$f(a+\mathrm{\Delta }x)-f(a)\approx D{f}_a\mathrm{\Delta }x=\sum _{i=1}^n\frac{\partial f}{\partial x_i}(a)\mathrm{\Delta }{x}_i.$

Heuristically, this suggests that if $d{x}_1,\dots ,d{x}_n$ are infinitesimal increments in the coordinate directions, then

$d{f}_a=\sum _{i=1}^n\frac{\partial f}{\partial x_i}(a)d{x}_i$

and this is the differential of $f$ at $a$.Template:SfnTemplate:Sfn In fact, the notion of the infinitesimal, which is merely symbolic here, can be equipped with extensive mathematical structure. Techniques, such as the theory of differential forms, effectively give analytical and algebraic descriptions of objects like infinitesimal increments, $d{x}_i$. For instance, $d{x}_i$ may be inscribed as a linear functional on the vector space ${ℝ}^n$. Evaluating $d{x}_i$ at a vector $h$ in ${ℝ}^n$ measures how much $h$ points in the $i$-th coordinate direction. The differential $d{f}_a$ is a linear combination of linear functionals and hence is itself a linear functional. The evaluation $d{f}_a(h)$ is the directional derivative of $f$ along $h$. This point of view makes the derivative an instance of the exterior derivative.Template:Sfn

Being a linear form, the differential is naturally a covector. In a Euclidean space, a covector is naturally dual to a vector (its transpose). This vector is called the gradient of $f$, and points in the direction in which $f$ increases most rapidly.

Suppose now that $f$ is a vector-valued function, that is, $f:{ℝ}^n\to {ℝ}^m$. In this case, the components $f_i$ of $f$ are real-valued functions, so they have associated differential forms $d{f}_i$. The differential $df$ amalgamates these forms into a single object and is therefore an instance of a vector-valued differential form.

If $f:M\to N$ is a mapping between differentiable manifolds, differentiability can be formulated by differentiability in any coordinate chart. Invariantly, the differential of $f$ at a point $p\in M$ is a linear map $d{f}_p:T_{p}M\to T_{f(p)}N$ from the tangent space of $M$ at $p$ to that of $N$ at $f(p)$.Template:SfnTemplate:Sfn This is also known as the pushforward. This is closely related to the derivative as a linear approximation, since evaluating $d{f}_p$ on a tangent vector gives the directional derivative of $f$ in that direction. Nevertheless, the differential is not quite the same thing as the derivative of a function between vector spaces: $f$ is a mapping of manifolds and not of the vector spaces $T_{p}M$ and $T_{f(p)}N$ where the linear approximation lives.

In applications such as thermodynamics, the language of differentials often has an additional conceptual role. Expressions such as $dU=TdS-PdV$ describe differentials of state functions and lead to questions about exactness, natural variables, and relations among partial derivatives. This use is related to the derivative of a multivariable function, but it is not just another name for the linear map $D{f}_a$ associated with an ordinary function.Template:SfnTemplate:Sfn

Total derivative

The term total derivative is also used in more than one way. In some mathematical texts it denotes the full derivative $D{f}_a$, as opposed to any one partial derivative. In that sense, the total derivative is the linear map that accounts for variation in all coordinate directions simultaneously.

In many applied contexts, however, "total derivative" refers instead to the derivative of a composite dependence. For example, if

$z=f(x,y),x=x(t),y=y(t),$

then the total derivative of $z$ with respect to $t$ is

$\frac{dz}{dt}=\frac{\partial f}{\partial x}\frac{dx}{dt}+\frac{\partial f}{\partial y}\frac{dy}{dt}.$

This is the ordinary derivative of the composite function $f(x(t),y(t))$, computed by the chain rule. Equivalently, it is obtained by applying the derivative of $f$ to the velocity vector of the path $t\mapsto (x(t),y(t))$:

$\frac{d}{dt}f(x(t),y(t))=D{f}_{(x(t),y(t))}(x^{\prime }(t),y^{\prime }(t)).$

This chain-rule sense of "total derivative" is common in physics, engineering, economics, and other applied fields. In mechanics, for instance, the total time derivative of a function $F(q,p,t)$ along a trajectory includes both explicit dependence on $t$ and implicit dependence through the time-dependent variables $q(t)$ and $p(t)$. In fluid mechanics, related terminology appears in the material derivative, which differentiates a quantity along the motion of a fluid parcel.Template:SfnTemplate:Sfn

Another example is in classical mechanics, where the total derivative of a function that depends on phase space parameters and time is its partial derivative in time plus its Poisson bracket with the Hamiltonian $H$:Template:Sfn $$\frac{df}{dt}=\frac{\partial f}{\partial t}+\{f,H\}$$ Like the material derivative, total derivative in mechanics has the property that it is the derivative of the composite when pulled back to any Hamiltonian trajectory, but is still treated as a function of all phase space coordinates and time.

In comparative statics, total derivatives often describe how endogenous variables change with respect to exogenous variables in an implicitly defined system of equations.Template:Sfn The endogeneous variables are generally not explicit functions of the exogeneous variables, other than through the implicit function theorem, and the total derivative is handled implicitly.

Thus, although "total derivative" can mean the derivative $D{f}_a$ in the sense above, the term also commonly refers to a derivative along a specified dependence or process. This ambiguity is one reason to distinguish between the derivative as a linear map and the various differential or total-derivative notations used in applications.

Chain rule

A form of the chain rule generalizes from one-variable calculus. It says that, for two functions $f$ and $g$, the derivative of the composite function $f\circ g$ at $a$ satisfies

$D(f\circ g{)}_a=D{f}_{g(a)}\circ D{g}_a$

where the composite on the right-hand side is the composition of linear maps. If the derivatives of $f$ and $g$ are identified with their Jacobian matrices, then the composite on the right-hand side is simply matrix multiplication.

Example: Differentiation with direct dependencies

Suppose that f is a function of two variables, x and y. If these two variables are independent, so that the domain of f is ${ℝ}^2$, then the behavior of f may be understood in terms of its partial derivatives in the x and y directions. However, in some situations, x and y may be dependent. For example, it might happen that f is constrained to a curve $y=y(x)$. In this case, we are actually interested in the behavior of the composite function $f(x,y(x))$. The partial derivative of f with respect to x does not give the true rate of change of f with respect to changing x because changing x necessarily changes y, while the partial derivative assumes y is fixed. However, the chain rule takes such dependencies into account. Write $\gamma (x)=(x,y(x))$. Then, the chain rule says

$D(f\circ \gamma {)}_{x_0}=D{f}_{(x_0,y(x_0))}\circ D{\gamma }_{x_0}.$

By expressing the derivative using Jacobian matrices such as$$\begin{array}{ccc}D{f}_{(x_0,y(x_0))}& =& {\left[\begin{array}{cc}\frac{\partial f}{\partial x}& \frac{\partial f}{\partial y}\end{array}\right]}_{(x_0,y(x_0))}\\ D{\gamma }_{x_0}& =& {\left[\begin{array}{c}\frac{\partial x}{\partial x}\\ \frac{\partial y}{\partial x}\end{array}\right]}_{x_0}.\end{array}$$This becomes:

$\frac{df(x,y(x))}{dx}(x_0)=\frac{\partial f}{\partial x}(x_0,y(x_0))\frac{dx}{dx}(x_0)+\frac{\partial f}{\partial y}(x_0,y(x_0))\frac{dy}{dx}(x_0).$

Suppressing the evaluation at $x_0$ for legibility, we may also write this as

$\frac{df(x,y(x))}{dx}=\frac{\partial f}{\partial x}\frac{dx}{dx}+\frac{\partial f}{\partial y}\frac{dy}{dx}.$

This gives a straightforward formula for the derivative of $f(x,y(x))$ in terms of the partial derivatives of $f$ and the derivative of $y(x)$.

For example, suppose

$f(x,y)=xy.$

The rate of change of f with respect to x is usually the partial derivative of f with respect to x; in this case,

$\frac{\partial f}{\partial x}=y.$

However, if y depends on x, the partial derivative does not give the true rate of change of f as x changes because the partial derivative assumes that y is fixed. Suppose we are constrained to the line

$y=x.$

Then

$f(x,y)=f(x,x)=x^2,$

and the total derivative of f with respect to x is

$\frac{df}{dx}=2x,$

which we see is not equal to the partial derivative $\partial f/\partial x$. Instead of immediately substituting for y in terms of x, however, we can also use the chain rule as above:

$\frac{df}{dx}=\frac{\partial f}{\partial x}+\frac{\partial f}{\partial y}\frac{dy}{dx}=y+x\cdot 1=x+y=2x.$

Example: Differentiation with indirect dependencies

While one can often perform substitutions to eliminate indirect dependencies, the chain rule provides for a more efficient and general technique. Suppose $L(t,x_1,\dots ,x_n)$ is a function of time $t$ and $n$ variables $x_i$ which themselves depend on time. Then, the time derivative of $L$ is

$\frac{dL}{dt}=\frac{d}{dt}L(t,x_1(t),\dots ,x_n(t)).$

The chain rule expresses this derivative in terms of the partial derivatives of $L$ and the time derivatives of the functions $x_i$:

$\frac{dL}{dt}=\frac{\partial L}{\partial t}+\sum _{i=1}^n\frac{\partial L}{\partial x_i}\frac{d{x}_i}{dt}=(\frac{\partial }{\partial t}+\sum _{i=1}^n\frac{d{x}_i}{dt}\frac{\partial }{\partial x_i})(L).$

This expression is often used in physics for a gauge transformation of the Lagrangian, as two Lagrangians that differ only by the total time derivative of a function of time and the $n$ generalized coordinates lead to the same equations of motion. An interesting example concerns the resolution of causality concerning the Wheeler–Feynman time-symmetric theory. The operator in brackets (in the final expression above) is also called the total derivative operator (with respect to $t$).

For example, the total derivative of $f(x(t),y(t))$ is

$\frac{df}{dt}=\frac{\partial f}{\partial x}\frac{dx}{dt}+\frac{\partial f}{\partial y}\frac{dy}{dt}.$

Here there is no $\partial f/\partial t$ term since $f$ itself does not depend on the independent variable $t$ directly.

Total differential equation

A total differential equation is a differential equation expressed in terms of total derivatives. Since the exterior derivative is coordinate-free, in a sense that can be given a technical meaning, such equations are intrinsic and geometric.

Application to equation systems

In economics, it is common for the total derivative to arise in the context of a system of equations.Template:Rp For example, a simple supply-demand system might specify the quantity q of a product demanded as a function D of its price p and consumers' income I, the latter being an exogenous variable, and might specify the quantity supplied by producers as a function S of its price and two exogenous resource cost variables r and w. The resulting system of equations

$q=D(p,I),$

$q=S(p,r,w),$

determines the market equilibrium values of the variables p and q. The total derivative $dp/dr$ of p with respect to r, for example, gives the sign and magnitude of the reaction of the market price to the exogenous variable r. In the indicated system, there are a total of six possible total derivatives, also known in this context as comparative static derivatives: Template:Math, Template:Math, Template:Math, Template:Math, Template:Math, and Template:Math. The total derivatives are found by totally differentiating the system of equations, dividing through by, say Template:Math, treating Template:Math and Template:Math as the unknowns, setting Template:Math, and solving the two totally differentiated equations simultaneously, typically by using Cramer's rule.

See also

• Template:Annotated link
• Template:Annotated link - generalization of the total derivative
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link
• Template:Annotated link

References

• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• Template:Citation.
• A. D. Polyanin and V. F. Zaitsev, Handbook of Exact Solutions for Ordinary Differential Equations (2nd edition), Chapman & Hall/CRC Press, Boca Raton, 2003. Template:Isbn
• From thesaurus.maths.org total derivative

External links

• Template:MathWorld
• Ronald D. Kriz (2007) Envisioning total derivatives of scalar functions of two dimensions using raised surfaces and tangent planes from Virginia Tech