Descent direction

In optimization, a descent direction is a vector $𝐩\in {ℝ}^n$ that points towards a local minimum ${𝐱}^{*}$ of an objective function $f:{ℝ}^n\to ℝ$.

Computing ${𝐱}^{*}$ by an iterative method, such as line search defines a descent direction ${𝐩}_k\in {ℝ}^n$ at the $k$th iterate to be any ${𝐩}_k$ such that $⟨{𝐩}_k,\mathrm{\nabla }f({𝐱}_k)⟩<0$, where $⟨,⟩$ denotes the inner product. The motivation for such an approach is that small steps along ${𝐩}_k$ guarantee that $f$ is reduced, by Taylor's theorem.

Using this definition, the negative of a non-zero gradient is always a descent direction, as $⟨-\mathrm{\nabla }f({𝐱}_k),\mathrm{\nabla }f({𝐱}_k)⟩=-⟨\mathrm{\nabla }f({𝐱}_k),\mathrm{\nabla }f({𝐱}_k)⟩<0$.

Numerous methods exist to compute descent directions, all with differing merits, such as gradient descent or the conjugate gradient method.

More generally, if $P$ is a positive definite matrix, then $p_k=-P\mathrm{\nabla }f(x_k)$ is a descent direction at $x_k$. This generality is used in preconditioned gradient descent methods.

See also

• Directional derivative