Singular value

Template:Short description In mathematics, in particular in functional analysis, the singular values of a compact operator $T:X\to Y$ acting between Hilbert spaces $X$ and Template:Tmath, are the square roots of the (necessarily non-negative) eigenvalues of the self-adjoint operator $T^{\ast }T$ (where $T^{\ast }$ denotes the adjoint of Template:Tmath).

The singular values are non-negative real numbers, usually listed in decreasing order Template:Tmath. The largest singular value ${\sigma }_1(T)$ is equal to the operator norm of $T$ (see Min-max theorem).

Visualization of a singular value decomposition (SVD) of a Template:Math-dimensional, real shearing matrix Template:Mvar. First, we see the unit disc in blue together with the two canonical unit vectors. We then see the action of Template:Mvar, which distorts the disc to an ellipse. The SVD decomposes Template:Mvar into three simple transformations: a rotation Template:Math, a scaling Template:Mvar along the rotated coordinate axes, and a second rotation Template:Mvar. Moreover, Template:Mvar is a (square, in this example) diagonal matrix containing in its diagonal the singular values of Template:Mvar, which represent the lengths Template:Math and Template:Math of the semi-axes of the ellipse.

If $T$ acts on a Euclidean space (for example, Template:Tmath), there is a simple geometric interpretation for the singular values: Consider the image by $T$ of the unit sphere; this is an ellipsoid, and the lengths of its semi-axes are the singular values of $T$ (the figure provides an example in Template:Tmath).

If Template:Tmath is a normal matrix, then its singular values are the absolute values of its eigenvalues. Indeed, in this case, the spectral theorem can be applied to obtain unitary diagonalization of $A$ as Template:Tmath. Then, by definition, Template:Tmath.

Most norms on Hilbert space operators studied are defined using singular values. For example, the Ky Fan Template:Tmath-norm is the sum of the first $k$ singular values, the trace norm is the sum of all singular values, and the Schatten norm is the Template:Tmath-th root of the sum of the Template:Tmath-th powers of the singular values. Note that each norm is defined only on a special class of operators, hence singular values can be useful in classifying different operators.

In the finite-dimensional case, a matrix can always be decomposed in the form Template:Tmath, where $U$ and $V^{\ast }$ are unitary matrices and $𝛴$ is a rectangular diagonal matrix with the singular values lying on its diagonal. This is the singular value decomposition.

Basic properties

For $A\in {ℂ}^{m\times n}$ and Template:Tmath:

Min-max theorem for singular values:
Here, $U$ is a subspace of Template:Tmath;

${\sigma }_i(A)=\underset{\dim (U)=n-i+1}{min}\underset{\underset{\Vert x{\Vert }_2=1}{x\in U}}{max}{\Vert Ax\Vert }_2.$

${\sigma }_i(A)=\underset{\dim (U)=i}{max}\underset{\underset{\Vert x{\Vert }_2=1}{x\in U}}{min}{\Vert Ax\Vert }_2.$

Matrix transpose and conjugate do not alter singular values:

Template:Tmath.

For any unitary matrices $U\in {ℂ}^{m\times m}$ and Template:Tmath,

Template:Tmath.

Relation to eigenvalues:

Template:Tmath.

Relation to trace:

Template:Tmath.

If $A^{\ast }A$ has full rank, the product of singular values is Template:Tmath.

If $A{A}^{\ast }$ has full rank, the product of singular values is Template:Tmath.

If $A$ is square and has full rank, the product of singular values is Template:Tmath.

If $A$ is normal, then Template:Tmath, that is, its singular values are the absolute values of its eigenvalues.

For a generic rectangular Template:Tmath matrix Template:Tmath, let $\tilde{A}=\left[\begin{array}{cc}0& A\\ A^{\ast }& 0\end{array}\right]$ be its augmented Template:Tmath matrix. Template:Tmath eigenvalues of Template:Tmath are $\pm {\sigma }_i(A)$ (where the ${\sigma }_i(A)$ are the singular values of Template:Tmath), and the $|m-n|$ remaining eigenvalues of Template:Tmath are zero. Let $A=U𝛴V^{\ast }$ be the singular value decomposition of Template:Tmath; then the eigenvectors of $\tilde{A}$ for Template:Tmath are Template:Tmath.Template:Pg

The smallest singular value

The smallest singular value of a matrix Template:Tmath is Template:Tmath. For a non-singular matrix Template:Tmath, it has the following properties:

• ${\Vert A^{-1}\Vert }_2={\sigma }_{\mathrm{n}}^{-1}(A)$.Template:Rp
• For all indices Template:Tmath, Template:Tmath Template:Tmath.Template:Rp

Intuitively, if Template:Tmath is small, then the rows of Template:Tmath are "almost" linearly dependent. If it is Template:Tmath, then the rows of Template:Tmath are linearly dependent and Template:Tmath is not invertible.

Inequalities about singular values

See also:

Singular values of sub-matrices

For Template:Tmath,

1. Let $B$ denote $A$ with one of its rows or columns deleted. Then $${\sigma }_{i+1}(A)\le {\sigma }_i(B)\le {\sigma }_i(A)$$
2. Let $B$ denote $A$ with two of its rows and columns deleted. Then $${\sigma }_{i+2}(A)\le {\sigma }_i(B)\le {\sigma }_i(A)$$
3. Let $B$ denote an $(m-k)\times (n-\ell )$ submatrix of $A$. Then $${\sigma }_{i+k+\ell }(A)\le {\sigma }_i(B)\le {\sigma }_i(A)$$

Singular values of Template:Math

For Template:Tmath,

1. $$\sum _{i=1}^k{\sigma }_i(A+B)\le \sum _{i=1}^k({\sigma }_i(A)+{\sigma }_i(B)),\text{where}k=min\{m,n\}.$$
2. $${\sigma }_{i+j-1}(A+B)\le {\sigma }_i(A)+{\sigma }_j(B),\mathrm{\forall }i,j\in {\mathbb{N}}^{\ast },i+j-1\le min\{m,n\}.$$

Singular values of Template:Math

For Template:Tmath,

1. $$\begin{array}{cc}\prod _{i=n}^{i=n-k+1}{\sigma }_i(A){\sigma }_i(B)& \le \prod _{i=n}^{i=n-k+1}{\sigma }_i(AB).\\ \prod _{i=1}^k{\sigma }_i(AB)& \le \prod _{i=1}^k{\sigma }_i(A){\sigma }_i(B).\\ \sum _{i=1}^k{\sigma }_i^p(AB)& \le \sum _{i=1}^k{\sigma }_i^p(A){\sigma }_i^p(B).\end{array}$$
2. $${\sigma }_n(A){\sigma }_i(B)\le {\sigma }_i(AB)\le {\sigma }_1(A){\sigma }_i(B),\mathrm{\forall }i=1,2,\dots ,n.$$

For Template:Tmath, $$2{\sigma }_i(A{B}^{\ast })\le {\sigma }_i(A^{\ast }A+B^{\ast }B),\mathrm{\forall }i=1,2,\dots ,n.$$

Singular values and eigenvalues

For Template:Tmath,

1. See:. $${\lambda }_i(A+A^{\ast })\le 2{\sigma }_i(A),\mathrm{\forall }i=1,2,\dots ,n.$$
2. Assume $|{\lambda }_1(A)|\ge \cdots \ge |{\lambda }_n(A)|$. Then for $k=1,2,\dots ,n$,
   1. Weyl's theorem: $$\prod _{i=1}^k|{\lambda }_i(A)|\le \prod _{i=1}^k{\sigma }_i(A).$$
   2. For $p>0$, $$\sum _{i=1}^k|{\lambda }_i^p(A)|\le \sum _{i=1}^k{\sigma }_i^p(A).$$

History

This concept was introduced by Erhard Schmidt in 1907. Schmidt called singular values "eigenvalues" at that time. The name "singular value" was first quoted by Smithies in 1937. In 1957, Allahverdiev proved the following characterization of the Template:Tmath-th singular number:

${\sigma }_n(T)=inf\{\Vert T-L\Vert :L\text{ is an operator of finite rank }<n\}.$

This formulation made it possible to extend the notion of singular values to operators in Banach spaces.

Note that there is a more general concept of s-numbers, which also includes Gelfand and Kolmogorov width.

See also

• Condition number
• Cauchy interlacing theorem or Poincaré separation theorem
• Schur–Horn theorem
• Singular value decomposition