Trace (linear algebra)

Template:Short description Template:More citations needed In linear algebra, the trace of a square matrix Template:Math, denoted Template:Math, is the sum of the elements on its main diagonal, $a_{11}+a_{22}+\dots +a_{nn}$. It is only defined for a square matrix (Template:Math).

The trace of a matrix is the sum of its eigenvalues (counted with multiplicities). Also, Template:Math for any matrices Template:Math and Template:Math of the same size. Thus, similar matrices have the same trace. As a consequence, one can define the trace of a linear operator mapping a finite-dimensional vector space into itself, since all matrices describing such an operator with respect to a basis are similar.

The trace is related to the derivative of the determinant (see Jacobi's formula).

Definition

The trace of an Template:Math square matrix Template:Math is defined asTemplate:Rp $$\mathrm{tr}(𝐀)=\sum _{i=1}^n{a}_{ii}=a_{11}+a_{22}+\dots +a_{nn}$$ where Template:Math denotes the entry on the Template:Nobr row and Template:Nobr column of Template:Math. The entries of Template:Math can be real numbers, complex numbers, or more generally elements of a field Template:Mvar. The trace is not defined for non-square matrices.

Example

Let Template:Math be a matrix, with $$𝐀=\left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right)=\left(\begin{array}{ccc}1& 0& 3\\ 11& 5& 2\\ 6& 12& -5\end{array}\right)$$

Then $$\mathrm{tr}(𝐀)=\sum _{i=1}^3{a}_{ii}=a_{11}+a_{22}+a_{33}=1+5+(-5)=1$$

Properties

Basic properties

The trace is a linear mapping. That is, $$\begin{array}{cc}\mathrm{tr}(𝐀+𝐁)& =\mathrm{tr}(𝐀)+\mathrm{tr}(𝐁)\\ \mathrm{tr}(c𝐀)& =c\mathrm{tr}(𝐀)\end{array}$$ for all square matrices Template:Math and Template:Math, and all scalars Template:Mvar.Template:Rp

A matrix and its transpose have the same trace:Template:Rp $$\mathrm{tr}(𝐀)=\mathrm{tr}\left({𝐀}^{𝖳}\right).$$

This follows immediately from the fact that transposing a square matrix does not affect elements along the main diagonal.

Trace of a product

The trace of a square matrix which is the product of two matrices can be rewritten as the sum of entry-wise products of their elements, i.e. as the sum of all elements of their Hadamard product. Phrased directly, if Template:Math and Template:Math are two Template:Math matrices, then: $$\mathrm{tr}\left({𝐀}^{𝖳}𝐁\right)=\mathrm{tr}\left(𝐀{𝐁}^{𝖳}\right)=\mathrm{tr}\left({𝐁}^{𝖳}𝐀\right)=\mathrm{tr}\left(𝐁{𝐀}^{𝖳}\right)=\sum _{i=1}^m\sum _{j=1}^n{a}_{ij}{b}_{ij}.$$

If one views any real Template:Math matrix as a vector of length Template:Mvar (an operation called vectorization) then the above operation on Template:Math and Template:Math coincides with the standard dot product. According to the above expression, Template:Math is a sum of squares and hence is nonnegative, equal to zero if and only if Template:Math is zero.Template:Rp Furthermore, as noted in the above formula, Template:Math. These demonstrate the positive-definiteness and symmetry required of an inner product; it is common to call Template:Math the Frobenius inner product of Template:Math and Template:Math. This is a natural inner product on the vector space of all real matrices of fixed dimensions. The norm derived from this inner product is called the Frobenius norm, and it satisfies a submultiplicative property, as can be proven with the Cauchy–Schwarz inequality: $$0\le {[\mathrm{tr}(𝐀𝐁)]}^2\le \mathrm{tr}\left({𝐀}^{𝖳}𝐀\right)\mathrm{tr}\left({𝐁}^{𝖳}𝐁\right),$$ if Template:Math and Template:Math are real matrices such that Template:Math is a square matrix. The Frobenius inner product and norm arise frequently in matrix calculus and statistics.

The Frobenius inner product may be extended to a hermitian inner product on the complex vector space of all complex matrices of a fixed size, by replacing Template:Math by its complex conjugate.

The symmetry of the Frobenius inner product may be phrased more directly as follows: the matrices in the trace of a product can be switched without changing the result. If Template:Math and Template:Math are Template:Math and Template:Math real or complex matrices, respectively, thenTemplate:Rp

$\mathrm{tr}(𝐀𝐁)=\mathrm{tr}(𝐁𝐀)$

This is notable both for the fact that Template:Math does not usually equal Template:Math, and also since the trace of either does not usually equal Template:Math. The similarity-invariance of the trace, meaning that Template:Math for any square matrix Template:Math and any invertible matrix Template:Math of the same dimensions, is a fundamental consequence. This is proved by $$\mathrm{tr}({𝐏}^{-1}(𝐀𝐏))=\mathrm{tr}((𝐀𝐏){𝐏}^{-1})=\mathrm{tr}(𝐀).$$ Similarity invariance is the crucial property of the trace in order to discuss traces of linear transformations as below.

Additionally, for real column vectors $𝐚\in {ℝ}^n$ and $𝐛\in {ℝ}^n$, the trace of the outer product is equivalent to the inner product:

$\mathrm{tr}\left(𝐛{𝐚}^{\mathsf{\text{T}}}\right)={𝐚}^{\mathsf{\text{T}}}𝐛$

Cyclic property

More generally, the trace is invariant under circular shifts, that is,

$\mathrm{tr}(𝐀𝐁𝐂𝐃)=\mathrm{tr}(𝐁𝐂𝐃𝐀)=\mathrm{tr}(𝐂𝐃𝐀𝐁)=\mathrm{tr}(𝐃𝐀𝐁𝐂).$

This is known as the cyclic property.

Arbitrary permutations are not allowed: in general, $$\mathrm{tr}(𝐀𝐁𝐂)\ne \mathrm{tr}(𝐀𝐂𝐁).$$

However, if products of three symmetric matrices are considered, any permutation is allowed, since: $$\mathrm{tr}(𝐀𝐁𝐂)=\mathrm{tr}\left({\left(𝐀𝐁𝐂\right)}^{𝖳}\right)=\mathrm{tr}(𝐂𝐁𝐀)=\mathrm{tr}(𝐀𝐂𝐁),$$ where the first equality is because the traces of a matrix and its transpose are equal. Note that this is not true in general for more than three factors.

Trace of a Kronecker product

The trace of the Kronecker product of two matrices is the product of their traces: $$\mathrm{tr}(𝐀\otimes 𝐁)=\mathrm{tr}(𝐀)\mathrm{tr}(𝐁).$$

Characterization of the trace

The following three properties: $$\begin{array}{cc}\mathrm{tr}(𝐀+𝐁)& =\mathrm{tr}(𝐀)+\mathrm{tr}(𝐁),\\ \mathrm{tr}(c𝐀)& =c\mathrm{tr}(𝐀),\\ \mathrm{tr}(𝐀𝐁)& =\mathrm{tr}(𝐁𝐀),\end{array}$$ characterize the trace up to a scalar multiple in the following sense: If $f$ is a linear functional on the space of square matrices that satisfies $f(xy)=f(yx),$ then $f$ and $\mathrm{tr}$ are proportional.

For $n\times n$ matrices, imposing the normalization $f(𝐈)=n$ makes $f$ equal to the trace.

Trace as the sum of eigenvalues

Given any Template:Math matrix Template:Math, there is

$\mathrm{tr}(𝐀)=\sum _{i=1}^n{\lambda }_i$

where Template:Math are the eigenvalues of Template:Math counted with multiplicity. This holds true even if Template:Math is a real matrix and some (or all) of the eigenvalues are complex numbers. This may be regarded as a consequence of the existence of the Jordan canonical form, together with the similarity-invariance of the trace discussed above.

Trace of commutator

When both Template:Math and Template:Math are Template:Math matrices, the trace of the (ring-theoretic) commutator of Template:Math and Template:Math vanishes: Template:Math, because Template:Math and Template:Math is linear. One can state this as "the trace is a map of Lie algebras Template:Math from operators to scalars", as the commutator of scalars is trivial (it is an Abelian Lie algebra). In particular, using similarity invariance, it follows that the identity matrix is never similar to the commutator of any pair of matrices.

Conversely, any square matrix with zero trace is a linear combination of the commutators of pairs of matrices. Moreover, any square matrix with zero trace is unitarily equivalent to a square matrix with diagonal consisting of all zeros.

Relationship to the characteristic polynomial

The trace of an $n\times n$ matrix $A$ is the coefficient of $t^{n-1}$ in the characteristic polynomial, possibly changed of sign, according to the convention in the definition of the characteristic polynomial.

Relationship to eigenvalues

If Template:Math is a linear operator represented by a square matrix with real or complex entries and if Template:Math are the eigenvalues of Template:Math (listed according to their algebraic multiplicities), then

$\mathrm{tr}(𝐀)=\sum _i{\lambda }_i$

This follows from the fact that Template:Math is always similar to its Jordan form, an upper triangular matrix having Template:Math on the main diagonal. In contrast, the determinant of Template:Math is the product of its eigenvalues; that is, $$\det (𝐀)=\prod _i{\lambda }_i.$$

Everything in the present section applies as well to any square matrix with coefficients in an algebraically closed field.

Derivative relationships

If Template:Math is a square matrix with small entries and Template:Math denotes the identity matrix, then we have approximately

$$\det (𝐈+\mathrm{\Delta }𝐀)\approx 1+\mathrm{tr}(\mathrm{\Delta }𝐀).$$

Precisely this means that the trace is the derivative of the determinant function at the identity matrix. Jacobi's formula

$$d\det (𝐀)=\mathrm{tr}(\mathrm{adj}(𝐀)\cdot d𝐀)$$

is more general and describes the differential of the determinant at an arbitrary square matrix, in terms of the trace and the adjugate of the matrix.

From this (or from the connection between the trace and the eigenvalues), one can derive a relation between the trace function, the matrix exponential function, and the determinant:$$\det (\exp (𝐀))=\exp (\mathrm{tr}(𝐀)).$$

A related characterization of the trace applies to linear vector fields. Given a matrix Template:Math, define a vector field Template:Math on Template:Math by Template:Math. The components of this vector field are linear functions (given by the rows of Template:Math). Its divergence Template:Math is a constant function, whose value is equal to Template:Math.

By the divergence theorem, one can interpret this in terms of flows: if Template:Math represents the velocity of a fluid at location Template:Math and Template:Mvar is a region in Template:Math, the net flow of the fluid out of Template:Mvar is given by Template:Math, where Template:Math is the volume of Template:Mvar.

The trace is a linear operator, hence it commutes with the derivative: $$d\mathrm{tr}(𝐗)=\mathrm{tr}(d𝐗).$$

Trace of a linear operator

In general, given some linear map Template:Math (where Template:Mvar is a finite-dimensional vector space), we can define the trace of this map by considering the trace of a matrix representation of Template:Mvar, that is, choosing a basis for Template:Mvar and describing Template:Mvar as a matrix relative to this basis, and taking the trace of this square matrix. The result will not depend on the basis chosen, since different bases will give rise to similar matrices, allowing for the possibility of a basis-independent definition for the trace of a linear map.

Such a definition can be given using the canonical isomorphism between the space Template:Math of linear maps on Template:Mvar and Template:Math, where Template:Math is the dual space of Template:Mvar. Let Template:Mvar be in Template:Mvar and let Template:Mvar be in Template:Mvar. Then the trace of the indecomposable element Template:Math is defined to be Template:Math; the trace of a general element is defined by linearity. The trace of a linear map Template:Math can then be defined as the trace, in the above sense, of the element of Template:Math corresponding to f under the above mentioned canonical isomorphism. Using an explicit basis for Template:Mvar and the corresponding dual basis for Template:Math, one can show that this gives the same definition of the trace as given above.

Numerical algorithms

Stochastic estimator

The trace can be estimated unbiasedly by "Hutchinson's trick":

Given any matrix

$𝑾\in {ℝ}^{n\times n}$

, and any random

$𝒖\in {ℝ}^n$

with

$𝔼[𝒖{𝒖}^{⊺}]=𝐈$

, we have

$𝔼[{𝒖}^{⊺}𝑾𝒖]=\mathrm{tr}𝑾$

.

For a proof expand the expectation directly.

Usually, the random vector is sampled from $\mathrm{N}(\mathrm{𝟎},𝐈)$ (normal distribution) or $\{\pm n^{-1/2}{\}}^n$ (Rademacher distribution).

More sophisticated stochastic estimators of trace have been developed.

Applications

If a 2 x 2 real matrix has zero trace, its square is a diagonal matrix.

The trace of a 2 × 2 complex matrix is used to classify Möbius transformations. First, the matrix is normalized to make its determinant equal to one. Then, if the square of the trace is 4, the corresponding transformation is parabolic. If the square is in the interval Template:Nowrap, it is elliptic. Finally, if the square is greater than 4, the transformation is loxodromic. See classification of Möbius transformations.

The trace is used to define characters of group representations. Two representations Template:Math of a group Template:Mvar are equivalent (up to change of basis on Template:Mvar) if Template:Math for all Template:Math.

The trace also plays a central role in the distribution of quadratic forms.

Lie algebra

The trace is a map of Lie algebras $\mathrm{tr}:{𝔤𝔩}_n\to K$ from the Lie algebra ${𝔤𝔩}_n$ of linear operators on an Template:Mvar-dimensional space (Template:Math matrices with entries in $K$) to the Lie algebra Template:Mvar of scalars; as Template:Mvar is Abelian (the Lie bracket vanishes), the fact that this is a map of Lie algebras is exactly the statement that the trace of a bracket vanishes: $$\mathrm{tr}([𝐀,𝐁])=0\text{ for each }𝐀,𝐁\in {𝔤𝔩}_n.$$

The kernel of this map, a matrix whose trace is zero, is often said to be Template:Visible anchor or Template:Visible anchor, and these matrices form the simple Lie algebra ${𝔰𝔩}_n$, which is the Lie algebra of the special linear group of matrices with determinant 1. The special linear group consists of the matrices which do not change volume, while the special linear Lie algebra is the matrices which do not alter volume of infinitesimal sets.

In fact, there is an internal direct sum decomposition ${𝔤𝔩}_n={𝔰𝔩}_n\oplus K$ of operators/matrices into traceless operators/matrices and scalars operators/matrices. The projection map onto scalar operators can be expressed in terms of the trace, concretely as: $$𝐀\mapsto \frac{1}{n}\mathrm{tr}(𝐀)𝐈.$$

Formally, one can compose the trace (the counit map) with the unit map $K\to {𝔤𝔩}_n$ of "inclusion of scalars" to obtain a map ${𝔤𝔩}_n\to {𝔤𝔩}_n$ mapping onto scalars, and multiplying by Template:Mvar. Dividing by Template:Mvar makes this a projection, yielding the formula above.

In terms of short exact sequences, one has $$0\to {𝔰𝔩}_n\to {𝔤𝔩}_n\stackrel{\mathrm{tr}}{\to }K\to 0$$ which is analogous to $$1\to {\mathrm{SL}}_n\to {\mathrm{GL}}_n\stackrel{\det }{\to }{K}^{*}\to 1$$ (where $K^{*}=K\setminus \{0\}$) for Lie groups. However, the trace splits naturally (via $1/n$ times scalars) so ${𝔤𝔩}_n={𝔰𝔩}_n\oplus K$, but the splitting of the determinant would be as the Template:Mvarth root times scalars, and this does not in general define a function, so the determinant does not split and the general linear group does not decompose: $${\mathrm{GL}}_n\ne {\mathrm{SL}}_n\times K^{*}.$$

Bilinear forms

The bilinear form (where Template:Math, Template:Math are square matrices) $$B(𝐗,𝐘)=\mathrm{tr}(\mathrm{ad}(𝐗)\mathrm{ad}(𝐘))\text{where }\mathrm{ad}(𝐗)𝐘=[𝐗,𝐘]=𝐗𝐘-𝐘𝐗$$ is called the Killing form, which is used for the classification of Lie algebras.

The trace defines a bilinear form: $$(𝐗,𝐘)\mapsto \mathrm{tr}(𝐗𝐘).$$

The form is symmetric, non-degenerate and associative in the sense that: $$\mathrm{tr}(𝐗[𝐘,𝐙])=\mathrm{tr}([𝐗,𝐘]𝐙).$$

For a complex simple Lie algebra (such as Template:Math), every such bilinear form is proportional to each other; in particular, to the Killing formTemplate:Citation needed.

Two matrices Template:Math and Template:Math are said to be trace orthogonal if $$\mathrm{tr}(𝐗𝐘)=0.$$

There is a generalization to a general representation $(\rho ,𝔤,V)$ of a Lie algebra $𝔤$, such that $\rho$ is a homomorphism of Lie algebras $\rho :𝔤\to \text{End}(V).$ The trace form ${\text{tr}}_V$ on $\text{End}(V)$ is defined as above. The bilinear form $$\varphi (𝐗,𝐘)={\text{tr}}_V(\rho (𝐗)\rho (𝐘))$$ is symmetric and invariant due to cyclicity.

Generalizations

The concept of trace of a matrix is generalized to the trace class of compact operators on Hilbert spaces, and the analog of the Frobenius norm is called the Hilbert–Schmidt norm.

If Template:Mvar is a trace-class operator, then for any orthonormal basis $(e_n{)}_n$, the trace is given by $$\mathrm{tr}(K)=\sum _n⟨e_n,K{e}_n⟩,$$ and is finite and independent of the orthonormal basis.

The partial trace is another generalization of the trace that is operator-valued. The trace of a linear operator Template:Mvar which lives on a product space Template:Math is equal to the partial traces over Template:Mvar and Template:Mvar: $$\mathrm{tr}(Z)={\mathrm{tr}}_A({\mathrm{tr}}_B(Z))={\mathrm{tr}}_B({\mathrm{tr}}_A(Z)).$$

For more properties and a generalization of the partial trace, see traced monoidal categories.

If Template:Mvar is a general associative algebra over a field Template:Mvar, then a trace on Template:Mvar is often defined to be any map Template:Math which vanishes on commutators; Template:Math for all Template:Math. Such a trace is not uniquely defined; it can always at least be modified by multiplication by a nonzero scalar.

A supertrace is the generalization of a trace to the setting of superalgebras.

The operation of tensor contraction generalizes the trace to arbitrary tensors.

Gomme and Klein (2011) define a matrix trace operator $\mathrm{trm}$ that operates on block matrices and use it to compute second-order perturbation solutions to dynamic economic models without the need for tensor notation.

Traces in the language of tensor products

Given a vector space Template:Mvar, there is a natural bilinear map Template:Math given by sending Template:Math to the scalar Template:Math. The universal property of the tensor product Template:Math automatically implies that this bilinear map is induced by a linear functional on Template:Math.

Similarly, there is a natural bilinear map Template:Math given by sending Template:Math to the linear map Template:Math. The universal property of the tensor product, just as used previously, says that this bilinear map is induced by a linear map Template:Math. If Template:Mvar is finite-dimensional, then this linear map is a linear isomorphism. This fundamental fact is a straightforward consequence of the existence of a (finite) basis of Template:Mvar, and can also be phrased as saying that any linear map Template:Math can be written as the sum of (finitely many) rank-one linear maps. Composing the inverse of the isomorphism with the linear functional obtained above results in a linear functional on Template:Math. This linear functional is exactly the same as the trace.

Using the definition of trace as the sum of diagonal elements, the matrix formula Template:Math is straightforward to prove, and was given above. In the present perspective, one is considering linear maps Template:Mvar and Template:Mvar, and viewing them as sums of rank-one maps, so that there are linear functionals Template:Math and Template:Math and nonzero vectors Template:Math and Template:Math such that Template:Math and Template:Math for any Template:Mvar in Template:Mvar. Then

$(S\circ T)(u)=\sum _i{\phi }_i(\sum _j{\psi }_j(u)w_j)v_i=\sum _i\sum _j{\psi }_j(u){\phi }_i(w_j)v_i$

for any Template:Mvar in Template:Mvar. The rank-one linear map Template:Math has trace Template:Math and so

$\mathrm{tr}(S\circ T)=\sum _i\sum _j{\psi }_j(v_i){\phi }_i(w_j)=\sum _j\sum _i{\phi }_i(w_j){\psi }_j(v_i).$

Following the same procedure with Template:Mvar and Template:Mvar reversed, one finds exactly the same formula, proving that Template:Math equals Template:Math.

The above proof can be regarded as being based upon tensor products, given that the fundamental identity of Template:Math with Template:Math is equivalent to the expressibility of any linear map as the sum of rank-one linear maps. As such, the proof may be written in the notation of tensor products. Then one may consider the multilinear map Template:Math given by sending Template:Math to Template:Math. Further composition with the trace map then results in Template:Math, and this is unchanged if one were to have started with Template:Math instead. One may also consider the bilinear map Template:Math given by sending Template:Math to the composition Template:Math, which is then induced by a linear map Template:Math. It can be seen that this coincides with the linear map Template:Math. The established symmetry upon composition with the trace map then establishes the equality of the two traces.

For any finite dimensional vector space Template:Mvar, there is a natural linear map Template:Math; in the language of linear maps, it assigns to a scalar Template:Mvar the linear map Template:Math. Sometimes this is called coevaluation map, and the trace Template:Math is called evaluation map. These structures can be axiomatized to define categorical traces in the abstract setting of category theory.

See also

• Trace of a tensor with respect to a metric tensor
• Characteristic function
• Field trace
• Golden–Thompson inequality
• Singular trace
• Specht's theorem
• Trace class
• Trace identity
• Trace inequalities
• von Neumann's trace inequality

References

• Template:Cite book

• Template:Cite book

• Template:Cite book

External links

• Template:Springer