Invertible matrix

In linear algebra, an invertible matrix (non-singular, non-degenerate or regular) is a square matrix that has an inverse. In other words, if a matrix is invertible, it can be multiplied by its inverse matrix to yield the identity matrix. Invertible matrices are the same size as their inverse.

The inverse of a matrix represents the inverse operation, meaning if a matrix is applied to a particular vector, followed by applying the matrix's inverse, the result is the original vector.

Definition

An Template:Mvar-by-Template:Mvar square matrix Template:Math is called invertible if there exists an Template:Mvar-by-Template:Mvar square matrix Template:Math such that$$𝐀𝐁=𝐁𝐀={𝐈}_n,$$where Template:Math denotes the Template:Mvar-by-Template:Mvar identity matrix and the multiplication used is ordinary matrix multiplication. If this is the case, then the matrix Template:Math is uniquely determined by Template:Math, and is called the inverse of Template:Math, denoted by Template:Math. Matrix inversion is the process of finding the matrix which when multiplied by the original matrix gives the identity matrix.

Basic idea

A matrix can be viewed as a rule for transforming vectors. For example, a real $n\times n$ matrix $A$ defines a linear transformation

$x\mapsto Ax$

from the set ${ℝ}^n$ of $n$-tuples of real numbers to itself. The matrix is invertible when this transformation can be undone by another linear transformation. In that case, there is a matrix $A^{-1}$ such that applying $A$ and then $A^{-1}$, or applying $A^{-1}$ and then $A$, returns every vector to where it started.

Geometrically, an invertible matrix does not collapse space into a lower-dimensional set. It sends distinct vectors to distinct vectors and reaches every vector in the target space. For a real matrix, this is reflected by its determinant: an invertible matrix has nonzero determinant, while a matrix with determinant zero collapses volume to zero and is not invertible.

Algebraically, invertibility means that the linear system

$Ax=b$

has a unique solution $x$ for every vector $b$. Equivalently, the columns of $A$ form a basis of the vector space. These and several other equivalent characterizations are summarized by the invertible matrix theorem.

Examples

An invertible matrix

Consider the following 2-by-2 matrix:

$𝐀=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)$

The matrix $𝐀$ is invertible, as it has inverse $𝐁=\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right),$ which can be confirmed by computing

$𝐀𝐁=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right)=\left(\begin{array}{cc}(-1)\times 2+\frac{3}{2}\times 2& (-1)\times 3+\frac{3}{2}\times 2\\ 1\times 2+(-1)\times 2& 1\times 3+(-1)\times 2\end{array}\right)=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)={𝐈}_2$

To check that it is invertible without finding an inverse, $det𝐀=-\frac{1}{2}$ can be computed, which is non-zero.

A non-invertible matrix

This is a non-invertible (singular) matrix:

$𝐂=\left(\begin{array}{cc}2& 4\\ 2& 4\end{array}\right)$

This matrix is not invertible, because different vectors $x$ and $x^{\prime }$ have the same value of $𝐂x$, for example, $x=\left[\begin{array}{c}2\\ -1\end{array}\right]$ and $x^{\prime }=\left[\begin{array}{c}0\\ 0\end{array}\right]$ both give $𝐂x=\left[\begin{array}{c}0\\ 0\end{array}\right]$. So it is impossible to reverse the transformation $x\mapsto 𝐂x$, since different inputs produce the same output. The determinant of $𝐂$ is 0, which is a necessary and sufficient condition for a matrix to be non-invertible.

Explicit formula for 2x2 matrices

For an arbitrary 2-by-2 matrix $𝐀$ defined as

$𝐀=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$,

its inverse is

${𝐀}^{-1}=\frac{1}{det𝐀}\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)$.

Criteria for invertibility

A matrix $𝐀$ (over a field such as the real numbers) is invertible if and only if the only solution to the equation $𝐀x=0$ is the zero vector. That is, the nullity of $𝐀$ is zero: its nullspace consists only of the zero vector. Equivalently, an $n\times n$ matrix is invertible if its rank is $n$, by the rank-nullity theorem. Such a matrix is said to be of full rank. Geometrically, this means that the column space of $𝐀$ is all of ${ℝ}^n$. The rank and nullity characterizations are related in the following way: having full rank means that the matrix maps onto all of ${ℝ}^n$ (surjectively), and having nullity zero means that the matrix is one-to-one. That is, it is a bijection, and therefore inveritble.

A more computational criterion for invertibility comes from the standard way of determining the rank and nullity of a matrix. For a square matrix, a matrix is invertible if and only if its row-reduced echelon form is the identity matrix. The reason is that this is the only row-reduced echelon matrix having full rank: all ones down the diagonal and zeros everywhere else.

Another criterion is that a matrix is invertible if and only if its determinant is non-zero.

Methods of matrix inversion

Template:Main There are many methods to compute the inverse matrix, when it exists. A basic method is to use Gaussian elimination, for example. This proceeds by considering the inverse matrix $A^{-1}$ as a product of elementary matrices, $A^{-1}=E_k{E}_{k-1}\dots E_1$ where the $E_i$ are the elementary matrices corresponding to the elementary row operations needed to put $A$ into is row-reduced echelon form. This method has the advantage that it will produce a row-reduced echelon form regardless of invertibility, and so gives a criterion to decide if a matrix is invertible which is often more efficient than computing the determinant: the matrix is invertible if and only if, at the end of the process, the echelon form is the identity matrix rather than some matrix of lower rank.

To convert this to a practical method for determining using this method, one uses an augmented matrix whose left side is the matrix to invert and whose right side is the identity matrix. Then, Gaussian elimination is used to convert the left side into the identity matrix, which causes the right side to become the inverse of the input matrix.

For example, take the following matrix: $$𝐀=\left(\begin{array}{cc}-1& \frac{3}{2}\\ 1& -1\end{array}\right)$$

The first step to compute its inverse is to create the augmented matrix $$\left(\begin{array}{cccc}-1& \frac{3}{2}& 1& 0\\ 1& -1& 0& 1\end{array}\right)$$

Call the first row of this matrix $R_1$ and the second row $R_2$. Then, add row 1 to row 2 $(R_1+R_2\to R_2).$ This yields $$\left(\begin{array}{cccc}-1& \frac{3}{2}& 1& 0\\ 0& \frac{1}{2}& 1& 1\end{array}\right)$$

Next, subtract row 2, multiplied by 3, from row 1 $(R_1-3R_2\to R_1),$ which yields $$\left(\begin{array}{cccc}-1& 0& -2& -3\\ 0& \frac{1}{2}& 1& 1\end{array}\right)$$

Finally, multiply row 1 by −1 $(-R_1\to R_1)$ and row 2 by 2 $(2R_2\to R_2).$ This yields the identity matrix on the left side and the inverse matrix on the right:$$\left(\begin{array}{cccc}1& 0& 2& 3\\ 0& 1& 2& 2\end{array}\right)$$

Thus, $${𝐀}^{-1}=\left(\begin{array}{cc}2& 3\\ 2& 2\end{array}\right)$$ It works because the process of Gaussian elimination can be viewed as a sequence of applying left matrix multiplication using elementary row operations using elementary matrices (${𝐄}_n$), such as ${𝐄}_n{𝐄}_{n-1}\cdots {𝐄}_2{𝐄}_1𝐀=𝐈$

Applying right-multiplication using ${𝐀}^{-1},$ we get ${𝐄}_n{𝐄}_{n-1}\cdots {𝐄}_2{𝐄}_1𝐈=𝐈{𝐀}^{-1}.$ And the right side $𝐈{𝐀}^{-1}={𝐀}^{-1},$ which is the inverse we want.

Properties

Singularity

Over a field, a square matrix that is not invertible is called singular or degenerate. A square matrix with entries in a field is singular if and only if its determinant is zero.

Invertible matrix theorem

Let Template:Math be a square Template:Mvar-by-Template:Mvar matrix over a field Template:Mvar (e.g., the field Template:Tmath of real numbers). The following statements are equivalent, i.e., they are either all true or all false for any given matrix:

• Template:Math is invertible, i.e. it has an inverse under matrix multiplication, i.e., there exists a Template:Math such that Template:Math. (In that statement, "invertible" can equivalently be replaced with "left-invertible" or "right-invertible" in which one-sided inverses are considered.)
• The linear transformation mapping Template:Math to Template:Math is invertible, i.e., it has an inverse under function composition. (There, again, "invertible" can equivalently be replaced with either "left-invertible" or "right-invertible".)
• The transpose Template:Math is an invertible matrix.
• Template:Math is row-equivalent to the Template:Mvar-by-Template:Mvar identity matrix Template:Math.
• Template:Math is column-equivalent to the Template:Mvar-by-Template:Mvar identity matrix Template:Math.
• Template:Math has Template:Mvar pivot positions.
• Template:Math has full rank: Template:Math.
• Template:Math has a trivial kernel: Template:Math
• The linear transformation mapping Template:Math to Template:Math is bijective; that is, the equation Template:Math has exactly one solution for each Template:Math in Template:Mvar. (There, "bijective" can equivalently be replaced with "injective" or "surjective".)
• The columns of Template:Math form a basis of Template:Mvar. (In this statement, "basis" can equivalently be replaced with either "linearly independent set" or "spanning set")
• The rows of Template:Math form a basis of Template:Mvar. (Similarly, here, "basis" can equivalently be replaced with either "linearly independent set" or "spanning set")
• The determinant of Template:Math is nonzero: Template:Math. In general, a square matrix over a commutative ring is invertible if and only if its determinant is a unit (i.e. multiplicatively invertible element) of that ring.
• The number 0 is not an eigenvalue of Template:Math. (More generally, a number $\lambda$ is an eigenvalue of Template:Math if the matrix $𝐀-\lambda 𝐈$ is singular, where Template:Math is the identity matrix.)
• The matrix Template:Math can be expressed as a finite product of elementary matrices.

Other properties

Furthermore, the following properties hold for an invertible matrix Template:Math:

• $({𝐀}^{-1}{)}^{-1}=𝐀$
• $(k𝐀{)}^{-1}=k^{-1}{𝐀}^{-1}$ for nonzero scalar Template:Mvar
• $(𝐀𝐱{)}^{+}={𝐱}^{+}{𝐀}^{-1}$ if Template:Math has orthonormal columns, where Template:Math denotes the Moore–Penrose inverse and Template:Math is a vector
• $({𝐀}^{\mathrm{T}}{)}^{-1}=({𝐀}^{-1}{)}^{\mathrm{T}}$
• For any invertible Template:Mvar-by-Template:Mvar matrices Template:Math and Template:Math, $(𝐀𝐁{)}^{-1}={𝐁}^{-1}{𝐀}^{-1}.$ More generally, if ${𝐀}_1,\dots ,{𝐀}_k$ are invertible Template:Mvar-by-Template:Mvar matrices, then $({𝐀}_1{𝐀}_2\cdots {𝐀}_{k-1}{𝐀}_k{)}^{-1}={𝐀}_k^{-1}{𝐀}_{k-1}^{-1}\cdots {𝐀}_2^{-1}{𝐀}_1^{-1}.$
• $det{𝐀}^{-1}=(det𝐀{)}^{-1}.$
• Left and right inverses are equal. That is, if $𝐋𝐀=𝐈$ and $𝐀𝐑=𝐈$ then $𝐋=𝐋(𝐀𝐑)=(𝐋𝐀)𝐑=𝐑$.

The rows of the inverse matrix Template:Math of a matrix Template:Math are orthonormal to the columns of Template:Math (and vice versa interchanging rows for columns). To see this, suppose that Template:Math where the rows of Template:Math are denoted as $v_i^{\mathrm{T}}$ and the columns of Template:Math as $u_j$ for $1\le i,j\le n.$ Then clearly, the Euclidean inner product of any two $v_i^{\mathrm{T}}{u}_j={\delta }_{i,j}.$ This property can also be useful in constructing the inverse of a square matrix in some instances, where a set of orthogonal vectors (but not necessarily orthonormal vectors) to the columns of Template:Math are known. In which case, one can apply the iterative Gram–Schmidt process to this initial set to determine the rows of the inverse Template:Math.

A matrix that is its own inverse (i.e., a matrix Template:Math such that Template:Math and consequently Template:Math) is called an involutory matrix.

In relation to its adjugate

The adjugate of a matrix Template:Math is a matrix $\mathrm{adj}(A)$ which exists independently of the invertibility of Template:Math. It satisfies the identity $$\mathrm{adj}(𝐀)𝐀=𝐀\mathrm{adj}(𝐀)=det(𝐀)I.$$ In particular, if Template:Math is invertible, then

${𝐀}^{-1}=\frac{1}{det(𝐀)}\mathrm{adj}(𝐀)$

In relation to the identity matrix

It follows from the associativity of matrix multiplication that if

$𝐀𝐁=𝐈$

for finite square matrices Template:Math and Template:Math, then also

$𝐁𝐀=𝐈$

This identity does not hold for non-square rectangular matrices, and need not be true for linear operators in infinite dimensions.

Density

Over the field of real numbers, the set of singular Template:Mvar-by-Template:Mvar matrices, considered as a subset of Template:Tmath is a null set, that is, has Lebesgue measure zero. That is true because singular matrices are the roots of the determinant function. It is a continuous function because it is a polynomial in the entries of the matrix. Thus in the language of measure theory, almost all Template:Mvar-by-Template:Mvar matrices are invertible.

Furthermore, the set of Template:Mvar-by-Template:Mvar invertible matrices is open and dense in the topological space of all Template:Mvar-by-Template:Mvar matrices. Equivalently, the set of singular matrices is closed and nowhere dense in the space of Template:Mvar-by-Template:Mvar matrices.

In practice, however, non-invertible matrices may be encountered. In numerical calculations, matrices that are invertible but close to a non-invertible matrix may still be problematic and are said to be ill-conditioned.

Derivative of the matrix inverse

Suppose that the invertible matrix A depends on a parameter t. Then the derivative of the inverse of A with respect to t is given by

$\frac{\mathrm{d}}{\mathrm{d}t}{𝐀}^{-1}=-{𝐀}^{-1}\frac{\mathrm{d}𝐀}{\mathrm{d}t}{𝐀}^{-1}$

To derive the above expression for the derivative of the inverse of A, one can differentiate the definition of the matrix inverse ${𝐀}^{-1}𝐀=𝐈$ using the product rule, and then solve for the derivative of the inverse of A:

$\mathrm{𝟎}=\frac{\mathrm{d}𝐈}{\mathrm{d}t}=\frac{\mathrm{d}({𝐀}^{-1}𝐀)}{\mathrm{d}t}=\frac{\mathrm{d}({𝐀}^{-1})}{\mathrm{d}t}𝐀+{𝐀}^{-1}\frac{\mathrm{d}𝐀}{\mathrm{d}t}$

Subtracting ${𝐀}^{-1}\frac{\mathrm{d}𝐀}{\mathrm{d}t}$ from both ends of this formula, and multiplying on the right by ${𝐀}^{-1}$ finishes the derivation.

If $\epsilon$ is a small number then the derivative formula gives:

${(𝐀+\epsilon 𝐗)}^{-1}={𝐀}^{-1}-\epsilon {𝐀}^{-1}𝐗{𝐀}^{-1}+𝒪({\epsilon }^2)$

Given a positive integer $n$,

$\begin{array}{cc}\frac{\mathrm{d}}{\mathrm{d}t}{𝐀}^n& =\sum _{i=1}^n{𝐀}^{i-1}\frac{\mathrm{d}𝐀}{\mathrm{d}t}{𝐀}^{n-i},\\ \frac{\mathrm{d}}{\mathrm{d}t}{𝐀}^{-n}& =-\sum _{i=1}^n{𝐀}^{-i}\frac{\mathrm{d}𝐀}{\mathrm{d}t}{𝐀}^{-(n+1-i)}\end{array}$

In particular,

$\begin{array}{cc}(𝐀+\epsilon 𝐗{)}^n& ={𝐀}^n+\epsilon \sum _{i=1}^n{𝐀}^{i-1}𝐗{𝐀}^{n-i}+𝒪\left({\epsilon }^2\right),\\ (𝐀+\epsilon 𝐗{)}^{-n}& ={𝐀}^{-n}-\epsilon \sum _{i=1}^n{𝐀}^{-i}𝐗{𝐀}^{-(n+1-i)}+𝒪\left({\epsilon }^2\right)\end{array}$

The derivative of a matrix determinant with respect to one of its elements is a matrix cofactor, which exhibits the transpose of the matrix inverse: $$\frac{\mathrm{d}}{\mathrm{d}{A}_{ij}}det(𝐀)={𝐂}_{ij}={A^{-1}}_{ji}det(𝐀)$$

Generalizations

Non-square matrices

Non-square matrices, i.e. Template:Mvar-by-Template:Mvar matrices for which Template:Math, do not have an inverse. However, in some cases such a matrix may have a left inverse or right inverse. If Template:Math is Template:Mvar-by-Template:Mvar and the rank of Template:Math is equal to Template:Math, (Template:Math), then Template:Math has a left inverse, an Template:Math-by-Template:Mvar matrix Template:Math such that Template:Math. If Template:Math has rank Template:Math (Template:Math), then it has a right inverse, an Template:Mvar-by-Template:Mvar matrix Template:Math such that Template:Math.

Some of the properties of inverse matrices are shared by generalized inverses (such as the Moore–Penrose inverse), which can be defined for any m-by-n matrix.

In Abstract algebra

While the most common case is that of matrices over the real or complex numbers, all of those definitions can be given for matrices over any algebraic structure equipped with addition and multiplication (i.e. rings). However, in the case of a ring being commutative, the condition for a square matrix to be invertible is that its determinant is invertible in the ring, which in general is a stricter requirement than it being nonzero. For a noncommutative ring, the usual determinant is not defined. The conditions for existence of left-inverse or right-inverse are more complicated, since a notion of rank does not exist over rings.

The set of Template:Math invertible matrices together with the operation of matrix multiplication and entries from ring Template:Mvar form a group, the general linear group of degree Template:Mvar, denoted Template:Math.

Practical Considerations

For most practical applications, it is not necessary to invert a matrix to solve a system of linear equations, since Gaussian elimination is computationally less expensive. However, for a unique solution to exist, it is necessary for the matrix involved to be invertible.

Matrix decomposition techniques like LU decomposition are much faster than inversion, and various fast algorithms for special classes of linear systems have also been developed.

Applications

Regression/least squares

Although an explicit inverse is not necessary to estimate the vector of unknowns, it is the easiest way to estimate their accuracy and is found in the diagonal of a matrix inverse (the posterior covariance matrix of the vector of unknowns). However, faster algorithms to compute only the diagonal entries of a matrix inverse are known in many cases.

Matrix inverses in real-time simulations

Matrix inversion plays a significant role in computer graphics, particularly in 3D graphics rendering and 3D simulations. Examples include screen-to-world ray casting, world-to-subspace-to-world object transformations, and physical simulations.

Matrix inverses in MIMO wireless communication

Matrix inversion also plays a significant role in the MIMO (Multiple-Input, Multiple-Output) technology in wireless communications. The MIMO system consists of N transmit and M receive antennas. Unique signals, occupying the same frequency band, are sent via N transmit antennas and are received via M receive antennas. The signal arriving at each receive antenna will be a linear combination of the N transmitted signals forming an N × M transmission matrix H. It is crucial for the matrix H to be invertible so that the receiver can figure out the transmitted information.

Diagonalization

Matrix inverses play a prominent role in diagonalization. An arbitrary matrix $𝐀$ is diagonalizable if and only if

$𝐀=𝐏𝐃{𝐏}^{-1}$

where $𝐃$ is a diagonal matrix and $𝐏$ is an invertible matrix. This expression allows arbitrary powers of $𝐀$ to be written in the compact form

${𝐀}^k=𝐏{𝐃}^k{𝐏}^{-1}$

where $k$ is a positive integer.

Further reading

• Template:Springer
• Template:Introduction to Algorithms
• Template:Cite book
• Template:Cite web

External links

• Template:Cite webTemplate:Cbignore
• Template:Cite web