Marchenko–Pastur distribution

Plot of the Marchenko-Pastur distribution for various values of lambda

Template:Probability distribution In the mathematical theory of random matrices, the Marchenko–Pastur distribution, or Marchenko–Pastur law, describes the asymptotic behavior of singular values of large rectangular random matrices. The theorem is named after Soviet Ukrainian mathematicians Volodymyr Marchenko and Leonid Pastur who proved this result in 1967.

If $X$ denotes an $m\times n$ random matrix whose entries are independent identically distributed random variables with mean 0 and variance ${\sigma }^2<\mathrm{\infty }$, let

$Y_n=\frac{1}{n}X{X}^T$

and let ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_m$ be the eigenvalues of $Y_n$ (viewed as random variables). Finally, consider the random measure

${\mu }_m(A)=\frac{1}{m}\mathrm{\#}\{{\lambda }_j\in A\},A\subset ℝ,$

which is the empirical distribution, counting the number of eigenvalues in the subset $A$ included in $ℝ$.

Theorem. Template:Citation needed Assume that $m,n\to \mathrm{\infty }$ so that the ratio $m/n\to \lambda \in (0,+\mathrm{\infty })$. Then ${\mu }_m\to \mu$ (in weak* topology in distribution), where

$\mu (A)=\{\begin{array}{cc}(1-\frac{1}{\lambda }){\mathrm{𝟏}}_{0\in A}+\nu (A),& \text{if }\lambda >1\\ \nu (A),& \text{if }0\le \lambda \le 1,\end{array}$

and

$d\nu (x)=\frac{1}{2\pi {\sigma }^2}\frac{\sqrt{({\lambda }_{+}-x)(x-{\lambda }_{-})}}{\lambda x}{\mathrm{𝟏}}_{x\in [{\lambda }_{-},{\lambda }_{+}]}dx$

with

${\lambda }_{\pm }={\sigma }^2(1\pm \sqrt{\lambda }{)}^2.$

The Marchenko–Pastur law also arises as the free Poisson law in free probability theory, having rate $1/\lambda$ and jump size $\lambda {\sigma }^2$.

Singular value bounds in the large system limit

As the dimensions of a random matrix $𝐗$ grow larger, the max/min singular values converge to $\Vert 𝐗{\Vert }_F(\frac{1}{\sqrt{min(m,n)}}\pm \frac{1}{\sqrt{max(m,n)}})$ .

These are useful approximations of singular value bounds for large matrices. For matrices of finite size as are typically encountered, they are more what you'd call "guidelines" than actual rules.

Moments

For each $k\ge 1$, its $k$-th moment isTemplate:Sfnm

$$\sum _{r=0}^{k-1}\frac{{\sigma }^{2k}}{r+1}(\genfrac{}{}{0ex}{}{k}{r})(\genfrac{}{}{0ex}{}{k-1}{r}){\lambda }^r=\frac{{\sigma }^{2k}}{k}\sum _{r=0}^{k-1}(\genfrac{}{}{0ex}{}{k}{r})(\genfrac{}{}{0ex}{}{k}{r+1}){\lambda }^r$$

Some transforms of this law

The Stieltjes transform is given by

$s(z)=\frac{{\sigma }^2(1-\lambda )-z-\sqrt{(z-{\sigma }^2(\lambda +1){)}^2-4\lambda {\sigma }^4}}{2\lambda z{\sigma }^2}$

for complex numbers Template:Mvar of positive imaginary part, where the complex square root is also taken to have positive imaginary part.Template:Sfnm It satisfies the quadratic equation

$\lambda {\sigma }^{2}zs(z{)}^2+(z-{\sigma }^2(1-\lambda ))s(z)+1=0.$

The Stieltjes transform can be repackaged in the form of the R-transform, which is given byTemplate:Sfnm

$R(z)=\frac{{\sigma }^2}{1-{\sigma }^2\lambda z}$

The S-transform is given byTemplate:Sfnm

$S(z)=\frac{1}{{\sigma }^2(1+\lambda z)}.$

For the case of $\sigma =1$, the $\eta$-transform Template:Sfnm is given by $𝔼\frac{1}{1+\gamma X}$ where $X$ satisfies the Marchenko-Pastur law.

$\eta (\gamma )=1-\frac{ℱ(\gamma ,\lambda )}{4\gamma \lambda }$

where $ℱ(x,z)={(\sqrt{x(1+\sqrt{z}{)}^2+1}-\sqrt{x(1-\sqrt{z}{)}^2+1})}^2$

For exact analysis of high dimensional regression in the proportional asymptotic regime, a convenient form is often $T(u):=\eta \left(\frac{1}{u}\right)$ which simplifies to

$T(u)=\frac{-1+\lambda -u+\sqrt{(1+u-\lambda {)}^2+4u\lambda }}{2\lambda }$

The following functions $B(u):=𝔼{\left(\frac{u}{X+u}\right)}^2$ and $V(u):=\frac{X}{(X+u{)}^2}$, where $X$ satisfies the Marchenko-Pastur law, show up in the limiting Bias and Variance respectively, of ridge regression and other regularized linear regression problems. One can show that $B(u)=T(u)-u\cdot T^{\prime }(u)$ and $V(u)=T^{\prime }(u)$.

Application to correlation matrices

For the special case of correlation matrices, we know that ${\sigma }^2=1$ and $\lambda =m/n$. This bounds the probability mass over the interval defined by

${\lambda }_{\pm }={(1\pm \sqrt{\frac{m}{n}})}^2.$

Since this distribution describes the spectrum of random matrices with mean 0, the eigenvalues of correlation matrices that fall inside of the aforementioned interval could be considered spurious or noise. For instance, obtaining a correlation matrix of 10 stock returns calculated over a 252 trading days period would render ${\lambda }_{+}={(1+\sqrt{\frac{10}{252}})}^2\approx 1.43$. Thus, out of 10 eigenvalues of said correlation matrix, only the values higher than 1.43 would be considered significantly different from random.

See also

• Wigner semicircle distribution
• Tracy–Widom distribution
• Wishart distribution

References

• Template:Cite book
• Template:Cite journal
• Template:Cite journal
• Template:Cite journal Link to free-access pdf of Russian version
• Template:Cite book Link to free download Another free access site
• Template:Cite journal
• Template:Cite journal