Bussgang theorem

Template:Short description In mathematics, the Bussgang theorem is a theorem of stochastic analysis. The theorem states that the cross-correlation between a Gaussian signal before and after it has passed through a nonlinear operation are equal to the signals auto-correlation up to a constant. It was first published by Julian J. Bussgang in 1952 while he was at the Massachusetts Institute of Technology.

Statement

Let $\{X(t)\}$ be a zero-mean stationary Gaussian random process and $\{Y(t)\}=g(X(t))$ where $g(\cdot )$ is a nonlinear amplitude distortion.

If $R_X(\tau )$ is the autocorrelation function of $\{X(t)\}$, then the cross-correlation function of $\{X(t)\}$ and $\{Y(t)\}$ is

$R_{XY}(\tau )=C{R}_X(\tau ),$

where $C$ is a constant that depends only on $g(\cdot )$.

It can be further shown that

$C=\frac{1}{{\sigma }^3\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}ug(u)e^{-\frac{u^2}{2{\sigma }^2}}du.$

Derivation for One-bit Quantization

It is a property of the two-dimensional normal distribution that the joint density of $y_1$ and $y_2$ depends only on their covariance and is given explicitly by the expression

$p(y_1,y_2)=\frac{1}{2\pi \sqrt{1-{\rho }^2}}{e}^{-\frac{y_1^2+y_2^2-2\rho y_1{y}_2}{2(1-{\rho }^2)}}$

where $y_1$ and $y_2$ are standard Gaussian random variables with correlation ${\varphi }_{y_1{y}_2}=\rho$.

Assume that $r_2=Q(y_2)$, the correlation between $y_1$ and $r_2$ is,

${\varphi }_{y_1{r}_2}=\frac{1}{2\pi \sqrt{1-{\rho }^2}}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{y}_{1}Q(y_2)e^{-\frac{y_1^2+y_2^2-2\rho y_1{y}_2}{2(1-{\rho }^2)}}d{y}_{1}d{y}_2$.

Since

${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{y}_1{e}^{-\frac{1}{2(1-{\rho }^2)}{y}_1^2+\frac{\rho y_2}{1-{\rho }^2}{y}_1}d{y}_1=\rho \sqrt{2\pi (1-{\rho }^2)}{y}_2{e}^{\frac{{\rho }^2{y}_2^2}{2(1-{\rho }^2)}}$,

the correlation ${\varphi }_{y_1{r}_2}$ may be simplified as

${\varphi }_{y_1{r}_2}=\frac{\rho }{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{y}_{2}Q(y_2)e^{-\frac{y_2^2}{2}}d{y}_2$.

The integral above is seen to depend only on the distortion characteristic $Q()$ and is independent of $\rho$.

Remembering that $\rho ={\varphi }_{y_1{y}_2}$, we observe that for a given distortion characteristic $Q()$, the ratio $\frac{{\varphi }_{y_1{r}_2}}{{\varphi }_{y_1{y}_2}}$ is $K_Q=\frac{1}{\sqrt{2\pi }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{y}_{2}Q(y_2)e^{-\frac{y_2^2}{2}}d{y}_2$.

Therefore, the correlation can be rewritten in the form

${\varphi }_{y_1{r}_2}=K_Q{\varphi }_{y_1{y}_2}$.

The above equation is the mathematical expression of the stated "Bussgang‘s theorem".

If $Q(x)=\text{sign}(x)$, or called one-bit quantization, then $K_Q=\frac{2}{\sqrt{2\pi }}{\int }_0^{\mathrm{\infty }}{y}_2{e}^{-\frac{y_2^2}{2}}d{y}_2=\sqrt{\frac{2}{\pi }}$.

Arcsine law

If the two random variables are both distorted, i.e., $r_1=Q(y_1),r_2=Q(y_2)$, the correlation of $r_1$ and $r_2$ is

${\varphi }_{r_1{r}_2}={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}Q(y_1)Q(y_2)p(y_1,y_2)d{y}_{1}d{y}_2$.

When $Q(x)=\text{sign}(x)$, the expression becomes,

${\varphi }_{r_1{r}_2}=\frac{1}{2\pi \sqrt{1-{\rho }^2}}[{\int }_0^{\mathrm{\infty }}{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2+{\int }_{-\mathrm{\infty }}^0{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2-{\int }_0^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2-{\int }_{-\mathrm{\infty }}^0{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2]$

where $\alpha =\frac{y_1^2+y_2^2-2\rho y_1{y}_2}{2(1-{\rho }^2)}$.

Noticing that

${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p(y_1,y_2)d{y}_{1}d{y}_2=\frac{1}{2\pi \sqrt{1-{\rho }^2}}[{\int }_0^{\mathrm{\infty }}{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2+{\int }_{-\mathrm{\infty }}^0{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2+{\int }_0^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2+{\int }_{-\mathrm{\infty }}^0{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2]=1$,

and ${\int }_0^{\mathrm{\infty }}{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2={\int }_{-\mathrm{\infty }}^0{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2$, ${\int }_0^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^0{e}^{-\alpha }d{y}_{1}d{y}_2={\int }_{-\mathrm{\infty }}^0{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2$,

we can simplify the expression of ${\varphi }_{r_1{r}_2}$ as

${\varphi }_{r_1{r}_2}=\frac{4}{2\pi \sqrt{1-{\rho }^2}}{\int }_0^{\mathrm{\infty }}{\int }_0^{\mathrm{\infty }}{e}^{-\alpha }d{y}_{1}d{y}_2-1$

Also, it is convenient to introduce the polar coordinate $y_1=R\cos \theta ,y_2=R\sin \theta$. It is thus found that

${\varphi }_{r_1{r}_2}=\frac{4}{2\pi \sqrt{1-{\rho }^2}}{\int }_0^{\pi /2}{\int }_0^{\mathrm{\infty }}{e}^{-\frac{R^2-2R^2\rho \cos \theta \sin \theta }{2(1-{\rho }^2)}}RdRd\theta -1=\frac{4}{2\pi \sqrt{1-{\rho }^2}}{\int }_0^{\pi /2}{\int }_0^{\mathrm{\infty }}{e}^{-\frac{R^2(1-\rho \sin 2\theta )}{2(1-{\rho }^2)}}RdRd\theta -1$.

Integration gives

${\varphi }_{r_1{r}_2}=\frac{2\sqrt{1-{\rho }^2}}{\pi }{\int }_0^{\pi /2}\frac{d\theta }{1-\rho \sin 2\theta }-1=-\frac{2}{\pi }\arctan \left(\frac{\rho -\tan \theta }{\sqrt{1-{\rho }^2}}\right){|}_0^{\pi /2}-1=\frac{2}{\pi }\arcsin (\rho )$，

This is called "Arcsine law", which was first found by J. H. Van Vleck in 1943 and republished in 1966. The "Arcsine law" can also be proved in a simpler way by applying Price's Theorem.

The function $f(x)=\frac{2}{\pi }\arcsin x$ can be approximated as $f(x)\approx \frac{2}{\pi }x$ when $x$ is small.

Price's Theorem

Given two jointly normal random variables $y_1$ and $y_2$ with joint probability function

$p(y_1,y_2)=\frac{1}{2\pi \sqrt{1-{\rho }^2}}{e}^{-\frac{y_1^2+y_2^2-2\rho y_1{y}_2}{2(1-{\rho }^2)}}$,

we form the mean

$I(\rho )=E(g(y_1,y_2))={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}g(y_1,y_2)p(y_1,y_2)d{y}_{1}d{y}_2$

of some function $g(y_1,y_2)$ of $(y_1,y_2)$. If $g(y_1,y_2)p(y_1,y_2)\to 0$ as $(y_1,y_2)\to 0$, then

$\frac{{\partial }^{n}I(\rho )}{\partial {\rho }^n}={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{{\partial }^{2n}g(y_1,y_2)}{\partial y_1^n\partial y_2^n}p(y_1,y_2)d{y}_{1}d{y}_2=E\left(\frac{{\partial }^{2n}g(y_1,y_2)}{\partial y_1^n\partial y_2^n}\right)$.

Proof. The joint characteristic function of the random variables $y_1$ and $y_2$ is by definition the integral

$\mathrm{\Phi }({\omega }_1,{\omega }_2)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p(y_1,y_2)e^{j({\omega }_1{y}_1+{\omega }_2{y}_2)}d{y}_{1}d{y}_2=\exp \{-\frac{{\omega }_1^2+{\omega }_2^2+2\rho {\omega }_1{\omega }_2}{2}\}$.

From the two-dimensional inversion formula of Fourier transform, it follows that

$p(y_1,y_2)=\frac{1}{4{\pi }^2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\mathrm{\Phi }({\omega }_1,{\omega }_2)e^{-j({\omega }_1{y}_1+{\omega }_2{y}_2)}d{\omega }_{1}d{\omega }_2=\frac{1}{4{\pi }^2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\exp \{-\frac{{\omega }_1^2+{\omega }_2^2+2\rho {\omega }_1{\omega }_2}{2}\}{e}^{-j({\omega }_1{y}_1+{\omega }_2{y}_2)}d{\omega }_{1}d{\omega }_2$.

Therefore, plugging the expression of $p(y_1,y_2)$ into $I(\rho )$, and differentiating with respect to $\rho$, we obtain

$\begin{array}{cc}\frac{{\partial }^{n}I(\rho )}{\partial {\rho }^n}& ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)p(y_1,y_2)d{y}_{1}d{y}_2\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)\left(\frac{1}{4{\pi }^2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{{\partial }^n\mathrm{\Phi }({\omega }_1,{\omega }_2)}{\partial {\rho }^n}{e}^{-j({\omega }_1{y}_1+{\omega }_2{y}_2)}d{\omega }_{1}d{\omega }_2\right)d{y}_{1}d{y}_2\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)(\frac{(-1{)}^n}{4{\pi }^2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\omega }_1^n{\omega }_2^n\mathrm{\Phi }({\omega }_1,{\omega }_2)e^{-j({\omega }_1{y}_1+{\omega }_2{y}_2)}d{\omega }_{1}d{\omega }_2)d{y}_{1}d{y}_2\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)(\frac{1}{4{\pi }^2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\mathrm{\Phi }({\omega }_1,{\omega }_2)\frac{{\partial }^{2n}{e}^{-j({\omega }_1{y}_1+{\omega }_2{y}_2)}}{\partial y_1^n\partial y_2^n}d{\omega }_{1}d{\omega }_2)d{y}_{1}d{y}_2\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)\frac{{\partial }^{2n}p(y_1,y_2)}{\partial y_1^n\partial y_2^n}d{y}_{1}d{y}_2\end{array}$

After repeated integration by parts and using the condition at $\mathrm{\infty }$, we obtain the Price's theorem.

$\begin{array}{cc}\frac{{\partial }^{n}I(\rho )}{\partial {\rho }^n}& ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}g(y_1,y_2)\frac{{\partial }^{2n}p(y_1,y_2)}{\partial y_1^n\partial y_2^n}d{y}_{1}d{y}_2\\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{{\partial }^{2}g(y_1,y_2)}{\partial y_1\partial y_2}\frac{{\partial }^{2n-2}p(y_1,y_2)}{\partial y_1^{n-1}\partial y_2^{n-1}}d{y}_{1}d{y}_2\\ & =\cdots \\ & ={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{{\partial }^{2n}g(y_1,y_2)}{\partial y_1^n\partial y_2^n}p(y_1,y_2)d{y}_{1}d{y}_2\end{array}$

Proof of Arcsine law by Price's Theorem

If $g(y_1,y_2)=\text{sign}(y_1)\text{sign}(y_2)$, then $\frac{{\partial }^{2}g(y_1,y_2)}{\partial y_1\partial y_2}=4\delta (y_1)\delta (y_2)$ where $\delta ()$ is the Dirac delta function.

Substituting into Price's Theorem, we obtain,

$\frac{\partial E(\text{sign}(y_1)\text{sign}(y_2))}{\partial \rho }=\frac{\partial I(\rho )}{\partial \rho }={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}4\delta (y_1)\delta (y_2)p(y_1,y_2)d{y}_{1}d{y}_2=\frac{2}{\pi \sqrt{1-{\rho }^2}}$.

When $\rho =0$, $I(\rho )=0$. Thus

$E(\text{sign}(y_1)\text{sign}(y_2))=I(\rho )=\frac{2}{\pi }{\int }_0^{\rho }\frac{1}{\sqrt{1-{\rho }^2}}d\rho =\frac{2}{\pi }\arcsin (\rho )$,

which is Van Vleck's well-known result of "Arcsine law".

Application

This theorem implies that a simplified correlator can be designed.Template:Clarify Instead of having to multiply two signals, the cross-correlation problem reduces to the gatingTemplate:Clarify of one signal with another.Template:Citation needed

Further reading

• E.W. Bai; V. Cerone; D. Regruto (2007) "Separable inputs for the identification of block-oriented nonlinear systems", Proceedings of the 2007 American Control Conference (New York City, July 11–13, 2007) 1548–1553