Khatri–Rao product

Template:Short description In mathematics, the Khatri–Rao product or block Kronecker product of two partitioned matrices $𝐀$ and $𝐁$ is defined as

$𝐀\ast 𝐁={({𝐀}_{ij}\otimes {𝐁}_{ij})}_{ij}$

in which the ij-th block is the Template:Nowrap sized Kronecker product of the corresponding blocks of A and B, assuming the number of row and column partitions of both matrices is equal. The size of the product is then Template:Nowrap.

For example, if A and B both are Template:Nowrap partitioned matrices e.g.:

$𝐀=\left[\begin{array}{cc}{𝐀}_{11}& {𝐀}_{12}\\ {𝐀}_{21}& {𝐀}_{22}\end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],𝐁=\left[\begin{array}{cc}{𝐁}_{11}& {𝐁}_{12}\\ {𝐁}_{21}& {𝐁}_{22}\end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],$

we obtain:

$𝐀\ast 𝐁=\left[\begin{array}{cc}{𝐀}_{11}\otimes {𝐁}_{11}& {𝐀}_{12}\otimes {𝐁}_{12}\\ {𝐀}_{21}\otimes {𝐁}_{21}& {𝐀}_{22}\otimes {𝐁}_{22}\end{array}\right]=\left[\begin{array}{cccc}1& 2& 12& 21\\ 4& 5& 24& 42\\ 14& 16& 45& 72\\ 21& 24& 54& 81\end{array}\right].$

This is a submatrix of the Tracy–Singh product of the two matrices (each partition in this example is a partition in a corner of the Tracy–Singh product).

Column-wise Kronecker product

The column-wise Kronecker product of two matrices is a special case of the Khatri-Rao product as defined above, and may also be called the Khatri–Rao product. This product assumes the partitions of the matrices are their columns. In this case Template:Nowrap, Template:Nowrap, Template:Nowrap and for each j: Template:Nowrap. The resulting product is a Template:Nowrap matrix of which each column is the Kronecker product of the corresponding columns of A and B. Using the matrices from the previous examples with the columns partitioned:

$𝐂=\left[\begin{array}{ccc}{𝐂}_1& {𝐂}_2& {𝐂}_3\end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],𝐃=\left[\begin{array}{ccc}{𝐃}_1& {𝐃}_2& {𝐃}_3\end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],$

so that:

$𝐂\ast 𝐃=\left[\begin{array}{ccc}{𝐂}_1\otimes {𝐃}_1& {𝐂}_2\otimes {𝐃}_2& {𝐂}_3\otimes {𝐃}_3\end{array}\right]=\left[\begin{array}{ccc}1& 8& 21\\ 2& 10& 24\\ 3& 12& 27\\ 4& 20& 42\\ 8& 25& 48\\ 12& 30& 54\\ 7& 32& 63\\ 14& 40& 72\\ 21& 48& 81\end{array}\right].$

This column-wise version of the Khatri–Rao product is useful in linear algebra approaches to data analytical processing and in optimizing the solution of inverse problems dealing with a diagonal matrix.

In 1996 the column-wise Khatri–Rao product was proposed to estimate the angles of arrival (AOAs) and delays of multipath signals and four coordinates of signals sources at a digital antenna array.

Face-splitting product

Face splitting product of matrices

An alternative concept of the matrix product, which uses row-wise splitting of matrices with a given quantity of rows, was proposed by V. Slyusar in 1996.

This matrix operation was named the "face-splitting product" of matrices or the "transposed Khatri–Rao product". This type of operation is based on row-by-row Kronecker products of two matrices. Using the matrices from the previous examples with the rows partitioned:

$𝐂=\left[\begin{array}{c}{𝐂}_1\\ {𝐂}_2\\ {𝐂}_3\end{array}\right]=\left[\begin{array}{ccc}1& 2& 3\\ 4& 5& 6\\ 7& 8& 9\end{array}\right],𝐃=\left[\begin{array}{c}{𝐃}_1\\ {𝐃}_2\\ {𝐃}_3\end{array}\right]=\left[\begin{array}{ccc}1& 4& 7\\ 2& 5& 8\\ 3& 6& 9\end{array}\right],$

the result can be obtained:

$𝐂\bullet 𝐃=\left[\begin{array}{c}{𝐂}_1\otimes {𝐃}_1\\ {𝐂}_2\otimes {𝐃}_2\\ {𝐂}_3\otimes {𝐃}_3\end{array}\right]=\left[\begin{array}{ccccccccc}1& 4& 7& 2& 8& 14& 3& 12& 21\\ 8& 20& 32& 10& 25& 40& 12& 30& 48\\ 21& 42& 63& 24& 48& 72& 27& 54& 81\end{array}\right].$

Main properties

Examples

Source:

$\begin{array}{cc}& (\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\bullet \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right])(\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\otimes \left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right])(\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\end{array}\right]\otimes \left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\end{array}\right])(\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\ast \left[\begin{array}{c}{y}_1\\ y_2\end{array}\right])\\ =& (\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\bullet \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right])(\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\otimes \left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right])\\ =& \left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 0\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\sigma }_1& 0\\ 0& {\sigma }_2\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]\circ \left[\begin{array}{cc}1& 0\\ 1& 0\\ 0& 1\end{array}\right]\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{cc}{\rho }_1& 0\\ 0& {\rho }_2\end{array}\right]\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right].\end{array}$

Theorem

Source:

If $M=T^{(1)}\bullet \dots \bullet T^{(c)}$, where $T^{(1)},\dots ,T^{(c)}$ are independent components a random matrix $T$ with independent identically distributed rows $T_1,\dots ,T_m\in {ℝ}^d$, such that

$E[(T_{1}x{)}^2]={\Vert x\Vert }_2^2$ and $E{[(T_{1}x{)}^p]}^{\frac{1}{p}}\le \sqrt{ap}\Vert x{\Vert }_2$,

then for any vector $x$

$|{\Vert Mx\Vert }_2-{\Vert x\Vert }_2|<\epsilon {\Vert x\Vert }_2$

with probability $1-\delta$ if the quantity of rows

$m=(4a{)}^{2c}{\epsilon }^{-2}\log 1/\delta +(2ae){\epsilon }^{-1}(\log 1/\delta {)}^c.$

In particular, if the entries of $T$ are $\pm 1$ can get

$m=O({\epsilon }^{-2}\log 1/\delta +{\epsilon }^{-1}{(\frac{1}{c}\log 1/\delta )}^c)$

which matches the Johnson–Lindenstrauss lemma of $m=O({\epsilon }^{-2}\log 1/\delta )$ when $\epsilon$ is small.

Block face-splitting product

Transposed block face-splitting product in the context of a multi-face radar model

According to the definition of V. Slyusar the block face-splitting product of two partitioned matrices with a given quantity of rows in blocks

$𝐀=\left[\begin{array}{cc}{𝐀}_{11}& {𝐀}_{12}\\ {𝐀}_{21}& {𝐀}_{22}\end{array}\right],𝐁=\left[\begin{array}{cc}{𝐁}_{11}& {𝐁}_{12}\\ {𝐁}_{21}& {𝐁}_{22}\end{array}\right],$

can be written as :

$𝐀[\bullet ]𝐁=\left[\begin{array}{cc}{𝐀}_{11}\bullet {𝐁}_{11}& {𝐀}_{12}\bullet {𝐁}_{12}\\ {𝐀}_{21}\bullet {𝐁}_{21}& {𝐀}_{22}\bullet {𝐁}_{22}\end{array}\right].$

The transposed block face-splitting product (or Block column-wise version of the Khatri–Rao product) of two partitioned matrices with a given quantity of columns in blocks has a view:

$𝐀[\ast ]𝐁=\left[\begin{array}{cc}{𝐀}_{11}\ast {𝐁}_{11}& {𝐀}_{12}\ast {𝐁}_{12}\\ {𝐀}_{21}\ast {𝐁}_{21}& {𝐀}_{22}\ast {𝐁}_{22}\end{array}\right].$

Main properties

1. Transpose:

   ${(𝐀[\ast ]𝐁)}^{\mathsf{\text{T}}}={\mathbf{\text{A}}}^{\mathsf{\text{T}}}[\bullet ]{𝐁}^{\mathsf{\text{T}}}$

Applications

The Face-splitting product and the Block Face-splitting product used in the tensor-matrix theory of digital antenna arrays. These operations are also used in:

• Artificial Intelligence and Machine learning systems to minimization of convolution and tensor sketch operations,
• A popular Natural Language Processing models, and hypergraph models of similarity,
• Generalized linear array model in statistics
• Two- and multidimensional P-spline approximation of data,
• Studies of genotype x environment interactions.

See also

• Kronecker product
• Hadamard product

References

• Template:Cite journal
• Template:Citation
• Template:Citation
• Template:Citation