Lei Mao bio photo

Lei Mao

Machine Learning, Artificial Intelligence, Computer Science.

Twitter Facebook LinkedIn GitHub   G. Scholar E-Mail RSS

Introduction

Tensor product and Kronecker product are very important in quantum mechanics. It also have practical physical meanings for quantum processes. One of the interesting properties of Kronecker product is that it is “almost commutative”.


In this blog post, I would like to informally discuss the “almost commutative” property for Kronecker product.

Outer Product

Given two vectors $u = \{u_0, u_1, \cdots, u_{m-1}\}$ and $v = \{v_0, v_1, \cdots, v_{n-1}\}$, the outer product of $u$ and $v$, denoted as $u \otimes v$, is defined as

Using index notation,

If $u$ and $v$ are column vectors, i.e., $u = [u_0, u_1, \cdots, u_{m-1}]^{\top}$ and $v = [v_0, v_1, \cdots, v_{n-1}]^{\top}$, the outer product could also be simply expressed using matrix multiplication.

The mapping of outer product could be described as

Tensor Product

Tensor product is essentially an general case of one dimensional array outer product. It is the outer product of two tensors, namely multidimensional tensors, that could have different dimensionality.


If $A$ is a $k$ dimensional tensor, $A \in \mathbb{C}^{ \prod_{i=0}^{k-1} m_i}$ and $B$ is a $k^{\prime}$ dimensional tensor, $B \in \mathbb{C}^{ \prod_{i=0}^{k^{\prime}-1} n_i}$, the tensor product $A \otimes B$ would have shape $A \otimes B \in \mathbb{C}^{ \prod_{i=0}^{k-1} m_i \prod_{i=0}^{k^{\prime}-1} n_i}$.

Kronecker Product

The outer product and Kronecker product are closely related. In fact the same symbol is commonly used to denote both operations.


If $u$ and $v$ are column vectors, i.e., $u = [u_0, u_1, \cdots, u_{m-1}]^{\top}$ and $v = [v_0, v_1, \cdots, v_{n-1}]^{\top}$, the Kronecker product could also be expressed as follows.

Note that there is an implicit dimensional reduction process in the above expression.


The mapping of outer product could be described as

More concretely, let $A \in \mathbb{C}^{m \times m^{\prime}}$ and $B \in \mathbb{C}^{n \times n^{\prime}}$. Then the Kronecker product of $A$ and $B$ is defined as the matrix

Almost Commutative

Kronecker product is not commutative, i.e., usually $A \otimes B \neq B \otimes A$. However, Kronecker product is almost commutative with some row and column exchanges in some dimensions. We define this as $A \otimes B \cong B \otimes A$.


Suppose we have $A \in \mathbb{C}^{m \times m^{\prime}}$ and $B \in \mathbb{C}^{n \times n^{\prime}}$, we have

We rearrange the columns of $A \otimes B$, we have

We further rearrange the rows of $A \otimes B$, we have

Therefore,

This means, there exist permutation matrices $P$ and $Q$ such that

Miscellaneous

In Numpy, the tensor product could be computed using numpy.ufunc.outer, and the Kronecker product could be computed using numpy.kron.