Tensor Product

The tensor product is a method for multiplying linear maps that computes the outer product of every pair of tensors.

From 2 matrices $ A=\begin{bmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{bmatrix} $ and $ B=\begin{bmatrix}b_{11}&b_{12}\\b_{21}&b_{22}\end{bmatrix} $ the tensor product noted $ \otimes $ is calculated $$ A \otimes B = \begin{bmatrix}a_{11}\begin{bmatrix}b_{11}&b_{12}\\b_{21}&b_{22}\end{bmatrix}&a_{12}\begin{bmatrix}b_{11}&b_{12}\\b_{21}&b_{22}\end{bmatrix} \\ a_{21}\begin{bmatrix}b_{11}&b_{12}\\b_{21}&b_{22}\end{bmatrix} & a_{22}\begin{bmatrix}b_{11}&b_{12}\\b_{21}&b_{22}\end{bmatrix}\end{bmatrix} = \begin{bmatrix}a_{11}b_{11}&a_{11}b_{12}&a_{12}b_{11}&a_{12}b_{12}\\a_{11}b_{21}&a_{11}b_{22}&a_{12}b_{21}&a_{12}b_{22}\\a_{21}b_{11}&a_{21}b_{12}&a_{22}b_{11}&a_{22}b_{12}\\a_{21}b_{21}&a_{21}b_{22}&a_{22}b_{21}&a_{22}b_{22}\end{bmatrix} $$

From 2 vectors $ \vec{a} = \begin{bmatrix}a_1 \\ a_2 \\ \vdots \\ a_n \end{bmatrix} $ and $ \vec{b} = \begin{bmatrix}b_1 \\ b_2 \\ \vdots \\ b_m \end{bmatrix} $ the tensor product noted $ \otimes $ is calculated $ [ a_{i} ] \otimes [ b_{j} ] ^ T $ by converting vectors to matrices by transposing the second vector so as to have a row vector and a column vector.

The resulting tensor will have dimensions of the multiplication of the number of elements in the original vectors.

$$ \vec{a} \otimes \vec{b} = \begin{bmatrix}a_1 b_1 & a_1 b_2 & \cdots &a_1 b_m \\ a_2 b_1 & a_2 b_2&\cdots &a_2 b_m \\ \vdots & \vdots & \ddots & \vdots \\ a_n b_1 & a_n b_2 & \cdots & a_n b_m \end{bmatrix} $$