TENSOR NETWORK

Tensor
Tensor: natural presentation of multi-dimensional array
A D-order Tensor is denoted as

Tensor Network
Tensor Network (TNs):
● Sparse data structures engineered for the efficient representation and manipulation of very high-dimensional Tensor.
● Graphical representation with Tensor Network Diagram.
Scalar
I1
I2
I3
I
J
I
I I J I1
I2
I3
Vector Matrix Cube
Rank-0 Tensor Rank-1 Tensor Rank-2 Tensor Rank-3 Tensor
Array form
Graphical
Diagram
Form

Tensor Network
Tensor Operations
A x
I J I
b=Ax A B C=AB
I J K I K
A B
I
J
K
L
M
P
C
L
M
P
I
J
Matrix-Vector
Multiplication
Matrix-Matrix
Multiplication
Matrix Trace
Tensor-Tensor
Contraction

Tensor Network
Tensor Decomposition
I1
I2
I3
I4
I5
I6
1
1
1
1
1
1
I1
I2
I3
I4
I5
I6
r
r r
r
r r
I1
I2
I3
I4
I5
I6
r
r
r
r
r
I1
I2
I3
I4
I5
I6
r
r
r
r
r
r
CP Decomposition Tucker Decomposition
Tensor Train Decomposition Tensor Ring Decomposition

Types of Tensor Network
Matrix Product States / Tensor Train Decomposition
Matrix Product States (MPS) is a factorization of a tensor with N indices into a chain-like product of three-index tensors.
MPS MPS with periodic boundary conditions

Types of Tensor Network
Matrix Product Operator
A matrix product operator (MPO) is a tensor network where each tensor has two external, uncontracted indices as well as
two internal indices contracted with neighboring tensors in a chain-like fashion.
MPO

Types of Tensor Network
Tree Tensor Network / Hierarchical Tucker Decomposition
Tree Tensor Network is class of Tensor Network presented by general tree-like structures.

Types of Tensor Network
Projected Entangled Pair States (PEPS)
The PEPS tensor network generalizes the Matrix Product State / Tensor Train Tensor Network from a one-dimensional
network to a network on an arbitrary graph.
The tensor diagram for a PEPS on a finite square lattice is:

Types of Tensor Network
Multi-scale Entanglement Renormalization Ansatz (MERA)
Compared with the PEPS formats, the main advantage of the MERA formats is that the order and size of each core tensor in
the internal tensor network structure is often much smaller, which dramatically reduces the number of free parameters and
provides more efficient distributed storage of huge-scale data tensors.

Applications of Tensor Network
Big domains:
● Representing Continuous Functions
● Machine Learning
● Probability and Statistic
● Quantum Physics
In Machine Learning:
● Quantum Computers
● Supervised/ Unsupervised/ Reinforcement Learning
● Expressivity & priors of TN based models
● Generative models
● Compressing weights of neural nets
● Optimization Methods
● Feature extraction & tensor completion
● …

Resource
https://tensornetwork.org/
[1] Roberts, Chase, et al. "Tensornetwork: A library for physics and machine learning." arXiv preprint
arXiv:1905.01330 (2019).
[2] Biamonte, Jacob. "Lectures on quantum tensor networks." arXiv preprint arXiv:1912.10049 (2019).
[3] Biamonte, Jacob, and Ville Bergholm. "Tensor networks in a nutshell." arXiv preprint arXiv:1708.00006
(2017).
[4] Orús, Román. "A practical introduction to tensor networks: Matrix product states and projected entangled
pair states." Annals of physics 349 (2014): 117-158.
[5] Bridgeman, Jacob C., and Christopher T. Chubb. "Hand-waving and interpretive dance: an introductory
course on tensor networks." Journal of physics A: Mathematical and theoretical 50.22 (2017): 223001.
[6] Cichocki, Andrzej, et al. "Low-rank tensor networks for dimensionality reduction and large-scale
optimization problems: Perspectives and challenges part 1." arXiv preprint arXiv:1609.00893 (2016).