Talk given at the GAMM Annual Meeting in Graz

1. 82nd GAMM Annual Scientific Conference
Graz, April 19, 2011
Preconditioned Inverse Iteration for
Hierarchical Matrices
Peter Benner and Thomas Mach
Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory
Magdeburg
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 1/28

2. Hierarchical (H-)Matrices PINVIT Numerical Results
Outline
1 Hierarchical (H-)Matrices
2 PINVIT
3 Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 2/28

3. Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Some dense matrices, e.g. BEM or FEM, can be approximated by
H-matrices in a data-sparse manner.
hierarchical tree TI block H-tree TI × I
I = {1, 2, 3, 4, 5, 6, 7, 8}
12345678 12345678 12345678 12345678
1 1 1 1
2 2 2 2
{1, 2, 3, 4} {5, 6, 7, 8} 3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
{1, 2} {3, 4} {5, 6} {7, 8} 7 7 7 7
8 8 8 8
{1}{2}{3}{4}{5}{6}{7}{8} dense matrices, rank-k-matrices
rank-k-matrix: Ms×t = AB T , A ∈ Rn×k , B ∈ Rm×k (k n, m)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 3/28

4. Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI × I , k) = M ∈ RI × I rank (Ma×b ) ≤ k ∀a × b admissible
3 3
22 7 10 14 8 11 9 8 5
3 7 19 10 11 8
3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8
14
8
11
11
11
8
10
11
31 11
31 11
9
3
16 11
7
3
8
6
15
5
6
7
13
5
8
8
15 9
15 12 13
adaptive rank k(ε)
19
13
11 61 11 10 7
8
9 16 11 11 8 11 8
13 13
9 3 25 10 13 8
7 11 10 19 11 6 5 8 11 9
11 16
3
storage NSt,H (T , k) = O(n log n k(ε))
8 11 11 31 11 15 6 9 15 11
11 8
8 8 15 10 10 15 9
5 8 11 61 3
6 14 9 11 10 10 7
13
6 15 10 3 6 25 10 6 13 8
12 5 6 10 14 6
10 19
10
10
31 10 16 9
11
8
8
15 11 12
20
13 8
complexity of approximate arithmetic
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8
13 13
7 13
11 5 6 10 16 10
9
7
3
3
11
25
10
10
19 9
13
8
8
11
11
11 8 3 7
8 15 8 39 10 10 10 6 5
10
3
9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11
13
7 10 7 22 7 10 11 9 8 5
12
O(n log n k(ε))
10 7 19 10 11 8
MH v
3
7 12 10 6 3 10 10 31 11 9 16 12 8 15 8
20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11
13 13
11 8 13 7
O(n log n k(ε)2 )
7 11 10 16 61 11 23
+H , −H
9 10 11
13 9 6 13 20 9 9 7
11 8 3 7
9 8 8 15 5 6 12 9 39 10 3 10 15 10
12 13
11 8
8 8 15 9 10 15 9
7 11 11 3 3
61 10
−1
∗H , HLU(·), (·)H O(n (log n)2 k(ε)2 )
5 8 7 7 10 9 11
19
13 9 6 13 20 9 9 7
9 8
13
8
8
11 5 6 10 15 10 9 34 10 13
12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28

5. Hierarchical (H-)Matrices PINVIT Numerical Results
H-Matrices [Hackbusch 1998]
Hierarchical matrices
H(TI × I , k) = M ∈ RI × I rank (Ma×b ) ≤ k ∀a × b admissible
3 3
22 7 10 14 8 11 9 8 5
3 7 19 10 11 8
3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8
14
8
11
11
11
8
10
11
31 11
31 11
9
3
16 11
7
3
8
6
15
5
6
7
13
5
8
8
15 9
15 12 13
adaptive rank k(ε)
19
13
11 61 11 10 7
8
9 16 11 11 8 11 8
13 13
9 3 25 10 13 8
7 11 10 19 11 6 5 8 11 9
11 16
3
storage NSt,H (T , k) = O(n log n k(ε))
8 11 11 31 11 15 6 9 15 11
11 8
8 8 15 10 10 15 9
5 8 11 61 3
6 14 9 11 10 10 7
13
6 15 10 3 6 25 10 6 13 8
12 5 6 10 14 6
10 19
10
10
31 10 16 9
11
8
8
15 11 12
20
13 8
complexity of approximate arithmetic
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8
13 13
7 13
11 5 6 10 16 10
9
7
3
3
11
25
10
10
19 9
13
8
8
11
11
11 8 3 7
8 15 8 39 10 10 10 6 5
10
3
9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11
13
7 10 7 22 7 10 11 9 8 5
12
O(n log n k(ε))
10 7 19 10 11 8
MH v
3
7 12 10 6 3 10 10 31 11 9 16 12 8 15 8
20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11
13 13
11 8 13 7
O(n log n k(ε)2 )
7 11 10 16 61 11 23
+H , −H
9 10 11
13 9 6 13 20 9 9 7
11 8 3 7
9 8 8 15 5 6 12 9 39 10 3 10 15 10
12 13
11 8
8 8 15 9 10 15 9
7 11 11 3 3
61 10
−1
∗H , HLU(·), (·)H O(n (log n)2 k(ε)2 )
5 8 7 7 10 9 11
19
13 9 6 13 20 9 9 7
9 8
13
8
8
11 5 6 10 15 10 9 34 10 13
12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51
T
B1 T
B2 T
B1
A1 + A2 = A1 A2 T
B2
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 4/28

6. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λi in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28

7. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 5/28

8. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
Definition
The function
x T Mx
µ(x) = µ(x, M) =
xT x
is called the Rayleigh quotient.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28

9. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
Definition
The function
x T Mx
µ(x) = µ(x, M) =
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
2
xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) ,
x x
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28

10. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
Residual r (x) = Mx − xµ(x).
Minimize the Rayleigh quotient by a gradient method:
2
xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) ,
x x
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28

11. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
Definition
The function
x T Mx
µ(x) = µ(x, M) =
xT x
is called the Rayleigh quotient.
Minimize the Rayleigh quotient by a gradient method:
2
xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)) ,
x x
+ preconditioning ⇒ update equation:
xi+1 := xi − B −1 (Mxi − xi µ(xi )) .
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28

12. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
Preconditioned residual B −1 r (x) = B −1 (Mx − xµ(x)).
+ preconditioning ⇒ update equation:
xi+1 := xi − B −1 (Mxi − xi µ(xi )) .
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 6/28

13. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr 2009]
xi+1 := xi − B −1 (Mxi − xi µ(xi ))
If
M ∈ Rn×n symmetric positive definite and
B −1 approximates the inverse of M, so that
I − B −1 M M
≤ c < 1,
then Preconditioned INVerse ITeration (PINVIT) converges and
the number of iterations is independent of n.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 7/28

14. Hierarchical (H-)Matrices PINVIT Numerical Results
Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]
The residual
ri = Mxi − xi µ(xi )
converges to 0, so that
ri 2 <
is a useful termination criterion.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 8/28

15. Hierarchical (H-)Matrices PINVIT Numerical Results
Variants of PINVIT
[Neymeyr 2001: A Hierarchy of Precond. Eigens. for Ellipt. Diff. Op.]
Classification by Neymeyr:
PINVIT(1): xi+1 := xi − B −1 ri .
PINVIT(2): xi+1 := arg minv ∈span{xi ,B −1 ri } µ(v ).
PINVIT(3): xi+1 := arg minv ∈span{xi−1 ,xi ,B −1 ri } µ(v ).
PINVIT(n): Analogously.
PINVIT(·,d): Replacing x by a rectangular full rank matrix
X ∈ Rn×d one gets the subspace version of PINVIT(·).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 9/28

16. Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n , X0 ∈ Rn×d (X0 X0 = I , e.g. randomly chosen)
T
Output: Xp ∈ R n×d , µ ∈ Rd×d , with MX − X µ ≤
p p
B −1 = (M)−1 or B −1 = L−T L−1
H H H
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Orthogonalize Xi−1 − B −1 R
R := MXi − Xi µ, µ = XiT MXi
end
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28

17. Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
The number of iterations is independent of matrix size n.
H-PINVIT(1,d)
Input: M ∈ Rn×n , X0 ∈ Rn×d (X0 X0 = I , e.g. randomly chosen)
T
Output: Xp ∈ R n×d , µ ∈ Rd×d , with MX − X µ ≤
p p
B −1 = (M)−1 or B −1 = L−T L−1
H H H O(n (log n)2 k (c)2 )
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Orthogonalize Xi−1 − B −1 R O(n (log n) k (c)2 )
R := MXi − Xi µ, µ = XiT MXi O(n (log n) k (c))
end
The complexity of the algorithm is determined by the H-matrix
inversion/Cholesky decomposition: ⇒ O(n (log n)2 k (c)2 ).
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 10/28

18. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28

19. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 11/28

20. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
How to find λi in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 12/28

21. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α k β )?
If i = n − d with d < O(log n),
use subspace version PINVIT(·,d ).
...
0λn λn−1 λn−2 λ1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28

22. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α k β )?
If i = n − d with d log n?
... ...
0λn λi+1 λi λi−1 λ1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28

23. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α k β )?
If i = n − d with d log n,
shift with σ near λi .
... ...
0λn λi+1 λi λi−1 λ1
σ
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28

24. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
How to find λi in O(n (log n)α k β )?
If i = n − d with d log n,
shift with σ near λi .
... ...
0λn λi+1 λi λi−1 λ1
σ
But (M − σI) is not positive definite.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 13/28

25. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
Folded Spectrum Method [Wang, Zunger 1994]
Mσ = (M − σI)2
Mσ is s.p.d., if M is s.p.d. and σ = λi .
Assume all eigenvalues of Mσ are simple.
Mv = λv ⇔ Mσ v = (M − σI)2 v
= M 2 v − 2σMv + σ 2 v
= λ2 v − 2σλv + σ 2 v
= (λ − σ)2 v
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 14/28

26. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ or LLT = Mσ .
3 Use PINVIT to find the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28

27. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ or LLT = Mσ .
3 Use PINVIT to find the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.
Mσ in linear-polylogarithmic complexity.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28

28. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ or LLT = Mσ .
3 Use PINVIT to find the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.
Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting is
prohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28

29. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ or LLT = Mσ .
3 Use PINVIT to find the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.
Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting is
prohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28

30. Hierarchical (H-)Matrices PINVIT Numerical Results
Folded Spectrum Method
1 Choose σ.
2 Compute
2
a) Mσ = (M − σI) and
−1
b) Mσ or LLT = Mσ .
3 Use PINVIT to find the smallest eigenpair (µσ , v ) of Mσ .
4 Compute µ = v T Mv /v T v .
(µ, v ) is the nearest eigenpair to σ.
H-arithmetic enables us to shift, square and invert M resp.
Mσ in linear-polylogarithmic complexity.
If M is sparse, then shifting, squaring and inverting is
prohibitive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 15/28

31. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
How to find λi in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28

32. Hierarchical (H-)Matrices PINVIT Numerical Results
Goal: Eigenvalues of symmetric H-Matrices
M = M T ∈ H(T , k),
Λ(M) = {λ1, λ2, . . . , λn },
λ1 ≥ λ2 ≥ · · · ≥ λn > 0
⇓
How to find λn in O(n (log n)α k β )?
How to find λi in O(n (log n)α k β )?
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 16/28

33. Hierarchical (H-)Matrices PINVIT Numerical Results
Numerical Results
Numerical Results
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 17/28

34. Hierarchical (H-)Matrices PINVIT Numerical Results
Hlib
Hlib ¨
[Borm, Grasedyck, et al.]
We use the Hlib1.3 (www.hlib.org) for the
H-arithmetic operations and some examples out of
the library for testing the eigenvalue algorithm.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 18/28

35. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1] × [−1, 1], precond.: H-inversion
ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17 27.08
FEM64 4 096 3.8009E-010 0.82 6.68 8.06
FEM128 16 384 4.4099E-010 5.84 7.09 5.44
FEM256 65 536 3.9651E-010 34.47 5.91 5.22
FEM512 262 144 3.7877E-010 194.00 5.63 6.51
d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28

36. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1] × [−1, 1], precond.: H-inversion
ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 5.6146E-010 0.01
FEM16 256 4.5918E-010 0.02 2.00 106.67
FEM32 1 024 3.7550E-010 0.12 6.17
  27.08
FEM64 4 096 3.8009E-010 ˆ
0.821 − λ1
λ 6.68 8.06
FEM128 16 384 4.4099E-010
 ˆ
λ2 − λ2 
5.84 7.09
 5.44
FEM256 65 536 3.9651E-010 λ 5.91
 ˆ
34.473 − λ3  5.22
FEM512 262 144 3.7877E-010 ˆ4
194.004 − λ
λ 5.63 2 6.51
d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 19/28

37. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
104
H-PINVIT(1,4) H-inversion
103 H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
102
CPU time in s
101
100
10−1
10−2
10−3
16 32 64 128 EM256 EM512
FEM FEM FEM FEM F F
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28

38. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
104
H-PINVIT(1,4) H-inversion
103 H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
102
CPU time in s
101
100
10−1
10−2
10−3
16 32 64 128 EM256 EM512
FEM FEM FEM FEM F F
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 20/28

39. Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
H-PINVIT(1,d)
Input: M ∈ Rn×n , X0 ∈ Rn×d (X0 X0 = I , e.g. randomly chosen)
T
Output: Xp ∈ R n×d , µ ∈ Rd×d , with MX − X µ ≤
p p
B −1 = (M)−1 or B −1 = L−T L−1
H H H O(n (log n)2 k (c)2 )
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Orthogonalize Xi−1 − B −1 R O(n (log n) k (c)2 )
R := MXi − Xi µ, µ = XiT MXi O(n (log n) k (c))
end
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28

40. Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
H-PINVIT(1,d)
Input: MCompetitive to Rn×d (X0 X0 = I , e.g. randomly chosen)
∈ Rn×n , X0 ∈ MATLAB eigs.
T
Output: Xp ∈ R n×d , µ ∈ Rd×d , with MX − X µ ≤
p p
B −1 = (M)−1 or B −1 = L−T L−1
H H H O(n (log n)2 k (c)2 )
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Orthogonalize Xi−1 − B −1 R O(n (log n) k (c)2 )
R := MXi − Xi µ, µ = XiT MXi O(n (log n) k (c))
end
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28

41. Hierarchical (H-)Matrices PINVIT Numerical Results
Algorithm and Complexity
H-PINVIT(1,d)
Input: M ∈ Rn×n , X0 ∈ Rn×d (X0 X0 = I , e.g. randomly chosen)
T
Output: Xp ∈ R n×d , µ ∈ Rd×d , with MX − X µ ≤
p p
B −1 = (M)−1 or B −1 = L−T L−1
H H H O(n (log n)2 k (c)2 )
T
R := MX0 − X0 µ, µ = X0 MX0
for (i := 1; R F > ; i + +) do
Xi := Orthogonalize Xi−1 − B −1 R O(n (log n) k (c)2 )
R := MXi − Xi µ, µ = XiT MXi O(n (log n) k (c))
end
Expensive.
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 21/28

42. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
2D Laplace over [−1, 1] × [−1, 1], precond.: H-Cholesky decomp.
ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 4.6920E-010 0.01
FEM16 256 4.7963E-010 0.02 2.00 106.67
FEM32 1 024 3.4696E-010 0.08 4.00 27.08
FEM64 4 096 4.6414E-010 0.48 6.00 8.06
FEM128 16 384 3.3206E-010 3.20 6.67 5.44
FEM256 65 536 3.8468E-010 13.90 4.34 5.22
FEM512 262 144 3.1353E-010 62.40 4.49 6.51
d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 22/28

43. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
104
H-PINVIT(1,4) H-Chol. decomp.
103 H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni ))
102
CPU time in s
101
100
10−1
10−2
10−3
16 32 64 128 EM256 EM512
FEM FEM FEM FEM F F
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28

44. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, smallest eigenvalues
104
H-PINVIT(1,4) H-Chol. decomp.
103 H-PINVIT(3,4) O(N(ni ) log ni )
O(N(ni )) MATLAB eigs
102
CPU time in s
101
100
10−1
10−2
10−3
16 32 64 128 EM256 EM512
FEM FEM FEM FEM F F
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 23/28

45. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn , FEM64
·10−2
6
4
·10−2
5 2
0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28

46. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−1 , FEM64
·10−2
6
4
·10−2
5 2
0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28

47. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−2 , FEM64
·10−2
6
4
·10−2
5 2
0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28

48. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, vn−3 , FEM64
·10−2
6
4
·10−2
5 2
0
0
−2
1
−5 −4
−1 0
−0.5 0
0.5 −6
1 −1
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 24/28

49. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1] × [−1, 1], precond.: H-inversion
ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 1.7219E-013 <0.01
FEM16 256 7.2388E-012 0.03 106.67
FEM32 1 024 6.0045E-013 0.12 3.57 27.08
FEM64 4 096 1.0915E-012 0.93 7.48 8.06
FEM128 16 384 1.5655E-012 6.08 6.50 5.44
FEM256 65 536 1.1976E-011 52.40 8.62 5.22
d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only H-PINVIT (without H-inversion)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28

50. Hierarchical (H-)Matrices PINVIT Numerical Results
2D Laplace, inner eigenvalues
2D Laplace over [−1, 1] × [−1, 1], precond.: H-Cholesky decomp.
ti N(ni )
Name ni error ti ti−1 N(ni−1 )
FEM8 64 1.5306E-013 <0.01
FEM16 256 9.8071E-012 0.02 106.67
FEM32 1 024 1.0577E-012 0.05 2.50 27.08
FEM64 4 096 1.5776E-012 0.31 6.20 8.06
FEM128 16 384 2.1416E-012 1.64 5.29 5.44
FEM256 65 536 6.7400E-012 9.73 5.93 5.22
FEM512 262 144 1.5002E-010 545.00 56.01 6.51
d = 4, c = 0.2, eig = 10−4 , N(ni ) = ni (log2 ni ) Csp Cid
Time t only H-PINVIT (without H-Cholesky decomposition)
Max Planck Institute Magdeburg Thomas Mach, Preconditioned Inverse Iteration for Hierarchical Matrices 25/28