This document discusses techniques for solving eigen problems, including the power method, inverse power method, QR decomposition, and QR algorithm. It provides details on implementing these techniques, such as the steps of the QR algorithm and ways to accelerate its convergence like deflation and ad hoc shifts. References are also included.

1. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
Understanding the QR Algorithm
Kenneth Mwangi
Northern Arizona University
December 3, 2008
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

2. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
Table of contents
1 Eigen Problem
2 Power and Inverse Power Method
Power Method
Inverse Power Method
3 QR Decomposition
QR Decomposition
4 QR Algorithm
QR Algorithm
5 Techniques used to Accelerate Convergence
6 References
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

3. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
Introduction to the Eigen Value Problem
The classical mathematical eigenvalue problem is defined as
the solution of the following equation:
Avn = λvn ; n = 1, 2, . . . , N
Eigen Problem reveals the following facts;
1 A is a n matrix either real or complex.
2 n is small (1000,2000,. . . )
3 Calculate all the Eigenvalue. Eigenvectors?
The generalised eigenvalue problem is; Avn = Bλvn
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

4. Eigen Problem
Power and Inverse Power Method
QR Decomposition Power Method
QR Algorithm Inverse Power Method
Techniques used to Accelerate Convergence
References
Power Method
The power iteration is a very simple algorithm.
It does not decompose the matrix and thus it can be used
when A is a large matrix.
Nevertheless, it will find only one eigenvalue (one with the
greatest absolute value) and it may converge only slowly.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

5. Eigen Problem
Power and Inverse Power Method
QR Decomposition Power Method
QR Algorithm Inverse Power Method
Techniques used to Accelerate Convergence
References
The power method algorithm is highlighted below;
u (0) = x0
k=1
Repeat
w = Au (k−1)
λ = entry of u with the largest magnitude.
u (k) = wλ
k ←k +1
until u (k) − u (k−1) <
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

6. Eigen Problem
Power and Inverse Power Method
QR Decomposition Power Method
QR Algorithm Inverse Power Method
Techniques used to Accelerate Convergence
References
Inverse Power Method
Inverse iteration is also an iterative eigenvalue algorithm.From the
power method, the method seeks to improve the performance
While the power method always converges to the largest
eigenvalue, inverse iteration also enables the choice of eigenvalue
to converge to.
The inverse iteration algorithm converges fairly quickly.However,
we can achieve even faster convergence by using a better
eigenvalue approximation each time (Rayleigh quotient iteration).
The inverse iteration algorithm requires solving a linear system at
each step. This would appear to require O(n3 ) operations.
However, this can be reduced to O(n2 ) if we first reduce A to
Hessenberg form.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

7. Eigen Problem
Power and Inverse Power Method
QR Decomposition Power Method
QR Algorithm Inverse Power Method
Techniques used to Accelerate Convergence
References
To get the eigenvalue of A closet to q
Eigenvalue of (A − qI )−1
1 1 1
= λ1 −q , λ2 −q , . . . , λn −q
Suppose λk is the eigenvalue of A closet to q. Thus the
modification on the power algorithm is;
w (k) = (A − qI )−1 u (k−1)
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

8. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
QR Decomposition
QR Decomposition is the factorization of a matrix into an
orthogonal and a right triangular matrix;
A = QR
where Q is an orthogonal matrix (meaning thatQ T Q = I ) and R is
an upper triangular matrix (also called right triangular matrix).
Although the QR factorization is more computationally rigorous
than the LU factorization the method offers superior stability
properties.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

9. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
The QR decomposition is usually calculated using the following
ways;
1 Gram-Schmidt process;
2 Householder reflections; and
3 Givens rotations.
Mostly used process is the Householder reflection because it has
greater numerical stability than the Gram-Schmidt method.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

10. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
Householder Transformation
If X and Y are vectors with the same norm, there exists an
orthogonal symmetric matrix P such that;
Y = PX where P = I − 2WW T and
X −Y
W = X −Y . Since P is both orthogonal and symmetric, it follows
that P −1 = P.
* * * * * * * * * *
* * * * * * * * *
Conclusion; X = * * * * * and Y = * * *
* * * * * * *
* * * * * *
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

11. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
QR Algorithm
Techniques used to Accelerate Convergence
References
QR Algorithm
The overall stability theory for the QR algorithm was given by
Wilkinson’s in 1965 in his work the algebraic eigenvalue problem.
We now investigate a well known and efficient method for finding
all the eigenvalues of a general n × n real matrix; but working with
a general matrix takes many iterations and becomes expensive and
time consuming.
Therefore the QR Method works much faster on special matrices,
hence the reason to decompose the general matrices to a
Hessenberg form.
Hessenberg matrix is one that is ”almost” triangular. To be exact,
an upper Hessenberg matrix has zero entries below the first
subdiagonal.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

12. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
QR Algorithm
Techniques used to Accelerate Convergence
References
QR Algorithm Implementation
To find eigenvalues of a real matrix A using the QR factorization,
we will generate a sequence of matrices A(m) that are orthogonal
and similar to A, the QR proceeds by forming a sequences of
matrices A = A(1) , A(2) , . . . , as follows.
1 A(1) = A is factored as a product A(1) = Q (1) R (1) , where Q (1) is
orthogonal and R (1) is upper triangular.
2 A(2) is defined as A(2) = R (1) Q (1) .
Thus if the eigenvalues satisfy |λ1 | > |λ2 | > . . . > |λ2 |, the iterates
converge to T , an upper triangular matrix with the eigenvalues on
the diagonal.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

13. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
QR Algorithm
Techniques used to Accelerate Convergence
References
QR Algorithm
Given A ∈ R n×n
Define A1 = A
For k = 1, 2, . . . , do
Calculate the QR decomposition Ak = Qk Rk ,
Define Ak+1 = Ak .
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

14. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
Techniques used to Accelerate Convergence
Deflation
Aggressive deflation (due to Francis 1961, Watkins 1995)
Ad hoc shifts
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm

15. Eigen Problem
Power and Inverse Power Method
QR Decomposition
QR Algorithm
Techniques used to Accelerate Convergence
References
References
1 Burden and Faires , Numerical Analysis , 8th Edition
2 Laurene V. Fausett, Applied Numerical Analysis Using MATLAB,
2nd Edition
3 Marco Latini ,The QR Algorithm, Past and Future
4 Wikipedia
5 Dr. Shafiu Jibrin, Numerical Analysis notes.
Kenneth Mwangi Northern Arizona University Understanding the QR Algorithm