Design and Analysis of Algorithms

1. CHIPTEB 7
Polynomials and the FFT
—=C;=——=€;=——=C;=——«;’“;=——=Q=——=Q=——=C;=——q:=r———4;=n—
2.3.1. Polynomials
A polynomial in the variable x over an algebraic fieldF is a function A(x)
that can be represented as follows:
.-I—|
Alfx] = ELM! .
.I 0
We call n the degree-bound of the polynomial, and we call the values a0, a1,
. ., an - l the coefficients of the polynomial. The coefficients are drawn from the
field F, typically the set C of complex numbers. A polynomial A(x) is said to
have degree k if its highest nonzero coefficient is 011.. The degree of a polynomial of
degree-boundn can be any integer between 0 andn - 1, inclusive. Conversely, a
polynomial of degree k is a polynomial of degree-bound n for any n > k.
There are a variety of operations we might wish to define for polynomials.
For polynomial addition, if A (x) andB (x) are polynomials of degree-bound n, we
say that their sum is a polynomial C (x), also of degree-bound n, such that C (x) :
A (x) + B (x) for all x in the underlying field. That is, if
.‘I—l
AU} = E of?"
JI:-.'L"
and
.'.'—]
8-inch = Z opt! .
_r II
then
:.-—|
("it“) = Z :rI.I-' ,
Fr.
where cj=aj+bjforj= 0,1, . . .,n- 1. For example, ifA(x) = 6x3+ 7X2- le+ 9
and B(x) = -2x3+ 4x - 5, then C(x) = 4x3+ 7x2- 6x + 4.
For polynomial multiplication, if A(x) and B(x) are polynomials of degree-
bound n, we say that their product C(x) is a polynomial of degree-bound 2n - 1 such
that C(x) =A(x)B(x) for all x in the underlying field. You have probably multiplied
polynomials before, by multiplying each term in A(x) by each term in B(x) and

2. combining terms with equal powers. For example, we can multiply A(x) = 6x3 + 7x2 -
10x + 9 and B(x) = -2x3+ 4x - 5 as follows:
6x3+ 7x2- 10x + 9
- 2x3 + 4x - 5
- 30x3- 35x2+ 50x - 45
24x4+ 28x3- 40x2+ 36x
- 12x6- 14x5+ 20x4 -18x3
- 12x6- 14x5+ 44x4- 20x3- 75x2+ 86x - 45
Another way to express the product C (x) is
3.1.";
Clfix} = Z c,.-.r-' .
3:0
where
c, — Elam): ,1. _
11"”)
Note that degree(C) = degree(A) + degree(B), implying
degree-bound(C) = degree-bound(A) + degree-bound(B) - 1
5 degree-bound(A) + degree-bound(B).
We shall nevertheless speak of the degree-bound of C as being the sum of
the degree-bounds of A and B, since if a polynomial has degree-bound kit also has
degree-bound k + 1.
2.3.2. Representation of polynomials
The coefficient and point-value representations of polynomials are in a
sense equivalent', that is, a polynomial in point-value form has a unique
counterpart in coefficient form.
Coefficient representation
A coefficient representation of a polynomial of degree-bound n is a
vector of coefficients a = (a0, a1, . . ., an-1). In matrix equations in this chapter,
we shall generally treat vectors as column vectors.

3. A(x) = an + alx + azxz +---+ an_1x"'1
3(1): bc| + blx + bzxz + ---+ bn_1x"‘1
Add : O(n) arithmetic operation
A(x) + B[x) = (a0 + bu ) + (a1 + b1)x + ---+ (an_1 + bn_1)x"_1
Evaluate. O (11) using Horner’s method
A(x) = no + (36(01 + 1:012 + -“+ I (an—2 + x (an—1119)
Multiply. 0 (n2) using Brute force algo
211—2 i
A(x) x 3(x) = Z cL-xi , where Ci = Z ajbi_j
5:0 jzo
Point-value representation
Fundamental theorem of algebra: A degree n polynomial with complex
coefficients has exactly 11 complex roots.
Corollary. A degree n — 1 polynomial A(x) is uniquely specified by its
evaluation at n distinct values of x
A(x) = (Ind/01 , (xfl—lJyfl—l)
B (x) = (xurzu ), : (In—1: Zn—l)
Add. O(n) arithmetic operations.
A(x) + BC?!) = (150.370 + 20). "':(xn-1:yn-1 + Zn-l)
Multiply (convolve). E) (11), but need 2n — 1 points.
A(x) X 305) = (350.310 X 20): "'. (x2n—11y2n—1 X 2211—1)
Evaluate. O(nZ) using Lagrange’s fomiula.
11—1 [.1 . x 7 x-
Arx) = y (—J"‘ ’ )2:” k Hjtk xk 7 xi
Theorem. For any set
Converting between two representations: brute force
Point-value => coefficient

4. x0, ---.xn_1 a0 + alx + + an_1x”“1Given n distinct points , Evaluate it at n
distinct points
l :01 XS - - - xfif ' .51.; 1'11.
1 x1 .1; - - - 1'1"] 111 1"
l .1'_.,._-_ .1'_-_ I - - - -"5.-li— ' Gin—I 1"11—:J
Running time. 1:9 (n2) for matrix-vector multiply (or n Horner’s)..
Coefficient => point-value.
Given n distinct points x0,-~,xn_1 and values 3’0""!3’11-1 , find unique
n—1
polynomial a0 + “ix + + an_1x ,that has given values at given points.
J. In Ill]: ' ' ' IE I (In 'L'.;_.
1 x. .1; - - - .1":"'] £11 ,1-'1
l .1'_.,._-_ .1'_§_| - - - .1;_,":'_ ' an—I 3"11—'.1
115.111: is him-eth'ble ifi'xi dishhcl
Running time. 9 (n3) for Gaussian elimination.
Theorem. For any set {(xo, yo), (x1,y1), . . ., (xn-1,yn-1)} of n point-value pairs,
there is a unique polynomial A(x) of degree-bound n such that y, = A(xk) for k =
0,1,...,n-1.
Proof: write out A(xo) = y0,A(x1) = yl... A(xn.1) = yn.1 as
" .I.' 'I
l :01 .15 - - - EU .51.; 11-,
I x. .1; - - - .1'f'] £11 .1-'1
l .1'_..._-_ .1'Ifi_| - - - 351':— J' 61.1— | 111—:
The matrix on the left is denoted V(x0,x1, . . .,x,,—1) and is known as a
Vandermonde matrix. this matrix has determinant
n11. — 1.1 .
Determinant (x0, x1, . . ., xn-l) ¢ 0 since all the x’s are distinct. So the system
has a solution
A: V (x0, x1, . - .1 xn-1)'1y

5. So the a’s can be uniquely found. Solving such alinear system takes'FJI (n3) tiem,
but we can interpolate faster.
LaGrange formula: given point — value
-'l | H” — X1]
1111:. — "‘ 111—- -—. -:1; Hum. -— x.)
1-1611
Can show that A (x) is a degree bound n polynomial and A(x.) = y1. this takes E)
(n2) teim
A(X) a {(xflryo): (X1,y1), ' ' ':(x"_12y"-1)}
B (X) 9{(x0,y’o), (x1,y’1), - - .1 (xn-l, Y'n-1)}
(note thatA and B are evaluated at the some n points), then a point-value
representation for
C (X) 9{(X01y0+ Y0): (X1: y1+Y'1)= - - ., (Xn.1,yn.1+y’1.1)}.
And conversion takes 0 (n) time. What about multiplication of A (x) and B (x)
both degree bound n polynomials?
C(x) : A(x)B(x) is degree bound 2n—1 , so we must augment the number of point
value pairs used to A (x) and B (x)
A(X) a {(xoryo): (X1,y1), ' ' '2 (x"_1:y"'1)}
B (X) 9 {(Xoa Y0): (X1: Y1): - - -= (Kn-1: Y'n-1)}
C (>09 {(xO,yoy’o),(xl,y1y’1),...,(X2n.1,y2n.1y’2n.1)}
Is 1-) (n) time.if we can find a fast way to convert coefficients of point value
pairs, use the E) (n) multiplication of point value pairs, and convert back, we
will have a very fast algorithm. We do so by choosing the points for the point
value pairs very carefully. These points are complex root of unity.
To multiply A(x)B(x) we:
1. Use the coefficient representations, but up to degree bound 2n by
2 Compute the point value of A(x) and B(x) at eh 2n roots of unity via FFT.
3. Do print wise multiplication
4 lnterpole back from the 2“‘11 roots of unity to a coefficient representation via
inverse DFT

6. — I ‘ ... .
‘1? '21' "" “11—! ' Omlinary I11u|rip|iL'a‘.io11 lg. r r.” 1 ' .5 (.tmllmlcnr
' . - . "i '9' '1’ -“—- -' rcprusenl'clhom
ha- i’[‘ _' 5.1.4 . Tum.- L'Ju! ) _ . J
A
111-1111111111 Inlurpnlafion
Time (:Jfri lg n} 3 Time 81.11 '.g n}
. g I'
111111;). 1111111.) (1.5.1; .1J |
' " i . — 1 - .
A(EUJFJ.B(M2INJ I’minlwiw11)1J|[ipllt'a:[l[m {(193,} '. Fowl-value
__ - —" Tin 0|”. )- - '3' l'cpl'cscnwnmn-e
.' =3 ‘ . .1 :
— "17 2 |
111%}: 11.311135» ‘1 .153sz: 1 |
This figure shows a graphical outline of an efficient polynomial-
multiplication process. Representations on the top are in coefficient form, while
those on the bottom are in point-value form. The arrows from left to right
correspond to the multiplication operation. The W2“ terms are complex (2n)th
roots of unity.
2.3.3. The DFT and FFT:
A complex nth root of unity is a complex number w such that
w" = .
There are exactly n complex nth roots of unity', these are e2 H W" for k = 0, 1, . . . , n -
1. To interpret this formula, we use the definition of the exponential of a complex
number:
ei" = cos(u) + 1' sin(u)
-. I“J4; (HE
w; cuff: 1.1.1
11 | '.'
.' I’J) . _ . . (U
The values of ”3‘ “- ’ 9‘
principal 8th root of unity.
The figure shows that the n complex roots of unity are equally spaced around the
circle of unit radius centered at the origin of the complex plane. The value
in the complex plane, where ws = e211 i/8 is the

7. w" = e217 i/n
The DFT (Discrete Fourier Transform)
Recall that we wish to evaluate a polynomial
P: --|
A(x) = 2: 1:13.114
i=0
[1 . I 2 11—1 _
Of degree-bound n at (”R‘- u 1-" a)“, ' ' ' -' m" (that 1s, at the n complex nth roots
of unity).2 Without loss of generality, we assume that n is a power of 2, since a
given degree-bound can always be raised--we can always add new high-order
zero coefficients as necessary. We assume thatA is given in coefficient
form: a = (a0, a1, . . . , an.1). Let us define the resultsyk, fork = 0, 1,. . . , n - 1,
by
11-.- = A1111:-
H l _
—- Z (.1;wa .
_,I'=IiJ
The vector y = (yo,y1, . . . ,y,,-1) is the Discrete Fourier Transform (DFT) of the
coefficient vector or = (a0, a1, . . . , an-l). We also writey = DFTn(a).
2The lengthn is actually what we referred to as Zn in previous section, since we
double the degree-bound of the given polynomials prior to evaluation. In the context
of polynomial multiplication, therefore, we are actually working with complex
(2n)th roots of unity.+
The FFT (Fast Fourier Transform)
By using a method known as the Fast Fourier Transform (FF 1), which
takes advantage of the special properties of the complex roots of unity, we can
compute DFT,,(a)in time E) (n lg n), as opposed to the 1-3012) time of the
straightforward method.
The FFT method employs a divide-and-conquer strategy, using the even-
index and odd-index coefficients of A(x) separately to define the two new degree-
bound n/Z polynomials A[°](x) and A[”(x):
A[0](x) = a0+a2x+a4x2+¢+¢+an-ZX”/2-1,
A[1](x) = a1+a3x+a5x2+***+an-1x”/2-1.

8. Note thatAm contains all the even-index coefficients of A (the binary representation
of the index ends in 0) andAm contains all the odd-index coefficients (the binary
representation of the index ends in 1). It follows that
14(16): 401062 ) + xAmOC2 )1
C‘ m' ....w"'"
so that the problem of evaluating A(x) at 111 ... ' LI reduces to
1. Evaluating the degree-bound n/Z polynomials A[°](x) and A[”(x) at the points
[cufi ]3. Ii :11]. 1], . . . , (raj:— ! 3'
And then combining the results according to the equation
By the halving lemma, the list of values consists not of n distinct values
but only of the n/Z complex (n/2)th roots of unity, with each root occurring
exactly twice. Therefore, the polynomials Am] and AH] of degree-bound n/Z are
recursively evaluated at the n/Z complex (n/2)th roots of unity. These
subproblems have exactly the same form as the original problem, but are half
the size. We have now successfully divided an n-element DFT" computation
into two n/Z-element DFTn/Z computations. This decomposition is the basis for
the following recursive FFT algorithm, which computes the DFT of an n-
element vector or = (a0, a1, . . . , an.1), where n is a power of 2.
Algorithm
RECUszE—FFHG}
1 n <— lerrgzhm] 1} n is a power of 2.
’2 if n : l
3 then return a
4 (11),: (_ €2.11”
5 w 1— 1
6 .1110] <— (£111,113... .,1’.’i,._9::|
7 allli—(flhafinqflnbfl
8 1'1”] 1— RECURSIvfi-FFTfalfllj
9 ym 1— RECURSIVE-mmmj
10 for 11’ — CF in 111,32 ---1
l 1 do yk .— ){LC'} + wyL”
L2 3J1: ! 1371"?) ‘_ yLUJ _ myil]
13 an 1— U) (1)..
[4 return 1‘ 1:1 y is assumed to be column vector.
The RECURSIVE-FFT procedure works as follows. Lines 2-3
represent the basis of the recursion', the DFT of one element is the element
itself, since in this case

9. yo = flow?
= ng'l
= {11].
Lines 6-7 define the coefficient vectors for the polynomials Am] and Am. Lines
4, 5, and 13 guarantee that w is updated properly so that whenever lines 11-12
are executed, (1.1 _ m" . (Keep1ng a running value ofwfrom 1terat10n to
iteration saves time over computing 50¢. from scratch each time through
the for loop.) Lines 8-9 perform the recursive DFTn/Z computations, setting,
fork=0,1,...,n/2-1,
'va = 11101111111,
3’1” = Amiwfigl.
or, since 51):” 2 1:11;," by the cancellation lemma,
1111 = 11011-1111.
11” = 111111111.
Lines 11-12 combine the results of the recursive DFTn/Z calculations.
Fory0,y1, . . . ,yn/Z - 1, line 11 yields
0 _ l
111 = 3111+ €111.11]
11111113111 wiA'WwfiH
A((ufi) .
where the last line of this argument follows from equation (32.9). For yn/Z, "/2+1, . .
.,y,,- 1, letting k= 0,1,. . ., n/Z - 1, line 12 yields
l'J') .
.l‘l-‘(H'PL-‘E: ' y}: I — Will;
= 1:10] I Willa-Ill ,".|
_Ir .1
= .zll'iil-iwi'“) - w: ' 3""-"‘1:'A:' '[cufi’i '1.
: ,1[-:1]IILII2_I.1+1-I I a): I '-"""3:.»1“aft-135'":‘1
: I,I.IIL.II:1I—::.I.-'2_III
The second line follows from the first since rm, I I I : fir” . The fourth line
_ .1. __ _ _ (11 = .11...- _
follows from the third because l(“i-'1 1 1mphes A)" (1)” . The last hne

10. follows from equation (32.9). Thus, the vector yreturned by RECURSIVE-FFT is
indeed the DFT of the input vector a.
To determine the running time of procedure RECURSIVE-FFT, we note that
exclusive of the recursive calls, each invocation takes time 1-3 (n), where n is the
length of the input vector. The recurrence for the running time is therefore
T(n) = 2T(n/2) + B(n)
= E) (n lg n) .
Thus, we can evaluate a polynomial of degree-bound n at the complex nth roots of
unity in time B1 (n lg n) using the Fast Fourier Transform.
The iterative F F T implementation runs in time E) (n lg n). The call to BIT-
REVERSE-COPY(a,A) certainly runs in O(n lg n) time, since we iterate n times and
can reverse an integer between 0 and n - 1, with lg nbits, in 0(lg n) time.
Egu 1" |
1.111} — Z E ;
.1=| '='::'.-
Ig -'l
: —- H__12,1-|
.2:131.1:
_ Z:_4.
1-
— @{nlgn}.
:5