Binomial Heaps and Fibonacci Heaps

Advanced
Data
Structures
PRIORITY QUEUES –
BINARY HEAPS,
BINOMIAL HEAPS
FIBONACCI HEAPS

 Instructor
 Prof. Amrinder Arora
 Contact me:
http://www.standardwisdom.com/contact-me/
 Other Links
 Software Blog:
http://sj.standardwisdom.com
 Algorithms Textbook:
http://www.standardwisdom.com/algorithms-book/
LOGISTICS
Heaps and Priority Queues Advanced Data Structures - Arora 2

Advanced Data
Structures
Basics
Record / Struct
Arrays / Linked
Lists / Stacks /
Queues
Graphs / Trees
/ BSTs
Advanced
Trie, B-Tree
Splay Trees
R-Trees
Heaps and PQs
Union Find,
Sets
WHERE WE ARE

 Robert Tarjan – Princeton
 Rada Mihalcea – UNT
 Dan Melamed – NYU
 Ata Türk – Bilkent Univ.
CREDITS

 Ice cream costs vary by flavor: $3 (vanilla), $4
(strawberry), or $5 (chocolate). All flavors of ice
cream are equally likely.
 Average cost = $4.
 Ice cream costs $10. Every 10th ice cream
additionally gives you a $60 voucher
(redeemable in cash).
 Amortized cost = $4.
BUT FIRST, AN ICE CREAM

 Ice cream costs $1
 Every 10th ice cream has a surcharge of $10.
 Amortized cost = $2
AMORTIZED VS. AVERAGE (CONT.)

 A binary tree T that satisfies two properties:
 MinHeap: key(parent)  key(child)
 [OR MaxHeap: key(parent)  key(child)]
 All levels are full, except the last one, which is left-filled
HEAPS – BINARY TREES

 To implement priority queues
 Priority queue = a queue where all elements have a
“priority” associated with them
 Remove in a priority queue removes the element
with the smallest priority
 insert
 removeMin
WHAT ARE HEAPS USEFUL FOR?

 A heap T storing n keys has height h = log(n), which
is O(log n). [The statement is true for almost
complete binary trees in general.]
HEAP PROPERTIES

 Using arrays.
 If indexed from 1: Parent = k ; Children = 2k , 2k+1
 Efficient! [No pointers. Just array indexes.]
HEAP IMPLEMENTATION
[4]
6
12 7
1918 9
6
9 7
10
30
31
[1]
[2] [3]
[5] [6]
[1]
[2] [3]
[4]
[1]
[2]

 Insert 3
HEAP INSERTION

 Just insert it at the first empty slot.
 Upheap (Shift the key up), if necessary
HEAP INSERTION (CONT.)

 Continue the upheap process, until:
 Either the key is smaller than the parent,
 Or it becomes the root. [We have a new minimum]
HEAP INSERTION (CONT.)

 Remove element from priority queues – removeMin()
or extractMin()
 Remove the root, replace with the last element.
 “Downheap” (Swap the node with the smaller of the
child nodes) if necessary
HEAP REMOVAL

 Terminate downheap when
 reach leaf level
 key parent is greater than key child
HEAP REMOVAL (CONT.)

 One can build a heap simply by doing n inserts.
 This takes O(n log n) time. (Why?)
 Instead, we can build a heap faster, in O(n) time
 Build it from ground up
BUILDING A HEAP

 Step 1: Build a heap: O(n) time
 Step 2: removeMin() n times
 Running time: O(n log n)
 Heap sort is generally quite efficient, and competes
effectively against quick sort. (Both heap sort and
quick sort are significantly faster than merge sort,
although of course, not in asymptotic terms.)
HEAP SORTING

Operation Binary Heap Tree
Insert O(log n)
Extract Minimum O(log n)
Find Minimum O(1)
HEAPS, SUMMARY
It is not easy to merge two binary heaps, especially if they
are of different sizes.
If we decrease the key of a node, upheap can be done in
O(log n) time.

Union,
Decrease
Key,
Random
Delete
MORE ACTIONS ON
HEAPS

 MAKE-HEAP ()
 Creates & returns a new heap with no elements.
 INSERT (H,x)
 Inserts a node x whose key field has already been filled into heap H.
 MINIMUM (H)
 Returns a pointer to the node in heap H whose key is minimum.
 EXTRACT-MIN (H)
 Deletes the node from heap H whose key is minimum.
 DECREASE-KEY (H, x, k)
 Decreases the key value of node x within heap H the new value k
 DELETE (H, x)
 Deletes node x from heap H.
 UNION (H1, H2)
 Creates and returns a new heap that contains all nodes of heaps H1
& H2.
 Heaps H1 & H2 are destroyed by this operation
MERGEABLE HEAPS
As
before
New
Oper-
ations

 A binomial heap is a collection of binomial trees.
 The binomial tree Bk is an ordered tree defined
recursively
 B0 – Consists of a single node
 Bk – Consists of two binominal trees Bk-1 linked together. Root
of one is the leftmost child of the root of the other.
BINOMIAL TREES

BINOMIAL TREES
B k-1
B k-1
B k
Bo

BINOMIAL TREES
B4
B0 B1 B2
B
1
B1
B3
B2
B1
B0
B3 B2
B1
B0

BINOMIAL TREES
Bk
Bk-2
Bk-1
B2
B1
Bo

For the binomial tree Bk
1. There are 2k nodes,
2. The height of tree is k,
3. There are exactly C(k,i) nodes at depth i for i=
0,1,..,k and
4. The root has the highest degree, k. This is higher
than the degree of any other node
5. Children of the root are the roots of subtrees B0, B1
B2, B3 … Bk-1
PROPERTIES OF BINOMIAL TREES

 PROOF: By induction on k
PROPERTIES OF BINOMIAL TREES (CONT.)

 COROLLARY: The maximum degree of any node in an
n-node binomial tree is lg(n)
 The term BINOMIAL TREE comes from the 3rd
property. C(k,i) is the binomial coefficient.
PROPERTIES OF BINOMIAL TREES (CONT.)

 A BINOMIAL HEAP H is a set of BINOMIAL TREES that
satisfies the following “Binomial Heap Properties”
 Each binomial tree in H is HEAP-ORDERED
 The key of a node is ≥ the key of the parent
 Root of each binomial tree in H contains the smallest key in
that tree.
BINOMIAL HEAPS

In a consolidated binomial heap, there is at most one
binomial tree in H whose root has a given degree.
 n-node binomial heap H consists of at most [lgn] + 1 binomial
trees. (Binary represantation of n has lg(n) + 1 bits)
BINOMIAL HEAPS (CONT.)

 Example: A binomial heap with n = 13 nodes
 3 2 1 0
 13 =< 1, 1, 0, 1>2
 Consists of B0, B2, B3
 head[H]
BINOMIAL HEAPS
10
25
1
12
18
6
298
38
14
27
1711
B0
B2
B3

 Each binomial tree within a binomial heap is stored
in the left-child, right-sibling representation
 Each node X contains POINTERS
 p[x] to its parent
 child[x] to its leftmost child
 sibling[x] to its immediately right sibling
 Each node X also contains the field degree[x] which
denotes the number of children of X.
REPRESENTATION OF BINOMIAL HEAPS

REPRESENTATION OF BINOMIAL HEAPS
HEAD [H]
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
10
parent
degree
key
child
sibling
ROOT LIST (LINKED LIST)

 Lazy Approach – Consolidate only when needed (data
structure can break some properties at times)
 Eager Approach – Consolidate often (keep the data
structure clean at all times)
CONSOLIDATION – LAZY VS. EAGER

 Lazy Approach
 Insert – Simply add to the list and update minimum if needed
 Union – Simply combine the list using pointers, and update
minimum if needed
 Delete – Delete the minimum, merge the lists and consolidate
 Eager Approach
 Insert – Create a new heap, union and consolidate
 Union – Combine the lists and consolidate
 Delete – Delete the minimum, merge the lists and consolidate
CONSOLIDATION, IN CONTEXT OF
BINOMIAL HEAPS

 Insert – O(log n) time to consolidate the lists
 Union – O(log n) time to consolidate the lists
 Delete – O(log n) time to consolidate the lists
BINOMIAL HEAP (EAGER APPROACH)
OPERATIONS

 Use a potential function: f(H) = trees(H), which is simply the
number of trees in H.
 Insert
 Actual cost: O(1)
 Change in potential function = 1.
 Amortized cost: O(1)
 Union
 Actual cost: O(1)
 Change in potential function = 0.
 Amortized cost: O(1)
 Delete
 Actual cost = O(number of trees in the heap + max degree)
= # of trees + O(log n)
 Change in potential function = O(log n) - # trees
 Amortized cost: O(log n) + O(max degree) = O(log n)
BINOMIAL HEAP (LAZY APPROACH) -
OPERATIONS

Dijkstra
and Prim
DECREASE KEY – A
CENTRAL PROBLEM

 Decrease key operation appears frequently in two
common algorithms:
 Dijkstra’s algorithm for Shortest Path
 Prim’s algorithm for Minimum Spanning Tree
 The data structures studied so far either do not support
the decrease key operation, or can be made to support it
in O(log n) time.
 Fibonacci heaps provide decrease key operation in O(1)
amortized time.
 This changes the time of those two algorithms from O(m
log n) to O(m + n log n), where m is the number of edges
and n is the number of vertices in the graph.
DECREASE KEY

 History. [Fredman and Tarjan, 1986]
 Ingenious data structure and analysis.
 Original motivation: improve Dijkstra's shortest path algorithm
from O(E log V ) to O(E + V log V ).
 Basic idea.
 Similar to binomial heaps, but less rigid structure.
 Uses the concept of a “mark” to support decrease key
operation
 Historical perspective/question: While now a days we
study both lazy and eager versions of binomial heaps, it
is this researcher’s (Arora’s) belief that binomial heaps
essentially meant eager version before the appearance
of Fibonacci heaps.
FIBONACCI HEAPS

 Fibonacci heap.
 Very similar to Binomial heap, it is a linked list of heap-ordered
trees.
 Maintain pointer to minimum element.
 Set of “marked” nodes (To be explained shortly)
FIBONACCI HEAPS: STRUCTURE
723
30
17
35
26 46
24
Heap H
39
4118 52
3
44
roots heap-ordered tree

 Notation.
 n = number of nodes in heap.
 rank(x) = number of children of node x.
 rank(H) = max rank of any node in heap H.
 trees(H) = number of trees in heap H.
 marks(H) = number of marked nodes in heap H.
FIBONACCI HEAPS: NOTATION
723
30
17
35
26 46
24
39
4118 52
3
44
rank = 3 min
Heap H
trees(H) = 5 marks(H) = 3
marked
n = 14

 We will use amortized analysis using potential
function technique
 Potential of heap H, (H) = trees(H) + 2  marks(H)
FIBONACCI HEAPS: POTENTIAL
FUNCTION
723
30
17
35
26 46
24
(H) = 5 + 23 = 11
39
4118 52
3
44
min
Heap H
trees(H) = 5 marks(H) = 3
marked

 Insert.
 Create a new singleton tree.
 Add to root list; update min pointer (if necessary).
FIBONACCI HEAPS: INSERT
723
30
17
35
26 46
24
39
4118 52
3
44
21
insert 21
min
Heap H

 Insert.
 Create a new singleton tree.
 Add to root list; update min pointer (if necessary).
FIBONACCI HEAPS: INSERT
39
41
723
18 52
3
30
17
35
26 46
24
44
21
min
Heap H
insert 21

 Actual cost. O(1)
 Potential of heap H, (H) = trees(H) + 2  marks(H)
 Change in potential. +1
 Amortized cost. O(1)
FIBONACCI HEAPS: INSERT ANALYSIS
39
41
7
18 52
3
30
17
35
26 46
24
44
2123
min
Heap H

 Linking operation. Make larger root be a child of
smaller root.
LINKING OPERATION
39
4118 52
3
4477
56 24
15
tree T1 tree T2
39
4118 52
3
44
77
56 24
15
tree T'
smaller rootlarger root still heap-ordere

 Union. Combine two Fibonacci heaps.
 Representation. Root lists are circular, doubly linked
lists.
FIBONACCI HEAPS: UNION
39
41
717
18 52
3
30
23
35
26 46
24
44
21
min min
Heap H' Heap H''

 Union. Combine two Fibonacci heaps.
 Representation. Root lists are circular, doubly linked
lists.
39
41
717
18 52
3
30
23
35
26 46
24
44
21
min
Heap H

 Actual cost. O(1)
 Change in potential. 0
(H) = trees(H) + 2  marks(H)
potential function
39
41
717
18 52
3
30
23
35
26 46
24
44
21
min
Heap H

 Delete min.
 Delete min; meld its children into root list; update min.
 Consolidate trees so that no two roots have same rank.
FIBONACCI HEAPS: DELETE MIN
39
4118 52
3
44
1723
30
7
35
26 46
24
min

 Delete min.
 Delete min; meld its children into root list; update min.
 Consolidate trees so that no two roots have same rank.
 This can be done using something like counting sort.
39
411723 18 52
30
7
35
26 46
24
44
min

39
411723 18 52
30
7
35
26 46
24
44
min current

39
411723 18 52
30
7
35
26 46
24
44
0 1 2 3
currentmin
rank

39
411723 18 52
30
7
35
26 46
24
44
0 1 2 3
min current
rank

39
411723 18 52
30
7
35
26 46
24
44
0 1 2 3
min
current
rank

39
411723 18 52
30
7
35
26 46
24
44
0 1 2 3
min
current
rank
link 23 into 17

39
4117
23
18 52
30
7
35
26 46
24
44
0 1 2 3
min
current
rank
link 17 into 7

39
417
30
18 52
17
35
26 46
24
44
0 1 2 3
23
current
min
rank
link 24 into 7

39
417
30
18 52
23
17
35
26 46
24 44
0 1 2 3
min
current
rank

7
30
52
23
17
35
26 46
24
min
3941
18
44
stop

 Actual cost. O(rank(H)) + O(trees(H))
 O(rank(H)) to meld min's children into root list.
 O(rank(H)) + O(trees(H)) to update min.
 O(rank(H)) + O(trees(H)) to consolidate trees.
 Change in potential. O(rank(H)) - trees(H)
 trees(H' )  rank(H) + 1 since no two trees have same rank.
 Child nodes from the min root in H are moved to the list of roots in H’
and are no longer marked.
 That is, marks(H’)  marks(H)
 That is, (H)  trees(H’) – trees(H)
 That is, (H)  rank(H) + 1 – trees(H).
 Amortized cost. O(rank(H)).
[We can scale the constants so that the constant of O notation
matches 1.]
ANALYSIS

 Is amortized cost of O(rank(H)) good?
 Yes, if rank(H)  O(log n)
 Therefore, our goal in Fibonacci trees is to bound
rank(H). If only insert and delete-min operations, all
trees are binomial trees and rank(H) is automatically
bounded by lg(n).
 We need to implement decrease key operation so
that rank(H) = O(log n), as well as cost of decrease
key is O(1).
ANALYSIS

 Intuition for deceasing the key of node x.
 If heap-order is not violated, just decrease the key of x.
 Otherwise, cut tree rooted at x and meld into root list.
 To keep trees flat: as soon as a node has its second child cut,
cut it off and meld into root list (and unmark it).
FIBONACCI HEAPS: DECREASE KEY
24
46
17
30
23
7
88
26
21
52
39
18
41
38
7235
min
marked node:
one child already cut

 If the parent node is already marked and the child
node is cut, then we need to cut the parent node
also, mark it’s parent. This is done recursively.
 Suppose this continues for “c” levels. Then, the
actual cost of decrease key operation is O(c).
FIBONACCI HEAPS: DECREASE KEY (CONT.)

 Actual cost. O(c)
 O(1) time for changing the key.
 O(1) time for each of c cuts, plus melding into root list.
 Change in potential. O(1) - c
 trees(H') = trees(H) + c.
 marks(H')  marks(H) - c + 2.
   c + 2  (-c + 2) = 4 - c.
This explains/motivates the potential function which
puts twice as many units on marks as on trees.
FIBONACCI HEAPS: DECREASE KEY
ANALYSIS

 Lemma. Fix a point in time. Let x be a node, and let y1, …, yk
denote
its children in the order in which they were linked to x. Then:
 Proof:
 When yi was linked into x, x had at least i -1 children y1, …, yi-1.
 Since only trees of equal rank are linked, at that time
rank(yi) = rank(xi)  i - 1.
 Since then, at most one child cut from yi, or it would have been cut
 Thus, right now rank(yi)  i - 2.
FIBONACCI HEAPS: BOUNDING THE
RANK

rank (yi ) 
0 if i 1
i 2 if i 1



x
y1 y2 yk…

• We want to bound how many nodes can be cut from a
tree before it loses a “rank”.
• Definition: Let Fk be smallest possible tree of rank k
satisfying property.
RANK (CONT.)
F0 F1 F2 F3 F4 F5
1 2 3 5 8 13

• We want to bound how many nodes can be cut from a
tree before it loses a “rank”.
• Definition: Let Fk be smallest possible tree of rank k
RANK (CONT.)
F4 F5
8 13
F6
8 + 13 = 21

 Definition. Let Fk be smallest possible tree of rank k
 Fibonacci fact. Fk  k, where  = (1 + 5) / 2 
1.618.
 Corollary. rank(H)  log n .
RANK

 Delete node x.
 decrease-key of x to -.
 delete-min element in heap.
 Amortized cost. O(rank(H))
 O(1) amortized for decrease-key.
 O(rank(H)) amortized for delete-min.
FIBONACCI HEAPS: RANDOM DELETE

PRIORITY QUEUES PERFORMANCE COST
SUMMARY
† amortizedn = number of elements in priority queue
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
Heap
log n
1
log n
n
log n
log n
1
Binomial
Heap
log n
log n
log n
log n
log n
log n
1
Fibonacci
Heap †
1
1
log n
1
1
log n
1
Linked
List
1
n
n
1
n
n
is-empty 1 1 11

Advanced Data
Structures
Basics
Record / Struct
Arrays / Linked
Lists / Stacks /
Queues
Graphs / Trees
/ BSTs
Advanced
Trie, B-Tree
Splay Trees
R-Trees
Heaps and PQs
Union Find,
Sets
WHERE WE ARE

 Algorithmic Puzzles
 http://www.slideshare.net/amrinderarora/algorithmi
c-puzzles
READ MORE

 Tries – An Excellent Data Structure for Strings
 http://www.slideshare.net/amrinderarora/tries-
string
READ EVEN MORE

Binomial Heaps and Fibonacci Heaps

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Binomial Heaps and Fibonacci Heaps

Semelhante a Binomial Heaps and Fibonacci Heaps (20)

Mais de Amrinder Arora

Mais de Amrinder Arora (20)

Último

Último (20)

Binomial Heaps and Fibonacci Heaps