SlideShare a Scribd company logo
1 of 105
Download to read offline
Challenge the future
Delft
University of
Technology
Course IN4303
Compiler Construction
Eduardo Souza, Guido Wachsmuth, Eelco Visser
LL Parsing
Traditional Parsing Algorithms
lessons learned
LL Parsing
Recap: Lexical Analysis
What are the formalisms to describe regular languages?
• regular grammars
• regular expressions
• finite state automata
Why are these formalisms equivalent?
• constructive proofs
How can we generate compiler tools from that?
• implement DFAs
• generate transition tables
2
today’s lecture
LL Parsing
Overview
3
today’s lecture
LL Parsing
Overview
efficient parsing algorithms
• predictive/recursive descent parsing
• LL parsing
3
today’s lecture
LL Parsing
Overview
efficient parsing algorithms
• predictive/recursive descent parsing
• LL parsing
grammar classes
• LL(k) grammars
• LR(k) grammars
3
today’s lecture
LL Parsing
Overview
efficient parsing algorithms
• predictive/recursive descent parsing
• LL parsing
grammar classes
• LL(k) grammars
• LR(k) grammars
more top-down parsing algorithms
• Parser Combinators
• Parsing Expression Grammars
• ALL(*)
3
LL Parsing
Predictive (Recursive Descent)
Parsing
I
4
formal languages
LL Parsing 5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
5
Recap: A Theory of Language
formal languages
LL Parsing
vocabulary Σ
finite, nonempty set of elements (words, letters)
alphabet
string over Σ
finite sequence of elements chosen from Σ
word, sentence, utterance
formal language λ
set of strings over a vocabulary Σ
λ ⊆ Σ*
5
Recap: A Theory of Language
formal grammars
LL Parsing 6
Recap: A Theory of Language
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)*
N (N∪Σ)*
× (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)*
N (N∪Σ)*
× (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
nonterminal symbol
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)*
N (N∪Σ)*
× (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
context
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)*
N (N∪Σ)*
× (N∪Σ)*
start symbol S∈N
6
Recap: A Theory of Language
replacement
formal grammars
LL Parsing
formal grammar G = (N, Σ, P, S)
nonterminal symbols N
terminal symbols Σ
production rules P ⊆ (N∪Σ)*
N (N∪Σ)*
× (N∪Σ)*
start symbol S∈N
grammar classes
type-0, unrestricted
type-1, context-sensitive: (a A c, a b c)
type-2, context-free: P ⊆ N × (N∪Σ)*
type-3, regular: (A, x) or (A, xB)
6
Recap: A Theory of Language
formal languages
LL Parsing 7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)*
× (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*
:
w=u p v ∧ w’=u q v
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)*
× (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*
:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ*
| S G
*
w}
7
Recap: A Theory of Language
formal languages
LL Parsing
formal grammar G = (N, Σ, P, S)
derivation relation G ⊆ (N∪Σ)*
× (N∪Σ)*
w G w’
∃(p, q)∈P: ∃u,v∈(N∪Σ)*
:
w=u p v ∧ w’=u q v
formal language L(G) ⊆ Σ*
L(G) = {w∈Σ*
| S G
*
w}
classes of formal languages
7
Recap: A Theory of Language
recursive descent
LL Parsing 8
Exp → “while” Exp “do” Exp
Predictive parsing
public void parseExp() {
consume(WHILE);
parseExp();
consume(DO);
parseExp();
}
look ahead
LL Parsing 9
Exp → “while” Exp “do” Exp
Exp → “if” Exp “then” Exp “else” Exp
Predictive parsing
public void parseExp() {
switch current() {
case WHILE: consume(WHILE); parseExp(); ...; break;
case IF : consume(IF); parseExp(); ...; break;
default : error();
}
}
parse table
LL Parsing 10
rows
• nonterminal symbols N
• symbol to parse
columns
• terminal symbols Σk
• look ahead k
entries
• production rules P
• possible conflicts
Predictive parsing
T1 T2 T3 ...
N1 N1 →... N1 →...
N2 N2 →...
N3 N3 →... N3 →...
N4 N4 →...
N5 N5 →...
N6 N6 →... N6 →...
N7 N7 →...
N8 N8 →... N8 →... N8 →...
...
automaton
LL Parsing
Predictive parsing
11
… tn $t1
$
S
input
parse table
stack
automaton
LL Parsing
Predictive parsing
12
x … $
$
x
…
automaton
LL Parsing
Predictive parsing
12
… $
$
…
automaton
LL Parsing
Predictive parsing
13
x … $
$
Xi
x
Xi
Xi → Y1... Yk
…
automaton
LL Parsing
Predictive parsing
13
x … $
$
x
Xi
Xi → Y1... Yk
…
Yk
…
Y1
LL Parsing
LL Parse Tables
II
14
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
letters that w can start with
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
w G
*
ε
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
filling the table
LL Parsing 15
Predictive parsing
letters that can follow X
nullable
LL Parsing
nullable(X)
(X, ε) ∈ P nullable(X)
(X0, X1 … Xk)∈P ∧ nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0)
nullable(w)
nullable(ε)
nullable(X1 … Xk) = nullable(X1) ∧ … ∧ nullable(Xk)
16
Predictive parsing
first sets
LL Parsing
FIRST(X)
X∈Σ : FIRST(X) = {X}
(X0, X1 … Xi … Xk)∈P ∧ nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1)
FIRST(w)
FIRST(ε) = {}
¬nullable(X) FIRST(Xw) = FIRST(X)
nullable(X) FIRST(Xw) = FIRST(X) ∪ FIRST(w)
17
Predictive parsing
follow sets
LL Parsing
FOLLOW(X)
(X0, X1 … Xi … Xk)∈P ∧ nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0)
(X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk)
18
Predictive parsing
LL Parsing 19
Example
nullable FIRST FOLLOW
Start
Exp
Exp’
Term
Term’
Fact
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable
LL Parsing 20
Example
(X, ε) ∈ P nullable(X)
(X0, X1 … Xk)∈P ∧
nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0)
nullable FIRST FOLLOW
Start
Exp
Exp’
Term
Term’
Fact
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable
LL Parsing 21
Example
(X, ε) ∈ P nullable(X)
(X0, X1 … Xk)∈P ∧
nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0)
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable FIRST FOLLOW
Start no
Exp no
Exp’ yes
Term no
Term’ yes
Fact no
FIRST sets
LL Parsing 22
Example (X0, X1 … Xi … Xk)∈P ∧
nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1)
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable FIRST FOLLOW
Start no
Exp no
Exp’ yes
Term no
Term’ yes
Fact no
FIRST sets
LL Parsing 23
Example (X0, X1 … Xi … Xk)∈P ∧
nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1)
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable FIRST FOLLOW
Start no Num (
Exp no Num (
Exp’ yes +
Term no Num (
Term’ yes *
Fact no Num (
FOLLOW sets
LL Parsing 24
Example
(X0, X1 … Xi … Xk)∈P ∧
nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0)
(X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk)
nullable FIRST FOLLOW
Start no Num (
Exp no Num (
Exp’ yes +
Term no Num (
Term’ yes *
Fact no Num (
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
FOLLOW sets
LL Parsing 25
Example
(X0, X1 … Xi … Xk)∈P ∧
nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0)
(X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk)
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
nullable FIRST FOLLOW
Start no Num (
Exp no Num ( ) EOF
Exp’ yes + ) EOF
Term no Num ( + ) EOF
Term’ yes * + ) EOF
Fact no Num ( * + ) EOF
LL parse table
LL Parsing 26
Example
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
+ * Num ( ) EOF
Start
Exp
Exp’
Term
Term’
Fact
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
LL parse table
LL Parsing 27
Example
entry (X, w)∈P at row X and column T
T∈ FIRST(w)
nullable(w) ∧ T∈ FOLLOW(X)
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
+ * Num ( ) EOF
Start p0 p0
Exp p1 p1
Exp’ p2 p3 p3
Term p4 p4
Term’ p6 p5 p6 p6
Fact p7 p8
parsing (4+3)*6 EOF
LL Parsing 28
Example
p0: Start → Exp EOF
p1: Exp → Term Exp’
p2: Exp’ → “+” Term Exp’
p3: Exp’ →
p4: Term → Fact Term’
p5: Term’ → “*” Fact Term’
p6: Term’ →
p7: Fact → Num
p8: Fact → “(” Exp “)”
+ * Num ( ) EOF
Start p0 p0
Exp p1 p1
Exp’ p2 p3 p3
Term p4 p4
Term’ p6 p5 p6 p6
Fact p7 p8
LL Parsing
Grammar classes
29
context-free grammars
LL Parsing
Grammar classes
29
context-free grammars
LL(0)
LL Parsing
Grammar classes
29
context-free grammars
LL(1)
LL(0)
LL Parsing
Grammar classes
29
context-free grammars
LL(k)
LL(1)
LL(0)
encoding precedence
LL Parsing 30
Exp → Num
Exp → “(” Exp “)”
Exp → Exp “*” Exp
Exp → Exp “+” Exp
Predictive parsing
Fact → Num
Fact → “(” Exp “)”
Term → Term “*” Fact
Term → Fact
Exp → Exp “+” Term
Exp → Term
eliminating left recursion
LL Parsing 31
Term → Term “*” Fact
Term → Fact
Exp → Exp “+” Term
Exp → Term
Predictive parsing
Term’ → “*” Fact Term’
Term’ →
Term → Fact Term’
Exp’ → “+” Term Exp’
Exp’ →
Exp → Term Exp’
left factoring
LL Parsing 32
Exp → “if” Exp “then” Exp “else” Exp
Exp → “if” Exp “then” Exp
Predictive parsing
Exp → “if” Exp “then” Exp Else
Else → “else” Exp
Else →
LL Parsing
Parser Combinators
III
33
LL Parsing
Parser Combinators
34
• Parsers are modeled as functions, and larger parsers are built
from smaller ones using higher-order functions.
• Can be implemented in any lazy functional language with
higher-order style type system (e.g. Haskell, Scala).
• Parser combinators enable a recursive descent parsing strategy
that facilitates modular piecewise construction and testing.
LL Parsing
Parser Combinators
35
type Parser = String -> Tree
LL Parsing
Parser Combinators
36
type Parser = String -> Tree
What about intermediate parsers?
LL Parsing
Parser Combinators
37
type Parser = String -> (Tree, String)
type Parser = String -> Tree
Return the remainder of the input!
LL Parsing
Parser Combinators
38
type Parser = String -> (Tree, String)
type Parser = String -> Tree
WWhat about parsers that fail?
LL Parsing
Parser Combinators
39
type Parser = String -> Tree
type Parser = String -> [(Tree, String)]
type Parser = String -> (Tree, String)
Returns either a singleton or 

an empty list!
LL Parsing
Parser Combinators
40
type Parser = String -> Tree
type Parser = String -> [(Tree, String)]
A parser p returns either a singleton list [(a, String)] indicating
success, or an empty list [] indicating failure. Returning a list of
results, also allow to express ambiguities.
type Parser = String -> (Tree, String)
type Parser a = String -> [(a, String)]
LL Parsing
Primitive Parsers
41
success success :: a -> Parser a
success v = inp -> [(v, inp)]
success :: a String -> [(a, String)]
success v inp -> [(v, inp)]
fail
match
match :: Parser Char
match = inp -> case inp of
[] -> []
(x:xs) -> [(x,xs)]
fail :: Parser a
fail = inp -> []
or
LL Parsing
Parser Combinators
42
alternation
sequencing
alt :: Parser a -> Parser a -> Parser a
p 'alt' q = inp -> (p inp ++ q inp)
seq :: Parser a -> Parser b -> Parser (a, b)
p ‘seq‘ q = inp -> [((v,w), inp’’) | (v, inp’) <- p inp
, (w, inp’’) <- q inp’]
LL Parsing
Parser Combinators
43
• Parser combinators can use semantic actions to manipulate
intermediate results, being able to produce values rather than
trees.
• More powerful combinators can be build to allow parsing
context-sensitive languages (by passing the context together
with parsing results).
• Parser Combinators in Scala:
A. Moors, F. Piessens, Martin Odersky. Parser combinators in
Scala. Technical Report Department of Computer Science,
K.U. Leuven, February 2008.
LL Parsing
Parsing Expression Grammars
IV
44
LL Parsing
Parsing Expression Grammars (PEGs)
45
A ← e
Rules are of the form:
where A is a non-terminal and e is a parsing expression.
LL Parsing
PEG operators
46
LL Parsing
Syntactic Predicates
47
&e and !e : preserves the knowledge whether e fails or not.
Comment ← '//' (!EndOfLine .)* EndOfLine
LL Parsing
Parsing Expression Grammars
48
S ← a b / a S ← a / a b
• Prioritized choice '/'.
LL Parsing
Parsing Expression Grammars
49
S ← a b / a ≠ S ← a / a b
• Prioritized choice '/'.
• Backtracks when parsing an alternative fails.
• Uses memoization of intermediate results to parse in linear time,
called packrat parsing.
• Does not allow left-recursion.
LL Parsing
ANTLR / ALL(*)
V
50
LL Parsing
LL(*)
51
• Parses LL-regular grammars: for any given non-terminal,
parsers can use the entire remaining of the input to
differentiate the alternative productions.
• Statically builds a DFA from the grammar productions to
recognize lookahead.
• When in conflict, chooses the first alternative and backtracks
when DFA lookahead recognition fails.
LL Parsing
LL(*) Grammar Analysis
52
S : ID
| ID = E
| unsigned* int ID
| unsigned* ID ID
LL Parsing
LL(*) Grammar Analysis
53
S : ID
| ID = E
| unsigned* int ID
| unsigned* ID ID
(1)
(2)
(3)
(4)
augmented recursive 

transition network (ATN)
s11
s
s1
s3
ε
s7
ε
ε
ε
s2
s4
s’
s8
s5
s10s9
s12 s14s13
ID
ID ID
IDint
unsigned
unsigned
ε
ε
ID =
s6
E
ε
ε
ε
ε
terminals: ID, =, int, unsigned
LL Parsing
LL(*) Grammar Analysis
54
S : ID
| ID = E
| unsigned* int ID
| unsigned* ID ID
(1)
(2)
(3)
(4)
terminals: ID, =, int, unsigned
converts ATN to DFA
s11
s
s1
s3
ε
s7
ε
ε
ε
s2
s4
s’
s8
s5
s10s9
s12 s14s13
ID
ID ID
IDint
unsigned
unsigned
ε
ε
ID =
s6
E
ε
ε
ε
ε
(i) Initial state = closure(s);
(ii) Create a final state for each
production;
(iii) While DFA state contains ATN states
from multiple paths, create transitions
with terminals (or EOF if DFA contains s’);
(iv) If DFA state contain ATN states from
a single path, add transition to
corresponding final DFA state;
s0’
s1’
s2’
s3’
=> 2
s4’
=> 1
s5’
=> 4
s6’
=> 3
ID
=
EOF
ID
ID
int
unsigned
unsigned
int
LL Parsing
LL(*) Grammar Analysis
55
(1)
(2)
(3)
(4)
S : ID
| ID = E
| unsigned* int ID
| unsigned* ID ID
s11
s
s1
s3
ε
s7
ε
ε
ε
s2
s4
s’
s8
s5
s10s9
s12 s14s13
ID
ID ID
IDint
unsigned
unsigned
ε
ε
ID =
s6
E
ε
ε
ε
ε
terminals: ID, =, int, unsigned
LL Parsing
Adaptive LL(*)
56
S : E = E ;
| E ;
E : E * E
| E + E
| E ( E )
| ID
void S() {
switch (adaptivePredict("S", callStack)){
case 1 :
E(); match('='); E();
match(‘;'); break;
case 2:
E(); match(';'); break;
}
}
LL Parsing
Adaptive LL(*)
57
Dynamically builds the lookahead DFA.
S : E = E ;
| E ;
E : E * E
| E + E
| E ( E )
| ID
terminals: =, ;, *, +, (, ), ID
s6 s8
E
s
s1
ε
ε
s2
s’
s3
E =
s4
E
ε
ε
s5
;
s7
;
LL Parsing
Adaptive LL(*)
58
D0 s1
ID
:1
=
s2 s4s3 :2
(
ID ) ;
D0 s1
ID
:1
=
After parsing:
x = y;
f(x);
Dynamically builds the lookahead DFA.
LL Parsing
Adaptive LL(*)
59
S : x B
| y C
B : A a
C : A b a
A : b
| ε
parse: yba
LL Parsing
Adaptive LL(*)
60
S : x B
| y C
B : A a
C : A b a
A : b
| ε
input: yba
stack: S
LL Parsing
Adaptive LL(*)
61
S : x B
| y C
B : A a
C : A b a
A : b
| ε
input: yba
stack: yC
LL Parsing
Adaptive LL(*)
62
S : x B
| y C
B : A a
C : A b a
A : b
| ε
input: ba
stack: C
LL Parsing
Adaptive LL(*)
63
S : x B
| y C
B : A a
C : A b a
A : b
| ε
input: ba
stack: A ba
LL Parsing
Adaptive LL(*)
64
S : x B
| y C
B : A a
C : A b a
A : b
| ε
input: ba
stack: A ba
b in First(A)
b in Follow(A)
A1 or A2?
A1
A2
LL Parsing
Adaptive LL(*)
65
• Tries to parse using strong LL (without looking at the call
stack), when there are multiple alternatives, splits the parser to
try all possibilities. 

• Parsing continues with a graph, handling multiple stacks in
paralell to explore all alternatives.
• Eventually, if the grammar is unambiguous, only one stack
"survives". If multiple stacks survive, an ambiguty has been
found and the parser picks the production with lower value.
LL Parsing
ANTLR 4
66
• Accepts as input any context-free grammar that does not
contain indirect or hidden left-recursion.
• Generates ALL(*) parses in Java or C#.
• Grammars contain lexical and syntactic rules and generate a
lexer and a parser.
• ANTLR4 grammars can be scannerless and composable by
using individual characters as input symbols.
LL Parsing
Summary
VI
67
lessons learned
LL Parsing
Summary
68
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
68
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
68
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
How can we generate compiler tools from that?
• implement automaton
• generate parse tables
68
lessons learned
LL Parsing
Summary
How can we parse context-free languages effectively?
• predictive parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
How can we generate compiler tools from that?
• implement automaton
• generate parse tables
What are other techniques for implementing top-down parsers?
• Parser Combinators
• PEGs
• ALL(*)
68
learn more
LL Parsing
Literature
69
learn more
LL Parsing
Literature
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata
Theory, Languages, and Computation. 2006
69
learn more
LL Parsing
Literature
formal languages
Noam Chomsky: Three models for the description of language. 1956
J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata
Theory, Languages, and Computation. 2006
syntactical analysis
Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in
Java, 2nd edition. 2002
Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers:
Principles, Techniques, and Tools, 2nd edition. 2006
69
learn more
LL Parsing
Literature
70
learn more
LL Parsing
Literature
ALL(*)
Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the
power of dynamic analysis. In OOPSLA 2014.
70
learn more
LL Parsing
Literature
ALL(*)
Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the
power of dynamic analysis. In OOPSLA 2014.
Parsing Expression Grammars
Bryan Ford. Parsing Expression Grammars: a recognition-based syntactic
foundation. In POPL 2004.
70
learn more
LL Parsing
Literature
ALL(*)
Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the
power of dynamic analysis. In OOPSLA 2014.
Parsing Expression Grammars
Bryan Ford. Parsing Expression Grammars: a recognition-based syntactic
foundation. In POPL 2004.
Parser Combinators
Graham Hutton. Higher-Order Functions for Parsing. Journal of Functional
Programming, 1992.
A. Moors, F. Piessens, Martin Odersky. Parser combinators in Scala. Technical
Report Department of Computer Science, K.U. Leuven, February 2008.
70
LL Parsing
copyrights
71
LL Parsing 72
copyrights
LL Parsing
Pictures
Slide 1: Book Scanner by Ben Woosley, some rights reserved
Slides 5-7: Noam Chomsky by Fellowsisters, some rights reserved
Slide 8, 9, 26-28: Tiger by Bernard Landgraf, some rights reserved
Slide 32: Ostsee by Mario Thiel, some rights reserved
73

More Related Content

What's hot

Declarative Semantics Definition - Term Rewriting
Declarative Semantics Definition - Term RewritingDeclarative Semantics Definition - Term Rewriting
Declarative Semantics Definition - Term RewritingGuido Wachsmuth
 
Compiler Components and their Generators - Lexical Analysis
Compiler Components and their Generators - Lexical AnalysisCompiler Components and their Generators - Lexical Analysis
Compiler Components and their Generators - Lexical AnalysisGuido Wachsmuth
 
Compiler Components and their Generators - Traditional Parsing Algorithms
Compiler Components and their Generators - Traditional Parsing AlgorithmsCompiler Components and their Generators - Traditional Parsing Algorithms
Compiler Components and their Generators - Traditional Parsing AlgorithmsGuido Wachsmuth
 
Static name resolution
Static name resolutionStatic name resolution
Static name resolutionEelco Visser
 
Introduction - Imperative and Object-Oriented Languages
Introduction - Imperative and Object-Oriented LanguagesIntroduction - Imperative and Object-Oriented Languages
Introduction - Imperative and Object-Oriented LanguagesGuido Wachsmuth
 
Declare Your Language: Syntactic (Editor) Services
Declare Your Language: Syntactic (Editor) ServicesDeclare Your Language: Syntactic (Editor) Services
Declare Your Language: Syntactic (Editor) ServicesEelco Visser
 
Declare Your Language: Type Checking
Declare Your Language: Type CheckingDeclare Your Language: Type Checking
Declare Your Language: Type CheckingEelco Visser
 
CS4200 2019 | Lecture 5 | Transformation by Term Rewriting
CS4200 2019 | Lecture 5 | Transformation by Term RewritingCS4200 2019 | Lecture 5 | Transformation by Term Rewriting
CS4200 2019 | Lecture 5 | Transformation by Term RewritingEelco Visser
 
Dynamic Semantics Specification and Interpreter Generation
Dynamic Semantics Specification and Interpreter GenerationDynamic Semantics Specification and Interpreter Generation
Dynamic Semantics Specification and Interpreter GenerationEelco Visser
 
Compiler Construction | Lecture 5 | Transformation by Term Rewriting
Compiler Construction | Lecture 5 | Transformation by Term RewritingCompiler Construction | Lecture 5 | Transformation by Term Rewriting
Compiler Construction | Lecture 5 | Transformation by Term RewritingEelco Visser
 
Parsing in Compiler Design
Parsing in Compiler DesignParsing in Compiler Design
Parsing in Compiler DesignAkhil Kaushik
 
Python (regular expression)
Python (regular expression)Python (regular expression)
Python (regular expression)Chirag Shetty
 
Python advanced 2. regular expression in python
Python advanced 2. regular expression in pythonPython advanced 2. regular expression in python
Python advanced 2. regular expression in pythonJohn(Qiang) Zhang
 
"That scripting language called Prolog"
"That scripting language called Prolog""That scripting language called Prolog"
"That scripting language called Prolog"Sergei Winitzki
 

What's hot (20)

Static Analysis
Static AnalysisStatic Analysis
Static Analysis
 
Declarative Semantics Definition - Term Rewriting
Declarative Semantics Definition - Term RewritingDeclarative Semantics Definition - Term Rewriting
Declarative Semantics Definition - Term Rewriting
 
Compiler Components and their Generators - Lexical Analysis
Compiler Components and their Generators - Lexical AnalysisCompiler Components and their Generators - Lexical Analysis
Compiler Components and their Generators - Lexical Analysis
 
Compiler Components and their Generators - Traditional Parsing Algorithms
Compiler Components and their Generators - Traditional Parsing AlgorithmsCompiler Components and their Generators - Traditional Parsing Algorithms
Compiler Components and their Generators - Traditional Parsing Algorithms
 
Static name resolution
Static name resolutionStatic name resolution
Static name resolution
 
Dynamic Semantics
Dynamic SemanticsDynamic Semantics
Dynamic Semantics
 
Introduction - Imperative and Object-Oriented Languages
Introduction - Imperative and Object-Oriented LanguagesIntroduction - Imperative and Object-Oriented Languages
Introduction - Imperative and Object-Oriented Languages
 
Declare Your Language: Syntactic (Editor) Services
Declare Your Language: Syntactic (Editor) ServicesDeclare Your Language: Syntactic (Editor) Services
Declare Your Language: Syntactic (Editor) Services
 
Declare Your Language: Type Checking
Declare Your Language: Type CheckingDeclare Your Language: Type Checking
Declare Your Language: Type Checking
 
Type analysis
Type analysisType analysis
Type analysis
 
CS4200 2019 | Lecture 5 | Transformation by Term Rewriting
CS4200 2019 | Lecture 5 | Transformation by Term RewritingCS4200 2019 | Lecture 5 | Transformation by Term Rewriting
CS4200 2019 | Lecture 5 | Transformation by Term Rewriting
 
Dynamic Semantics Specification and Interpreter Generation
Dynamic Semantics Specification and Interpreter GenerationDynamic Semantics Specification and Interpreter Generation
Dynamic Semantics Specification and Interpreter Generation
 
Compiler Construction | Lecture 5 | Transformation by Term Rewriting
Compiler Construction | Lecture 5 | Transformation by Term RewritingCompiler Construction | Lecture 5 | Transformation by Term Rewriting
Compiler Construction | Lecture 5 | Transformation by Term Rewriting
 
Regular expression examples
Regular expression examplesRegular expression examples
Regular expression examples
 
Parsing in Compiler Design
Parsing in Compiler DesignParsing in Compiler Design
Parsing in Compiler Design
 
Python (regular expression)
Python (regular expression)Python (regular expression)
Python (regular expression)
 
Python advanced 2. regular expression in python
Python advanced 2. regular expression in pythonPython advanced 2. regular expression in python
Python advanced 2. regular expression in python
 
Perl6 signatures
Perl6 signaturesPerl6 signatures
Perl6 signatures
 
Regular Expression
Regular ExpressionRegular Expression
Regular Expression
 
"That scripting language called Prolog"
"That scripting language called Prolog""That scripting language called Prolog"
"That scripting language called Prolog"
 

Viewers also liked

Viewers also liked (6)

Language
LanguageLanguage
Language
 
Register Allocation
Register AllocationRegister Allocation
Register Allocation
 
Garbage Collection
Garbage CollectionGarbage Collection
Garbage Collection
 
LR Parsing
LR ParsingLR Parsing
LR Parsing
 
Programming languages
Programming languagesProgramming languages
Programming languages
 
Software languages
Software languagesSoftware languages
Software languages
 

Similar to LL Parsing

Compiler components and their generators traditional parsing algorithms
Compiler components and their generators  traditional parsing algorithmsCompiler components and their generators  traditional parsing algorithms
Compiler components and their generators traditional parsing algorithmsPeshrowKareem1
 
The Ring programming language version 1.5.4 book - Part 20 of 185
The Ring programming language version 1.5.4 book - Part 20 of 185The Ring programming language version 1.5.4 book - Part 20 of 185
The Ring programming language version 1.5.4 book - Part 20 of 185Mahmoud Samir Fayed
 
TI1220 Lecture 9: Parsing & interpretation
TI1220 Lecture 9: Parsing & interpretationTI1220 Lecture 9: Parsing & interpretation
TI1220 Lecture 9: Parsing & interpretationEelco Visser
 
2Bytesprog2 course_2014_c5_pointers
2Bytesprog2 course_2014_c5_pointers2Bytesprog2 course_2014_c5_pointers
2Bytesprog2 course_2014_c5_pointerskinan keshkeh
 
Scope Graphs: A fresh look at name binding in programming languages
Scope Graphs: A fresh look at name binding in programming languagesScope Graphs: A fresh look at name binding in programming languages
Scope Graphs: A fresh look at name binding in programming languagesEelco Visser
 
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...Paulo Miguel Almeida
 
Meta-learning and the ELBO
Meta-learning and the ELBOMeta-learning and the ELBO
Meta-learning and the ELBOYoonho Lee
 
Expressiveness and Model of the Polymorphic λ Calculus
Expressiveness and Model of the Polymorphic λ CalculusExpressiveness and Model of the Polymorphic λ Calculus
Expressiveness and Model of the Polymorphic λ Calculusevastsdsh
 
The Ring programming language version 1.7 book - Part 23 of 196
The Ring programming language version 1.7 book - Part 23 of 196The Ring programming language version 1.7 book - Part 23 of 196
The Ring programming language version 1.7 book - Part 23 of 196Mahmoud Samir Fayed
 
Open course(programming languages) 20150225
Open course(programming languages) 20150225Open course(programming languages) 20150225
Open course(programming languages) 20150225JangChulho
 
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)codin9cafe
 

Similar to LL Parsing (20)

Compiler components and their generators traditional parsing algorithms
Compiler components and their generators  traditional parsing algorithmsCompiler components and their generators  traditional parsing algorithms
Compiler components and their generators traditional parsing algorithms
 
The Ring programming language version 1.5.4 book - Part 20 of 185
The Ring programming language version 1.5.4 book - Part 20 of 185The Ring programming language version 1.5.4 book - Part 20 of 185
The Ring programming language version 1.5.4 book - Part 20 of 185
 
TI1220 Lecture 9: Parsing & interpretation
TI1220 Lecture 9: Parsing & interpretationTI1220 Lecture 9: Parsing & interpretation
TI1220 Lecture 9: Parsing & interpretation
 
Ch2 (1).ppt
Ch2 (1).pptCh2 (1).ppt
Ch2 (1).ppt
 
Ch2
Ch2Ch2
Ch2
 
2Bytesprog2 course_2014_c5_pointers
2Bytesprog2 course_2014_c5_pointers2Bytesprog2 course_2014_c5_pointers
2Bytesprog2 course_2014_c5_pointers
 
Scope Graphs: A fresh look at name binding in programming languages
Scope Graphs: A fresh look at name binding in programming languagesScope Graphs: A fresh look at name binding in programming languages
Scope Graphs: A fresh look at name binding in programming languages
 
Left factor put
Left factor putLeft factor put
Left factor put
 
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...
Auckland Programming Algorithms and Performance Meetup about Stacks & LeetCod...
 
Meta-learning and the ELBO
Meta-learning and the ELBOMeta-learning and the ELBO
Meta-learning and the ELBO
 
Expressiveness and Model of the Polymorphic λ Calculus
Expressiveness and Model of the Polymorphic λ CalculusExpressiveness and Model of the Polymorphic λ Calculus
Expressiveness and Model of the Polymorphic λ Calculus
 
The Ring programming language version 1.7 book - Part 23 of 196
The Ring programming language version 1.7 book - Part 23 of 196The Ring programming language version 1.7 book - Part 23 of 196
The Ring programming language version 1.7 book - Part 23 of 196
 
Probability Theory 5
Probability Theory 5Probability Theory 5
Probability Theory 5
 
chapter9.ppt
chapter9.pptchapter9.ppt
chapter9.ppt
 
Primitive-Roots.pptx
Primitive-Roots.pptxPrimitive-Roots.pptx
Primitive-Roots.pptx
 
Open course(programming languages) 20150225
Open course(programming languages) 20150225Open course(programming languages) 20150225
Open course(programming languages) 20150225
 
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)
codin9cafe[2015.02.25]Open course(programming languages) - 장철호(Ch Jang)
 
Bottomupparser
BottomupparserBottomupparser
Bottomupparser
 
Bottomupparser
BottomupparserBottomupparser
Bottomupparser
 
Bottomupparser
BottomupparserBottomupparser
Bottomupparser
 

More from Eelco Visser

CS4200 2019 | Lecture 4 | Syntactic Services
CS4200 2019 | Lecture 4 | Syntactic ServicesCS4200 2019 | Lecture 4 | Syntactic Services
CS4200 2019 | Lecture 4 | Syntactic ServicesEelco Visser
 
CS4200 2019 | Lecture 3 | Parsing
CS4200 2019 | Lecture 3 | ParsingCS4200 2019 | Lecture 3 | Parsing
CS4200 2019 | Lecture 3 | ParsingEelco Visser
 
CS4200 2019 | Lecture 2 | syntax-definition
CS4200 2019 | Lecture 2 | syntax-definitionCS4200 2019 | Lecture 2 | syntax-definition
CS4200 2019 | Lecture 2 | syntax-definitionEelco Visser
 
CS4200 2019 Lecture 1: Introduction
CS4200 2019 Lecture 1: IntroductionCS4200 2019 Lecture 1: Introduction
CS4200 2019 Lecture 1: IntroductionEelco Visser
 
A Direct Semantics of Declarative Disambiguation Rules
A Direct Semantics of Declarative Disambiguation RulesA Direct Semantics of Declarative Disambiguation Rules
A Direct Semantics of Declarative Disambiguation RulesEelco Visser
 
Declarative Type System Specification with Statix
Declarative Type System Specification with StatixDeclarative Type System Specification with Statix
Declarative Type System Specification with StatixEelco Visser
 
Compiler Construction | Lecture 17 | Beyond Compiler Construction
Compiler Construction | Lecture 17 | Beyond Compiler ConstructionCompiler Construction | Lecture 17 | Beyond Compiler Construction
Compiler Construction | Lecture 17 | Beyond Compiler ConstructionEelco Visser
 
Domain Specific Languages for Parallel Graph AnalytiX (PGX)
Domain Specific Languages for Parallel Graph AnalytiX (PGX)Domain Specific Languages for Parallel Graph AnalytiX (PGX)
Domain Specific Languages for Parallel Graph AnalytiX (PGX)Eelco Visser
 
Compiler Construction | Lecture 15 | Memory Management
Compiler Construction | Lecture 15 | Memory ManagementCompiler Construction | Lecture 15 | Memory Management
Compiler Construction | Lecture 15 | Memory ManagementEelco Visser
 
Compiler Construction | Lecture 14 | Interpreters
Compiler Construction | Lecture 14 | InterpretersCompiler Construction | Lecture 14 | Interpreters
Compiler Construction | Lecture 14 | InterpretersEelco Visser
 
Compiler Construction | Lecture 13 | Code Generation
Compiler Construction | Lecture 13 | Code GenerationCompiler Construction | Lecture 13 | Code Generation
Compiler Construction | Lecture 13 | Code GenerationEelco Visser
 
Compiler Construction | Lecture 12 | Virtual Machines
Compiler Construction | Lecture 12 | Virtual MachinesCompiler Construction | Lecture 12 | Virtual Machines
Compiler Construction | Lecture 12 | Virtual MachinesEelco Visser
 
Compiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone FrameworksCompiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone FrameworksEelco Visser
 
Compiler Construction | Lecture 10 | Data-Flow Analysis
Compiler Construction | Lecture 10 | Data-Flow AnalysisCompiler Construction | Lecture 10 | Data-Flow Analysis
Compiler Construction | Lecture 10 | Data-Flow AnalysisEelco Visser
 
Compiler Construction | Lecture 9 | Constraint Resolution
Compiler Construction | Lecture 9 | Constraint ResolutionCompiler Construction | Lecture 9 | Constraint Resolution
Compiler Construction | Lecture 9 | Constraint ResolutionEelco Visser
 
Compiler Construction | Lecture 8 | Type Constraints
Compiler Construction | Lecture 8 | Type ConstraintsCompiler Construction | Lecture 8 | Type Constraints
Compiler Construction | Lecture 8 | Type ConstraintsEelco Visser
 
Compiler Construction | Lecture 7 | Type Checking
Compiler Construction | Lecture 7 | Type CheckingCompiler Construction | Lecture 7 | Type Checking
Compiler Construction | Lecture 7 | Type CheckingEelco Visser
 
Compiler Construction | Lecture 6 | Introduction to Static Analysis
Compiler Construction | Lecture 6 | Introduction to Static AnalysisCompiler Construction | Lecture 6 | Introduction to Static Analysis
Compiler Construction | Lecture 6 | Introduction to Static AnalysisEelco Visser
 
Compiler Construction | Lecture 4 | Parsing
Compiler Construction | Lecture 4 | Parsing Compiler Construction | Lecture 4 | Parsing
Compiler Construction | Lecture 4 | Parsing Eelco Visser
 
Compiler Construction | Lecture 3 | Syntactic Editor Services
Compiler Construction | Lecture 3 | Syntactic Editor ServicesCompiler Construction | Lecture 3 | Syntactic Editor Services
Compiler Construction | Lecture 3 | Syntactic Editor ServicesEelco Visser
 

More from Eelco Visser (20)

CS4200 2019 | Lecture 4 | Syntactic Services
CS4200 2019 | Lecture 4 | Syntactic ServicesCS4200 2019 | Lecture 4 | Syntactic Services
CS4200 2019 | Lecture 4 | Syntactic Services
 
CS4200 2019 | Lecture 3 | Parsing
CS4200 2019 | Lecture 3 | ParsingCS4200 2019 | Lecture 3 | Parsing
CS4200 2019 | Lecture 3 | Parsing
 
CS4200 2019 | Lecture 2 | syntax-definition
CS4200 2019 | Lecture 2 | syntax-definitionCS4200 2019 | Lecture 2 | syntax-definition
CS4200 2019 | Lecture 2 | syntax-definition
 
CS4200 2019 Lecture 1: Introduction
CS4200 2019 Lecture 1: IntroductionCS4200 2019 Lecture 1: Introduction
CS4200 2019 Lecture 1: Introduction
 
A Direct Semantics of Declarative Disambiguation Rules
A Direct Semantics of Declarative Disambiguation RulesA Direct Semantics of Declarative Disambiguation Rules
A Direct Semantics of Declarative Disambiguation Rules
 
Declarative Type System Specification with Statix
Declarative Type System Specification with StatixDeclarative Type System Specification with Statix
Declarative Type System Specification with Statix
 
Compiler Construction | Lecture 17 | Beyond Compiler Construction
Compiler Construction | Lecture 17 | Beyond Compiler ConstructionCompiler Construction | Lecture 17 | Beyond Compiler Construction
Compiler Construction | Lecture 17 | Beyond Compiler Construction
 
Domain Specific Languages for Parallel Graph AnalytiX (PGX)
Domain Specific Languages for Parallel Graph AnalytiX (PGX)Domain Specific Languages for Parallel Graph AnalytiX (PGX)
Domain Specific Languages for Parallel Graph AnalytiX (PGX)
 
Compiler Construction | Lecture 15 | Memory Management
Compiler Construction | Lecture 15 | Memory ManagementCompiler Construction | Lecture 15 | Memory Management
Compiler Construction | Lecture 15 | Memory Management
 
Compiler Construction | Lecture 14 | Interpreters
Compiler Construction | Lecture 14 | InterpretersCompiler Construction | Lecture 14 | Interpreters
Compiler Construction | Lecture 14 | Interpreters
 
Compiler Construction | Lecture 13 | Code Generation
Compiler Construction | Lecture 13 | Code GenerationCompiler Construction | Lecture 13 | Code Generation
Compiler Construction | Lecture 13 | Code Generation
 
Compiler Construction | Lecture 12 | Virtual Machines
Compiler Construction | Lecture 12 | Virtual MachinesCompiler Construction | Lecture 12 | Virtual Machines
Compiler Construction | Lecture 12 | Virtual Machines
 
Compiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone FrameworksCompiler Construction | Lecture 11 | Monotone Frameworks
Compiler Construction | Lecture 11 | Monotone Frameworks
 
Compiler Construction | Lecture 10 | Data-Flow Analysis
Compiler Construction | Lecture 10 | Data-Flow AnalysisCompiler Construction | Lecture 10 | Data-Flow Analysis
Compiler Construction | Lecture 10 | Data-Flow Analysis
 
Compiler Construction | Lecture 9 | Constraint Resolution
Compiler Construction | Lecture 9 | Constraint ResolutionCompiler Construction | Lecture 9 | Constraint Resolution
Compiler Construction | Lecture 9 | Constraint Resolution
 
Compiler Construction | Lecture 8 | Type Constraints
Compiler Construction | Lecture 8 | Type ConstraintsCompiler Construction | Lecture 8 | Type Constraints
Compiler Construction | Lecture 8 | Type Constraints
 
Compiler Construction | Lecture 7 | Type Checking
Compiler Construction | Lecture 7 | Type CheckingCompiler Construction | Lecture 7 | Type Checking
Compiler Construction | Lecture 7 | Type Checking
 
Compiler Construction | Lecture 6 | Introduction to Static Analysis
Compiler Construction | Lecture 6 | Introduction to Static AnalysisCompiler Construction | Lecture 6 | Introduction to Static Analysis
Compiler Construction | Lecture 6 | Introduction to Static Analysis
 
Compiler Construction | Lecture 4 | Parsing
Compiler Construction | Lecture 4 | Parsing Compiler Construction | Lecture 4 | Parsing
Compiler Construction | Lecture 4 | Parsing
 
Compiler Construction | Lecture 3 | Syntactic Editor Services
Compiler Construction | Lecture 3 | Syntactic Editor ServicesCompiler Construction | Lecture 3 | Syntactic Editor Services
Compiler Construction | Lecture 3 | Syntactic Editor Services
 

Recently uploaded

Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Bert Jan Schrijver
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...masabamasaba
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationJuha-Pekka Tolvanen
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfkalichargn70th171
 
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benonimasabamasaba
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park masabamasaba
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfonteinmasabamasaba
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension AidPhilip Schwarz
 
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnVTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnAmarnathKambale
 
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Hararemasabamasaba
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...masabamasaba
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park masabamasaba
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyviewmasabamasaba
 
The title is not connected to what is inside
The title is not connected to what is insideThe title is not connected to what is inside
The title is not connected to what is insideshinachiaurasa2
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...chiefasafspells
 
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech studentsHimanshiGarg82
 
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...masabamasaba
 

Recently uploaded (20)

Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...
 
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Atlanta Psychic Readings, Attraction spells,Brin...
 
What Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the SituationWhat Goes Wrong with Language Definitions and How to Improve the Situation
What Goes Wrong with Language Definitions and How to Improve the Situation
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
 
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni%in Benoni+277-882-255-28 abortion pills for sale in Benoni
%in Benoni+277-882-255-28 abortion pills for sale in Benoni
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnVTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
 
%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare%in Harare+277-882-255-28 abortion pills for sale in Harare
%in Harare+277-882-255-28 abortion pills for sale in Harare
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
 
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park %in kempton park+277-882-255-28 abortion pills for sale in kempton park
%in kempton park+277-882-255-28 abortion pills for sale in kempton park
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
 
The title is not connected to what is inside
The title is not connected to what is insideThe title is not connected to what is inside
The title is not connected to what is inside
 
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
Love witchcraft +27768521739 Binding love spell in Sandy Springs, GA |psychic...
 
Microsoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdfMicrosoft AI Transformation Partner Playbook.pdf
Microsoft AI Transformation Partner Playbook.pdf
 
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students
 
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
%+27788225528 love spells in Toronto Psychic Readings, Attraction spells,Brin...
 

LL Parsing

  • 1. Challenge the future Delft University of Technology Course IN4303 Compiler Construction Eduardo Souza, Guido Wachsmuth, Eelco Visser LL Parsing Traditional Parsing Algorithms
  • 2. lessons learned LL Parsing Recap: Lexical Analysis What are the formalisms to describe regular languages? • regular grammars • regular expressions • finite state automata Why are these formalisms equivalent? • constructive proofs How can we generate compiler tools from that? • implement DFAs • generate transition tables 2
  • 4. today’s lecture LL Parsing Overview efficient parsing algorithms • predictive/recursive descent parsing • LL parsing 3
  • 5. today’s lecture LL Parsing Overview efficient parsing algorithms • predictive/recursive descent parsing • LL parsing grammar classes • LL(k) grammars • LR(k) grammars 3
  • 6. today’s lecture LL Parsing Overview efficient parsing algorithms • predictive/recursive descent parsing • LL parsing grammar classes • LL(k) grammars • LR(k) grammars more top-down parsing algorithms • Parser Combinators • Parsing Expression Grammars • ALL(*) 3
  • 7. LL Parsing Predictive (Recursive Descent) Parsing I 4
  • 8. formal languages LL Parsing 5 Recap: A Theory of Language
  • 9. formal languages LL Parsing vocabulary Σ finite, nonempty set of elements (words, letters) alphabet 5 Recap: A Theory of Language
  • 10. formal languages LL Parsing vocabulary Σ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance 5 Recap: A Theory of Language
  • 11. formal languages LL Parsing vocabulary Σ finite, nonempty set of elements (words, letters) alphabet string over Σ finite sequence of elements chosen from Σ word, sentence, utterance formal language λ set of strings over a vocabulary Σ λ ⊆ Σ* 5 Recap: A Theory of Language
  • 12. formal grammars LL Parsing 6 Recap: A Theory of Language
  • 13. formal grammars LL Parsing formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N 6 Recap: A Theory of Language
  • 14. formal grammars LL Parsing formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N 6 Recap: A Theory of Language nonterminal symbol
  • 15. formal grammars LL Parsing formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N 6 Recap: A Theory of Language context
  • 16. formal grammars LL Parsing formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N 6 Recap: A Theory of Language replacement
  • 17. formal grammars LL Parsing formal grammar G = (N, Σ, P, S) nonterminal symbols N terminal symbols Σ production rules P ⊆ (N∪Σ)* N (N∪Σ)* × (N∪Σ)* start symbol S∈N grammar classes type-0, unrestricted type-1, context-sensitive: (a A c, a b c) type-2, context-free: P ⊆ N × (N∪Σ)* type-3, regular: (A, x) or (A, xB) 6 Recap: A Theory of Language
  • 18. formal languages LL Parsing 7 Recap: A Theory of Language
  • 19. formal languages LL Parsing formal grammar G = (N, Σ, P, S) 7 Recap: A Theory of Language
  • 20. formal languages LL Parsing formal grammar G = (N, Σ, P, S) derivation relation G ⊆ (N∪Σ)* × (N∪Σ)* w G w’ ∃(p, q)∈P: ∃u,v∈(N∪Σ)* : w=u p v ∧ w’=u q v 7 Recap: A Theory of Language
  • 21. formal languages LL Parsing formal grammar G = (N, Σ, P, S) derivation relation G ⊆ (N∪Σ)* × (N∪Σ)* w G w’ ∃(p, q)∈P: ∃u,v∈(N∪Σ)* : w=u p v ∧ w’=u q v formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w} 7 Recap: A Theory of Language
  • 22. formal languages LL Parsing formal grammar G = (N, Σ, P, S) derivation relation G ⊆ (N∪Σ)* × (N∪Σ)* w G w’ ∃(p, q)∈P: ∃u,v∈(N∪Σ)* : w=u p v ∧ w’=u q v formal language L(G) ⊆ Σ* L(G) = {w∈Σ* | S G * w} classes of formal languages 7 Recap: A Theory of Language
  • 23. recursive descent LL Parsing 8 Exp → “while” Exp “do” Exp Predictive parsing public void parseExp() { consume(WHILE); parseExp(); consume(DO); parseExp(); }
  • 24. look ahead LL Parsing 9 Exp → “while” Exp “do” Exp Exp → “if” Exp “then” Exp “else” Exp Predictive parsing public void parseExp() { switch current() { case WHILE: consume(WHILE); parseExp(); ...; break; case IF : consume(IF); parseExp(); ...; break; default : error(); } }
  • 25. parse table LL Parsing 10 rows • nonterminal symbols N • symbol to parse columns • terminal symbols Σk • look ahead k entries • production rules P • possible conflicts Predictive parsing T1 T2 T3 ... N1 N1 →... N1 →... N2 N2 →... N3 N3 →... N3 →... N4 N4 →... N5 N5 →... N6 N6 →... N6 →... N7 N7 →... N8 N8 →... N8 →... N8 →... ...
  • 26. automaton LL Parsing Predictive parsing 11 … tn $t1 $ S input parse table stack
  • 29. automaton LL Parsing Predictive parsing 13 x … $ $ Xi x Xi Xi → Y1... Yk …
  • 30. automaton LL Parsing Predictive parsing 13 x … $ $ x Xi Xi → Y1... Yk … Yk … Y1
  • 31. LL Parsing LL Parse Tables II 14
  • 32. entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) filling the table LL Parsing 15 Predictive parsing
  • 33. entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) filling the table LL Parsing 15 Predictive parsing letters that w can start with
  • 34. entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) filling the table LL Parsing 15 Predictive parsing w G * ε
  • 35. entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) filling the table LL Parsing 15 Predictive parsing letters that can follow X
  • 36. nullable LL Parsing nullable(X) (X, ε) ∈ P nullable(X) (X0, X1 … Xk)∈P ∧ nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0) nullable(w) nullable(ε) nullable(X1 … Xk) = nullable(X1) ∧ … ∧ nullable(Xk) 16 Predictive parsing
  • 37. first sets LL Parsing FIRST(X) X∈Σ : FIRST(X) = {X} (X0, X1 … Xi … Xk)∈P ∧ nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1) FIRST(w) FIRST(ε) = {} ¬nullable(X) FIRST(Xw) = FIRST(X) nullable(X) FIRST(Xw) = FIRST(X) ∪ FIRST(w) 17 Predictive parsing
  • 38. follow sets LL Parsing FOLLOW(X) (X0, X1 … Xi … Xk)∈P ∧ nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0) (X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk) 18 Predictive parsing
  • 39. LL Parsing 19 Example nullable FIRST FOLLOW Start Exp Exp’ Term Term’ Fact p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)”
  • 40. nullable LL Parsing 20 Example (X, ε) ∈ P nullable(X) (X0, X1 … Xk)∈P ∧ nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0) nullable FIRST FOLLOW Start Exp Exp’ Term Term’ Fact p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)”
  • 41. nullable LL Parsing 21 Example (X, ε) ∈ P nullable(X) (X0, X1 … Xk)∈P ∧ nullable(X1) ∧ … ∧ nullable(Xk) nullable(X0) p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” nullable FIRST FOLLOW Start no Exp no Exp’ yes Term no Term’ yes Fact no
  • 42. FIRST sets LL Parsing 22 Example (X0, X1 … Xi … Xk)∈P ∧ nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1) p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” nullable FIRST FOLLOW Start no Exp no Exp’ yes Term no Term’ yes Fact no
  • 43. FIRST sets LL Parsing 23 Example (X0, X1 … Xi … Xk)∈P ∧ nullable(X1 … Xi) FIRST(X0) ⊇ FIRST(Xi+1) p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” nullable FIRST FOLLOW Start no Num ( Exp no Num ( Exp’ yes + Term no Num ( Term’ yes * Fact no Num (
  • 44. FOLLOW sets LL Parsing 24 Example (X0, X1 … Xi … Xk)∈P ∧ nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0) (X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk) nullable FIRST FOLLOW Start no Num ( Exp no Num ( Exp’ yes + Term no Num ( Term’ yes * Fact no Num ( p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)”
  • 45. FOLLOW sets LL Parsing 25 Example (X0, X1 … Xi … Xk)∈P ∧ nullable(Xi+1 … Xk) FOLLOW(Xi) ⊇ FOLLOW(X0) (X0, X1 … Xi … Xk)∈P FOLLOW(Xi) ⊇ FIRST(Xi+1 … Xk) p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” nullable FIRST FOLLOW Start no Num ( Exp no Num ( ) EOF Exp’ yes + ) EOF Term no Num ( + ) EOF Term’ yes * + ) EOF Fact no Num ( * + ) EOF
  • 46. LL parse table LL Parsing 26 Example entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) + * Num ( ) EOF Start Exp Exp’ Term Term’ Fact p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)”
  • 47. LL parse table LL Parsing 27 Example entry (X, w)∈P at row X and column T T∈ FIRST(w) nullable(w) ∧ T∈ FOLLOW(X) p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” + * Num ( ) EOF Start p0 p0 Exp p1 p1 Exp’ p2 p3 p3 Term p4 p4 Term’ p6 p5 p6 p6 Fact p7 p8
  • 48. parsing (4+3)*6 EOF LL Parsing 28 Example p0: Start → Exp EOF p1: Exp → Term Exp’ p2: Exp’ → “+” Term Exp’ p3: Exp’ → p4: Term → Fact Term’ p5: Term’ → “*” Fact Term’ p6: Term’ → p7: Fact → Num p8: Fact → “(” Exp “)” + * Num ( ) EOF Start p0 p0 Exp p1 p1 Exp’ p2 p3 p3 Term p4 p4 Term’ p6 p5 p6 p6 Fact p7 p8
  • 52. LL Parsing Grammar classes 29 context-free grammars LL(k) LL(1) LL(0)
  • 53. encoding precedence LL Parsing 30 Exp → Num Exp → “(” Exp “)” Exp → Exp “*” Exp Exp → Exp “+” Exp Predictive parsing Fact → Num Fact → “(” Exp “)” Term → Term “*” Fact Term → Fact Exp → Exp “+” Term Exp → Term
  • 54. eliminating left recursion LL Parsing 31 Term → Term “*” Fact Term → Fact Exp → Exp “+” Term Exp → Term Predictive parsing Term’ → “*” Fact Term’ Term’ → Term → Fact Term’ Exp’ → “+” Term Exp’ Exp’ → Exp → Term Exp’
  • 55. left factoring LL Parsing 32 Exp → “if” Exp “then” Exp “else” Exp Exp → “if” Exp “then” Exp Predictive parsing Exp → “if” Exp “then” Exp Else Else → “else” Exp Else →
  • 57. LL Parsing Parser Combinators 34 • Parsers are modeled as functions, and larger parsers are built from smaller ones using higher-order functions. • Can be implemented in any lazy functional language with higher-order style type system (e.g. Haskell, Scala). • Parser combinators enable a recursive descent parsing strategy that facilitates modular piecewise construction and testing.
  • 58. LL Parsing Parser Combinators 35 type Parser = String -> Tree
  • 59. LL Parsing Parser Combinators 36 type Parser = String -> Tree What about intermediate parsers?
  • 60. LL Parsing Parser Combinators 37 type Parser = String -> (Tree, String) type Parser = String -> Tree Return the remainder of the input!
  • 61. LL Parsing Parser Combinators 38 type Parser = String -> (Tree, String) type Parser = String -> Tree WWhat about parsers that fail?
  • 62. LL Parsing Parser Combinators 39 type Parser = String -> Tree type Parser = String -> [(Tree, String)] type Parser = String -> (Tree, String) Returns either a singleton or 
 an empty list!
  • 63. LL Parsing Parser Combinators 40 type Parser = String -> Tree type Parser = String -> [(Tree, String)] A parser p returns either a singleton list [(a, String)] indicating success, or an empty list [] indicating failure. Returning a list of results, also allow to express ambiguities. type Parser = String -> (Tree, String) type Parser a = String -> [(a, String)]
  • 64. LL Parsing Primitive Parsers 41 success success :: a -> Parser a success v = inp -> [(v, inp)] success :: a String -> [(a, String)] success v inp -> [(v, inp)] fail match match :: Parser Char match = inp -> case inp of [] -> [] (x:xs) -> [(x,xs)] fail :: Parser a fail = inp -> [] or
  • 65. LL Parsing Parser Combinators 42 alternation sequencing alt :: Parser a -> Parser a -> Parser a p 'alt' q = inp -> (p inp ++ q inp) seq :: Parser a -> Parser b -> Parser (a, b) p ‘seq‘ q = inp -> [((v,w), inp’’) | (v, inp’) <- p inp , (w, inp’’) <- q inp’]
  • 66. LL Parsing Parser Combinators 43 • Parser combinators can use semantic actions to manipulate intermediate results, being able to produce values rather than trees. • More powerful combinators can be build to allow parsing context-sensitive languages (by passing the context together with parsing results). • Parser Combinators in Scala: A. Moors, F. Piessens, Martin Odersky. Parser combinators in Scala. Technical Report Department of Computer Science, K.U. Leuven, February 2008.
  • 68. LL Parsing Parsing Expression Grammars (PEGs) 45 A ← e Rules are of the form: where A is a non-terminal and e is a parsing expression.
  • 70. LL Parsing Syntactic Predicates 47 &e and !e : preserves the knowledge whether e fails or not. Comment ← '//' (!EndOfLine .)* EndOfLine
  • 71. LL Parsing Parsing Expression Grammars 48 S ← a b / a S ← a / a b • Prioritized choice '/'.
  • 72. LL Parsing Parsing Expression Grammars 49 S ← a b / a ≠ S ← a / a b • Prioritized choice '/'. • Backtracks when parsing an alternative fails. • Uses memoization of intermediate results to parse in linear time, called packrat parsing. • Does not allow left-recursion.
  • 73. LL Parsing ANTLR / ALL(*) V 50
  • 74. LL Parsing LL(*) 51 • Parses LL-regular grammars: for any given non-terminal, parsers can use the entire remaining of the input to differentiate the alternative productions. • Statically builds a DFA from the grammar productions to recognize lookahead. • When in conflict, chooses the first alternative and backtracks when DFA lookahead recognition fails.
  • 75. LL Parsing LL(*) Grammar Analysis 52 S : ID | ID = E | unsigned* int ID | unsigned* ID ID
  • 76. LL Parsing LL(*) Grammar Analysis 53 S : ID | ID = E | unsigned* int ID | unsigned* ID ID (1) (2) (3) (4) augmented recursive 
 transition network (ATN) s11 s s1 s3 ε s7 ε ε ε s2 s4 s’ s8 s5 s10s9 s12 s14s13 ID ID ID IDint unsigned unsigned ε ε ID = s6 E ε ε ε ε terminals: ID, =, int, unsigned
  • 77. LL Parsing LL(*) Grammar Analysis 54 S : ID | ID = E | unsigned* int ID | unsigned* ID ID (1) (2) (3) (4) terminals: ID, =, int, unsigned converts ATN to DFA s11 s s1 s3 ε s7 ε ε ε s2 s4 s’ s8 s5 s10s9 s12 s14s13 ID ID ID IDint unsigned unsigned ε ε ID = s6 E ε ε ε ε (i) Initial state = closure(s); (ii) Create a final state for each production; (iii) While DFA state contains ATN states from multiple paths, create transitions with terminals (or EOF if DFA contains s’); (iv) If DFA state contain ATN states from a single path, add transition to corresponding final DFA state;
  • 78. s0’ s1’ s2’ s3’ => 2 s4’ => 1 s5’ => 4 s6’ => 3 ID = EOF ID ID int unsigned unsigned int LL Parsing LL(*) Grammar Analysis 55 (1) (2) (3) (4) S : ID | ID = E | unsigned* int ID | unsigned* ID ID s11 s s1 s3 ε s7 ε ε ε s2 s4 s’ s8 s5 s10s9 s12 s14s13 ID ID ID IDint unsigned unsigned ε ε ID = s6 E ε ε ε ε terminals: ID, =, int, unsigned
  • 79. LL Parsing Adaptive LL(*) 56 S : E = E ; | E ; E : E * E | E + E | E ( E ) | ID void S() { switch (adaptivePredict("S", callStack)){ case 1 : E(); match('='); E(); match(‘;'); break; case 2: E(); match(';'); break; } }
  • 80. LL Parsing Adaptive LL(*) 57 Dynamically builds the lookahead DFA. S : E = E ; | E ; E : E * E | E + E | E ( E ) | ID terminals: =, ;, *, +, (, ), ID s6 s8 E s s1 ε ε s2 s’ s3 E = s4 E ε ε s5 ; s7 ;
  • 81. LL Parsing Adaptive LL(*) 58 D0 s1 ID :1 = s2 s4s3 :2 ( ID ) ; D0 s1 ID :1 = After parsing: x = y; f(x); Dynamically builds the lookahead DFA.
  • 82. LL Parsing Adaptive LL(*) 59 S : x B | y C B : A a C : A b a A : b | ε parse: yba
  • 83. LL Parsing Adaptive LL(*) 60 S : x B | y C B : A a C : A b a A : b | ε input: yba stack: S
  • 84. LL Parsing Adaptive LL(*) 61 S : x B | y C B : A a C : A b a A : b | ε input: yba stack: yC
  • 85. LL Parsing Adaptive LL(*) 62 S : x B | y C B : A a C : A b a A : b | ε input: ba stack: C
  • 86. LL Parsing Adaptive LL(*) 63 S : x B | y C B : A a C : A b a A : b | ε input: ba stack: A ba
  • 87. LL Parsing Adaptive LL(*) 64 S : x B | y C B : A a C : A b a A : b | ε input: ba stack: A ba b in First(A) b in Follow(A) A1 or A2? A1 A2
  • 88. LL Parsing Adaptive LL(*) 65 • Tries to parse using strong LL (without looking at the call stack), when there are multiple alternatives, splits the parser to try all possibilities. 
 • Parsing continues with a graph, handling multiple stacks in paralell to explore all alternatives. • Eventually, if the grammar is unambiguous, only one stack "survives". If multiple stacks survive, an ambiguty has been found and the parser picks the production with lower value.
  • 89. LL Parsing ANTLR 4 66 • Accepts as input any context-free grammar that does not contain indirect or hidden left-recursion. • Generates ALL(*) parses in Java or C#. • Grammars contain lexical and syntactic rules and generate a lexer and a parser. • ANTLR4 grammars can be scannerless and composable by using individual characters as input symbols.
  • 92. lessons learned LL Parsing Summary How can we parse context-free languages effectively? • predictive parsing algorithms 68
  • 93. lessons learned LL Parsing Summary How can we parse context-free languages effectively? • predictive parsing algorithms Which grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages 68
  • 94. lessons learned LL Parsing Summary How can we parse context-free languages effectively? • predictive parsing algorithms Which grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages How can we generate compiler tools from that? • implement automaton • generate parse tables 68
  • 95. lessons learned LL Parsing Summary How can we parse context-free languages effectively? • predictive parsing algorithms Which grammar classes are supported by these algorithms? • LL(k) grammars, LL(k) languages How can we generate compiler tools from that? • implement automaton • generate parse tables What are other techniques for implementing top-down parsers? • Parser Combinators • PEGs • ALL(*) 68
  • 97. learn more LL Parsing Literature formal languages Noam Chomsky: Three models for the description of language. 1956 J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata Theory, Languages, and Computation. 2006 69
  • 98. learn more LL Parsing Literature formal languages Noam Chomsky: Three models for the description of language. 1956 J. E. Hopcroft, R. Motwani, J. D. Ullman: Introduction to Automata Theory, Languages, and Computation. 2006 syntactical analysis Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java, 2nd edition. 2002 Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers: Principles, Techniques, and Tools, 2nd edition. 2006 69
  • 100. learn more LL Parsing Literature ALL(*) Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the power of dynamic analysis. In OOPSLA 2014. 70
  • 101. learn more LL Parsing Literature ALL(*) Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the power of dynamic analysis. In OOPSLA 2014. Parsing Expression Grammars Bryan Ford. Parsing Expression Grammars: a recognition-based syntactic foundation. In POPL 2004. 70
  • 102. learn more LL Parsing Literature ALL(*) Terence John Parr, Sam Harwell, Kathleen Fisher. Adaptive LL(*) parsing: the power of dynamic analysis. In OOPSLA 2014. Parsing Expression Grammars Bryan Ford. Parsing Expression Grammars: a recognition-based syntactic foundation. In POPL 2004. Parser Combinators Graham Hutton. Higher-Order Functions for Parsing. Journal of Functional Programming, 1992. A. Moors, F. Piessens, Martin Odersky. Parser combinators in Scala. Technical Report Department of Computer Science, K.U. Leuven, February 2008. 70
  • 105. copyrights LL Parsing Pictures Slide 1: Book Scanner by Ben Woosley, some rights reserved Slides 5-7: Noam Chomsky by Fellowsisters, some rights reserved Slide 8, 9, 26-28: Tiger by Bernard Landgraf, some rights reserved Slide 32: Ostsee by Mario Thiel, some rights reserved 73