Visit to a blind student's school🧑🦯🧑🦯(community medicine)
Compiler Components and their Generators - LR Parsing
1. Compiler Components & Generators
Traditional Parsing Algorithms
Guido Wachsmuth
Delft
Course IN4303
University of
Technology Compiler Construction
Challenge the future
2. Recap: Traditional Parsing Algorithms
lessons learned
How can we parse context-free languages effectively?
• predictive parsing algorithms
• LR parsing algorithms
Which grammar classes are supported by these algorithms?
• LL(k) grammars, LL(k) languages
• LR(k) grammars, LR(k) languages
How can we generate compiler tools from that?
• implement automata
• generate parse tables
Traditional Parsing Algorithms 2
13. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
Traditional Parsing Algorithms 5
14. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 5
15. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 5
16. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
Traditional Parsing Algorithms 5
17. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 5
18. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 5
19. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
Traditional Parsing Algorithms 5
20. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
Traditional Parsing Algorithms 5
21. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
Traditional Parsing Algorithms 5
22. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
Traditional Parsing Algorithms 5
23. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
Traditional Parsing Algorithms 5
24. Recap: LR parsing
example
$ 7 * 3 + 7 * 3 $
$ E * 3 + 7 * 3 $
$ E * E + 7 * 3 $
$ E + 7 * 3 $
$ E + E * 3 $
$ E + E * E $
$ E + E $
$ E $
$ S
Traditional Parsing Algorithms 5
25. Recap: LR parsing
parse table
rows T1 ... N1 ...
• states of a DFA 1 s3
2 a g5
columns
3 r1
• topmost stack symbol, Σ, N
4 r2
entries 5
• reduce, rule number 6 g1
• shift, goto state 7 s1
• goto state
8
• accept
...
Traditional Parsing Algorithms 6
27. LR(0) parse tables
items, closure & goto
S’ → . S $
S→x
S→(L)
L→S
L→L,S
Traditional Parsing Algorithms 7
28. LR(0) parse tables
items, closure & goto
item
S’ → . S $
S→x
S→(L)
L→S
L→L,S
Traditional Parsing Algorithms 7
29. LR(0) parse tables
items, closure & goto
item closure
• for every item A → α . X β
S’ → . S $
• for every rule X → γ
• add item X → . γ
S→x
S→(L)
L→S
L→L,S
Traditional Parsing Algorithms 7
30. LR(0) parse tables
items, closure & goto
closure
• for every item A → α . X β
S’ → . S $
S→.x
• for every rule X → γ
S→.(L) • add item X → . γ
S→x
S→(L)
L→S
L→L,S
Traditional Parsing Algorithms 7
31. LR(0) parse tables
items, closure & goto
S’ → . S $
S→.x
S→.(L)
S→x
S→(L)
L→S
L→L,S
Traditional Parsing Algorithms 7
32. LR(0) parse tables
items, closure & goto
S’ → . S $
S→.x
S→.(L)
S S→x
S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
33. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x
S→.(L)
S S→x
S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
34. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x
S→.(L) (
S→(.L)
S S→x
S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
35. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x
S→.(L) (
S→(.L)
L→.S
L→.L,S
S→.x S→x
S
S→.(L) S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
36. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) (
S→(.L)
L→.S
L→.L,S
S→.x S→x
S
S→.(L) S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
37. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) (
S→(.L)
L→.S
L→.L,S
( S→.x
S S→x
S→.(L) S→(L)
L→S
L→L,S
S’ → S . $
Traditional Parsing Algorithms 7
38. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) (
S→(.L)
L→.S
L→.L,S
( S→.x
S S→x
S→.(L) S→(L)
S L→S
L→L,S
S’ → S . $ L→S.
Traditional Parsing Algorithms 7
39. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) (
S→(.L)
L→.S
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S L→S
L→L,S
S’ → S . $ L→S.
Traditional Parsing Algorithms 7
40. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) (
S→(.L)
L→.S
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
41. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $
S→.x x
S→.(L) ( L→L,.S
S→(.L)
L→.S ,
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
42. LR(0) parse tables
items, closure & goto
x
S→x.
S’ → . S $ L→L,.S
S→.x x S→.x
S→.(L) ( S→.(L)
S→(.L)
L→.S ,
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
43. LR(0) parse tables
items, closure & goto
x x
S→x.
S’ → . S $ L→L,.S
S→.x x S→.x
S→.(L) ( S→.(L)
S→(.L)
L→.S ,
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
44. LR(0) parse tables
items, closure & goto
x x
S→x.
S’ → . S $ L→L,.S
S→.x x S→.x
S→.(L) ( ( S→.(L)
S→(.L)
L→.S ,
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
45. LR(0) parse tables
items, closure & goto
L→L,S.
S
x x
S→x.
S’ → . S $ L→L,.S
S→.x x S→.x
S→.(L) ( ( S→.(L)
S→(.L)
L→.S ,
L→.L,S
( S→.x
S L S→(L.) S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ L→S. S→(L).
Traditional Parsing Algorithms 7
46. LR(0) parse tables
items, closure & goto
L→L,S.9
S
x x
S→x. 2
S’ → . S $ 1 L→L,.S8
S→.x x S→.x
S→.(L) ( ( S→.(L)
S→(.L)3
L→.S ,
L→.L,S
( S→.x
S L S→(L.)5 S→x
S→.(L) L→L.,S S→(L)
S ) L→S
L→L,S
S’ → S . $ 4 L→S. 6 S→(L).7
Traditional Parsing Algorithms 7
64. SLR parse tables
shift-reduce conflicts
E
S→E.$
S→.E$
E→.T+E
E→.T T E→T.+E
T→.x
E→T.
T +
x E→T+.E
E→.T+E E→T+E
E→.T E→T
T→x. x T→.x E E→T+E. T→x
Traditional Parsing Algorithms 9
65. SLR parse tables
shift-reduce conflicts
E
S→E.$ 2
S→.E$
E→.T+E
E→.T T E→T.+E
T→.x 1
E→T. 3
T +
x E→T+.E
E→.T+E E→T+E
E→.T E→T
T→x. x T→.x E E→T+E.
6
5 4 T→x
Traditional Parsing Algorithms 9
66. SLR parse tables
shift-reduce conflicts
x + $ E T
1 s5 g2 g3
E
S→E.$ 2 2 a
S→.E$
E→.T+E 3 r2 ? r2
E→.T T 4 s5 g6 g3
E→T.+E
T→.x 1 5 r3 r3 r3
E→T. 3
6 r1 r1 r1
T +
x E→T+.E
E→.T+E E→T+E
E→.T E→T
T→x. x T→.x E E→T+E.
6
5 4 T→x
Traditional Parsing Algorithms 9
67. SLR parse tables
shift-reduce conflicts
x + $ E T
1 s5 g2 g3
E
S→E.$ 2 2 a
S→.E$
E→.T+E 3 s4 r2
E→.T T 4 s5 g6 g3
E→T.+E
T→.x 1 5 r3 r3
E→T. 3
6 r1
T +
x E→T+.E
E→.T+E E→T+E
E→.T E→T
T→x. x T→.x E E→T+E.
6
5 4 T→x
Traditional Parsing Algorithms 9
68. LR(1) parse tables
look-ahead
E
S→E.$ ?
S→.E$ ?
E→.T+E $
E→.T $ T E→T.+E $
T→.x +$
E→T. $
T +
x E→T+.E $
E→.T+E $ E→T+E
E→.T $ E→T
T→x. +$ x T→.x +$ E E→T+E. $ T→x
Traditional Parsing Algorithms 10
69. LR(1) parse tables
look-ahead
x + $ E T
1 s5 g2 g3
E
S→E.$ ? 2 a
S→.E$ ?
E→.T+E $ 3 s4 r2
E→.T $ T 4 s5 g6 g3
E→T.+E $
T→.x +$ 5 r3 r3
E→T. $
6 r1
T +
x E→T+.E $
E→.T+E $ E→T+E
E→.T $ E→T
T→x. +$ x T→.x +$ E E→T+E. $ T→x
Traditional Parsing Algorithms 10
70. LALR(1) parse tables
state space reduction
unify states
• with same items
• and same outgoing transitions
• but different look-ahead sets
might introduce new conflicts
Traditional Parsing Algorithms 11
124. public boolean
authenticate(String user, String pw) {
SQL stm = <| SELECT id FROM Users
WHERE name = ${user}
AND password = ${pw} |>;
return executeQuery(stm).size() != 0;
}
156. public boolean
authenticate(String user, String pw) {
SQL stm = <| SELECT id FROM Users
WHERE name = ${user}
AND password = ${pw} |>;
return executeQuery(stm).size() != 0;
}
166. Summary
lessons learned
How can we generate LR parse tables?
• items, closure, goto
How can we improve LR(0) parse table generation?
• SLR: consider FOLLOW sets to avoid shift-reduce conflicts
• LR(1): consider look-ahead in states
• LALR(1): unify LR(1) states to reduce state space
Traditional Parsing Algorithms 89
167. Summary
lessons learned
How can we generate LR parse tables?
• items, closure, goto
How can we improve LR(0) parse table generation?
• SLR: consider FOLLOW sets to avoid shift-reduce conflicts
• LR(1): consider look-ahead in states
• LALR(1): unify LR(1) states to reduce state space
Why are efficient parsing algorithms problematic?
• not longer pure, declarative, beautiful
• paradise lost: seven plagues
• paradise regained: scannerless generalised parsing
Traditional Parsing Algorithms 89
168. Summary
lessons learned
How can we generate LR parse tables?
• items, closure, goto
How can we improve LR(0) parse table generation?
• SLR: consider FOLLOW sets to avoid shift-reduce conflicts
• LR(1): consider look-ahead in states
• LALR(1): unify LR(1) states to reduce state space
Why are efficient parsing algorithms problematic?
• not longer pure, declarative, beautiful
• paradise lost: seven plagues
• paradise regained: scannerless generalised parsing
Traditional Parsing Algorithms 89
170. Literature
learn more
syntactical analysis
Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java,
2nd edition. 2002
Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers:
Principles, Techniques, and Tools, 2nd edition. 2006
Traditional Parsing Algorithms 90
171. Literature
learn more
syntactical analysis
Andrew W. Appel, Jens Palsberg: Modern Compiler Implementation in Java,
2nd edition. 2002
Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman, Monica S. Lam: Compilers:
Principles, Techniques, and Tools, 2nd edition. 2006
generalised parsing
Eelco Visser: Syntax Definition for Language Prototyping. PhD thesis 1997
M.G.J. van den Brand, J. Scheerder, J.J. Vinju, and E. Visser: Disambiguation
Filters for Scannerless Generalized LR Parsers. CC 2002
Lennart C. L. Kats, Eelco Visser, Guido Wachsmuth: Pure and Declarative
Syntax Definition - Paradise Lost and Regained. SPLASH 2010
Traditional Parsing Algorithms 90
172. Outlook
coming next
lectures
• Lecture 14: Beyond grammarware
Question & Answer Jan 5
• 10 questions, submit & vote
Lab Dec 10
• support expressions
• support statements
• support ordinary classes & fields
Traditional Parsing Algorithms 91
175. Pictures
copyrights
Slide 1:
Book Scanner by Ben Woosley, some rights reserved
Slide 12:
West Cornwall Pasty Co. by Dominica Williamson, some rights reserved
Slide 91:
Ostsee by Mario Thiel, some rights reserved
Slide 92:
Questions by Oberazzi, some rights reserved
Slide 93:
Too Much Credit by Andres Rueda, some rights reserved
Slides 14-87:
Pure and Declarative Syntax Definition: Paradise Lost and Regained, some rights reserved
Traditional Parsing Algorithms 94
Notas do Editor
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n
computer science: lexical syntax\n\ncan write this as a regular grammar\n