Conference presentation at the GRaMoT 2008 workshop, co-located with ICSE 2008.

1. Incremental pattern matching in the
VIATRA d lt f ti t
VIATRA model transformation system
Gábor Bergmann
András Ökrös
István Ráth (rath@mit.bme.hu)
Dániel Varró
Dániel Varró Department of Measurement and
Department of Measurement and
Gergely Varró Information Systems
Budapest University of Technology and
University of Technology and
Economics

2. Overview
• Introduction
• Concepts
•P f
Performance analysis
l i
• Future work
• Summary

3. Introduction

4. Incremental model transformations
model transformations
• Key usage scenarios for MT:
▫ Mapping between languages
▫ Intra‐domain model manipulation
Model execution
Validity checking (constraint evaluation)
Validity checking (constraint evaluation)
• They work with evolving models.
▫ Users are constantly changing/modifying them.
▫ Users usually work with large models.
Users usually work with large
• Problem: transformations are slow
▫ To execute… (large models)
▫ and to re‐execute again and again (always starting from scratch).
• Solution: incrementality
▫ Take the source model, and its mapped counterpart;
▫ Use the information about how the source model was changed;
▫ M
Map and apply the changes (but ONLY the changes) to the target model.
d l h h (b ONLY h h ) h d l

5. Towards incrementality
Towards incrementality
• How to achieve incrementality?
How to achieve incrementality?
▫ Incremental updates: avoid re‐generation.
Don t recreate what is already there.
Don’t recreate what is already there
Use reference (correspondence) models.
▫ Incremental execution: avoid re‐computation.
Incremental execution: avoid re computation.
Don’t recalculate what was already computed.
How?

6. Incremental graph pattern matching
Incremental graph pattern matching
• Graph transformations require pattern matching
p q p g
• Goal: retrieve the matching set quickly
• How?
▫ Store (cache) matchings
▫ Update them as the model changes
Update precisely (incrementality)
U d t i l (i t lit )
• Expected results: good, if…
▫ There is enough memory ( )
There is enough memory (*)
▫ Queries are dominant
▫ Model changes are relatively sparse (**)
▫ e.g. synchronization, constraint evaluation, …

7. Operational overview
XForm
pattern interpreter model manipulation
matching
Incremental
event
VIATRA
pattern
notification Model space
matcher
updates

8. Architecture
LS pattern
LS tt
Model XForm matcher
parser parser
XML XForm
VIATRA
V
serializer interpreter
Incremental
Native importer & pattern
A2 Framewo
loader interface
l d i t f matcher
Core interfaces
ork
VIATRA Model space Program model store

9. Concepts

10. Core idea: use RETE nets
• RETE network INPUT
▫ node: (partial) matches of a
node: (partial) matches of a
(sub)pattern
t3 Model space
Model space p1 p2 p3 t1 t2 k1 k2 t3
▫ edge: update propagation t3 t3 t3
• Demonstrating the principle
Demonstrating the principle
▫ input: Petri net Input nodes
: Place
p1 p2 p3
d : Token
k1 k2
: Transition
t1 t2 t3
▫ pattern: fireable transition
▫ Model change: new transition
t3
(t3)
t1
Intermediate
p1, k1 p2, k2
p1
nodes p1, k1, t1 p2, k2, t3
p3
3 p2
t2 p2, k2, t3
t3
Production node
Production node
p1, k1, t1, p3 p2, k2, t2, p3

11. RETE network construction
RETE network construction
• Key: pattern decomposition
y p p
▫ Pattern = set of constraints (defined over pattern variables)
▫ Types of constraints: type, topology (source/target),
hierarchy (containment), attribute value, generics
h h ( ) b l
(instanceOf/supertypeOf), injectivity, [negative] pattern
calls, …
calls, …
• Construction algorithm (roughly)
▫ 1. Decompose the pattern into elementary constraints (*)
▫ 2. Process the elementary constraints and connect them
with appropriate intermediate nodes (JOIN, MINUS‐JOIN,
UNION, …)
UNION )
▫ 3. Create terminator production node

12. Key RETE components
Key RETE components
INPUT
• JOIN node
JOIN node INPUT INPUT
▫ ~relational algebra:
natural join
natural join
JOIN
• MINUS‐JOIN
▫ Negative existence
Negative existence JOIN
PRODUCTION
sourcePlace

13. Other VIATRA features
Other VIATRA features
• Pattern calls
▫ Simply connect the production nodes
▫ Pattern recursion is fully supported
• OR patterns
OR‐patterns
▫ UNION intermediate nodes
• Check conditions
▫ check (value(X) % 5 == 3)
▫ check (length(name(X)) < 4)
▫ check (myFunction(name(X))!=‘myException’)
check (myFunction(name(X))!= myException )
▫ Filter and term evaluator nodes
• Result: full VIATRA transformation language support; any
pattern can be matched incrementally.

14. Updates
• Needed when the model space changes
Needed when the model space changes
• VIATRA notification mechanism (EMF is also possible)
▫ Transparent: user modification, model imports, results of a
Transparent: user modification, model imports, results of a
transformation, external modification, … RETE is always
updated!
• Input nodes receive elementary modifications and
release an update token
▫ Represents a change in the partial matching (+/‐)
• Nodes process updates and propagate them if needed
▫ PRECISE update mechanism

15. Performance analysis
Performance analysis

16. Performance
• In theory…
▫ Building phase is slow (“warm‐up”)
How slow?
▫ Once the network is built, pattern matching is an
,p g
“instantaneous” operation.
Excluding the linear cost of reading the result set.
▫ But… there is a performance penalty on model manipulation.
p p y p
How much?
• Dependencies?
▫ Pattern size
Pattern size
▫ Matching set size
▫ Model size
▫ …?
?

17. Benchmarking
• Example transformation: Petri net simulation
▫ One complex pattern for the enabledness condition
▫ Two graph transformation rules for firing
▫ As‐long‐as‐possible (ALAP) style execution (“fire at will”)
▫ Model graphs:
A “large” Petri net actually used in a research project (~60 places, ~70
transitions, ~300 arcs)
Scaling up: automatic generation preserving liveness ( t 100000
S li t ti ti i li (up to 100000
places, 100000 transitions, 500000 arcs)
• Analysis
▫ Measure execution time (average multiple runs)
Measure execution time (average multiple runs)
▫ Take “warm‐up” runs into consideration
• Profiling
▫ Measure overhead network construction time
Measure overhead, network construction time
▫ “Normalize” results

18. Profiling results
Profiling results
• Model manipulation overhead: ~15% (of overall CPU
p (
time)
▫ Depends largely on the transformation!
• Memory overhead
Memory overhead
▫ Petri nets (with RETE networks) up to ~100000 fit into 1‐
1.5GB RAM (VIATRA model space limitations)
▫ Grows linearly with model size (as expected)
( )
▫ Nature of growth is pattern‐dependent
• Network construction overhead
Network construction overhead
▫ Similar to memory; pattern‐dependent.
▫ PN: In the same order as VIATRA’s LS heuristics
initialization.
initialization

19. Execution times
Execution times Matches/outperforms
Sparse Petri net benchmark GrGEN.NET for large
models and high
d l d hi h
1000000 iteration counts.
Viatra/RETE
Three orders of (x1k)
( 1k)
100000 magnitude and Viatra/LS (x1k)
growing…
ms)
10000
on time (m
GrGenNET (x1k)
G G NET ( 1k)
1000
Viatra/RETE
Executio
100
(x1M)
( 1M)
GrGen.NET
10 (x1M)
100 1000 10000 100000
Petri net size

20. Benchmarking summary
Benchmarking summary
• Predictable near‐linear growth
Predictable near linear growth
▫ As long as there is enough memory
▫ Certain problem classes: constant execution time ☺
Certain problem classes: constant execution time ☺
▫ A ga

21. Improving performance
Improving performance
• Strategies
▫ Improve the construction algorithm
Memory efficiency (node sharing)
Memory efficiency (node sharing)
Heuristics‐driven constraint enumeration (based on
p
pattern [and model space] content)
[ p ] )
▫ Parallelism
Update the RETE network in parallel with the
transformation
Parallel network construction
▫?

22. Future work
Future work

23. More benchmarking…
More benchmarking
• Ongoing research
Ongoing research
▫ Extending the Varro benchmark
Mutex STS/LTS
ORM
▫ Extended benchmarking use cases
Extended benchmarking use cases
Simulation (model execution)
Synchronization
y
Constraint evaluation
▫ Parallel transformations

24. Event‐driven live transformations
Event‐driven live transformations
• Problem: MT is mostly batch‐like
y
▫ But models are constantly evolving Frequent re‐
transformations are needed for
mapping
synchronization
constraint checking
…
• An incremental PM can solve the performance problem,
but a formalism is needed
▫ to specify when to (re)act
▫ and how.
• Ideally the formalism should be MT‐like
Ideally, the formalism should be MT like.

25. Event‐driven live transformations (cont d)
Event‐driven live transformations (cont’d)
• An idea: represent events as model elements.
p
• Our take: represent events as changes in the matching
set of a pattern.
▫ ~generalization
• Live transformations
▫ maintain the context (variable values, global variables, …);
i t i th t t ( i bl l l b l i bl )
▫ run as a “daemon”, react whenever necessary;
▫ as the models change the system can react instantly since
as the models change, the system can react instantly, since
everything needed is there in the RETE network: no re‐
computation is necessary.
• Paper accepted at ICMT2008.

26. Summary
• Incremental pattern matching support integrated
Incremental pattern matching support integrated
into VIATRA2 R3
▫ Based on the RETE algorithm
Based on the RETE algorithm
▫ Provides full support for the pattern language
▫ High performance in certain problem classes
High performance in certain problem classes
• Future
▫ Performance will be further improved
Performance will be further improved
▫ New applications in live transformations