3. 3
OVERVIEW
Who I Am
The Assignment
Renovating applications
The Strategy
Lessons Learned
Unsolved Problems
4. 4
WHO AM I
Eric Jan Malotaux
Software Architect with Ordina, ICT service provider in the Netherlands
Trained as a musician and musicologist
27 years experience in software development
What I learned
Automated procedures (software factory) as important as programming skills
Communication and trust within a team even more important
Specialties – anything that speeds up software development
Software factories – automating as much as possible
Agile development (extreme programming, Scrum)
Model-driven code generation
6. 6
THE ASSIGNMENT
THE APPLICATION
A very succesful mortgage application
185 screens
100 users
Quarterly releases
Sizeable model: 200,000 lines (10Mb) of XMI
Using a proprietary model-driven code generator
Several interfaces to external systems: queues
Several native Windows DLL's for complicated calculations
7.
8. 8
THE ASSIGNMENT
THE MODELER/GENERATOR
Old, but very complete
Model in relational tables (DB2)
Modeler written in MS-Access
C++ Class Tree to keep model in memory
Loader containing mapping (DB → Class Tree) knowledge
Hand-written C++ framework, modeled
Persistence support classes, including lazy loading
User Interface support classes
Action Language: COOL (Common Object Oriented Language)
Methods on objects
Actions on UI elements, like buttons
Implicit metamodel (as far as known)
9. 9
THE ASSIGNMENT
WHY TRANSFORM?
Dependent on outdated C++ compiler and libraries
Scarce expertise
Model-driven techniques in general
The proprietary modeler/generator in particular
C++
Modeler/generator maintained, but not further development
Doesn't fit in Enterprise Architecture
Difficult to offshore
Want a mainstream Java web application
And no code generator anymore please, thank you very much.
No dependency on proprietary tools
Current developers where quite happy, though
13. Legacy application Renovated application
Goals
Requirements
FAST FAST Mod4J
FAST
model C++ C++ C++
extractor
model Xtend2
mogram
model
extractor extractor generator
(database) (xmi) (textual DSL)
(xmi)
Xtend2
Xpand
C++ generator C++ generator
generator
C++ Java
implementatio implementatio
n n
15. Old Intermediate New
(Enterprise Architect)
SQL C++ UML Ecore
M3
expressed in Abstract expressed in expressed in expressed in
Syntax
FAST FAST
EA C++ FAST EA ecore
M2 metamodel “Class Tree”
import metamodel export Expo
metamodel
(relational) rt (ecore)
gene
ra
to
r
expressed in expressed in expressed in
(xten
d
FMA
2)
gen
FMA FMA e
FMA Model FMA
expor
M1 model gen
load
e
object
t
r model
(database) graph a (xmi)
r (C++) t
a o
t r
o s
r (xte
s n
(C+ d
+ 2
FMA FMA
M0 user data
) )
application
16. Ext
r
a
Enterpri
se
c Legenda
t
Architect
o given
r
FAST gen
FAST
manual
FAST e EMF
“Class metamo tools
metamo r generato
Tree del
del a r
” (ecore) generated
t
o
r
FMA (xte
n FMA FAST
model Load/
d model model
FM
(databas extract
2 (XMI) Java API
A
e)
) gen
e
r
a
FAST t
FMA editor o
model r
(xte
n
d
JNA FMA 2
FMA )
adapters example
(Java)
(Java) (Java)
FAST FAST
maven/
framewo framewo
javac/
rk rk
build
(C++) (Java)
External External
(MQ) (MQ) FMA
clas clas (JWS/jar FMA
ses ses ) data
(C++) (Java)
18. 18
LESSONS LEARNED
The value of an explicit metamodel
Help in understanding the model
Generated export program
Generated tree editor
Generated model API
Basis for a new DSL
19. 19
LESSONS LEARNED
Generator architecture
Consistent naming
Use an intermediate model
20. 20
LESSONS LEARNED
Incompatible source/target architecture
Layering: classic client → server, modern three-tiered
Lazy loading assumed in the COOL Action Language
One Action Language for UI and Class Methods
21. 21
LESSONS LEARNED
Know your tools
Know your tools
EMF is a great tool, once you obtain a copy of the EMF book
Xtext2, needed for COOL Parser, difficult to integrate
Xtend2, nice but still buggy
22. 22
REMAINING UNSOLVED PROBLEMS
Eclipse is a hard to tame beast!
Working combination of (versions of) plugins
Hard to get rid of red crosses
Continuous integration
Maven ↔ OSGi: two worlds
Eclipse artifacts (not) in public maven repositories
23. 23
REMAINING CHALLENGE
Maintaining the migrated model
Use the EMF-generated tree editor
already as good as, or better than, the Access application
Derive an Xtext grammar from the metamodel
Manual adjustments
Have a DSL almost for free
Familiar concepts: the metamodel (language) was not changed.
This presentation is tcase study about “Transforming a 15 year old model-driven application from C++ to Java”
Let me first give an overview of my presentation.
First I will introduce myself. Then I will describe the assignment that this presentation is about Since the assignment was essentially a renovation of an existing application, I will describe how renovation of old applications is done in general. Next I will describe how we applied different strategies in succession as we learned on the job. Finally I will try to draw some lessons from the experience and mention a number of problems that we did not yet find a good solution for.
What I learned in those 27 years: - Automated procedures as important as programming skills or architecture - Automated build - Version management - Automated deployment - (Automatic unit testing) - Communication and trust within team Specialties: - Software factories 25+ years - used to be new but now more common - Model-driven code generation 10+ years - Agile Development - eXtreme Programming 10+ years - Scrum 3+ years
What was our assignment?
A very succesful mortgage application - 185 screens (or parts of screens) - 100 users - regular quarterly releases - sizeable model: 200.000 lines, 10Mb of XMI Using a proprietary model-driven C++ generator written in C++ Several interfaces to external systems, queues Several native Windows DLL's for complicated calculations
Implicit metamodel (as far as known) Model in relational tables (DB2) C++ “Class Tree” to load model in memory - analogous to the EMF generated Java API Loader containing the mapping knowledge Hand-written C++ framework - persistence support classes, including lazy loading - user interface support classes Action Language: COOL (Common Object Oriented Language), used everywhere: - methods of objects - actions on UI elements (buttons, fields)
Scarce expertise - model-driven techniques in general - this particular modeler/generator in particular - C++ Dependent on outdated C++ compiler and libraries Modeler/generator maintained but not further developed: dead end Difficult to offshore Wanted: a regular Java webapplication No code generator - dependency on proprietary tool - Current developers where quite happy with the tool and productive
Start to establish goals for a new application Collect requirements Make specification Implement However: in many cases there is only source code, and if there is more documentation, it is often outdated. How to renovate: - automatic source translation? No idiomatic Java. - Better: reverse engineer specifications and re-implement - Even better (but more difficult): reverse engineer reqs. - End to Goals.
But in our case we had specification in the form of an executable (therefore correct) model. 1) Write a new C++ generator to generate Java - Problem: large conceptual gap to bridge: - C++ to Java (fluency in both needed) - Different architecture - On top of model driven skills 2) Use Mod4J as intermediate model - Smaller conceptual gap - Built-in Java architecture - Based on written reference architecture 3) Migrate model to EMF for productivity - Better tools - Better software factory 4) Mod4J incompatibly architecture - fat client ↔ layering - COOL language everywhere - lazy loading
Want to use EMF - Need an ecore metamodel - Derive from “Class Tree” - Via Enterprise Architect im/export - Generate an export program - Generate our new application - Loading time: 1,5 second - But initially 25 seconds (!) - Need a HashMap for id's
An intermediate model provides a natural place for names
Classic client server → modern three tiered layered Lazy loading assumed in de Action Language One Scope with UI scripts and Class Methods
Know your tools - EMF is great, but you need the book - Xtext2 for the COOL Parser difficult to integrate - Xtend2, nice but buggy and slow
Eclipse is difficult - Working combination of (versions of) plugins - Hard to get rid of the red crosses Continuous integration - Maven ↔ OSGi: two worlds - Solve a similar problem - Different audiences: tool/application builders - But MDCD overlap the two - Eclipse artifacts not automatically in maven repositories.
How to maintain the migrated model: - Use EMF tree editor - already quite good - Derive an Xtext grammar from the metamodel - beautify - DSL for “free” - The language is still the same - Familiar for the maintainers