1. From big
GORM-centered
into a cloud of
fast nodes
Jorge Uriarte Aretxaga
@jorgeuriarte
http://www.slideshare.com/gailen/

2. About me

3. @jorgeuriarte
jorge.uriarte@gailen.es
About me

4. Origins

5. In the beginning...

6. In the beginning...

7. Well-known territory

8. Well-known territory
JVM based!

9. Well-known territory
JVM based!
Old good SQL
& RDBMs

10. Well-known territory
JVM based!
Old good SQL
& RDBMs
GORM magic!

12. Productive development environment

13. Productive development environment
Affordable learning curve

14. Productive development environment
Affordable learning curve
Good-enough performance

15. Evolution

19. “Mmmhhh... I wouldn’t
build it that way
nowadays...”
Enrique Amodeo

20. “Mmmhhh... I wouldn’t
build it that way
nowadays...”
Enrique Amodeo

21. “Mmmhhh... I wouldn’t
build it that way
nowadays...”
Enrique Amodeo
“Mmmhhh...
nowadays...
neither would I!”
Me

25. New brands
& countries

26. Reality Check

27. Reality bites

28. Reality bites

34. crementalMode
-XX: +UseTLAB -XX:+CMSIn
seConcMarkSweepGC ingThreshold=0
-XX: +UseParNewGC -XX:+U tyCy cle=10 -XX:MaxTenur
X:MaxPermSize=128m X:CMSIncrementalDu
-s erver -Xmx1024m -X ment alDutyCycleMin=0 -X itGC
acing -XX:CMSIncre -XX:+DisableExplic
-X X:+CMSIncrementalP gOcc upancyFraction=60
6 -XX:CMSInitiatin
-X X:SurvivorRatio=25

35. C
P arNewG e
-X X:+Use ntalMod
e =128m SIncreme
MaxP ermSiz -XX:+CM 0
l-XX:in= crementalMode
eM +CMSIn
-X se: rNewGC -XX:+U AB rkSwee alDut e=10 c MaxTen 0
y
4g X:+UX PaX:+UseTLseConcMammentpGC tyCyclyC -XX:old=uringThreshold=0
x -X
-XX:+UseTLAB
3g X:-Xm 8m mentC DutyCycleMin=0M-XX:ncrere entalDu GCgThresh
-server -Xmx r -Xms -X CMSIncre pG al -
1024m -XX:MaxPermS
ize=12 X CMSInc
SI +Disableenurit n 60
X:+CserveentalP
MSIncrem acing kSweegOccupng FractionC 0 -XX: MaxT Explic i
cMar atin ancy -XX: =6 a ction=
ancyFr
-X -
X:SurvivorRatio=25on
s eC 6 -XX:CMSInitialPaci - XX: p
-XX:+U SIncrement utyCycle=10 tiatingOccu
-X
i
- XX:+CM ncrementalD 6 -XX:CMSIn
I 5
- XX:CMS ivorRatio=2 GC
-X X:Surv bleExplicit
sa
- XX:+Di

36. Productivity slows down...

37. Fragility...

38. Evolution stopped...

40. Slow development

41. Slow development
Fragile system... prone to errors

42. Slow development
Fragile system... prone to errors
Tuning hell with JVM heap

43. Slow development
Fragile system... prone to errors
Tuning hell with JVM heap
Terracota cluster issues

44. Slow development
Fragile system... prone to errors
Tuning hell with JVM heap
Terracota cluster issues
Overall excessive complexity

45. Change?
Surgery?
Refactor?
keep tuning?

46. Meanwhile... in another
mental thread...

49. A solution looking for a problem?

50. But get back to the previous point...

51. But get back to the previous point...
Will you dare jump off it?

54. • New side-project
• Potentially deadly traffic
• Still to be tighly connected with
our core system

55. • New side-project
• Potentially deadly traffic
• Still to be tighly connected with
our •core system
Solutions?
•Dedicated system?
•More machines & tuning?
•Catch that train!

62. Proof of concept and... JUMP!

66. Some details?

67. Fast importer from
Quick & dirty (1w)
existing XMLs
Elasticsearch
Prototype
- show me the code -
Redis Groovy!!!
Fast Controllers (XML parsing, JSON
Ajax (fewer requests) rendering, grails
controllers, ...)

69. • Simply groovy.util.XmlSlurper

70. • Simply groovy.util.XmlSlurper
• Two importers, really

71. • Simply groovy.util.XmlSlurper
• Two importers, really
• Bulk

72. • Simply groovy.util.XmlSlurper
• Two importers, really
• Bulk
• Incremental (continuous)

73. • Simply groovy.util.XmlSlurper
• Two importers, really
• Bulk
• Incremental (continuous)
• Born as a temporary hack...

74. • Simply groovy.util.XmlSlurper
• Two importers, really
• Bulk
• Incremental (continuous)
• Born as a temporary hack...
soon to give way to a better system

76. • Redis master/slaves replication

77. • Redis master/slaves replication
• Fast

78. • Redis master/slaves replication
• Fast
• Trivial

79. • Redis master/slaves replication
• Fast
• Trivial
• Fast

80. • Redis master/slaves replication
• Fast
• Trivial
• Fast
• Trivial

81. • Redis master/slaves replication
• Fast
• Trivial
• Fast
• Trivial
• Did I mention fast and trivial?

82. Deadly simple Grails app

83. Deadly simple Grails app

84. Deadly simple Grails app

85. Deadly simple Grails app
“Look mum, no GORM!”

86. From ‘searchable’ to ElasticSearch

87. • Fat-trimmed new fronts
• Fast, *light* application
• Information still safe in the core
system
• Much more capacity with less
resources
• Simple integration

88. Old system
New system
Wait... where did the red arrows go?

89. Subdomain based write-through

90. Subdomain based write-through
• Login, session, tracking...

91. Subdomain based write-through
• Login, session, tracking...
• Purchases, payments, subscriptions

92. Subdomain based write-through
• Login, session, tracking...
• Purchases, payments, subscriptions
• Everything goes to our *good old core
system*

93. Subdomain based write-through
• Login, session, tracking...
• Purchases, payments, subscriptions
• Everything goes to our *good old core
system*
• ACID, consistency, GORM, SQL,
transactions...

94. Subdomain based write-through
• Login, session, tracking...
• Purchases, payments, subscriptions
• Everything goes to our *good old core
system*
• ACID, consistency, GORM, SQL,
transactions...
• ...but now with a small fraction of the
load

95. Evaluation

96. Terracota/Clustering
approach vs “new thing”
Evaluation

97. Terracota/Clustering
approach vs “new thing”
Evaluation
Memory, speed, size,
codebase

98. Terracota/Clustering
approach vs “new thing”
Evaluation
Memory, speed, size,
codebase
Decoupled search
system

99. Terracota/Clustering
approach vs “new thing”
Evaluation
Memory, speed, size,
codebase New problems,
synchronization
Decoupled search
system

100. Productivity increasing again...

101. New landscape

102. New landscape
Get rid of XML importer
(event driven pub/subs...?)

103. New landscape
Get rid of XML importer
(event driven pub/subs...?)
Delete old code (yeah!)

104. Keep pushing for speed
New landscape
Get rid of XML importer
(event driven pub/subs...?)
Delete old code (yeah!)

105. Keep pushing for speed
Keep it simple (this time)
New landscape
Get rid of XML importer
(event driven pub/subs...?)
Delete old code (yeah!)

106. Just ask me... in two years

107. @jorgeuriarte

108. "From big GORM-centered into a cloud of fast
nodes" por Jorge Uriarte se encuentra bajo una
Licencia Creative Commons Atribución-
CompartirIgual 3.0 Unported.
External sources
PITA
http://www.quicksales.com.au/ad/p-i-t-a-pain-in-the-ass-car-number-plate/2606513
Hamster
http://www.flickr.com/photos/sualk61/3117477410/
Cliff view
http://www.flickr.com/photos/epsos/4376727123/sizes/l/
Knight in the white horse
http://www.flickr.com/photos/photoplod/7982134342/
Snail
http://www.flickr.com/photos/raphaelquinet/693813637/
Fast snail
http://www.flickr.com/photos/zorro13/4086173313
Timanfaya devil
http://www.flickr.com/photos/yelacis/6941891989/
Light patterns
http://www.flickr.com/photos/aerosolhalos/6676192013/
Kick in the top of the cliff
http://www.flickr.com/photos/4elevenpix/5056089861/
Homer evolution
http://www.flickr.com/photos/photonquantique/1858685882/
Lego machine
http://www.flickr.com/photos/djimison/3058818283/
Landscape
http://www.flickr.com/photos/blmiers2/6112610781/
Exam
http://www.flickr.com/photos/albertogp123/5843577306/
Future is past
http://www.flickr.com/photos/tind/7533681980/