Lukáš Zapletal Katello on Torquebox / JRuby talk for Developer Conference 2013 Brno Czech Republic (English)

1. KATELLO ON TORQUEBOX
Java Loves Ruby

4. ABOUT ME
Lukáš Zapletal

5. ABOUT ME
@lzap

6. ABOUT ME
@lzap_CZ @lzap80

8. THEME SELECTION
Blah blah blah:
Sky - Beige - Simple - Serif - Night - Default

9. KATELLO

10. TORQUEBOX

11. RUBY

12. JRUBY

13. JAVA

14. JAVA

15. CATS-FREE TALK

16. WHAT IS KATELLO

17. KATELLO IS

18. A OPEN-SOURCE

19. CONTENT AND SYSTEM

20. MANAGEMENT STACK

21. FOR DATACENTERS

22. AND CLOUD

23. IF YOU TAKE ...

24. AND CLOUD

25. FOR DATACENTERS

26. AND CLOUD

27. FOR DATACENTERS

28. AND CLOUD

29. ENOUGH FUN!
What the cloud is?

30. NIST
National Institute of Standards and Technology

31. NIST DEFINITION
the NIST definition of cloud computing

32. NIST DEFINITION

33. NIST DEFINITION

34. NIST DEFINITION

35. NIST DEFINITION

36. NIST DEFINITION

37. NIST DEFINITION

38. NIST DEFINITION

39. CLOUD DEFINITION
Cloud computing is a model for enabling ubiquitous,
convenient, on-demand network access to a shared pool of
configurable computing resources.

40. CLOUD DEFINITION
It can be rapidly provisioned and released with minimal
management effort or service provider interaction.

41. ESSENTIAL CHARACTERISTICS
on-demand service

42. ESSENTIAL CHARACTERISTICS
broad network access

43. ESSENTIAL CHARACTERISTICS
resource pooling

44. ESSENTIAL CHARACTERISTICS
rapid elasticity

45. ESSENTIAL CHARACTERISTICS
measured service

46. SERVICE MODELS
SaaS
PaaS
Iaas

47. DEPLOYMENT MODELS
private cloud
community cloud
hybrid cloud
public cloud

48. WHAT YOU CAN DO WITH KATELLO?

49. RED HAT SUBSCRIPTION
what is it and how it works

50. SUBSCRIPTION MANAGEMENT
import Red Hat subscriptions from Portal
create your own products and subscriptions
register machines and consume them
see some statistics and graphs

51. CONTENT MANAGEMENT
sync RPM content from CDN
sync RPM content from other repositories
sync puppet content from Puppet Forge
separate content into environments and content views
promote content
consume content using yum or puppet
remote install/upgrade content

52. THERE IS FOREMAN

53. PROVISIONING
register installation trees
prepare provisioning templates
provision bare-metal/virtual systems
maintain registry of all systems

54. CONFIGURATION MANAGEMENT
import Puppet classes into Foreman database
assign classes to hosts (existing or provisioned)
assign parameters to classes
collect info from Facter and Puppet
create statistics and graphs

55. KATELLO UI

56. KATELLO UI

57. KATELLO UI

58. RED HAT PRODUCTS
Subscription Asset Manager (SAM)
CloudForms System Engine

59. WHAT IS TORQUEBOX AND JRUBY

60. JRUBY
Ruby 1.8/1.9 on JVM
mature and stable project
JIT and AOT
bidirectional
is in Fedora

61. TORQUEBOX
application platform for Ruby on Rails, Sinatra...
runs atop of JBoss AS
offers services like messaging, scheduling, caching
allows use of clustering, load-ballancing and HA
uses standards where possible

62. WHY SHOULD WE CARE?
why to port to JVM

63. WHY SHOULD WE CARE?
memory :-)

64. WHY SHOULD WE CARE?
performance (skipping for this talk)

65. WHY SHOULD WE CARE?

66. MEMORY IS THE ISSUE
before that we need to cover threads

67. MRI RUBY 1.8
green threads

68. MRI RUBY 1.9
native threads with GIL

69. GLOBAL INTERPRETER LOCK
any time one thread is running _Ruby_ code
no other thread can be running _Ruby_ code

70. GLOBAL INTERPRETER LOCK

71. GLOBAL INTERPRETER LOCK
significant barrier to parallelism
does _not_ limit I/O by the design
but many native rubygems also limits I/O

72. STATE OF THREADING IN MRI RUBY
not the best

73. FORKING SERVERS IN MRI RUBY
threads are not the only options for web concurency
process forking can do the thing too
Linux is good in forking
unfortunately MRI Ruby can't leverage COW memory

74. FORKING SERVERS IN MRI RUBY
Ruby Enterprise Edition solves this for 1.8
Ruby 1.9 has many REE optimalizations (but not COW)
Ruby 2.0 will finally deliver COW-friendly forking
(bitmaps)

75. STATE OF FORKING IN MRI RUBY
not the best

76. DEPLOYMENT OPTIONS WITH RUBY
The Ruby community has always insisted that performance
is not an issue while constantly searching for higher
performance web servers and application stacks. -- Greg
Weber

77. DEPLOYMENT OPTIONS WITH RUBY
evented programming (reactor pattern) brings some
parallelism

78. DEPLOYMENT OPTIONS WITH RUBY
forking - phusion passenger, unicorn
evented - thin, goliath, vert.x
threaded - mongrel, torquebox

79. DEPLOYMENT OPTIONS WITH RUBY
combination of forking + evented
combination of threaded + evented
combination of threaded + forking

80. DEPLOYMENT OPTIONS WITH RUBY
the issue with evented servers (thin) is granularity

81. DEPLOYMENT OPTIONS WITH RUBY
controller - sql* - render - response*
controller - sql* - render - response*
controller - sql* - render - response*

82. DEPLOYMENT OPTIONS WITH RUBY
to unleash power of evented processing, you need to rewrite
your application (fibers, goliath, node.js, vert.x, async
sinatra)

83. DEPLOYMENT OPTIONS WITH RUBY
there are not many options for threading setups

84. SO WHEN TO CONSIDER JRUBY?
you have an app that is not build around evented pattern
your app takes decent amount of memory
your app also contains lots of I/O operations (SQL,
messaging, REST calls)
you want to scale up

85. WARNING
I did not cover Rubinius or REE which partially solves some
of these issues

86. BY THE WAY
the following languages have concurrency built in the
runtime
erlang
haskell
google go

87. AND WHAT'S JAVA

88. BEFORE YOU START

89. TRY WITH JRUBY FIRST
instead of torquebox

90. SLOW START
jruby start a little bit slower

91. OPTIMIZE JRUBY START
#JV_PS"cin -juycmiemd=F"
AAOT=-let Drb.opl.oeOF
bnl ee rissre
ude xc al evr

92. OPTIMIZE JRUBY START
#JUYOT=-19--X+MCasnodnEald
RB_PS"-. JX:CSlsUlaignbe
--X+sCnMrSepC
JX:UeocakweG
--XMxemie26 --m10m
JX:aPrSz=5m JXx80"
bnl ee rissre
ude xc al evr

93. OPTIMIZE JRUBY START
#juy-n-evr&
rb -gsre
#bnl ee rissre
ude xc al evr

94. KATELLO START IN DEV
$tm bnl ee rk evrnet
ie ude xc ae niomn
ra 01.7s
el m986
ue 01.4s
sr m824
ss0074
y m.6s

95. KATELLO START IN PROD
$tm rk evrnet
ie ae niomn
ra 01.2s
el m332
ue 0999
sr m.7s
ss0287
y m.1s

96. RUBYGEMS ARE SLOW
multiple directories approach
ruby needs to walk the tree
many stat/open calls with ENOENT
bundler adds more dirs
rvm/rbenv adds even more dirs

97. HOW RUBY HANDLES REQUIRE
#src rk evrnet2& |ge EON
tae ae niomn >1 rp NET
..
.
oe(xla_lf-../i/igeo.b,OROL)=- EON
pn"/dpfuf013lbsnltnr" _DNY 1 NET
oe(xntla-../i/igeo.b,OROL)=- EON
pn"/e-dp031lbsnltnr" _DNY 1 NET
oe(xjhnr-../i/igeo.b,OROL)=- EON
pn"/sitb021lbsnltnr" _DNY 1 NET
oe(xj-ots062lbsnltnr" OROL)=- EON
pn"/srue-../i/igeo.b, _DNY 1 NET
oe(xjmi-../i/igeo.b,OROL)=- EON
pn"/amt065lbsnltnr" _DNY 1 NET
oe(xyicmrso-../i/igeo.b,OROL)=- EON
pn"/u-opesr096lbsnltnr" _DNY 1 NET
oe(xi8_aa033lbsnltnr" OROL)=- EON
pn"/1ndt-../i/igeo.b, _DNY 1 NET
..
.

98. HOW RUBY HANDLES REQUIRE
O(n^2)

99. HOW RUBY HANDLES REQUIRE
optimized in ruby 2.0

100. HOW RUBY 2.0 HANDLES REQUIRE
O(n^2) - k

101. HOW RUBY HANDLES REQUIRE
enough theory!

102. RUBYGEMS IN KATELLO
#bnl isal|w -
ude ntl c l
10
2

103. KATELLO STAT/OPEN MISSES IN PROD
#src bnl ee rk evrnet2& |ge EON |w -
tae ude xc ae niomn >1 rp NET c l
42
03

104. KATELLO STAT/OPEN MISSES IN DEV
#src bnl ee rk evrnet2& |ge EON |w -
tae ude xc ae niomn >1 rp NET c l
124
732

105. KATELLO STAT/OPEN MISSES IN DEV

106. RUBYGEMS ARE SLOW
and it's not getting better

107. RUBYGEMS ARE SLOW
avoid bundler
avoid rvm/rbenv
use bundler_ext

108. PORTING ISSUES

109. BINARY FILES
writing to a binary file needs b-flag

110. BINARY FILES
Fl.pn"hfl.i" 'b)d ||
ieoe(teiebn, w' o f
fwiesuf
.rt(tf)
ed
n

111. ACTIVERECORD
install proper gems

112. ACTIVERECORD
i dfnd JUYVRIN
f eie? RB_ESO
gm'cieeodjb-dpe'
e atvrcr-dcaatr
gm'dcpsge'
e jb-otrs,
gm'cieeodjbpsgeq-dpe'
e atvrcr-dcotrslaatr,
es
le
gm'g
e p'
ed
n

113. ACTIVERECORD
various versions (rails 3.0 vs new adapter)

114. ACTIVERECORD
ERRudfndmto `olc'fr"rae_tDS"Srn (oehd
RO neie ehd clet o cetda EC:tig NMto
Err
ro)
..atvrcr-dcaatr126lbajb/otrslaatrr:2
./cieeodjb-dpe-../i/rdcpsgeq/dpe.b6
0i `itnt
:n dsic'
..
.

115. OTHER ISSUES
improper rails namespace
:-)

116. RUBY AND SYSTEMTAP

117. WHAT IS SYSTEMTAP
free software infrastructure to simplify the gathering of
information about the running Linux system

118. WHY SYSTEMTAP IS USEFUL
no need to modify your app
no need to restart it

119. WHY SYSTEMTAP IS USEFUL
steep learning curve
C-like syntax

120. WHY SYSTEMTAP IS USEFUL
very low-level
supports high-level (JVM, Python, Ruby)

121. WHY SYSTEMTAP IS USEFUL
part of RHEL and Fedora
kernels are systemtap ready
Ruby extension part of RHEL 6.2 (RHSA-2011-1581)

122. WHY SYSTEMTAP IS USEFUL
project documentation and wiki
RHEL6 SystemTap Beginners Guide

123. INSTALL SYSTEMTAP
#ym- isal
u y ntl
ssetp
ytma
ssetprnie
ytma-utm
kre-euif-uae-`
enldbgno`nm r
kre-euif-omn`nm -``nm -`
enldbgnocmo-uae i-uae r
kre-ee-uae-`
enldvl`nm r

124. UC1: HUNTING FILE CHANGE
#tuh/et
oc ts

125. UC1: HUNTING FILE CHANGE
#l - /et
s i ts
24
7

126. UC1: HUNTING FILE CHANGE
#l /e/d0
l dvm-
bwr--.1ro ds 23 0Ar1 1:3/e/m0
r-w-- ot ik 5, p 7 02 dvd-

127. UC1: HUNTING FILE CHANGE
#ctflcag.t
a iehnesp
goa AT_OE=1
lbl TRMD
poekre.ucin"eat_oy),
rb enlfnto(sttrcp"!
kre.ucin"eei_eat"!
enlfnto(gnrcsttr),
kre.ucin"nd_eat" {
enlfnto(ioesttr)
dvn =$nd-is-sdv
e_r ioe>_b>_e
ioen =$nd-iio
nd_r ioe>_n
i (e_r= MDV$,2 #mjrmnrdvc
f dvn = KE(1$) ao/io eie
& ioen = $
& nd_r = 3
& $tr>avld&AT_OE
& at-i_ai TRMD)
pit (% %(d % 0%/u% %"
rnf "d s%) s xx% o dn,
gtiefa_s) eenm(,pd) poeuc)
etmodyu(, xcae) i(, rbfn(,
dvn,ioen,$tr>amd,ud)
e_r nd_r at-i_oe i()
}

128. UC1: HUNTING FILE CHANGE
#sa - flcag.t 23024&
tp v iehnesp 5 7
#cmd60/et
ho 0 ts
13662012 cmd65)gnrcsttr0f00024100 0
34792123 ho(17 eei_eat xd00/7 060

129. UC2: DOWN THE RUBY STACK
#ctfcoilr
a atra.b
dffcoiln
e atra
f=1 frii 1.;f* i ed f
; o n .n = ; n;
ed
n
pt fcoil4
us atra 2

130. UC2: DOWN THE RUBY STACK
#ctclssp
a al.t
poerb.ucinety
rb uyfnto.nr
{
pit(% = %.si %:dn,tra_net1,
rnf"s > s% n s%" hedidn()
casae mtonm,fl,ln)
lsnm, ehdae ie ie;
}
poerb.ucinrtr
rb uyfnto.eun
{
pit(% < %.si %:dn,tra_net-)
rnf"s = s% n s%" hedidn(1,
casae mtonm,fl,ln)
lsnm, ehdae ie ie;
}

131. UC2: DOWN THE RUBY STACK
#sa clssp- "uyfcoilr"
tp al.t c rb atra.b
10061727884120241593340000
45017589953466451696800000
0rb(66) = Mdl.ehdaddi fcoilr:
uy110: > ouemto_de n atra.b1
1 rb(66) < Mdl.ehdaddi fcoilr:
3 uy110: = ouemto_de n atra.b1
0rb(66) = Ojc.atra i fcoilr:
uy110: > betfcoil n atra.b5
2 rb(66) = Rneec i fcoilr:
5 uy110: > ag.ah n atra.b2
6 rb(66)
1 uy110: = Fxu. i fcoilr:
> inm* n atra.b2
..
.
75rb(66)
0 uy110: < Bgu. i fcoilr:
= inm* n atra.b2
72rb(66) < Rneec i fcoilr:
1 uy110: = ag.ah n atra.b2
78rb(66) < Ojc.atra i fcoilr:
1 uy110: = betfcoil n atra.b2
0rb(66) = Ojc.usi fcoilr:
uy110: > betpt n atra.b5
2 rb(66) = Bgu.osi fcoilr:
0 uy110: > inmt_ n atra.b5
3 rb(66) < Bgu.osi fcoilr:
8 uy110: = inmt_ n atra.b5
5 rb(66) = I.rt i fcoilr:
3 uy110: > Owie n atra.b5
7 rb(66) < I.rt i fcoilr:
4 uy110: = Owie n atra.b5
8 rb(66) = I.rt i fcoilr:
1 uy110: > Owie n atra.b5
9 rb(66) < I.rt i fcoilr:
9 uy110: = Owie n atra.b5
16rb(66) < Ojc.usi fcoilr:
0 uy110: = betpt n atra.b5

132. UC2: DOWN THE RUBY STACK
#ctrbcutsp
a uyon.t
goa f_al;
lbl ncls
poerb.ucinety
rb uyfnto.nr
{
f_al[lsnm,mtonm]<<1
nclscasae ehdae < ;
}
poeed{
rb n
frah(casae mtonm]i f_al-lmt3){
oec [lsnm, ehdae n ncls ii 0
pit(%x%.sn,
rnf"d s%"
@on(nclscasae mtonm],
cutf_al[lsnm, ehdae)
casae mtonm)
lsnm, ehdae;
}
dlt f_al;
eee ncls
}

133. UC2: DOWN THE RUBY STACK
#sa rbcutsp- "uyfcoilr"
tp uyon.t c rb atra.b
10061727884120241593340000
45017589953466451696800000
2xBgu.
1 inm*
2xFxu.
1 inm*
2 I.rt
x Owie
1 Mdl.ehdadd
x ouemto_de
1 Rneec
x ag.ah
1 Bgu.os
x inmt_
1 Ojc.us
x betpt
1 Ojc.atra
x betfcoil

134. UC3: RUBY "TOP"
#ct.rb-o-oiidsp
a /uytpmdfe.t
goa f_al[04]
lbl ncls120;
poerb.ucinety{
rb uyfnto.nr
i (snt(ie "ael")f_al[i(,
f iisrfl, ktlo) nclspd)
fl,mtonm,ln]<<1
ie ehdae ie < ;
}
poetmrm(00 {
rb ie.s40)
as_la_cen)
nicersre(
pit(%s%0 %s%5 %sn,
rnf"6 8s 6 2s 6"
"I" "IEAE,"IE,"UCIN,"AL"
PD, FLNM" LN" FNTO" CLS)
frah(pdflnm,ucaelnn]i f_al-lmt1){
oec [i,ieaefnnm,ieo n ncls ii 5
pit(%d%0 %d%5 %dn,
rnf"6 8s 6 2s 6"
pd flnm,lnn,fnnm,
i, ieae ieo ucae
@on(nclspd flnm,fnnm,lnn])
cutf_al[i, ieae ucae ieo);
}
dlt f_al;
eee ncls
}

135. BUNDLER_EXT

136. BUNDLER_EXT
http://rubygems.org/gems/bundler_ext
https://github.com/aeolus-incubator/bundler_ext

137. BUNDLER_EXT
#ctGmie
a efl
gm'al' '..0
e ris, 301'
gm'sn
e jo'
gm'etcin' :eur = 'etcin'
e rs-let, rqie > rs_let
gm'amt,'=054
e jmi' > ..'
gm'al_adn,'=052
e riswre' > ..'
gm'e-dp
e ntla'
gm'at'
e ouh
gm'dpfuf
e la_lf'

138. BUNDLER_EXT
i Fl.xs?Fl.xadpt(./.Gmiei' _FL_)
f ieeit(ieepn_ah'../efl.n, _IE_)
rqie'ude_x'
eur bnlret
Bnlrx.ytmrqieFl.xadpt(./.Gmiei' _FL
udeEtsse_eur(ieepn_ah'../efl.n, _IE
_) :ru1 :ru2 Risev
_, gop, gop, al.n)
es
le
Bnlrrqie:ru1 :ru2 Risev
ude.eur gop, gop, al.n
ed
n

139. WE ARE DONE

140. CREDITS
Greg Weber - http://blog.gregweber.info/posts/2011-06-
16-high-performance-rb-part3
Ilya Grigorik -
http://www.igvita.com/2008/11/13/concurrency-is-a-
myth-in-ruby/
inc.com - finish line pic
and world-famouse memegenerator.net