Using software modules today is the default way of working for most systems and frameworks. With the advent of many software languages and OSS frameworks, new module systems are constantly created and new module ecosystems start to prevail. This trend is horizontal and covers operating system packages, language libraries and application modules (plugins). But while some module systems are nicer to use, others are repeating past mistakes and are a daily source for developer agony and pain. In this short talk I will present the \"lessons learned\" at JFrog, where we make software for managing software libraries and deal with many types of module systems. This talk will show what works and what doesn\'t work in a module system; what features can make a module ecosystem thrive or fail; and why, despite all downsides, modules are here to stay and conquer more space as the Cloud continues to grow.

1. Using Software Modules -
Welcome to Hell!

2. Who’s talking?
2

3. Agenda
25 minutes
125 slides
30 memes

6. Packages & Modules
Modules Packages

7. Packages & Modules
Modules Packages
API
development

8. Packages & Modules
Modules Packages
API
development
Installable
runtime

9. For the sake of this talk

10. Modules, modules,
modules

11. Modules, modules,
modules
System dpkg, RPM, tgz, pacman, ipkg, pkgutils, PETget,
Upkg, PISI, nix, Equo, Conary, tazpkg, App Store,
MacPorts, fink, Homebrew, Google Play, GetJar,
Amazon Appstore, Cygwin, Npackd, Steam,
Chocolately, NSIS, IPS

12. Modules, modules,
modules
System dpkg, RPM, tgz, pacman, ipkg, pkgutils, PETget,
Upkg, PISI, nix, Equo, Conary, tazpkg, App Store,
MacPorts, fink, Homebrew, Google Play, GetJar,
Amazon Appstore, Cygwin, Npackd, Steam,
Chocolately, NSIS, IPS

13. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam

14. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Cabal, Composer, CPAN, CRAN, Python eggs, Ivy,
Maven, Gradle, sbt, leiningen, PyPI, NuGet,
RubyGems, Quicklisp, CocoaPods

15. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Cabal, Composer, CPAN, CRAN, Python eggs, Ivy,
Maven, Gradle, sbt, leiningen, PyPI, NuGet,
RubyGems, Quicklisp, CocoaPods

16. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods

17. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods
Framewor
k
RoR, Grails, jQuery, Node.js, Spring Roo, JBoss
Forge, Play, Chef, Puppet, Eclipse

18. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods
Framewor
k
RoR, Grails, jQuery, Node.js, Spring Roo, JBoss
Forge, Play, Chef, Puppet, Eclipse

19. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods
Framewor
k
RoR, Grails, jQuery,
Node.js, Spring Roo, Chef,
Puppet, Eclipse

20. Modules, modules,
modules
System dpkg, RPM, App Store,
Homebrew, Google Play,
Steam
Dev. Ivy, Maven, Gradle, sbt,
PyPI, NuGet, RubyGems,
CocoaPods
Framewor
k
RoR, Grails, jQuery,
Node.js, Spring Roo, Chef,
Puppet, Eclipse
Applicatio Any pluggable application!

22. DRAGON STORIES AHEAD!
fACDdb

23. STORY NUMBER ONE
f

24. , A STORY OF INDEXES +

25. , A STORY OF INDEXES +
< KEPT IN WRONG PLACE >

26. Once Upon A Time…

27. Once Upon A Time…
Can’t
serve…
searches…

28. Not anymore…

29. Look ma, no index download!

33. - Have to download huge files
Before Searching

34. - Have to download huge files
Before Searching
- Updated rarely

35. - Have to download huge files
Before Searching
- Updated rarely
- Requires special client

36. - Have to download huge files
Before Searching
- Updated rarely
- Requires special client

39. Who’s good, who’s not?
NuGet
RubyGem
s
Deb
RPM
Maven

40. Who’s good, who’s not?
NuGet
RubyGem
s
Deb
RPM
Maven
Good
Excuse:
old tools

41. Lesson learned
Index on the server-
side,
expose query API

42. STORY NUMBER TWO
A

44. Heinrich Heine

45. Heinrich Heine

54. STORY NUMBER TWO
(AND A HALF)
AA

55. He cut his ear off
and died unappreciated

56. Lesson learned
Authenticate by public
identity;
Give credit to the
developer

58. Identified, credited. Well
Done.

64. STORY NUMBER THREE
C

65. Imagine a typical windows
user

68. Depending on name only

69. Depending on name only
Where’s the
version,
dude?!

70. TIME FOR THE BOROMIR
MEME!
You knew it is coming!

74. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around

75. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all

76. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy

77. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w

78. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w
Platfor Classpa

79. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w
Platfor Classpa Transitive

80. Single Namespace Troubles
Scope Troubl
e
Examples Work-
around
System Can’t
use
multipl
e
version
s,
update
all
RPMs,
Ruby,
Groovy
RVM, GVM,
Homebre
w
Platfor Classpa Transitive Uberjar

81. Interlude…

82. Maven’s Conflict Manager
A v1

83. Maven’s Conflict Manager
A v1
E v1

84. Maven’s Conflict Manager
A v1
E v1
D v1

85. Maven’s Conflict Manager
A v1
B v1E v1
D v1

86. Maven’s Conflict Manager
A v1
C v1
B v1E v1
D v1

87. Maven’s Conflict Manager
A v1
C v1
B v1
D v2
E v1
D v1

88. Maven’s Conflict Manager
A v1
C v1
B v1
D v2
E v1
D v1

89. Maven’s Conflict Manager

90. Maven’s Conflict Manager

91. Maven’s Conflict Manager

93. BOROMIR RETURNS!
Reprise…

98. Just not very usable…

99. Lesson learned
Provide isolation and
easy uninstall

100. STORY NUMBER FOUR

101. Version ranges

104. Stuff Suddenly stops
working

105. Lesson learned
Require explicit
dependency versions

106. STORY NUMBER FIVE
D

107. Like
vodka
and
beer…

108. …Config
.
and
Metada
ta…

109. …Just
won’t
mix.

112. So, a user had a project…

113. My
Project
‘A1’

114. My
Project
‘A1’
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo

115. My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo

116. My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo
Project
‘B1’

117. My
Project
‘A1’
<dependency>B1</dependency>
<alwaysUse>
<repository>MyRepo</repository>
</alwaysUse>
MyRepo
Project
‘B1’
BadRepo
<repository>BadRepo</repository>
<dependency>C1</dependency>


118. My
Project
‘A1’
<dependency>B1</dependency>
<dependency>C1</dependency>
<repository>MyRepo</repository>
<repository>BadRepo</repository>
MyRepo
Project
‘B1’
BadRepo

119. My
Project
‘A1’
<dependency>B1</dependency>
<dependency>C1</dependency>
<repository>MyRepo</repository>
<repository>BadRepo</repository>
MyRepo
Project
‘B1’
BadRepo

121. Lesson learned
Don’t mix vodka and
beer configuration and
metadata

123. NO MORE STORIES,
ONE MORE THINGOOO
fACDdb

124. Doc predicts

125. All you need is cloud