1. The Cathedral, the Bazaar and
the Commissar
The Evolution of Innovation in
Enterprise Java
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.

2. Topics
• The Sources of Innovation
• A History of Innovation in Enterprise
Java
• Where next?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 2

3. Assumption:
Innovation matters and is essential to
the flourishing of a technology platform
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.

4. What Drives Innovation?
• There has been much research into the
drivers of innovation
• Key themes identified include
– Creativity
– Experimentation
– Competition
– Economic motivation
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 4

5. Sources of Creativity
• Source: National Center of Education and the
Economy (2006)
• Creative Thinking
– Comfort in disagreeing with others and trying
solutions that depart from the status quo
– Combining knowledge from previously disparate fields
– Ability to persevere through difficult problems and dry
spells
– Ability to step away from an effort and return later
with a fresh perspective (“incubation”)
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 5

6. Value of Experimentation in
Innovation
• Stefan Thomke of Harvard Business School argues that
every company’s ability to innovate depends on a series
of experiments [successful or not], that help create new
products and services or improve old ones. That period
between the earliest point in the design cycle and the
final release should be filled with experimentation,
failure, analysis, and yet another round of
experimentation.
• Experimentation can only work if failure is punished and
failed ideas are killed off
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 6

7. Innovation and competition
• The Department of Justice and the
Federal Trade Commission have
frequently raised innovation concerns as
reasons to challenge mergers
• Harvard Business School – “Competition
and Innovation” paper
– Empirical work…has pointed to a positive
correlation between product market
competition and innovative output
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 7

8. The ages of Enterprise Java
• Before J2EE
• The promise of J2EE
• The decline of J2EE
• The rise of open source
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 8

9. Before J2EE
• Mid 1990s
– Java gradually moves to the server side
• Largely unregulated
• Many competing products in different areas
– NetDynamics
– TopLink
– Silverstream
– Persistence PowerTier
– Apple WebObjects
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 9

10. Before J2EE…
• Good and Bad
– Innovation and choice of approaches
– Fragmented server-side market
– Real danger of vendor lock-in
– Many solutions very expensive,
• No impact from open source
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 10

11. The Glory Days of J2EE
• 1999-2003
– The JCP becomes dominant in the space
• TopLink and other “non-standard”
technologies cannot compete with J2EE
standards
– ORM versus EJB entity beans
– Velocity vs JSF
– WebObjects vs web tier
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 11

12. The Glory Days of J2EE
• Good and bad
– A market is created
– Vendor lock-in is reduced, but not
eliminated
– Increasing thought control strangles
innovation
– Flaws in the model take years to be
resolved, while many projects fail
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 12

13. The decline of J2EE
• 2003- • Proportion of enterprise Java
users using Tomcat
• Move away from
70
traditional application 60
server towards 50
lighter-weight 40
Springframework.org
30
solutions such as 20
BZ Research
Tomcat 10
0
– Tomcat now clear WAS JBoss WLS Tomcat
leader in enterprise
Java deployments
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 13

14. The rise of open source
• Fewer and fewer organizations develop
enterprise Java applications without
using open source
– Those that do face increasing competitive
disadvantage
• Numerous open source projects help to
shape the future
– Eclipse
– Spring
– AspectJ
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 14

15. The three key sources of
innovation in Enterprise Java
• The Cathedral
• The Bazaar
• The Commissar
• Let’s understand each
• Understand why none alone is sufficient
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 15

16. The Cathedral or Bazaar?
• Cathedral model has software built by
relatively few people, with centralized
design
• Bazaar has many deveolopers who lay out
their wares
• Classic statement on open source by Eric
Raymond, 1996
– Linux is subversive. Who would have thought
even five years ago (1991) that a world-class
operating system could coalesce as if by magic
out of part-time hacking by several thousand
developers scattered all over the planet,
connected only by the tenuous strands of the
Internet?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 16

17. Reality check
• As Linux has matured into enterprise
use, it’s no longer predominantly
developed this way
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 17

18. Case Study: Linux
• Dispelling the perception that Linux is cobbled together by a
large cadre of lone hackers working in isolation, the individual
in charge of managing the Linux kernel said that most Linux
improvements now come from corporations.
“People’s stereotype [of the typical Linux developer] is of a
male computer geek working in his basement writing code in
his spare time, purely for the love of his craft. Such people
were a significant force up until about five years ago,” said
Andrew Morton, whose role is maintaining the Linux kernel in
its stable form.
Morton said contributions from such enthusiasts, “is waning.”
Instead, most code is generated by programmers
punching the corporate time clock.
About 1,000 developers contribute changes to Linux on a
regular basis, Morton said. Of those 1,000 developers, about
100 are paid to work on Linux by their employers. And those
100 have contributed about 37,000 of the last 38,000 changes
made to the operating system.
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 18

19. Not everyone wants to play
• As open source becomes mainstream, more and more users of
open source won’t want to contribute and shouldn’t need to
• “The myth of open source software is the aura of freedom that
surrounds it. Download the source code and play with it if you
wish. And best of all, you won’t have to pay the freight.
Although that’s the public image of open source, the reason
why open source software is growing popular within
enterprises has nothing to do with open source itself. Few
enterprises care about whether they can monkey around with
source code because the relative minority that still have active
internal software development staffs have more important
things to do.”
Tony Baer, Datamonitor Computerwire
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 19

20. Myth: Community magically
generates open source software
• All we need is lots of
developers
• With all those fingers on
all those keyboards,
great software just
happens
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 20

21. Reality check
• Innovation is not a numbers game
• The bazaar model encourages
competition in implementation but may
not produce innovation
• The cathedral model is more likely to
produce innovation
• Neither is a complete solution
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 21

22. The Commissar
• Java has its own somewhat
unique model
– The Commissar
• In this model, the politburo
knows what’s best for the
proletariat (you)
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 22

23. The Commissar Knows Best
• JCP expert groups talk
largely in private
• Typically composed of
software vendors
• Relatively slow pace of
change, like Soviet 5
year plans
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 23

24. Why standards are needed
• Standards can create markets
• Standards can provide a base on which
competing open source and commercial
alternatives can flourish
– JTA
– Servlet API
– JMS
• Standards can protect customers from lock in
to a proprietary technology
• To ensure interoperability
– Web Services
– IIOP
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 24

25. How much standardization is
too much?
• In the Java world we have an unhealthy
obsession with standards
• Desire to standardize everything
• Failure to critically evaluate standard
technologies
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 25

26. Where Standards Don’t Work
• Jim Waldo (Sun Distinguished Engineer)
– Kowtowing to the god of standards is, I believe, doing
great damage to our industry, or craft, and our
science. It turns technical discussions into political
debates. It misunderstands the role that standards
have played in the past. Worst of all, it is leading us
down absurd technological paths in the quest to
follow standards which have never been implemented
and aren’t the right thing for the problems at hand.
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 26

27. Where standards don’t work
• CORBA history (1990s)
– Death by committee
– Attempts to innovate by committee
(distributed persistent objects)
• When they’re too slow
• When they’re divorced from reality
– Ivory castle
• When they are about politics, not
technology
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 27

28. Politics
• Like the Soviet Union, the JCP has also seen
its great purges
• JDO
– Arguably still the best technology choice for
ORM
– JPA is promising, but JDO 2.x remains
superior
– Yet JDO is dead in the market
• Database vendors didn’t like it and they are
influential in the JCP
• No application server vendor liked it as it
competed with EJB technology
• …so it was taken out and shot
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 28

29. Today’s Political Struggle:
OSGi vs JSR-277
• OSGi: Proven technology for dynamic
modularization
• From the OSGi Alliance
• Yet JSR-277 attempts to reinvent
modularization for Java
• Make or break issue for the JCP
• IBM, Oracle, BEA and many other vendors are
building their middleware strategy on OSGi
– They are not going to walk away just because Sun
doesn’t like it
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 29

30. Enterprise Java is no longer a
one party state
• Not just the Party (the JCP)
• OASIS
– SCA
– Web Services standards
• OSGi Alliance
– Dynamic modularization standards
– More enterprise standards
• Open source projects
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 30

31. The JCP must look at wider world and accept
that it doesn’t need to reinvent everything
JCP technology Ignored existing Negative consequences
technology
• HistoryTopLink and all otherwhen complete failures (EJB 1.x and 2.x)
Entity beans of failure •Two this has not
happenedsolutions
ORM •ORM in Java loses at least 6 years
•Billions of dollars of wasted development
effort from customers
Commons Log4J Added complexity of pointless abstraction
Logging layers such as Commons Logging
EJB (DI) Spring, PicoContainer, Limited DI functionality in EJB 3 specification
Hivemind misses opportunity to match best practice
EJB3 Spring, AOP Alliance, Lack of knowledge of AOP in the expert group
(interception) AspectJ, AspectWerkz produces fragile, clunky API missing central
AOP concepts
JSR 277 OSGi •Ignoring input and experience from OSGi
(modularization) •May split JCP as many organizations are
deeply committed to OSGi
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 31

32. The standards check list
1. Will the pace of change and innovation required
by met in a standards process cycle
2. Do we benefit from competing implementations?
3. Does this affect wire protocols (in which standards
are probably outside Java)
4. Is there an entrenched open source solution, in
which case competition may not occur?
5. Is the field mature and well understood
1. Has the proposal been tested in the market?
2. Do not want design by committee?
6. Is the standards committee representative of the
users of the technology?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 32

33. Does the standard exist to
lock out newcomers?
• When standards become too complex, like
J2EE, they effectively lock out new entrants
and benefit existing franchises, not consumers
• Study of the NHS
– In open markets the threat of entry by newcomers
not only puts pressure on prices; it also acts as a
pressure towards innovation because if, say, a
hospital – or new provider – pioneered new
techniques that provide higher quality, more cost-
effective, care, it should both attract more patients
and make greater profit
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 33

34. Competition is diminishing in
Java EE
• Oracle purchase of
BEA leaves IBM, Oracle
dominant in Java EE
• Since Red Hat
acquisition, JBoss
momentum has
stalled
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 34

35. Just the right amount of
competition
• There must be a reward for innovation
• Bad solutions must die
• Standard economic theory predicts that
innovation should decline with
competition, as more competition
reduces the monopoly rents that reward
successful innovators
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 35

36. The JCP and creativity
Creativity enabler JCP Open Cathedral
source
Comfort in disagreeing with others
and trying solutions that depart from
the status quo
Combing knowledge from previously
disparate fields
Ability to persevere through difficult
problems and dry spells
Ability to step away from an effort
and return later with a fresh
perspective
Competition and elimination of bad
ideas through market pressure
Create market
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 36

37. JCP: Conclusion
• The JCP is unlikely to produce
innovation but should focus on what it
can succeed at
– Creating a market where innovators can
compete above fundamental stanards
• Innovation by committee is a bad idea,
and has traditionally produced poor
results
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 37

38. Innovation needs
Competition: External Forces
• Ruby on Rails
• .NET
• External competition is very important
to Java
• Good to Great
– One of the companies profiled succeeded
because they chose to compete with a highly
efficient larger competitor, with the aim of
lifting their own performance
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 38

39. The Commissar doesn’t like
Competition
• Communism views competition as
wasteful and the root of all evil
– The proletarian liberates himself by
abolishing competition, private property,
and all class differences
• Friedrich Engels - The Principles of
Communism (1847)
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 39

40. The Reality
• Competition leads to the best performance,
even among communists
• Ironically, the Soviet Union only ever
succeeded when forced to compete to survive
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 40

41. But lack of inherent competition
in the system brought it down
• What really destroyed the USSR was the rapid
advancement of technology developed within
the fires of cooperative competition among
Western nations. The Soviets had brilliant
scientists and researchers, but they couldn’t
outmaneuver the efforts of thousands of
individual companies, whose efforts were being
selected by merit in a free market by millions
of consumers.
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 41

42. Things need to become faster
• Competition and experimentation needs
to occur rapidly
• As with the Soviet Union, technology
change and the increasing pace of
business leaves 2-3 years committee-
driven cycles looking less and less
relevant
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 42

43. Open Source implications:
MySQL
• Open source produces fast
experimentation/review cycles
• Biggest event in the future of the JCP is
not connected to Java
• Sun is becoming an open source
company
• The JCP is not a true standards body, so
is inevitably shaped by Sun’s agenda
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 43

44. Evidence of positive change
• JCP sessions, chaired by Patrick Curran
(JCP chair) show that Sun is opening up
• The JCP has gradually evolved over time
to be more open and less bureaucratic
• Needs to change much more and much
faster, but there is real hope…
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 44

45. What does Tomorrow Look Like?
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.

46. The standardization cycle
Breakdown of standards
process
Lack of innovation
No standards
Lock out new entrants /
Produce lowest common
denominator
Standardization
Create
coherent
market
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 46

47. The future
• No longer a one party state
– No one vendor or organization (even Sun)
will control all the pieces
• OSGi is shaping up as a make-or-break test
• Three sources of innovation
– Cathedral (proprietary vendors)
– Bazaar (open source)
– The Commissar (JCP)
• All three are needed
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 47

48. The future
• Open source innovation will continue
– Apache
– Eclipse
– Spring
• Change needs to be more rapid
– JCP needs to adapt to survive
– Likely to be run according to true open
source
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 48

49. The future
• Remember that you can be more than a
spectator
1. Evaluate technologies on merit, not
necessarily because of where they come
from
2. Choose the technologies you want to use
3. Take the opportunity to participate in the
JCP if you want to help keep it relevant
1. It’s not just a question of Sun “fixing” the JCP
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 49

50. Q&A
Copyright 2007 SpringSource. Copying, publishing or distributing without express written permission is prohibited.