30 minute version of the talk given in Bangalore and India in Sept 2012

1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
http://plainoldobjects.com/
1

2. Presentation goal
How decomposing
applications improves
deployability and
scalability
2

3. vmc push About-Chris
Developer Advocate for
CloudFoundry.com
Signup at http://cloudfoundry.com
cfopentour2012
3

4. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
4

5. Let’s imagine you are building
an e-commerce application
5

6. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to Tomcat
develop
test
deploy
scale 6

7. But there are problems with
a monolithic architecture
7

8. Users expect a rich, dynamic
and interactive experience
h
oug
en
ood
’tg
HTTP Request
isn
ure
ect
Java Web
Browser
it
HTML/Javascript Application
Ia rch
ty le U
s
Old
Real-time web ≅ NodeJS
8

9. Obstacle to frequent deployments
§Need to redeploy everything to change one component
§Interrupts long running background (e.g. Quartz) jobs
§Increases risk of failure
Fear of change
§Updates will happen less often
§e.g. Makes A/B testing UI really difficult 9

10. Overloads your IDE and container
Slows down development 10

11. Obstacle to scaling development
Accounting team
E-commerce
Engineering
application
Shipping team
11

12. Obstacle to scaling development
WAR
UI team StoreFrontUI
Accounting team Accounting
Inventory team InventoryService
Shipping team Shipping
12

13. Obstacle to scaling development
Lots of coordination and
communication required 13

14. Requires long-
term commitment
to a technology
stack
14

15. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
15

16. 16

17. The scale cube
Y axis -
functional
decomposition
Scale by
im ing
splitting
g s on
r
ila
tin iti
different things
lit art
p
gs y ata
th ale - d
sp
i s
ax
in b
Z
X axis - horizontal
Sc
duplication
17

18. Y-axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
InventoryService
Shipping
Service
18

19. Y-axis scaling - application level
accounting web application
Accounting
Service
Store front web application inventory web application
InventoryService
StoreFrontUI
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service 19

20. Partitioning strategies
§Partition by verb, e.g. shipping service
§Partition by noun, e.g. inventory service
§Single Responsibility Principle
§Unix utilities - do one focussed thing well
Something of an art
20

21. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to build a
page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
21

22. There are drawbacks
22

23. Complexity
See Steve Yegge’s Google
Platforms Rant re Amazon.com
23

24. Multiple databases
=
Transaction management
challenges
24

25. When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful 25

26. But there are many benefits
§Scales development: develop, deploy and scale
each service independently
§Update UI independently
§Improves fault isolation
§Eliminates long-term commitment to a single
technology stack
Modular, polyglot, multi-
framework applications
26

27. Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
27

28. If services are small...
§Regularly rewrite using a better technology stack
§Adapt system to changing requirements and
better technology without a total rewrite
§Pick the best developers rather than best <pick
a language> developers polyglot culture
Fred George
“Developer Anarchy”
28

29. The human body as a system
29

30. 50 to 70 billion of your cells die each
day
30

31. Yet you (the system) remain you
31

32. Can we build software systems
with these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
32

33. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
33

34. Inter-service communication options
§Synchronous HTTP asynchronous AMQP
§Formats: JSON, XML, Protocol Buffers, Thrift, ...
§Even via the database
Asynchronous is preferred
JSON is fashionable but binary
format is more efficient
34

35. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
35

36. Benefits
§Decouples caller from server
§Caller unaware of server’s coordinates (URL)
§Message broker buffers message when server is
down/slow
36

37. Drawbacks
§Additional complexity of message broker
§RPC using messaging is more complex
37

38. Writing code that calls
services
38

39. The need for parallelism
Service B
b = serviceB()
Call in parallel
c = serviceC()
Service A Service C
d = serviceD(b, c)
Service D
39

40. Java Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
40

41. Akka’s composable
futures are even better
41

42. Using Akka futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
And then invokes D
} yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
42

43. Spring Integration
§Implements EAI patterns
§Provides the building blocks for
a pipes and filters architecture
§Enables development of
application components that are
•loosely coupled
•insulated from messaging
infrastructure
§Messaging defined declaratively
43

44. Handling failure
Service A Service B
Errors happen
in distributed systems
44

45. About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
45

46. Use timeouts and retries
Never wait forever
Errors can be transient retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
46

47. Use per-dependency bounded thread pool
Service A
Runnable 1 Task 1
Runnable 2 Task 2
Service B
Runnable ... Task ...
bounded queue bounded thread pool
Fails fast if Limits number of
service is slow or down outstanding requests
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 47

48. Use a circuit breaker
High error rate stop calling temporarily
Down wait for it to come back up
Slow gives it a chance to recover
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 48

49. On failure
Return cached data
Avoid
Failing
Return default data
Fail fast
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 49

50. Summary
50

51. Monolithic applications are
simple to develop and deploy
BUT have significant
drawbacks
51

52. Apply the scale cube
§Modular, polyglot, and
scalable applications
§Services developed,
Y axis -
functional
decomposition
ng
deployed and scaled
independently
i
on
iti
rt
pa
ta
da
is-
ax
Z
X axis - horizontal duplication
52

53. @crichardson crichardson@vmware.com
http://slideshare.net/chris.e.richardson/
Questions?
www.cloudfoundry.com promo code: cfopentour2012