Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it's no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable services. We will also discuss how an event-based approach addresses the key challenges of developing applications with this architecture.

1. @crichardson
Microservices: Decomposing
Applications for
Deployability and Scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com

2. @crichardson
Presentation goal
How decomposing applications
improves deployability and
scalability
and
simplifies the adoption of new
technologies

3. @crichardson
About Chris

4. @crichardson
About Chris
Founder of a buzzword compliant (stealthy, social, mobile,
big data, machine learning, ...) startup
Consultant helping organizations improve how they
architect and deploy applications using cloud computing,
micro services, polyglot applications, NoSQL, ...

5. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client service interaction design
Decentralized data management

6. @crichardson
Let’s imagine you are
building an online store

7. @crichardson
Tomcat
Traditional application
architecture
Browser/
Client
WAR/EAR
MySQL
Database
Review Service
Product Info
Service
Recommendation
Service
StoreFrontUI
develop
test
deploy
Simple to
Load
balancer
scale
Spring MVC
Spring
Hibernate
Order Service
HTML
REST/JSON

8. @crichardson
But large, complex, monolithic
applications
problems

9. @crichardson
Intimidates developers

10. @crichardson
Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in
one basket

11. @crichardson
Overloads your IDE and
container
Slows down development

12. @crichardson
Lots of coordination and
communication required
Obstacle to scaling
development
I want
to update the UI
But
the backend is not working
yet!

13. @crichardson
Requires long-term commitment
to a technology stack

14. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client service interaction design
Decentralized data management

15. @crichardson

16. @crichardson
The scale cube
X axis
- horizontal duplication
Z
axis
-data
partitioning
Y axis -
functional
decomposition
Scale
by
splitting
sim
ilar
things
Scale by
splitting
different things

17. @crichardson
Y-axis scaling - application level
WAR
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

18. @crichardson
Y-axis scaling - application level
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

19. @crichardson
Product Info
Y-axis scaling - application level
Product Info
Service
Recommendation
Service
Review
Service
Order
Service
Browse Products
UI
Checkout UI
Order management
UI
Account
management UI
Apply X-axis and Z-axis scaling
to each service independently

20. @crichardson
Service deployment options
VM or Physical Machine
Docker/Linux container
JVM
JAR/WAR/OSGI bundle/...
Isolation, manageability
Density/efficiency

21. @crichardson
Partitioning strategies...
Partition by noun, e.g. product info service
Partition by verb, e.g. Checkout UI
Single Responsibility Principle
Unix utilities - do one focussed thing well

22. @crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art

23. @crichardson
Example micro-service using
Spring Boot
For more on micro-services see
http://microservices.io

24. @crichardson
But more realistically...
Focus on building cohesive services that
make development and deployment easier
- not just tiny services

25. @crichardson
Real world examples
http://highscalability.com/amazon-architecture
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
~600 services
100-150 services to build a page

26. @crichardson
There are drawbacks

27. @crichardson
Complexity of developing
and managing a distributed
system
Using a PaaS can significantly
simplify deployment
http://contino.co.uk/blog/2013/03/31/microservices-no-free-lunch.html

28. @crichardson
Multiple databases
&
Transaction management

29. @crichardson
Deploying features that span
multiple services requires
careful coordination

30. @crichardson
When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful

31. @crichardson
But there are many benefits

32. @crichardson
Smaller, simpler apps
Easier to understand and develop
Reduced startup time - important for GAE
Less jar/classpath hell - who needs OSGI?

33. @crichardson
Scales development:
develop, deploy and scale
each service independently

34. @crichardson
Improves fault isolation

35. @crichardson
Eliminates long-term commitment
to a single technology stack
Modular, polyglot, multi-
framework applications

36. @crichardson
Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving

37. @crichardson
Easily try other technologies
... and fail safely

38. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client service interaction design
Decentralized data management

39. @crichardson
How do clients of the system
interact with the services?

40. @crichardson
Directly connecting the front-end to the backend
Model
View Controller
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
Model
View Controller
Browser/Native App
Traditional server-side
web application
Chatty API
Web unfriendly
protocols

41. @crichardson
Use an API gateway
Model
View Controller
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
API
Gateway
Model
View Controller
Browser/Native App
Single entry point
Client
specific APIs
Protocol
translation
Traditional server-side
web application

42. @crichardson
Optimized client-specific
APIs
Web
application
Mobile
App
NodeJS
API
Gateway
REST
proxy
Event
publishing
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
getProductInfo()
getRecomm...()
getReviews()
getProductDetails()

43. @crichardson
Netflix API Gateway
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific
end points

44. @crichardson
API gateway design
challenges
Performance and scalability
Non-blocking I/O
Asynchronous, concurrent code
Handling partial failures
....
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

45. @crichardson
How does a browser
interact with the partitioned
web application?

46. @crichardson
Partitioned web app no
longer a single base URL
Browse Products
UI
Checkout UI
Order management
UI
Account
management UI
/products
/checkout
/orders
/account
Browser
?

47. @crichardson
The solution: single entry point
that routes based on URL
Browse Products
UI
Checkout UI
Order management
UI
Account
management UI
/products
/checkout
/orders
/account
Content
Router
Browser
http://acme.com/<service>/...
Hidden from
browser
Single entry
point

48. @crichardson
How do the services
communicate?

49. @crichardson
Inter-service communication
options
Synchronous HTTP asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Asynchronous is preferred
JSON is fashionable but binary format
is more efficient

50. Pros and cons of messaging
Pros
Decouples client from
server
Message broker buffers
messages
Supports a variety of
communication patterns
Cons
Additional complexity of
message broker
Request/reply-style
communication is more
complex

51. Pros and cons of HTTP
Pros
Simple and familiar
Request/reply is easy
Firewall friendly
No intermediate broker
Cons
Only supports request/
reply
Server must be
available
Client needs to
discover URL(s) of
server(s)

52. @crichardson
Discovery option #1:
Internal load balancer
Load
Balancer
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client/
API gateway
Services register
with load balancer
Client talks to
load balancer
Has a well-known
location
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/USVPC_creating_basic_lb.html

53. @crichardson
Discovery option #2:
client-side load balancing
REST
Client
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client
Service
Registry
Services register
with registry
Client polls
registry
http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html
http://techblog.netflix.com/2012/09/eureka.html

54. @crichardson
Lots of moving parts!
Product Info
Product Info
Service
Recommendation
Service
Review
Service
Order
Service
Browse Products UI
Checkout UI
Order management
UI
Account
management UI
API
Gate
way
Service registry
Content
Router
HTML
Browser
REST
Client
Ext.
LB
Ext.
LB

55. @crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client service interaction design
Decentralized data management

56. @crichardson
Decomposed services
decomposed databases
Order management Customer management
Order
Database
Customer
Database
Separate databases less coupling

57. @crichardson
Decomposed databases
polyglot persistence
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

58. @crichardson
Customer management
Untangling orders and customers
Order management
Order Service
placeOrder()
Customer Service
availableCredit()
updateCustomer()
Customer
creditLimit
...
has ordersbelongs toOrder
total
Invariant:
sum(order.total) <= creditLimit
available credit= creditLimit -
sum(order.total)
Trouble!

59. @crichardson
Problems
Reads
Service A needs to read data owned by service B
Updates
Transaction must update data owned by multiple
services

60. @crichardson
Handling reads: requesting
credit limit
Order management
placeOrder()
Customer management
getCreditLimit()

61. @crichardson
Pulling data
Benefits
Simple to implement
Ensures data is fresh
Drawbacks
Reduces availability
Increases response time

62. @crichardson
Customer management
Handling reads: replicating the
credit limit
Order management
Order Service
placeOrder()
Customer
creditLimit
...
Order
total
Customer’
creditLimit
changeCreditLimit()
sum(order.total) <=
creditLimit
Customer Service
updateCustomer()
Simplified

63. @crichardson
Useful idea: Bounded context
Different services have a different view of a
domain object, e.g.
User Management = complex view of user
Rest of application: User = PK + ACL + Name
Different services can have a different domain
model

64. @crichardson
Replicating data
Benefits
Improved availability for reads
Improves latency
Drawbacks
Additional complexity of replication mechanism

65. @crichardson
How to handle updates
(including of replicated
data)?

66. @crichardson
Use distributed transactions
Benefits
Guarantees consistency
Drawbacks
Complex
Reduced availability

67. @crichardson
Use eventual consistency
How
Services publish events when data changes
Subscribing services update their data
Benefits:
Simpler
Better availability
Drawbacks:
Application has to handle inconsistencies

68. @crichardson
How do services publish
events?

69. @crichardson
To maintain consistency the
application must
atomically publish an event
whenever
a domain object changes

70. @crichardson
Change tracking options
Database triggers
Hibernate event listener
Ad hoc event publishing code mixed into business logic
Domain events - “formal” modeling of events
Event Sourcing

71. @crichardson
Event sourcing
An event-centric approach to designing domain models
Aggregates handle commands by generating events
Apply events update aggregate state
Persist events NOT state
Replay events to recreate the current state of an
aggregate
Event Store ≃ database + message broker

72. @crichardson
EventStore API
trait EventStore {
def save[T](entityId: Id, events: Seq[Event]): T
def update[T](entityId: Id,
version: EntityVersion, events: Seq[Event]): T
def load[T](entityType: Class[T], entityId: EntityId): T
def subscribe(...) : ...
..
}

73. @crichardson
Using event sourcing
Event Store
CustomerCreditLimitUpdatedEvent
Order management
Order
total
Customer’
creditLimit
CustomerCreditLimitUpdatedEvent(...)
Customer management
Customer
creditLimit
...
Customer Service
updateCustomer()
UpdateCreditLimitCommand

74. @crichardson
Customer aggregate
case class Customer(customerId: String, creditLimit: BigDecimal)
extends ValidatingAggregate[Customer, CustomerCommands.CustomerCommand] {
def this() = this(null, null)
override def validate = {
case CreateCustomerCommand(customerId, creditLimit) =>
Seq(CustomerCreatedEvent(customerId, creditLimit))
case UpdateCreditLimitCommand(newLimit) if newLimit >= 0 =>
Seq(CustomerCreditLimitUpdatedEvent(newLimit))
}
override def apply = {
case CustomerCreatedEvent(customerId, creditLimit) =>
copy(customerId=customerId, creditLimit=creditLimit)
case CustomerCreditLimitUpdatedEvent(newLimit) =>
copy(creditLimit=newLimit)
}
}
Command
Events
Event
Updated
state

75. @crichardson
Unfamiliar but it solves many
problems
Eliminates O/R mapping problem
Supports both SQL and NoSQL databases
Publishes events reliably
Reliable eventual consistency framework
...

76. @crichardson
Summary

77. @crichardson
Monolithic applications are
simple to develop and deploy
BUT have significant
drawbacks

78. @crichardson
Apply the scale cube
Modular, polyglot, and
scalable applications
Services developed,
deployed and scaled
independently

79. @crichardson
Use a modular, polyglot architecture
Model
View Controller Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
API
Gateway
Model
View Controller
Server-side web
application
Browser/Native
application

80. @crichardson
Start refactoring your
monolith
Monolith Service
Anti-corruption
layer
Glue code
Pristine

81. @crichardson
Questions?
@crichardson chris@chrisrichardson.net
http://plainoldobjects.com