Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Decomposing applications for deployability and scalability #springone2gx #s12gx
1. Decomposing applications for
deployability and scalability
Chris Richardson,
Author of POJOs in Action, Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
http://plainoldobjects.com/
2. Presentation goal
How decomposing applications
improves deployability and
scalability
and
How Cloud Foundry helps
2
11. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to
Tomcat
develop
test
deploy
scale 11
12. But there are problems with a
monolithic architecture
12
13. Users expect a rich, dynamic
and interactive experience
h
oug
en
HTTP Request od
’t go
Browser isn Java Web Application
re
ctu
HTML/Javascript
e
rc hit
I a
ty le U
s
Old
Real-time web ≅ NodeJS
13
15. Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often
e.g. Makes A/B testing UI really difficult
15
23. The scale cube
Y axis -
functional
decomposition
ila ng
Scale by
i m ni
g s itio
r
splitting
tin rt
lit pa
different things
sp a
gs y dat
th ale is -
Sc ax
in b
X axis Z
- horizontal duplication 23
24. Y-axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
InventoryService
Shipping
Service
24
25. Y-axis scaling - application level
accounting web application
Accounting
Service
Store front web application inventory web application
StoreFrontUI InventoryService
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service 25
26. Partitioning strategies
Partition by verb, e.g. shipping service
Partition by noun, e.g. inventory service
Single Responsibility Principle
Unix utilities - do one focussed thing well
Something of an art
26
27. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to build a page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
27
31. When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful
31
32. But there are many benefits
Scales development: develop, deploy and scale each service
independently
Update UI independently
Improves fault isolation
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-framework
applications
32
33. Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
33
34. If services are small...
Regularly rewrite using a better technology stack
Adapt system to changing requirements and better technology
without a total rewrite
Pick the best developers rather than best <pick a language>
developers polyglot culture
Fred George
“Developer Anarchy”
34
38. Can we build software systems with
these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
38
40. Inter-service communication options
Synchronous HTTP asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Even via the database
Asynchronous is preferred
JSON is fashionable but binary format is
more efficient
40
41. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
41
42. Benefits
Decouples caller from server
Caller unaware of server’s coordinates (URL)
Message broker buffers message when server is down/slow
42
45. The need for parallelism
Service B
b = serviceB()
Call in parallel
c = serviceC()
Service A Service C
d = serviceD(b, c)
Service D
45
46. Java Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
46
47. Using Java Futures
public class Client {
private ExecutorService executorService;
private RemoteServiceProxy remoteServiceProxy;
public void doSomething() throws ... { Eventually contains result
Future<Integer> result =
executorService.submit(new Callable<Integer>() {
@Override
public Integer call() throws Exception {
return remoteServiceProxy.invokeRemoteService();
}
});
/// Do other things
When needed wait for result
int r = result.get(500, TimeUnit.MILLISECONDS);
System.out.println(r);
47
}
}
49. Composable Futures
val f1 = Future { ... ; 1 }
val f2 = Future { ... ; 2 }
Transforms Future
val f4 = f2.map(_ * 2)
assertEquals(4, Await.result(f4, 1 second))
Combines two futures
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html 49
50. Using Akka futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d And then invokes D
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
50
51. Spring Integration
Provides the building blocks for a pipes and
filters architecture
Enables development of application
components that are
•loosely coupled
•insulated from messaging infrastructure
Messaging defined declaratively
51
52. Handling failure
Service A Service B
Errors happen
in distributed systems
52
53. About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
53
54. How to run out of threads
HTTP Request
X
Thread 1
X
Thread 2
...
X
X
Thread N
Execute thread
Service A
XService B
pool
Eventually all threads If service B is down
then thread will be
Tomcat will be blocked blocked
54
55. Use timeouts and retries
Never wait forever
Errors can be transient retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
55
56. Use per-dependency bounded thread pool
Service A
Runnable 1 Task 1
Runnable 2 Task 2
Service B
Runnable ... Task ...
bounded queue bounded thread pool
Fails fast if Limits number of
service is slow or down outstanding requests
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 56
57. Use a circuit breaker
High error rate stop calling temporarily
Down wait for it to come back up
Slow gives it a chance to recover
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 57
58. On failure
Return cached data
Avoid
Failing
Return default data
Fail fast
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 58
59. Aspects + Actors
Dependency
Caller Invoker
Aspect Implements circuit
breaker state
machine
CircuitBreaker
Equivalent of thread Actor
pool
Worker Dependency
Worker
Actor Stub
Actor
59
66. Why NodeJS?
Familiar Javascript
High-performance, scalable event-driven, non-blocking I/O model
Over 13,000 modules developed by the community
Many JavaScript client frameworks have a NodeJS counterpart, e.g.
socket.io
66
67. NodeJS example
Handle HTTP Handle file read
var http = require('http');
request
var fs = require("fs");
http.createServer(function (req, res) {
fs.readFile('somefile.txt', function (err, data) {
if (err) throw err;
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end(data);
});
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
67
68. Socket.io - server-side
var express = require('express')
, http = require('http')
, app = express() Serve static content from public dir
, server = http.createServer(app)
, io = require('socket.io').listen(server)
;
app.configure(function(){ Listen on port 8081
app.use(express.static(__dirname + '/public'));
});
server.listen(8081);
Create a timer for
socket.io client
io.sockets.on('connection', function (socket) {
var counter = 0;
function tick() {
counter = counter + 1; Publish event
socket.emit('tick', counter);
};
setInterval(tick, 1000);
});
68
69. Socket.io - client-side
<html>
<body>
The time is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script>
<script src="/knockout-2.0.0.js"></script>
<script src="/clock.js"></script>
</body> Bind to model Connect to
</html>
socket.io server
clock.js
var socket = io.connect(location.hostname);
function ClockModel() {
self.ticker = ko.observable(1); Subscribe to tick
socket.on('tick', function (data) { event
self.ticker(data);
});
};
Update model
ko.applyBindings(new ClockModel()); 69
70. NodeJS isn’t the only game in town
JVM-based http://vertx.io/
70
71. A modern web application
Service 1
RESTful WS Node JS
Browser
HTML 5 Server Application
Application Service 2
Events
Socket.io Socket.io
client server
...
71
72. NodeJS - using RESTful WS and AMQP
REST
Service
REST
Requests
Node JS
Events
socket.io
AMQP AMQP
RabbitMQ Service
72
74. Original architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Tomcat
74
75. Modern architecture
Desktop Browser Native Mobile application HTML5 mobile application
StoreUI StoreUI StoreUI
NodeJS
Asynchronous, scalable NodeJS Javascript
communication StoreUI
RabbitMQ
Spring/Scala web
application Inventory Service Shipping Service
Standalone
Inventory Service Shipping Service “headless” Spring/
Java applications
MySQL Customer Redis Inventory Mongo Order
Billing Service
Database Database Database
75
77. Developing modular apps is more difficult
Many more moving parts to manage
•Platform services: SQL, NoSQL, RabbitMQ
•Application services: your code
Who is going to setup the environments:
•the developer sandbox?
•...
•QA environments?
But Cloud Foundry helps...
77
78. Easy polyglot application deployment and service
provisioning
OSS community
vFabric
Postgres
Ap
p
lica
Private
'o
Data Services Clouds
n
S
erv
i
ce
In
ter
fac
vFabric
e
RabbitMQTM Public
Clouds
Msg Services
Micro
Clouds
Other Services
Additional partners services …
79. Creating a platform service instance
$ vmc create-service mysql --name mysql1
Creating Service: OK
$ vmc services
......
=========== Provisioned Services ============
+-------------+---------+
| Name | Service |
+-------------+---------+
| mysql1 | mysql |
+-------------+---------+
81. Multi-application manifest - part 2
store/target:
Path to application
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required platform services
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
81
82. One command to create platform services and deploy
application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
vmc push:
Processing resources: OK •Reads the manifest file
Packing application: OK
Uploading (12K): OK
•Creates the required platform services
Push Status: OK •Deploys all the applications
Staging Application 'inventory': OK
Starting Application 'inventory': OK
Pushing application 'store'...
Creating Application: OK
Binding Service [si-mongo]: OK
Binding Service [si-rabbit]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK 82
83. Micro Cloud Foundry: new developer sandbox
App Instances Services
Open source Platform as a Service project
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
83
85. Caldecott = TCP over HTTP
native
protocol native
HTTP
Service Caldecott protocol
Caldecott gem Service
client Port application
NNN
Your computer Cloud Foundry
85
86. Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2
87. …Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS --
password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
mysql>
88. Running JUnit test with Caldecott
Configure your test code to use port + connection info
88
91. Apply the scale cube
Modular, polyglot, and scalable
applications
Y axis -
functional
decomposition
Services developed, deployed
and scaled independently
ngi
on
iti
rt
pa
a
at
-d
is
ax
Z
X axis - horizontal duplication
91
92. Cloud Foundry helps
Ap
.js
p
lic
a'
on
Private
Se
Data Services Clouds
rv
ice
In
te
rfa
ce
Public
Clouds
ce
rfa
Msg Services
te
In
er
vid
ro
d
P
ou
Micro
Cl
Clouds
Other Services