Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers.
In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as Spring and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
14. @crichardson
Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often
e.g. Makes A/B testing UI really difficult
18. @crichardson
Lots of coordination and
communication required
Obstacle to scaling
development
I want
to update the UI
But
the backend is not working
yet!
22. @crichardson
The scale cube
X axis
- horizontal duplication
Z
axis
-data
partitioning
Y axis -
functional
decomposition
Scale
by
splitting
sim
ilar
things
Scale by
splitting
different things
23. @crichardson
Y-axis scaling - application level
WAR
Shipping
Service
Accounting
Service
Inventory
Service
StoreFrontUI
24. @crichardson
Y-axis scaling - application level
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
Accounting
Service
StoreFrontUI
accounting web application
Shipping
Service
Inventory
Service
26. @crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art
32. @crichardson
When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful
33. @crichardson
But there are many benefits
Scales development: develop, deploy and scale each service
independently, e.g. update UI independently
Simplifies distributed development and outsourcing
Improves fault isolation
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-
framework applications
34. @crichardson
Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
35. @crichardson
If services are small...
Regularly rewrite using a better technology stack
Adapt system to changing requirements and better
technology without a total rewrite
Pick the best developers rather than best <pick a
language> developers polyglot culture
39. @crichardson
Can we build software systems
with these characteristics?
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
44. @crichardson
Example service: NodeJS
var express = require('express')
, http = require('http')
, amqp = require(‘amqp’)
....;
server.listen(8081);
...
var amqpCon = amqp.createConnection(...);
io.sockets.on('connection', function (socket) {
function amqpMessageHandler(message, headers, deliveryInfo) {
var m = JSON.parse(message.data.toString());
socket.emit(‘tick’, m);
};
amqpCon.queue(“”, {},
function(queue) {
queue.bind(“myExchange”, “”);
queue.subscribe(amqpMessageHandler);
});
});
Simple portable way to
deliver AMQP messages
to the browser
45. @crichardson
Example service: Sinatra
require 'sinatra'
post '/' do
phone_number = params[:From]
registration_url =
"#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}"
<<-eof
<Response>
<Sms>To complete registration please go to #{registration_url}</Sms>
</Response>
eof
end
46. @crichardson
Service deployment options
VM or Physical Machine
Linux Container/LXC
JVM
JAR/WAR/OSGI bundle/...
Isolation, manageability
Density/efficiency
You could do this yourself but ...
47. @crichardson
PaaS dramatically simplifies deployment
Applica'on
Service
Interface
OSS community
Private
Clouds
Public
Clouds
Micro
Clouds
Data Services
Other Services
Msg Services
vFabric
Postgres
vFabric
RabbitMQTM
Additional partners services …
48. @crichardson
Cloud Foundry features
One step deployment: vmc push
Single application
Service-oriented application
Easy platform service provisioning: vmc create-service
Simple scaling: vmc instances app-name +/- N
Health monitoring and automated recovery
49. @crichardson
Benefits of PaaS
Simplifies and automates deployment
Eliminates barriers to adding new service
Eliminates barriers to using a new platform service
Imposes conventions: packaging, configuration and
deployment
Enforces consistency
Eliminates accidental complexity
53. @crichardson
Benefits
Decouples client from server: client unaware of server’s
coordinates (URL)
Message broker buffers message when server is down/
slow
Supports a variety of communication patterns, e.g. point-
to-point, pub-sub, ...
55. @crichardson
Spring Integration
Provides the building blocks for a pipes
and filters architecture
Enables development of application
components that are
loosely coupled
insulated from messaging infrastructure
Messaging defined declaratively
57. Pros and cons of REST
Pros
Simple and familiar
Request/reply is easy
Browser and firewall
friendly
No intermediate broker
Cons
Only supports request/
reply
Server must be
available
Client needs to know
URL(s) of server(s)
59. @crichardson
About Hypertext As The Engine Of
Application State (HATEOAS)
$ curl http://cf-auto-scaler.cloudfoundry.com
{"links": [
{"rel":"autoscaledapps", "href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps"}
]}
The well
known URL
Linklink type
$ curl http://cf-auto-scaler.cloudfoundry.com/autoscaledapps
{"content":[
{"name":"vertx-clock",
"links":[
{"rel":"self","href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock"},
{"rel":"rules","href":"http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules"}
]
}],
...
}
Links to act on this app
60. @crichardson
Spring HATEOAS
@Controller
@RequestMapping(value = "/autoscaledapps")
public class AutoscaledAppController {
@RequestMapping(value = "/{appName}", method = RequestMethod.GET)
public HttpEntity<AutoscaledAppResource> get(@PathVariable String appName) {
AutoscaledAppResource ar = new AutoscaledAppResource(appName);
ar.add(linkTo(AutoscaledAppController.class)
.slash(appName).withSelfRel());
ar.add(linkTo(AutoscaledAppController.class)
.slash(appName).slash("rules").withRel("rules"));
return new HttpEntity<AutoscaledAppResource>(ar);
}
...
}
https://github.com/SpringSource/spring-hateoas
public class AutoscaledAppResource
extends ResourceSupport {
private String name;
62. @crichardson
The Spring REST shell$ rest-shell
http://localhost:8080:> baseUri http://cf-auto-scaler.cloudfoundry.com
http://cf-auto-scaler.cloudfoundry.com:> discover
rel href
=======================================================================
autoscaledapps http://cf-auto-scaler.cloudfoundry.com/autoscaledapps
http://cf-auto-scaler.cloudfoundry.com:> follow --rel autoscaledapps
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps:> post --from src/test/resources/examplejson/
createapp1.json --follow true
1 files uploaded to the server using POST
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock:> discover
rel href
================================================================================
self http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock
rules http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock:> follow --rel rules
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules:> post
--from src/test/resources/examplejson/createrule1.json --follow true
1 files uploaded to the server using POST
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules/idle:> up
http://cf-auto-scaler.cloudfoundry.com/autoscaledapps/vertx-clock/rules:> up
64. @crichardson
The need for parallelism
Product
Details
Controller
Product Details
Recommendations
Reviews
getProductDetails()
getRecomendations()
getReviews()
Call in
parallel
Display
Product
65. @crichardson
Futures are a great
concurrency abstraction
An object that will contain the result of a concurrent
computation http://en.wikipedia.org/wiki/Futures_and_promises
Hides the underlying concurrency mechanism: threads or
async
Future<Integer> result =
executorService.submit(new Callable<Integer>() {... });
Java has basic futures. We
can do much better....
66. @crichardson
Better: Futures with callbacks
val f : Future[Int] = Future { ... }
f onSuccess {
case x : Int => println(x)
}
f onFailure {
case e : Exception => println("exception thrown")
}
Guava ListenableFutures,
Java 8 CompletableFuture, Scala Futures
67. @crichardson
Even better: Composable Futures
val f1 = Future { ... ; 1 }
val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)
assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}
val f = Future.sequence((1 to 5).map { x => asyncOp(x) })
assertEquals(List(1, 4, 9, 16, 25),
Await.result(f, 1 second))
Scala Futures
Transforms Future
Combines two futures
Transforms list of futures to a
future containing a list
68. @crichardson
Using Scala futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ...
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d
val result = Await.result(future, 1 second)
Two calls execute in parallel
And then invokes D
Get the result of DScala Futures
70. @crichardson
About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
71. @crichardson
How to run out of threads
Tomcat
Execute thread
pool
HTTP Request
Thread 1
Thread 2
Thread 3
Thread n
Service A Service B
If service B is
down then thread
will be blocked
XX
X
X
X
Eventually all threads
will be blocked
72. @crichardson
Their approach
Network timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
https://github.com/Netflix/Hystrix