1. Spring into the Cloud
Josh Long Chris Richardson
@starbuxman @crichardson
josh.long@springsource.com chris.richardson@springsource.com
Friday, July 13, 12
2. About Josh Long
Spring Developer Advocate
twitter: @starbuxman
josh.long@springsource.com
2
Friday, July 13, 12
3. About Josh Long
Spring Developer Advocate
@starbuxman
josh.long@springsource.com
th
s i
is
im
po
rta
nt
!
Free Book!
SpringSource.org/roo
CONFIDENTIAL
Friday, July 13, 12 3
21. Main Risk: Lock In Welcome to the hotel california
Such a lovely place
Such a lovely face
Plenty of room at the hotel california
Any time of year, you can find it here
Last thing I remember, I was
Running for the door
I had to find the passage back
To the place I was before
’relax,’ said the night man,
We are programmed to receive.
You can checkout any time you like,
But you can never leave!
-the Eagles
CONFIDENTIAL
Friday, July 13, 12 15
25. Cloud Foundry: Clouds
§ AppFog.com
• community lead for PHP
• PaaS for PHP
§ Joyent
• community lead for Node.js
§ ActiveState
• community lead for Python, Perl
• Providers of Stackato private PaaS
CONFIDENTIAL
Friday, July 13, 12 19
33. Cloud Foundry: Services
§ Services are one of the extensibility planes in Cloud Foundry
• there are more services being contributed by the community daily!
§ MySQL, Redis, MongoDB, RabbitMQ, PostgreSQL
§ Services may be shared across applications
§ Cloud Foundry abstracts the provisioning aspect of services through
a uniform API hosted in the cloud controller
§ It’s very easy to take an app and add a service to the app in a uniform
way
• Cassandra? COBOL / CICS, Oracle
CONFIDENTIAL
Friday, July 13, 12 27
34. Cloud Foundry: Services
§ Take Advantage of Services
• they cost nothing to setup
• they deliver value
§ They Encourage Better Architectures
• Need a fast read-write cache? Redis is ready to go!
• Need to store long-tail documents? Give MongoDB a try
• Need to decouple what applications do from when they do it?
Use messaging and RabbitMQ
CONFIDENTIAL
Friday, July 13, 12 28
37. Accessing Your Services
§ Debugging and accessing the data locally
• Caldecott --> Service tunneling. Access your Cloud Foundry service as if it was local.
CONFIDENTIAL
Friday, July 13, 12 31
38. Demo: Using the tunnel to talk to your services
CONFIDENTIAL
Friday, July 13, 12 32
39. Cloud Foundry
architecture
CONFIDENTIAL
Friday, July 13, 12 33
40. Application Deployment Flow
VMC Cloud Controller
DEA
DEA Application
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
41. Application Deployment Flow
vmc push
VMC Cloud Controller
DEA
DEA Application
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
42. Application Deployment Flow
vmc push
VMC Cloud Controller
DEA
DEA Application
Framework
detection
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
43. Application Deployment Flow
vmc push
create app
VMC Cloud Controller
DEA
DEA Application
Framework
detection
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
44. Application Deployment Flow
vmc push
create app
VMC Cloud Controller
DEA
DEA Application
Framework
detection
Framework
specific
staging plugin
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
45. Application Deployment Flow
find DEA for app
vmc push
create app
VMC Cloud Controller
DEA
DEA Application
Framework
detection
Framework
specific
staging plugin
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
46. Application Deployment Flow
find DEA for app
vmc push I’m available
create app
VMC Cloud Controller
DEA
DEA Application
Framework
detection
Framework
specific
staging plugin
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
47. Application Deployment Flow
find DEA for app
vmc push I’m available
create app
VMC Cloud Controller
DEA
DEA Application
deploy droplet
Framework
detection
Framework
specific
staging plugin
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
48. Application Deployment Flow
find DEA for app
vmc push I’m available
create app
start
VMC Cloud Controller
DEA
DEA Application
deploy droplet
Framework
detection
Framework
specific
staging plugin
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
49. Application Deployment Flow
find DEA for app
vmc push I’m available
create app
start
VMC Cloud Controller
DEA
DEA Application
deploy droplet
Framework
detection
Framework
specific
staging plugin
update routes
Creates
Droplet
Router
Router
CONFIDENTIAL
Friday, July 13, 12 34
50. About the staging plugin
§ Framework specific
§ Creates a zip file = black box
• Contains everything necessary for DEA to run the application
• Two scripts: start.sh, stop.sh
CONFIDENTIAL
Friday, July 13, 12 35
51. Staging plugin for the Spring framework
§ Adds CloudAutoStagingBeanFactoryPostProcessor bean:
• Modifies bean definitions
• Implements auto-reconfiguration
§ Defines CloudApplicationContextInitializer in web.xml
• Activates cloud profile
• Defines PropertySource for CloudEnvironment properties
CONFIDENTIAL
Friday, July 13, 12 36
54. Self Healing Infrastructure
§If an application instance crashes
•DEA detects unexpected exit => DEA broadcasts message
•Routers remove instance from routing
•Health manager notifies Cloud Controller
•Cloud Controller re-launches instance
§If an DEA VM crashes
•Application instances become unavailable
•Health Manager notices the missing instances and notifies the Cloud
Controller
•Cloud Controller requests application instances to be started
•Existing DEA will reply and start the applications
CONFIDENTIAL
Friday, July 13, 12
55. But where are the VMs?!?
§ Cloud Foundry has a layered architecture
• Cloud Controller, DEA, … layer = processes
• VM management layer
§ Benefits: decoupling, simplicity, ….
§ CloudFoundry.com creates vSphere VMs running different Cloud
Foundry components
CONFIDENTIAL
Friday, July 13, 12
56. using
Micro Cloud Foundry
CONFIDENTIAL
Friday, July 13, 12 40
57. What is in Micro Cloud Foundry?
App Instances Services
Open source Platform as a Service project
10.04
A cloud packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
CONFIDENTIAL
Friday, July 13, 12 41
58. Pre-requisites for using Micro Cloud Foundry
Minimum 1 GB Minimum 8 GB Internet Connectivity
RAM Disk (w/DHCP is ideal)
VMC STS
Command line GUI
CONFIDENTIAL
Friday, July 13, 12 42
59. Register and login to CloudFoundry.com
CONFIDENTIAL
Friday, July 13, 12 43
60. Select a domain name
*.cloudfoundry.me => IP address of your machine where Micro Cloud Foundry is running
CONFIDENTIAL
Friday, July 13, 12 44
67. Logging into Micro Cloud Foundry
$ vmc target api.cer-micro1.cloudfoundry.me
Successfully targeted to [http://api.cer-micro1.cloudfoundry.me]
CONFIDENTIAL
Friday, July 13, 12 48
68. Logging into Micro Cloud Foundry
$ vmc target api.cer-micro1.cloudfoundry.me
Successfully targeted to [http://api.cer-micro1.cloudfoundry.me]
$ vmc register
Email: crichardson@vmware.com
Password: ********
Verify Password: ********
Creating New User: OK
Attempting login to [http://api.cer-micro1.cloudfoundry.me]
Successfully logged into [http://api.cer-micro1.cloudfoundry.me]
CONFIDENTIAL
Friday, July 13, 12 48
69. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
49
Friday, July 13, 12
70. Cloud Foundry provides NoSQL-aaS
But what’s a NoSQL database?
Why would you want to use it?
How do you use it?
50
Friday, July 13, 12
71. Relational databases are great...
§ SQL
• High-level
• Sorting
• Aggregation
§ ACID semantics
§ Well supported
• JDBC
• Hibernate/JPA
• Spring
§ Well understood
• Developers
• Operators
51
Friday, July 13, 12
72. ... but they have limitations
§ Object/relational impedance mismatch
§ Complicated to map rich domain model to relational schema
§ Difficult to handle semi-structured data, e.g. varying attributes
§ Schema changes
§ Extremely difficult/impossible to scale
§ Poor performance for some use cases
52
Friday, July 13, 12
73. Solution: Spend Money
OR
http://upload.wikimedia.org/wikipedia/commons/e/e5/Rising_Sun_Yacht.JPG
http://www.trekbikes.com/us/en/bikes/road/race_performance/madone_5_series/madone_5_2/#
53
Friday, July 13, 12
76. Future = multi-paradigm data storage for enterprise applications
e.g. Netflix
• RDBMS
• SimpleDB
• Cassandra
• Hadoop/Hbase
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
56
Friday, July 13, 12
77. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
57
Friday, July 13, 12
78. Redis
§ Advanced key-value store
§ Very fast
§ Optional persistence
§ Transactions with optimistic locking
§ Master-slave replication
§ Sharding using client-side consistent hashing
58
Friday, July 13, 12
79. Redis
§ Advanced key-value store
§ Very fast
§ Optional persistence K1 V1
§ Transactions with optimistic locking K2 V2
§ Master-slave replication
K3 V2
§ Sharding using client-side consistent hashing
58
Friday, July 13, 12
80. Adding members to a sorted set
Redis Server
Value
Score
Key
zadd myset 5.0 a
59
Friday, July 13, 12
81. Adding members to a sorted set
Redis Server
Value
Score
Key
zadd myset 5.0 a
59
Friday, July 13, 12
82. Adding members to a sorted set
Redis Server
Value
Score
Key
a
zadd myset 5.0 a myset
5.0
59
Friday, July 13, 12
83. Adding members to a sorted set
Redis Server
a b
zadd myset 10.0 b myset
5.0 10.
60
Friday, July 13, 12
84. Adding members to a sorted set
Redis Server
c a b
zadd myset 1.0 c myset
1.0 5.0 10.
61
Friday, July 13, 12
85. Retrieving members by index range
Start Stop
Index Index
Key Redis Server
zrange myset 0 1
c a b
myset
1.0 5.0 10.
62
Friday, July 13, 12
86. Retrieving members by index range
Start Stop
Index Index
Key Redis Server
zrange myset 0 1
c a b
myset
1.0 5.0 10.
c a
62
Friday, July 13, 12
87. Retrieving members by score
Key Min. Max.
Score Score
Redis Server
zrangebyscore myset 1 6
c a b
myset
1.0 5.0 10.
63
Friday, July 13, 12
88. Retrieving members by score
Key Min. Max.
Score Score
Redis Server
zrangebyscore myset 1 6
c a b
myset
1.0 5.0 10.
c a
63
Friday, July 13, 12
89. Redis use cases
§ Drop-in replacement for Memcached
• Session state
• Cache of data retrieved from SOR
§ Replica of SOR for queries needing high-performance
§ Handling tasks that overload an RDBMS
• Hit counts - INCR
• Most recent N items - LPUSH and LTRIM
• Randomly selecting an item – SRANDMEMBER
• Queuing – Lists with LPOP, RPUSH, ….
• High score tables – Sorted sets and ZINCRBY
•…
§ Notable users: github, guardian.co.uk, ….
64
Friday, July 13, 12
90. MongoDB
§ Document-oriented database
• JSON-style documents: Lists, Maps, primitives
• Schema-less
§ Transaction = update of a single document
§ Rich query language for dynamic queries
§ Very fast
§ Writes are asynchronous!
§ Highly scalable and available
65
Friday, July 13, 12
91. Data model = Binary JSON documents
{
"name" : "Sahn Maru",
"type" : ”Korean",
"serviceArea" : [
"94619",
"94618"
],
"openingHours" : [
{ Sequence of
"dayOfWeek" : "Wednesday",
"open" : 1730,
bytes on
"close" : 2230 disk è fast
}
], i/o
"_id" : ObjectId("4bddc2f49d1505567c6220a0")
}
66
Friday, July 13, 12
92. Data model = Binary JSON documents
Collection: Restaurants
{
"name" : "Sahn Maru",
"type" : ”Korean",
"serviceArea" : [
"94619",
"94618"
],
"openingHours" : [
{ Sequence of
"dayOfWeek" : "Wednesday",
"open" : 1730,
bytes on
"close" : 2230 disk è fast
}
], i/o
"_id" : ObjectId("4bddc2f49d1505567c6220a0")
}
66
Friday, July 13, 12
93. Data model = Binary JSON documents
Database: Food To Go
Collection: Restaurants
{
"name" : "Sahn Maru",
"type" : ”Korean",
"serviceArea" : [
"94619",
"94618"
],
"openingHours" : [
{ Sequence of
"dayOfWeek" : "Wednesday",
"open" : 1730,
bytes on
"close" : 2230 disk è fast
}
], i/o
"_id" : ObjectId("4bddc2f49d1505567c6220a0")
}
66
Friday, July 13, 12
94. Data model = Binary JSON documents
Server
Database: Food To Go
Collection: Restaurants
{
"name" : "Sahn Maru",
"type" : ”Korean",
"serviceArea" : [
"94619",
"94618"
],
"openingHours" : [
{ Sequence of
"dayOfWeek" : "Wednesday",
"open" : 1730,
bytes on
"close" : 2230 disk è fast
}
], i/o
"_id" : ObjectId("4bddc2f49d1505567c6220a0")
}
66
Friday, July 13, 12
96. MongoDB query by example
{ Find a restaurant
serviceArea:"94619", that serves the
openingHours: { 94619 zip code and
is open at 6pm on a
$elemMatch : { Monday
"dayOfWeek" : "Monday",
"open": {$lte: 1800},
"close": {$gte: 1800}
}
}
}
DBCursor cursor = collection.find(qbeObject);
while (cursor.hasNext()) {
DBObject o = cursor.next();
…
}
68
Friday, July 13, 12
97. MongoDB use cases
§ Use cases
• High volume writes
• Complex data
• Semi-structured data
§ Who is using it?
• Shutterfly, Foursquare
• Bit.ly Intuit
• SourceForge, NY Times
• GILT Groupe, Evite,
• SugarCRM
69
Friday, July 13, 12
98. Other NoSQL databases
Type Examples
Extensible columns/Column-oriented Hbase
SimpleDB
DynamoDB
Graph Neo4j
Key-value Membase
Voldemort
Document CouchDb
http://nosql-database.org/ lists 122+ NoSQL databases
70
Friday, July 13, 12
99. Other NoSQL databases
Type Examples
Extensible columns/Column-oriented Hbase
ri te
vo
SimpleDB
DynamoDB
fa
Graph Neo4j
yo ur
o ut
Key-value left Membase
I
if Voldemort
or ry
Document
S CouchDb
http://nosql-database.org/ lists 122+ NoSQL databases
70
Friday, July 13, 12
100. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
71
Friday, July 13, 12
101. Spring Data is here to help
For
NoSQL databases
http://www.springsource.org/spring-data
72
Friday, July 13, 12
102. Spring Data sub-projects
§ SQL: Spring Data JPA, JDBC extensions
§ Commons: Polyglot persistence
§ Key-Value: Redis, Riak
§ Document: MongoDB
§ Graph: Neo4j
§ GORM for NoSQL
73
Friday, July 13, 12
103. What you get
§ Template classes that hide the boilerplate code
§ Auto-generated (generic) repositories
§ Java NoSQL mapping
§ Cross Store Persistence
§ Support in Roo and Grails
74
Friday, July 13, 12
104. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
75
Friday, July 13, 12
105. Redis challenges
§ Connection management
• Need to get and close connections
§ Data mapping
• Redis = binary/strings
• Application = objects
§ Multiple client libraries
• Gratuitously different APIs
76
Friday, July 13, 12
106. Spring Data for Redis
§ Low-level - RedisConnection(Factory)
• Supports Jedis, Jredis and Rjc
• Insulates client code from underlying library
§ High-level - RedisTemplate
• Builds on RedisConnection(Factory)
• Connection management
• Pluggable Java binary conversion
§ Support classes:
• Collections-backed by RedisTemplate
• Atomic Counters
77
Friday, July 13, 12
107. Low-level API = RedisConnection(Factory)
78
Friday, July 13, 12
108. Using RedisConnectionFactory
public class LowLevelRedisTest {
@Autowired private RedisConnectionFactory redisConnectionFactory;
@Test
public void testLowLevel() {
Library independent code J
RedisConnection con = null;
try {
con = redisConnectionFactory.getConnection();
Ugly byte arrays L
byte[] key = "foo".getBytes();
byte[] value = "bar".getBytes();
con.set(key, value);
byte[] retrievedValue = con.get(key);
Assert.assertArrayEquals(value, retrievedValue);
} finally {
if (con != null) { con.close(); } Need to clean up L
}
}
79
Friday, July 13, 12
109. Configuring RedisConnectionFactory
@Configuration
public class RedisConfiguration {
@Value("${databaseHostName}")
protected String databaseHostName;
@Bean
public RedisConnectionFactory jedisConnectionFactory() {
JedisConnectionFactory factory = new JedisConnectionFactory();
factory.setHostName(databaseHostName);
factory.setPort(6379);
factory.setUsePool(true);
return factory;
}
}
80
Friday, July 13, 12
110. High-level API = RedisTemplate
§ Builds on RedisConnection(Factory)
§ Analogous to JdbcTemplate
§ Parameterized type
• K - Key type
• V – Value type
§ Handles Java Key/Value Redis byte[]
§ Maps Redis exceptions DataAccessException
§ StringRedisTemplate
• Extends RedisTemplate<String, String>
• Keys and values are Strings
81
Friday, July 13, 12
111. Using StringRedisTemplate
public class RedisTemplateTest {
@Autowired private StringRedisTemplate stringRedisTemplate;
@Test Returns KV type specific interface
public void testGetAndSet() {
stringRedisTemplate.opsForValue().set("foo", "bar");
assertEquals("bar", stringRedisTemplate.opsForValue().get("foo"));
}
@Test Converts between Strings and byte[]
public void testHashOps() {
stringRedisTemplate.opsForHash().put("myHash", "myKey", "value");
assertEquals("value",
stringRedisTemplate.opsForHash().get("myHash", "myKey"));
assertEquals(Collections.singleton("myKey"),
stringRedisTemplate.opsForHash().keys("myHash"));
assertEquals(Collections.singletonMap("myKey", "value"),
stringRedisTemplate.opsForHash().entries("myHash"));
}
82
Friday, July 13, 12
112. Configuring StringRedisTemplate
@Configuration
public class RedisConfiguration {
@Bean
public RedisConnectionFactory jedisConnectionFactory() {
…
}
@Bean
public StringRedisTemplate stringRedisTemplate(RedisConnectionFactory
factory) {
StringRedisTemplate template = new StringRedisTemplate();
template.setConnectionFactory(factory);
return template;
}
}
83
Friday, July 13, 12
113. RedisTemplate: Java objects binary data
§ RedisTemplate has multiple Serializers:
• DefaultSerializer - defaults to JdkSerializationRedisSerializer
• KeySerializer
• ValueSerializer
• HashKeySerializer
• HashValueSerializer
84
Friday, July 13, 12
115. Register serializers to override the default behavior
Converted to JSON by RedisTemplate
86
Friday, July 13, 12
116. Redis caching support
KVs = <prefix + K,V>
Template needs to (de)serialize K and V Sorted set of all keys for clear()
87
Friday, July 13, 12
117. Other Spring data for Redis features
§ Redis-backed collections
§ Atomic counters
§ Support for Redis Pub/sub
88
Friday, July 13, 12
118. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
89
Friday, July 13, 12
119. MongoDB API usage patterns
§ Create and store Mongo singleton
§ Externalized server host, port etc.
§ Inserts/Updates
• Map application POJO DBObject
• mongo.getDatabase(…).getCollection(…)
• Partial document updates
• Configure asynchronous vs. synchronous writes
§ Queries
• Construct query object
• mongo.getDatabase(…).getCollection(…)
• Iterate through Cursor
• Map DBObject application POJO
Higher-level than JDBC but still repetitive, …
90
Friday, July 13, 12
120. Spring Data - MongoDB
§ MongoTemplate
§ Generic repositories
§ Querydsl integration
§ Cross-store persistence
91
Friday, July 13, 12
122. Example entity
public class Restaurant {
private String id;
private String name;
private List<MenuItem> menuItems;
public Restaurant() {
}
public class MenuItem {
public Restaurant(String name) { private String name;
this.name = name; private double price;
…
} public MenuItem() {
}
public String getName() { return name; }
public MenuItem(String name,
public void setName(String name) { double price) {
this.name = name; this.name = name;
} this.price = price;
}
…getters and setters…
…getters and setters…
93
Friday, July 13, 12
123. Example data access code
@Repository
public class RestaurantRepository {
@Autowired
private MongoTemplate mongoTemplate;
public static final String RESTAURANTS_COLLECTION = "restaurants";
public void add(Restaurant restaurant) {
mongoTemplate.save(RESTAURANTS_COLLECTION, restaurant);
}
public List<Restaurant> findRestaurantsByName(String restaurantName) {
return mongoTemplate.find(RESTAURANTS_COLLECTION,
new Query(where("name").is(restaurantName)),
Restaurant.class);
}
94
Friday, July 13, 12
125. Spring MongoDB Example - Config 1
@Configuration public class MongoDbExampleConfig {
private @Value("#{mongoDbProperties.databaseName}") String mongoDbDatabase;
private @Value("#{mongoDbProperties.host}") String mongoDbHost;
@Bean public Mongo mongo() throws Exception {
return new Mongo(mongoDbHost);
}
@Bean public MongoTemplate mongoTemplate(Mongo mongo) {
MongoTemplate mongoTemplate = new MongoTemplate(mongo, mongoDbDatabase);
mongoTemplate.setWriteConcern(WriteConcern.SAFE);
mongoTemplate.setWriteResultChecking(WriteResultChecking.EXCEPTION);
return mongoTemplate;
}
…
<beans>
<context:annotation-config/>
External Config <context:component-scan
base-package="net.chrisrichardson.mongodb.example"/>
<util:properties id="mongoDbProperties"
mongodb.properties: location="mongodb.properties"/>
</beans>
databaseName=demo1
host=192.168.253.150
96
Friday, July 13, 12
126. Spring MongoDB Example - Config 2
<bean id="mongoTemplate"
class="org.springframework.data.mongodb.core.MongoTemplate">
<constructor-arg ref="mongoFactory"/>
</bean>
<mongo:db-factory id="mongoFactory"
host= "#{mongoDbProperties.host}"
dbname="#{mongoDbProperties.databaseName}" />
<util:properties
id="mongoDbProperties"
location="mongodb.properties"/>
97
Friday, July 13, 12
127. Summarize other features
§ In-place updates
§ Callbacks
§ Generic repositories
§ Annotation-driven mapping
§ Support for QueryDSL
§ Cross-store persistence
98
Friday, July 13, 12
128. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
• Why NoSQL?
• Overview of NoSQL databases
• Introduction to Spring Data
• Using Spring Data for Redis
• Using Spring Data for Mongo
• Deploying on Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
§ Wrap up
99
Friday, July 13, 12
129. Using Mongo and Redis with Cloud Foundry
§ Create a service - Mongo or Redis
§ Bind it to your application
§ Use <cloud:*/> namespace to access the bound service
• when cloud profile is active
100
Friday, July 13, 12
134. About <cloud:redis-connection-factory/>
<cloud:redis-connection-factory
id="redisConnectionFactory"
Use when multiple
[ service-name="redis1" ]
services are bound
/>
105
Friday, July 13, 12
138. About <cloud:mongo-db-factory/>
Use when multiple
services are bound
<cloud:mongo-db-factory
id="mongoFactory"
[ service-name="mongo1" ]
[ write-concern="SAFE" ]
Whether to wait
>
[ <cloud:mongo-options for writes to
complete
connections-per-host="..."
max-wait-time="..."
/> ]
</cloud:mongo-db-factory>
109
Friday, July 13, 12
140. Uses MySQL and MongoDB
@Entity
public class Customer { Stored in MySQL
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String firstName;
private String lastName;
Stored in Mongo
@RelatedDocument
private SurveyInfo surveyInfo;
public class SurveyInfo { public class Survey {
private List<Survey> questionsAndAnswers = new ArrayList<Survey>(); String question;
public List<Survey> getQuestionsAndAnswers() { String answer;
return questionsAndAnswers;
}
111
Friday, July 13, 12
141. Cross store configuration
@Configuration
@ComponentScan(basePackageClasses = CrossStoreCustomerRepository.class)
@EnableTransactionManagement(mode = AdviceMode.ASPECTJ)
public class ServicesConfiguration {
private String mongoDatabaseServiceName = "survey-mongo";
private String mysqlDatabaseServiceName = "survey-mysql";
@Bean
public CloudEnvironment cloudEnvironment() {
return new CloudEnvironment();
}
@Bean
public MongoServiceInfo mongoServiceInfo() {
return cloudEnvironment().getServiceInfo(mongoDatabaseServiceName, MongoServiceInfo.class);
}
@Bean
public MongoDbFactory mongoDbFactory() {
MongoServiceCreator mongoServiceCreator = new MongoServiceCreator();
return mongoServiceCreator.createService(mongoServiceInfo());
}
@Bean
public DataSource dataSource() {
RdbmsServiceInfo rdbmsServiceInfo = cloudEnvironment().getServiceInfo(mysqlDatabaseServiceName, RdbmsServiceInfo.class);
RdbmsServiceCreator rdbmsServiceCreator = new RdbmsServiceCreator();
DataSource dataSource = rdbmsServiceCreator.createService(rdbmsServiceInfo);
return dataSource;
}
...
112
Friday, July 13, 12
143. NoSQL and Caldecott
§ Caldecott let’s you tunnel to a NoSQL service
§ Use Redis CLI
• redis-cli
• Explore database, adhoc operations
• ...
§ Use Mongo CLI etc
• Explore database, adhoc operations
• Mongo dump/restore
• ...
114
Friday, July 13, 12
144. NoSQL wrap up
§ Cloud Foundry supports Mongo and Redis
§ For some use cases, NoSQL databases offer some combination of:
• Higher scalability
• Higher performance
• Richer data models
• Schema less
§ Spring Data simplifies the development of NoSQL applications
Cloud Foundry Easy development
= and
+
deployment of
Spring Data NoSQL applications
115
Friday, July 13, 12
145. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
•Why messaging?
•Messaging with RabbitMQ and AMQP
•Using Spring Integration
•Cloud Foundry and RabbitMQ
§ Wrap up
116
Friday, July 13, 12
146. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
•Why messaging?
•Messaging with RabbitMQ and AMQP
•Using Spring Integration
•Cloud Foundry and RabbitMQ
§ Wrap up
117
Friday, July 13, 12
148. But why messaging? Why RabbitMQ?
Application A Application B
RabbitMQ
Traditional application integration
119
Friday, July 13, 12
149. But why messaging? Why RabbitMQ?
Application
Service A Service B Service ...
RabbitMQ
• Essential component of our new scalable and fault tolerant architecture
• Integration mechanism for the services
• Enables services to discover each other
120
Friday, July 13, 12
150. Let’s imagine you are building an e-
commerce application
121
Friday, July 13, 12
151. wgrus-monolithic.war
@Controller
StoreFront
Widget Gadget Accounting Shipping
InventoryService InventoryService Service Service
RDBMS
122
Friday, July 13, 12
152. It’s simple to develop but ....
§ Lack of scalability
• Scale through replication
• Non-replicable component => nothing can be replicated
• Can’t scale different parts of the application differently
§ Lack of deployability
• Deploy it all in one go
• Increased risk of something breaking
§ Applications are brittle
• Store can’t accept orders unless all services are available
• Failure (e.g. memory leak) in one component can take down every other
§ Monolingual
• Can’t use non-JVM server-side technologies: NodeJS, Rails,
123
Friday, July 13, 12
154. wgrus-billing.war
Accounting
Service
wgrus-shipping.war
wgrus-store.war Shipping
Synchronous
Service
Spring
StoreFront Remoting MySQL
Spring Web
Services wgrus-inventory.war
Widget
InventoryService
wgrus-inventory.war
Gadget
InventoryService
125
Friday, July 13, 12
155. Benefits and Drawbacks
§ Benefits:
• Scale each service independently
• Deploy each service independently
• Mix JVM and non-JVM languages
§ Drawbacks
• Application is still brittle
• Store can’t accept orders unless all services are available
• Failure (e.g. memory leak) in one component can take down every other
126
Friday, July 13, 12
156. Solution:
Asynchronous Architecture
127
Friday, July 13, 12
157. wgrus-billing.war
Accounting
Service
wgrus-inventory.war
wgrus-store.war Widget
InventoryService
Message
StoreFront Broker MySQL
wgrus-inventory.war
Gadget
InventoryService
wgrus-shipping.war
Shipping
Service
128
Friday, July 13, 12
158. Benefits and Drawbacks
§ Benefits:
• Scale each service independently
• Deploy each service independently
• Mix JVM and non-JVM languages
• Improved availability
• Front-end keeps working even when backend services are down
• Messaging broker can buffer traffic and smooth out spikes
§ Drawbacks
• Yet another moving part
• Sometimes synchronous RPC is a better fit
129
Friday, July 13, 12
159. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
•Why messaging?
•Messaging with RabbitMQ and AMQP
•Using Spring Integration
•Cloud Foundry and RabbitMQ
§ Wrap up
130
Friday, July 13, 12
160. RabbitMQ – Messaging that Just Works
Robust
High-performance
Easy to use
AMQP LEADER
Friday, July 13, 12
161. Why AMQP?
A
Protocol,
not
an
API
•A defined set of
messaging capabilities
called the AMQ model
•A network wire-level
protocol, AMQP
On
commodity
hardware
•10-‐25
thousand
messages
per
second
is
rou>ne
*
•The
NIC
is
usually
the
boDleneck
*
Non-‐persistent
messages
17
Friday, July 13, 12
162. AMQP Architecture
café deliveries
queue
café NA deliveries
queue
café WW deliveries
queue
20
Friday, July 13, 12
163. AMQP Architecture
café deliveries
queue
café NA deliveries
queue
café WW deliveries
queue
20
Friday, July 13, 12
178. AMQP Architecture
all_drinks
queue
k.*
drin
drink.cold cold_drinks
queue
dri
nk.
ho hot_drinks
t
queue
21
Friday, July 13, 12
179. AMQP Architecture
all_drinks
queue
k.*
drin
drink.cold cold_drinks
queue
dri
nk.
ho hot_drinks
t
queue
Message Routing Keys:
1.drink.hot
2.drink.cold
3.drink.warm
21
Friday, July 13, 12
180. AMQP Architecture
all_drinks
1 2 3
queue
k.*
drin
drink.cold cold_drinks
2
queue
dri
nk.
ho hot_drinks
t
1
queue
Message Routing Keys:
1.drink.hot
2.drink.cold
3.drink.warm
21
Friday, July 13, 12
181. Spring AMQP
§ Encapsulates low-level details
§ Simplifies sending and receiving Producer Consumer
of messages
Amqp Listener
Template Container
Spring AMQP
AMQP
Friday, July 13, 12
182. Sending AMQP messages
@Component public class MessageSender {
@Autowired
private volatile AmqpTemplate amqpTemplate;
public void send(String message) {
this.amqpTemplate.convertAndSend(
"myExchange", "some.routing.key", message);
}
}
137
Friday, July 13, 12
183. Receiving AMQP messages
public class MyComponent {
@Autowired
private AmqpTemplate amqpTemplate;
public void read() throws Exception {
...
String value = amqpTemplate.receiveAndConvert("myQueueName");
...
}
}
138
Friday, July 13, 12
184. Spring AMQP: SimpleMessageListenerContainer
l Asynchronous message receiver
l POJO handlers
l Handles re-connection and listener failure (rollback, redelivery)
l Message conversion and error handling strategies
<listener-container connection-factory="rabbitConnectionFactory">
<listener ref="handler" method="handle" queue-names="my.queue">
</listener-container>
139
Friday, July 13, 12
185. Spring configuration
<rabbit:template id="rabbitTemplate"
connection-factory="rabbitConnectionFactory"/>
<rabbit:connection-factory
id="rabbitConnectionFactory"/>
140
Friday, July 13, 12
186. Spring AMQP is flexible and dynamic
BUT
It’s very low level
141
Friday, July 13, 12
187. Agenda
§ Why Cloud? Why PaaS?
§ Introducing Cloud Foundry
§ Cloud Foundry for Spring developers
§ Building Java applications on Cloud Foundry
§ Moving Spring applications to the Cloud
§ Developing NoSQL applications for Cloud Foundry
§ Application integration with RabbitMQ and Spring AMQP
•Why messaging?
•Messaging with RabbitMQ and AMQP
•Using Spring Integration
•Cloud Foundry and RabbitMQ
§ Wrap up
142
Friday, July 13, 12
188. Spring Integration
§ Builds on Spring framework
§ High-level of abstraction for building message
based applications
§ Implements EAI patterns
§ Provides plumbing for exchanging messages
between application components
§ Promotes loosely coupled components
§ Integrates with external messaging
infrastructure: JMS, AMQP, HTTP, Email, File
transfer
143
Friday, July 13, 12
189. Spring Integration concepts
§ Message channel
• Virtual pipe connecting producer and consumer
§ Message endpoints
• The filter of a pipes-and-filter architecture
• Read from and/or write to channel
§ Endpoint types:
• Transformer
• Filter
• Router
• Splitter
• Aggregator
• ServiceActivator
• Inbound channel adapter - read from external source, writes to channel
• Outbound channel adapter - read from channel write to external destination
144
Friday, July 13, 12
190. Example of reconfigurability - local
@Service
public class OrderServiceImpl { @Service
public class ShippingServiceImpl {
@Autowired
private ShippingService shippingService; public void shipOrder(String orderId) {
System.out.println("shipped order: " +
public void placeOrder() { orderId);
String orderId = generateOrderId(); }
…
shippingService.shipOrder(orderId); }
}
}
Order Shipping
Service service
Messaging
Gateway
Channel Service
Activator
145
Friday, July 13, 12
191. Example of reconfigurability - distributed
Code unchanged in new deployment
Order Shipping
Service service
Messaging
Gateway RabbitMQ
Channel AMQP AMQP Channel Service
Activator
146
Friday, July 13, 12