Cloud architectural patters and Microsoft Azure tools (a presentation at Atlanta code camp - 2014)

1. Welcome
Cloud Architectural Patterns
(using Microsoft Azure tools)
Pushkar Chivate

2. Agenda
• Quick Look: Microsoft Azure as a platform
• Overview of patterns for cloud
applications
• Specifically Data-Management, Resiliency and Messaging
patterns
• Demo
• Azure Tables,
• DocumentDB,
• Azure Service bus (Queues and Pub-Sub architecture)

3. What is the Cloud?
IT
Capacity
Time
Actual
Load
Load
Forecast
Under capacity
Allocated
IT-capacity
Over
capacity

4. How the Cloud Helps?
IT
Capacity
Time
Actual
Load
Load
Forecast
Capacity on
Demand

5. Cloud Models
IaaS
Infrastructure as a
service
(e.g. Azure VMs)
PaaS
Platform as a service
(e.g. Azure Storage,
Service Bus etc.)
SaaS
Software as a Service
(e.g. Office 365)

6. Microsoft Azure is a cloud platform
● On-Demand
● Self-service
● Pay per use
● Highly automated
● Designed for Failure
● Designed for Scale

7. Microsoft Azure
Designed for Failure
Network load balancer
Fault Domain
1
VM1 VM2 VM3 VM4 VM5 VM6
VM7 VM8
Fault Domain
2
Fault Domain
3
UD1
UD2

8. Microsoft Azure
Designed for Scale
Scale out
Network load balancer
VM1 VM2 VM3 VM4

9. Azure VMs, Websites
&
Rethinking AppsA look at various features
Azure offers…

10. Architectural Pattern
An architectural pattern is a general,
reusable solution to a commonly
occurring problem within a given
context.

11. Cloud Patterns
Data-Management Patterns
● Index Table & Sharding Pattern
Create indexes over the fields in data stores that are frequently referenced by query
criteria. This pattern can improve query performance by allowing applications to
more quickly retrieve data from a data store. Divide a data store into a set of
horizontal partitions shards
● Materialized View Pattern
Generate prepopulated views over the data in one or more data stores when the data
is formatted in a way that does not favor the required query operations. This
pattern can help to support efficient querying and data extraction, and improve
performance
● Command and Query Responsibility
Segregation (CQRS) Pattern
Segregate operations that read data from operations that update data by using
separate interfaces.

12. Cloud Patterns
Messaging Patterns
● Pipes and Filter Pattern
Decompose a task that performs complex processing into a series of discrete
elements that can be reused.
● Priority Queue Pattern
Prioritize requests sent to services so that requests with a higher priority are
received and processed more quickly than those of a lower priority
.
● Queue-based Load Leveling Pattern
Use a queue that acts as a buffer between a task and a service that it invokes in
order to smooth intermittent heavy loads that may otherwise cause the service to
fail or the task to timeout.

13. Data Storage
Various data storage techniques
● SQL Azure
● Azure Table
● Blob storage
● Document DB

14. Materialized View Pattern
This pattern can be used to support efficient
querying and data extraction, and improve
application performance
What is a materialized view?
Materialized view is a database object that
contains results of a query

15. Materialized View
OrderID Account
1 A
2 B
ItemID Name Stock
100 Shirts 100
101 Pants 125
OrderID ItemID Qty
Ordered
1 100 2
1 101 3
2 100 2
Materialized View
Database
tables
ItemID Name Total Qty
Ordered
100 Shirts 4
101 Pants 3
Denormalized data
Data refreshes
periodically

16. Materialized Views Vs Regular Views
Materialized View
● Stored as a physical
database object
● Indexes on the base
tables are not used ,
can create indexes on
materialized view
● Faster performance
● Needs to be updated
Regular View
● Stored as a query
against the base object
● Indexes on the base
tables are used
● No performance
improvement
● Need not be updated

17. Materialized view
When to use materialized
views?
When underlying data is
complex and difficult to query
directly
Simplify queries - expose
data in a way that doesn’t
require knowledge of
underlying data structure
Provide access to specific
subset of data (security
reasons)
Bridging the disjoint when
using different data stores

18. Materialized view
Snapshot
s
Database
1
Table
1
Materialized view
1
Database
2
Table
1
Materialized view
2
Materialized
view
Snapsho
t

19. Databases supporting Materialized views
Oracle
Called Indexed Views
IBM DB2
SQL
Server
MySQL
No Native support
Can be implemented
using triggers
Called Materialized Query
Table
Called Materialized View

20. Materialized view
When not to use materialized views?
● The source data is simple and easy to query.
● The source data changes very quickly, or can be accessed
without using a view. The processing overhead of creating views
may be avoidable in these cases.
● Consistency is a high priority. The views may not always be fully
consistent with the original data

21. Azure Tables
What are Azure Tables?
● Non relational entities (not RDBMS).
● Tables are nothing but collection of entities
● Entities are comprised of properties
● Properties are name value pairs

22. Azure Tables
Azure Storage
Account
Employees
(Table)
Entity
Property
Entity
Property
Property Property Property

23. Azure Table Entities
Azure Storage
Account
Employees
(Table) Entity
Property
<Name,Value
>
Property
<Name,Value
>
Entities
● Partition Key
● Row Key
● Timestamp
● … other properties
● Each entity can have different number
and type of properties
Partition Key + Row Key = Unique Identifier
(No support for secondary index in Azure Table)
Property
<Name,Value
>

24. Azure Tables Partitions
(Entity)
EmployeeName=…
CertName=…
BirthDate=…
CertNumber=…
EmployeeName=…
CertName=…
BirthDate=…
CertNumber=…
FavoriteTeam=…
(Entity)
CertName=…
EmployeeName=…
CertNumber=…
BirthDate=…
CertName=…
EmployeeName=…
CertNumber=…
BirthDate=…
Employees (Table) Certifications (Table)
PK = EmployeeName
RK = CertName
PK = CertName
RK = EmployeeName
How do you Partition the data?
• EmployeeName or CertName?

25. Azure Tables Partitions
How do you Partition the data?
• EmployeeName or CertName?
(PK) EmployeeName(E1), (RK) CertName (C1)
(PK) EmployeeName(E1), (RK) CertName
(C2)
(PK) EmployeeName(E2), (RK) CertName
(C1)
(PK) EmployeeName(E2), (RK) CertName
(C3)
(PK) EmployeeName(E3), (RK) CertName
(C2)
(PK) EmployeeName(E3), (RK) CertName
(C3)
(PK) CertName (C1), (RK)
EmployeeName(E1)
(PK) CertName (C1), (RK)
EmployeeName(E2)
(PK) CertName (C2), (RK)
EmployeeName(E1)
(PK) CertName (C2), (RK)
EmployeeName(E3)
(PK) CertName (C3), (RK)
EmployeeName(E2)
(PK) CertName (C3), (RK)
EmployeeName(E3)
O
R
Employees (Table) Certifications (Table)

26. Azure Tables
No Fixed Schema for
entities
Employee table
EmployeeName BirthDate FavoriteTeam
David Anderson 1/1/1970
Nancy Wilson 4/15/1965 Atlanta Falcons
John Doe April 1, 1989

27. Azure Table
Demo

28. Azure Tables
Summary
● No nice relational schema model that we get in RDBMS
● Easy to work with
● Built to scale
● Great pricing

29. DocumentDB
• Azure Document DB - relatively new service
• It’s fully managed Document database as a service
• It stores the data in JSON
• The Azure Tables are extremely scalable and cheap, but if you start querying
on any other attributes then you start running into problems.
• It’s massively scalable but fully query-able by all parts of JSON tree.
• Capacity units are 10GB in size and predictable reads and writes / second.

30. DocumentDB

31. DocumentDB
Demo

32. Azure Service Bus
● Secure messaging
capabilities
● Enable loosely coupled
solutions
● Achieve Publish-Subscribe
scenarios

33. Tightly Coupled
Store Front
End
Order
processing
Shipping
Store Back
End

34. Loosely Coupled
Store Front
End
Order
processing
Shipping
Store Back
End
Order Queue

35. Loosely Coupled
Store Front
End
Shipping
Order
Order Queue processing
Store Back
End

36. Service Bus – Queue
Demo

37. Publish - Subscribe
Store Front
End
Order
processing
Store Back
End
Order Printers
Subscription
1
Subscription
2
Order Queue
Shipping
Order
Papers

38. Service Bus – AMQP
• Service Bus supports the Advanced Message Queueing Protocol (AMQP) 1.0.
• AMQP enables you to build cross-platform, hybrid applications using an open
standard protocol
• Reliable: The AMQP 1.0 protocol allows messages to be exchanged with a
range of reliability guarantees, from fire-and-forget to reliable, exactly-once
acknowledged delivery.
• Using AMQP from .Net:
Endpoint=sb://[namespace].servicebus.windows.net;SharedSecretIssuer=[issuer
name];SharedSecretValue=[issuer key];TransportType=Amqp
• Using AMQP from Java:
# servicebus.properties - sample JNDI configuration
# Register a ConnectionFactory in JNDI using the form:
# connectionfactory.[jndi_name] = [ConnectionURL]
connectionfactory.SBCF = amqps://[username]:[password]@[namespace].servicebus.windows.net

39. Service Bus – AMQP

40. Cloud Patterns
Resiliency Patterns
● Circuit Breaker Pattern
Handle faults that may take a variable amount of time to rectify when connecting to a
remote service or resource. This pattern can improve the stability and resiliency of an
application.
● Retry Pattern
Enable an application to handle temporary failures when connecting to a service or
network resource by transparently retrying the operation in the expectation that the
failure is transient. This pattern can improve the stability of the application.
● Compensating Transaction Pattern
Undo the work performed by a series of steps, which together define an eventually
consistent operation, if one or more of the operations fails. Operations that follow
the eventual consistency model are commonly found in cloud-hosted
applications that implement complex business processes and workflows.

41. Summary
We Looked at…
- Azure as Infrastructure (VMs)
- Azure as Platform
-Websites
- Storage - Azure Tables, DocumentDB
- Messaging – Service Bus, Queues, Topics, AMQP
- Azure CDN

42. Summary
Network load balancer
VM1 VM2 VM3 VM4
SQL Database
Service Bus Queue
Subscription 1 Subscription 2
Worker Role 1 Worker Role 2
Azure
Tables
DocumentDB
Cach
e

43. References
- MSDN (cloud architectural patterns)
- azure.microsoft.com (Learning videos and documentation)
- some images from www