The document discusses various patterns for building scalable applications on Microsoft Azure including queue-based load leveling, competing consumers, priority queue, throttling, materialized view, CQRS, and sharding. It provides the context, solution, and considerations for each pattern. Code examples and demos are presented for some patterns.
Patterns for Scalability in Windows Azure Applications (Alex Mang)
1. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Patterns for Scalability in
Microsoft Azure Applications
Alex Mang
http://alexmang.ro
@mangalexandru
2. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Huge thanks to our sponsors & partners!
4. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Alex Mang
• CEO @ KeyTicket Solutions
–Microsoft BizSpark Plus
• Azure Advisor
• MS, MCP, MCSD
Speaker.Bio.ToString()
5. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Common requirements for cloud apps:
–Availability
–Data management
–Design and implementation
–Messaging
–Management and monitoring
–Performance and scalability
–Resiliency
–Security
What are patterns?
6. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Performance
–‘indication of responsiveness of a system to
execute any action within a given time interval’
• Scalability
–‘ability of a system to handle increases in load
without impact on performance’
Performance and Scalability Patterns
= ???
= ???
7. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Performance and Scalability Patterns
Cache-aside
Competing consumersCQRS
Event sourcing
Index table
Materialized view
Priority Queue
Queue based
load leveling
Sharding
Static content
Throttling
9. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Queue-Based Load Leveling Pattern (Context)
• Cloud app require external services
• High load on cloud app means high load on
services
• External services may be less scalable
• High load on cloud app could result in
failing external services
• Possible self-throttling
10. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Queue-Based Load Leveling Pattern (Solution)
• Force the processing of request inside a
queue
• Thus, load-leveled service requests
• Additional advantage: queue also works as
a buffer
11. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Queue-Based Load Leveling Pattern (Solution)
12. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Queue-Based Load Leveling Pattern (Consid.)
• Make sure services are scaled correctly
• Task senders may wait service replies
14. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Asynchronously process requests
• The number of concurrent requests over
time varies
• The time required for processing varies
Competing Consumers Pattern (Context)
15. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Competing Consumers Pattern (Solution)
16. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Competing Consumers Pattern (Considerations)
• Ordering
• Poisoned messages
• Result handling
• Message queue scaling
• Reliability
18. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Asynchronous processing via queues
–Queues can’t sort messages (most of the
times)
• Push notification (15K) vs. e-mail (15K)
Priority Queue Pattern (Context)
19. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Queues with different priorities
• Consumers based on queue priority
Priority Queue Pattern (Solution)
21. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Priority Queue Pattern (Considerations)
• What is ‘high priority’ vs ‘low priority’
• (Single pool consumers) high first, low after
• (Single pool consumers) elevate old
messages
• Multiple queues work best for less priority
definitions
22. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Priority Queue Pattern (When To Use)
Push first, send after example
Multi-tenant applications
Different SLAs / customers
23. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Priority Queue Pattern (When NOT To Use)
Messages have similar priority
No burst of messages in the queue ever
exists
Costs must be kept down
26. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Throttling Pattern (Context)
• Cloud application load varies
–# active users (mostly during work hours)
–Type of activities (analysis at end of month)
• Sudden unanticipated bursts
–Poor performance
–Eventual failures
• SLA requirements
27. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Throttling Pattern (Solution)
• Auto-scaling, for starters…
• Define resource soft limits
• Monitor resource usage
• Throttle users
–Based on business impact (tiers / plans)
–Based on users’ concurrent requests
• Degrade functionality
• Load-leveling pattern / priority-queue
pattern
30. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Throttling Pattern (Considerations)
• Architectural decision: consider it while
designing
• Quick monitoring technique
• Notify accordingly
31. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Throttling Pattern (When To Use)
• Meet SLA
• Prevent single user monopolize everything
• Gracefully handle activity bursts
• Control costs by limiting max. resource
usage
35. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Most developers think about how data is
stored
• In NoSQL, we usually store everything in a
single entity
• In SQL, we have size constraints
• End-up in:
–Performance impact
–High prices
Materialized View Pattern (Context)
36. Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Generate views in advance, containing data
on a per-requirement basis
• Only contain data required by query
• Include current values of calculated
columns or data items
• May be optimized for a single query
• Updated a.s.a.p. (schedule / triggered)
Materialized View Pattern (Solution)
38. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Materialized View Pattern (When To Use)
Queries are complex
Data difficult to query directly
Temporary views dramatically improve perf.
Temporary views act as DTOs for UI,
reporting etc.
Data store not always available
Security or privacy reasons
39. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Materialized View Pattern (When NOT To Use)
Data source is simple to query
Data changes quickly
Consistency is most important
42. Premium community conference on Microsoft technologies itcampro@ itcamp14#
COMMAND AND QUERY
RESPONSIBILITY SEGREGATION
PATTERN
43. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (Context)
• Traditional CRUD system do everything over
the same data store
• Typically, same entity for DB <--> UI <--> DB
45. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (Context)
• Many concurrent connections FAIL
• Complex business logic FAIL
• Too much data passed around
• Performance impact @ high load, due to
complex querying
• Security issues may arise
46. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (Solution)
• Segregate read (queries) from write
(commands)
• Models for querying and for updating are
different
• Possible to access same store, better not to
48. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (Considerations)
• Additional complexity
• Consistency considerations
• CQRS for parts of the application
• Use in conjuction with Event Source pattern
49. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (When To Use)
Multiple concurrent operations
Already familiar with Domain-Driven-
Design techniques
Read performance ≠ write performance
Different teams (read vs. write)
App. lifecycle: model update, business logic
update
50. Premium community conference on Microsoft technologies itcampro@ itcamp14#
CQRS Pattern (When NOT To Use)
Simple business rules
Simple CRUD-style UI are enough
Across the whole system
52. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Sharding Pattern (Context)
• Why scale out compute, and not scale out
data?
• Must scale out data because:
–Storage limitations
–Concurrent requests
–Network bandwidth
–Geography
53. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Sharding Pattern (Solution)
• Horizontal partitions of data
–(a.k.a. shards)
• Same schema, different data
• Runs on its own server
• Benefits:
–Scale out data service
–Use commodity hardware
–Better performance
–Closely located geographically
54. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Sharding Pattern (Solution)
• Lookup strategy• Range strategy• Hash strategy
55. Premium community conference on Microsoft technologies itcampro@ itcamp14#
Strategy Advantages Considerations
Lookup • More control
• Easy shard rebalance
• Shard lookup may
create additional
overhead
Range • Easy to implement
• Works well on range
queries
• Easy management
• Suboptimal balance
• Shard rebalance is
difficult
Hash • Best balance
• Request routing directly via
hashing alg.
• Calculating hash may
create additional
overhead
• Rebalance is difficult
Sharding Pattern (Solution)