Presentation from Application Developers Days 3 conference (Moscow, May 2012) about best practices in scalable application development on AWS.

1. Scalable Application
Development on AWS
Mikalai Alimenkou
http://xpinjection.com
11.05.2012

2. Background
• Java Technical Lead/Scrum Master at Zoral
Labs
• 7+ years in software development
• 5+ years of working by Agile
methodologies
• Expert in Agile engineering practices
• Founder and coach at XP Injection
• Organizer of Selenium Camp, JEEConf, XP
Days Ukraine, IT Brunch conferences

3. No
Future!

4. Cloud computing
• Scale capacity on demand
• Turn fixed costs into variable costs
• Clean APIs and conceptual models
• Always available
• Cost-effective
• Reduced time to market
• Focus on product

5. • Interesting startup
• Completely in Java
• Customer wants to use AWS
New project arrived

6. AWS structure
• Elastic Compute Cloud • Simple Storage Service (S3) • SimpleDB
(EC2) • Elastic Block Store (EBS) • DynamoDB
• Elastic Map Reduce (EMR) • Import/Export • Relational Database
• Auto Scaling • Storage Gateway Service (RDS)
• Elastic Load Balancing • ElastiCache
Compute Storage Database
• Route 53 • Simple Queue Service (SQS) • CloudWatch
• Virtual Private Cloud • Simple Notification Service • Elastic Beanstalk
(VPC) (SNS) • CloudFormation
• Direct Connect • Simple Email Service (SES) • Identity and Access
• CloudFront • Simple Workflow Service Management (IAM)
(SWS)
• CloudSearch
Networking App Services Deployment

7. Why Java?
• Runs everywhere
• Full technological stack
• Lots of tools for scalability
• Many frameworks for cloud development

8. EC2 quick introduction
$.08-$2.4 per
server hour
$.05 - $.12 per GB
data transfer

9. Simple boot model AMI
Instant-store AMI
• Stateless
Instance
• Cheaper
• Better IO
• Easier to scale
Cloud
EBS boot AMI
• Stateful
• Stop/start support
• Quick migration
• Faster loading

10. Different API access levels
AWS management console AWS console tools
REST API clients Plugins to IDEs and browsers

11. Elasticfox – my superhero!

12. Everything on AWS
• Infrastructure
• Testing
• Staging
• Production
• Monitoring

13. • Easy and reliable server configuration with AMI
• No work for system administrator
• Any number of servers available
• No waits for hardware
• More experimenting and testing
• Easy security model
Happiness…

14. Look ma, no hands!

15. What about money?

16. EC2 charging model

18. What I have for such money?
Testing
Server
DB
Server
Production
Server

19. Ways to save money
• Good and responsible team 
• Reuse instances when possible rounding
usage time
• Release instances when not more needed
• Monitor and control running instances
• Use reserved instances for continuously
running jobs

20. EC2 physical distribution

21. • Hourly payment is invisible
• Instances are virtual
• Same type of machine, but
different performance
• Instance not immediately
available
• Images creation require
“magical” skills
• No multicast messaging

22. Amazon cares about
you … and other people

23. Everything is unreliable
• Retry policies everywhere
• Timeouts for all operations
• Automatic recovery

24. EBS quick introduction
• Raw data volumes from 1GB to 1TB
• Replicated automatically
• Snapshots to S3
• May be used with different instances

25. EBS architecture

26. EBS fault tolerance

27. EBS cost model
• Not so cheap
• Additional costs for snapshots in S3

28. EBS != hard drive
• Slow for time critical tasks
• Bottleneck for Lucene indexing and DB
– Use parallel EBS volumes
– Separate data storage between EC2 instances
– Don’t use RDS
• Limited to 1TB
– Virtual RAID
– Data distribution

29. S3 quick introduction
$.055 - $.125
per GB month
storage
$.05 - $.12 per
GB data transfer

30. Load balancing with
JMS or AMQP
• JMS and AMQP are very scalable and flexible
• Costs are less than SQS (Simple Queue
Service)
• Configurable load balancing

31. Deployment options

32. Self-distributed system
• Host manager works via Java AWS API
• Centralized deployment
• Code distribution to EC2 hosts

33. Reduce coupling to AWS

34. Conclusions
• AWS is great for startups or research projects
• AWS increases development speed in times
• It is easy to create scalable and highly available
Java application on AWS
• Development, testing and production may be
done on AWS
• AWS save development costs
• AWS support Agile developmernt approaches

35. Useful books

36. @xpinjection
http://xpinjection.com
mikalai.alimenkou@xpinjection.com