2. Introduction
Cloud Computing Industry is growing
According to Gartner, worldwide cloud services revenue is on
pace to surpass $56.3 billion in 2009, a 21.3% increase from
2008 revenue of $46.4 billion, according to Gartner, Inc. The
market is expected to reach $150.1 billion in 2013.
Businesses are increasing Cloud adoption
"We expect a great deal of migration towards cloud computing
within the federal government in addition to the already robust
private sector growth. The growth of the cloud should not
outpace our ability to protect the data that goes into it..." ~
FormerWhite House advisor Paul Kurtz, partner with Good Harbor
Consulting, LLC
How can IT leaders ensure security in the cloud?
IWillStudy Cloud Summit Ahmedabad
4. Cloud Basics
Cloud Characteristics
Service Models
SaaS
IaaS
PaaS
Deployment Models
Public
Private
Community
Hybrid
IWillStudy Cloud Summit Ahmedabad
5. Cloud Computing: Why Now?
Experience with very large datacenters
Unprecedented economies of scale
Transfer of risk
Technology factors
Pervasive broadband Internet
Maturity in Virtualization Technology
Business factors
Minimal capital expenditure
Pay-as-you-go billing model
IWillStudy Cloud Summit Ahmedabad
6. And in a non-Cloud view, there are inefficiencies in
addressing those issues
Allocated Load
IT-capacities Forecast
“Under-supply“
of capacities
IT CAPACITY
“Waste“ of Fixed cost of
capacities IT-capacities
Barrier for
innovations Actual Load
TIME
7. However, in a Cloud View
Load
Allocated IT Forecast
capacities
No “under-supply“
IT CAPACITY
Reduction of Possible
“over-supply“ reduction of
IT-capacities
in case of
Reduction reduced load
of initial
investments
Actual Load
Time
9. Cloud Service Models
Software as a Service
(SaaS)
Platform as a Service
(PaaS)
Infrastructure as a Service
(IaaS)
IWillStudy Cloud Summit Ahmedabad
16. “OpenStack is a collection of open source software
projects that enterprises/service providers can use to
setup and run their cloud compute and storage
infrastructure.”
— docs.openstack.org
The OpenStack Consortium has grown rapidly in the
past year:
• NASA • Intel
• Rackspace • Cisco
• Citrix • HP
• Dell • Over 140 others
• AMD
OpenStack services are available via Amazon’s S3
and EC2 APIs. Applications written for Amazon Web
Services will work with OpenStack.
IWillStudy Cloud Summit Ahmedabad
17. Introducing Windows Azure
Web applications WEB & CLOUDS Third party cloud
Compute Storage Management Relational data Management Connectivity Access control
Composite applications ON-PREMISES LOB Applications
IWillStudy Cloud Summit Ahmedabad
18. Google App Engine
Based on non-proprietary technology:
Java servlet API version 2.5 or
Python
Easy to learn
Rich tooling
Extendable (can use 3rd party tools like GWT, Guice,
Spring, JSF, Struts, Grails, Restlet)
Commercial support
Google has proven they are good at this stuff
IWillStudy Cloud Summit Ahmedabad
20. GAE building blocks
Java Servlet Environment running in Java 6 sandbox
Version 2.5
Storing data
Object database != relational: denormalized -> optimized for reading
Designed for searching in very large datasets
Multiple flavours: JDO (and JPA or native), @nnotation based
Relationships: owned (1:1, 1:n) and unowned (1:n, n:1, n:n)
Entity groups
Transactions
Indexes
URL fetch
Integration with other systems through HTTP
Mail
Send and receive email
XMPP
Send and receive instant messages
Channel
IWillStudy Cloud Summit Ahmedabad
Persistent connection: no need for client side polling
21. GAE building blocks (continued)
Blobstore
Manage large data chunks (like files, images)
Image processing
Transform, resize, change format
Memcache
Minimize datastore hits
Cron
Schedule repetitive, non-interactive jobs
Task Queues
Asynchronous, non-interactive background processing
Authentication and SSO
Google account
Open ID
Multi-tenancy
Support different user groups
Many 3rd party libraries
But not all!
IWillStudy Cloud Summit Ahmedabad
22. AWS Management Console
Central location to all your aws cloud apps
EC2
Beanstalk
S3
RDS
…
IWillStudy Cloud Summit Ahmedabad
23. Setting up EC2 account and tools
Create AMI signing certificate
mkdir ~/.ec2
cd ~/.ec2
openssl genrsa -des3 -out pk-<group>.pem 2048
openssl rsa -in pk-<group>.pem -out pk-unencrypt-<group>.pem
openssl req -new -x509 -key pk-<group>.pem -out cert-<group>.pem -days
1095
Share all three .pem files manually with group members
Troubleshooting: If your client date is wrong your certs will not work
Upload certificate to AWS via IAM page
Login at: https://283072064258.signin.aws.amazon.com/console
Account: 283072064258
Username: group** (e.g. group1, group10, group18)
Password: In email from Dr. Jin (12 digits, something like N9EzPxXGw0Gg)
Click IAM tab -> users -> select yourself (use right arrow if needed)
In bottom pane select “Security Credentials” tab and click “Manage Signing Certificates”
Click “Upload Signing Certificate”
cat ~/.ec2/cert-<group>.pem
Copy contents into „Certificate Body‟ textbox and click „OK‟
25. Setting up EC2 account and tools
Retrieve and unpack AWS tools
wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
unzip ec2-api-tools.zip
Create ec2 initialization script
vi ec2-init.sh (you can use your preferred editor)
export JAVA_HOME=/usr
export EC2_HOME=~/ec2-api-tools-1.5.2.4
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=~/.ec2/pk-unencrypt-<group>.pem
export EC2_CERT=~/.ec2/cert-<group>.pem
source ec2-init.sh
This will need to be done every login
Alternately, put it in ~/.profile to have it done automatically on login
Test it out
ec2-describe-regions
ec2-describe-images -o self -o amazon
Troubleshooting
http://docs.amazonwebservices.com/AmazonEC2/gsg/2007-01-03/
26. Setting up EC2 account and tools
Create a new keypair (allows cluster login)
ec2-add-keypair <group>-keypair | grep –v KEYPAIR > ~/.ec2/id_rsa-<group>-
keypair
chmod 600 ~/.ec2/id_rsa-<group>-keypair
Only do this once! It will create a new keypair in AWS every time you run it
Share private key file between group members, keep it private
Don’t delete other groups’ keypairs!
Everyone has access to everyone else’s keypairs from the AWS console
EC2 tab ->Network and Security -> Keypairs
Troubleshooting
http://docs.amazonwebservices.com/AmazonEC2/gsg/2007-01-03/
27. Cleanup
Terminate your clusters when you’re done!
They cost <personX> grant money ($1/hour for a full cluster of 9 nodes)
You can always create more later
hadoop-ec2 terminate <group>-cluster
They can also be terminated manually from the AWS->EC2 console
28. GAE Demo
Course Builder on Google App engine
http://www.iwillstudy.com/group/cloudcomputing
IWillStudy Cloud Summit Ahmedabad