O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Amazon S3 Masterclass

4.711 visualizações

Publicada em

These are the slides from my 2015 Amazon S3 Masterclass webinar

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Amazon S3 Masterclass

  1. 1. Masterclass ianmas@amazon.com @IanMmmm Ian Massingham — Technical Evangelist Amazon S3
  2. 2. Masterclass Intended to educate you on how to get the best from AWS services Show you how things work and how to get things done A technical deep dive that goes beyond the basics 1 2 3
  3. 3. Amazon S3 Secure, durable, highly-scalable object storage
 Accessible via a simple web services interface Store & retrieve any amount of data Use alone or together with other AWS services
  4. 4. Amazon S3 Durable Available Low Cost Scalable High Performance Secure Integrated Easy to Use
  5. 5. Amazon S3 Backup & Archiving Big Data Analytics Static Website HostingDisaster Recovery Content Storage & Distribution Cloud-native Application Data
  6. 6. Agenda Amazon S3 Concepts & Fundamentals
 Namespaces Access Controls Storage Classes
 Encryption & Other Security Features Versioning & Cross-Region Replication Lifecycle Rules Website Hosting
  7. 7. AMAZON S3 CONCEPTS
  8. 8. BUCKETS Containers for objects stored in S3 Serve several purposes: Organise the Amazon S3 namespace at the highest level Identify the account responsible for charges Play a role in access control Serve as the unit of aggregation for usage reporting
  9. 9. Fundamental entities stored in Amazon S3 Consist of data & metadata Data portion is opaque to Amazon S3 Metadata is a set of name-value pairs that describe the object Object is uniquely identified within a bucket by a key (name) and a version ID OBJECTS
  10. 10. Unique identifier for an object within a bucket. Every object in a bucket has exactly one key Combination of a bucket, key & version ID uniquely identify each object KEYS http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl
  11. 11. The geographical region where Amazon S3 will store the buckets that you create Choose a region to optimise latency, minimise costs, or address regulatory requirements. REGIONS
  12. 12. AMAZON S3 FUNDAMENTALS A web store, not a file system Highly scalable data storage Fast Economical Access via APIs Highly available & durable
  13. 13. Access via APIshttp://aws.amazon.com/documentation/s3/
  14. 14. Access via SDKshttp://aws.amazon.com/tools/
  15. 15. Access via SDKs https://github.com/awslabs/aws-sdk-go
  16. 16. Access via AWS CLI List buckets List buckets contents Copy a file to an object Stream the contents of an object to STDOUT Delete an object Sync a directory with a bucket List buckets contents Delete buckets contents AWS S3 CLI help AWS CLI
  17. 17. Access via AWS CLI Detailed help on a specific command AWS CLI
  18. 18. AMAZON S3 FUNDAMENTALS A web store, not a file system Highly scalable data storage Fast Economical Access via APIs Highly available & durable
  19. 19. A web store, not a file system Eventually consistent Write once, Read many
  20. 20. A web store, not a file system Eventually consistent Write once, Read many
  21. 21. A web store, not a file system Eventually consistent Write once, Read many Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers
  22. 22. A web store, not a file system Eventually consistent Write once, Read many Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers
  23. 23. A web store, not a file system Eventually consistent Write once, Read many Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers
  24. 24. A web store, not a file system Eventually consistent Write once, Read many Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers
  25. 25. Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers A web store, not a file system Eventually consistent Write once, Read many
  26. 26. Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers A web store, not a file system Eventually consistent Write once, Read many
  27. 27. A web store, not a file system Eventually consistent Write once, Read many Region Availability.Zone Indexing Storage Load.balancers Web.servers Availability.Zone Indexing Storage Load.balancers Web.servers
  28. 28. A web store, not a file system Eventually consistent Write once, Read many
  29. 29. A web store, not a file system Eventually consistent Write once, Read many New Objects Updates Deletes Synchronously stores your data across multiple facilities before returning SUCCESS Read-after-write consistency, except US-STANDARD region Write then read: could report key does not exist Write then list: might not include key in list Overwrite then read: old data could be returned Delete then read: could still get old data Delete then list: deleted key could be included in list Find out more here: docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html
  30. 30. NAMESPACES
  31. 31. Bucket Name + Object Name (key) Globally Unique
  32. 32. Bucket Name + Object Name (key) Globally Unique Amazon S3 bucket object object bucket object object bucket object object
  33. 33. Bucket Name + Object Name (key) Globally Unique Amazon S3 ianm-aws-bootstrap wp/bootstrap.sh wp/credentials.txt ianm-aws-docs s3-webinar.pptx vid/s3-webinar.mp4 aws-exampl.es index.html logo.png
  34. 34. Object key Unique within a bucket
  35. 35. Object key Unique within a bucket Max 1024 bytes UTF-8 Including ‘path’ prefixes
  36. 36. Object key Unique within a bucket Max 1024 bytes UTF-8 Including ‘path’ prefixes assets/js/jquery/plugins/jtables.js   an example object key
  37. 37. Throughput Optimisation 2134857/gamedata/start.png   2134857/gamedata/resource.rsrc   2134857/gamedata/results.txt   2134858/gamedata/start.png   2134858/gamedata/resource.rsrc   2134858/gamedata/results.txt   2134859/gamedata/start.png   2134859/gamedata/resource.rsrc   2134859/gamedata/results.txt mynewgame Bucket Object keys S3 automatically partitions based upon key prefix:
  38. 38. Throughput Optimisation 2134857/gamedata/start.png   2134857/gamedata/resource.rsrc   2134857/gamedata/results.txt   2134858/gamedata/start.png   2134858/gamedata/resource.rsrc   2134858/gamedata/results.txt   2134859/gamedata/start.png   2134859/gamedata/resource.rsrc   2134859/gamedata/results.txt mynewgame Bucket Object keys S3 automatically partitions based upon key prefix: Incrementing game IDs
  39. 39. Throughput Optimisation 2134857/gamedata/start.png   2134857/gamedata/resource.rsrc   2134857/gamedata/results.txt   2134858/gamedata/start.png   2134858/gamedata/resource.rsrc   2134858/gamedata/results.txt   2134859/gamedata/start.png   2134859/gamedata/resource.rsrc   2134859/gamedata/results.txt mynewgame Bucket Object keys S3 automatically partitions based upon key prefix: Partition: mynewgame/2
  40. 40. Throughput Optimisation 7584312/gamedata/start.png   7584312/gamedata/resource.rsrc   7584312/gamedata/results.txt   8584312/gamedata/start.png   8584312/gamedata/resource.rsrc   8584312/gamedata/results.txt   9584312/gamedata/start.png   9584312/gamedata/resource.rsrc   9584312/gamedata/results.txt mynewgame Bucket Object keys S3 automatically partitions based upon key prefix: Reversed game ID
  41. 41. Throughput Optimisation 7584312/gamedata/start.png   7584312/gamedata/resource.rsrc   7584312/gamedata/results.txt   8584312/gamedata/start.png   8584312/gamedata/resource.rsrc   8584312/gamedata/results.txt   9584312/gamedata/start.png   9584312/gamedata/resource.rsrc   9584312/gamedata/results.txt mynewgame Bucket Object keys S3 automatically partitions based upon key prefix: Partitions: mynewgame/7 mynewgame/8 mynewgame/9
  42. 42. ACCESS CONTROLS
  43. 43. SECURE BY DEFAULT You decide what to share Apply policies to buckets and objects Policies, ACLs & IAM Use S3 policies, ACLs or IAM to define rules
  44. 44. Fine grained Administer as part of role based access Apply policies to S3 at role, user & group level Allow   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* Bob Jane IAM Policies Find out more here: aws.amazon.com/iam
  45. 45. Fine grained Administer as part of role based access Apply policies to S3 at role, user & group level Allow   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* Bob Jane Find out more here: aws.amazon.com/iam IAM Policies
  46. 46. Fine grained Administer as part of role based access Apply policies to S3 at role, user & group level Allow   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* Bob Jane Find out more here: aws.amazon.com/iam IAM Policies
  47. 47. Fine grained Administer as part of role based access Apply policies to S3 at role, user & group level Allow   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* Bob Jane Bucket Policies Fine grained Apply policies at the bucket level in S3 Incorporate user restrictions without using IAM Allow   Bob,  Jane   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* mybucket IAM Policies
  48. 48. Bucket Policies Fine grained Apply policies at the bucket level in S3 Incorporate user restrictions without using IAM Allow   Bob,  Jane   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* mybucket {      "Version":"2012-­‐10-­‐17",      "Statement":[          {              "Sid":"AddPerm",              "Effect":"Allow",              "Principal":  "*",              "Action":["s3:GetObject"],              "Resource":["arn:aws:s3:::examplebucket/*"]          }      ]   } Granting Read-Only Permission to an Anonymous User
  49. 49. {      "Version":"2012-­‐10-­‐17",      "Statement":[          {              “Sid":"AddPerm",              "Effect":"Allow",              "Principal":  "*",              "Action":["s3:GetObject"],              "Resource":["arn:aws:s3:::examplebucket/*"]          }      ]   } Granting Read-Only Permission to an Anonymous User Who the policy applies to. * means everyone The actions allowed/denied by this policy The AWS resource that this policy applies to. In this case all objects in example bucket The effect of the policy, allow or deny
  50. 50. Bucket Policies Fine grained Apply policies at the bucket level in S3 Incorporate user restrictions without using IAM Allow   Bob,  Jane   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* mybucket {      "Version":  "2012-­‐10-­‐17",      "Id":  "S3PolicyId1",      "Statement":  [          {              "Sid":  "IPAllow",              "Effect":  "Allow",              "Principal":  "*",              "Action":  "s3:*",              "Resource":  "arn:aws:s3:::examplebucket/*",              "Condition":  {                    "IpAddress":  {"aws:SourceIp":  "54.240.143.0/24"},                    "NotIpAddress":  {"aws:SourceIp":  "54.240.143.188/32"}                }            }        ]   } Restricting Access to Specific IP Addresses
  51. 51. Bucket Policies Fine grained Apply policies at the bucket level in S3 Incorporate user restrictions without using IAM Allow   Bob,  Jane   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* mybucket Other Example Use-Cases for Bucket Policies • Granting Permissions to Multiple Accounts with Added Conditions • Restricting Access to a Specific HTTP Referrer • Granting Permission to an Amazon CloudFront Origin Identity • Adding a Policy to Require MFA Authentication • Granting Cross-Account Permissions to Upload Objects While Ensuring the Bucket Owner Has Full Control Find out more here: docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html
  52. 52. Fine grained Administer as part of role based access Apply policies to S3 at role, user & group level Allow   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* Bob Jane Bucket Policies ACLs Fine grained Apply policies at the bucket level in S3 Incorporate user restrictions without using IAM Coarse grained Apply access control rules at the bucket and/or object level in S3 Allow   Bob,  Jane   Actions   PutObject   Resource   arn:aws:s3:::mybucket/* mybucket Allow   Everyone,  Bob,  Jane   Actions   Read   mybucket myobject IAM Policies
  53. 53. ACLs Coarse grained Apply access control rules at the bucket and/or object level in S3 Allow   Everyone,  Bob,  Jane   Actions   Read   mybucket myobject You can use ACLs to grant basic read/write permissions to other AWS accounts. There are limits to managing permissions using ACLs. For example, you can grant permissions only to other AWS accounts, you cannot grant permissions to users in your account.
  54. 54. DEMO: GETTING STARTED
  55. 55. STORAGE CLASSES
  56. 56. Designed to provide 99.999999999% durability and 99.99% availability of objects over a given year Designed to sustain the concurrent loss of data in two facilities Reduces costs by storing data at lower levels of redundancy than the Standard storage Designed to provide 99.99% durability and 99.99% availability of objects over a given year Suitable for archiving data, where data access is infrequent and a retrieval time of several hours is acceptable Uses the very low-cost Amazon Glacier storage service, but managed through Amazon S3 Standard Reduced Redundancy Storage Glacier
  57. 57. Moving Objects between S3 storage classes You can specify the storage class of an object when uploading or creating it $  aws  s3  cp  aws_uki.txt  s3://aws-­‐ianm-­‐s3-­‐masterclass/  -­‐-­‐storage-­‐ class  REDUCED_REDUNDANCY AWS CLI
  58. 58. Moving Objects between S3 storage classes You can change the storage class of an object that is already stored in Amazon S3 by copying it to the same key name in the same bucket $  aws  s3  cp  s3://aws-­‐ianm-­‐s3-­‐masterclass/aws_uki.txt  s3://aws-­‐ ianm-­‐s3-­‐masterclass/aws_uki.txt  -­‐-­‐storage-­‐class  STANDARD AWS CLI
  59. 59. Moving Objects between storage classes AWS Console
  60. 60. Moving Objects between storage classes $  python   >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  mybucket  =  conn.get_bucket(‘aws-­‐ianm-­‐s3-­‐masterclass’)   >>>  mybucket.copy_key('aws_uki.txt','aws-­‐ianm-­‐s3-­‐ masterclass','aws_uki.txt',storage_class='REDUCED_REDUNDANCY') Python
  61. 61. What about Amazon Glacier? We will come to this when we talk about Lifecycle Management
  62. 62. ENCRYPTION
  63. 63. Securing Data in Transit Securely upload or download your data via SSL-encrypted endpoints using HTTPS Alternatively, use a client encryption library such as the Amazon S3 Encryption Client to encrypt your data before uploading to Amazon S3 http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3EncryptionClient.html
  64. 64. SECURING DATA AT REST
  65. 65. Amazon S3 Server Side Encryption (SSE) Amazon S3 will automatically encrypt your data on write and decrypt your data on retrieval Uses Advanced Encryption Standard (AES) 256-bit symmetric keys There are three different ways to mange encryption keys:
  66. 66. SSE with Amazon S3 Key Management (SSE-SE) With SSE-S3, Amazon S3 will encrypt your data at rest and manage the encryption keys for you Data bucket Encrypted data Encrypted per-object key Key management (monthly rotation) Per-object key Master key
  67. 67. SSE with Customer-Provided Keys (SSE-C) With SSE-C, Amazon S3 will encrypt your data at rest using the custom encryption keys that you provide Amazon S3 doesn’t store your encryption key anywhere; the key is immediately discarded after Amazon S3 completes your requests Data bucket Encrypted data Customer Provided Key
  68. 68. SSE with AWS KMS (SSE-KMS) With SSE-KMS, Amazon S3 will encrypt your data at rest using keys that you manage in the AWS Key Management Service (KMS) AWS KMS provides an audit trail so you can see who used your key to access which object and when
  69. 69. ADDITIONAL SECURITY FEATURES
  70. 70. AUDIT LOGS
  71. 71. MULTI-FACTOR AUTHENTICATION DELETE
  72. 72. TIME-LIMTED ACCESS TO OBJECTS
  73. 73. Signed URLs Provide time-limited access to specific objects that expires after a set period Access Permissions Use on objects in non-public buckets to prevent access once the signed URL has expired https://ianmas-­‐aws.testbucket.s3.amazonaws.com/testfile.txt   ?Signature=JHCa39GV1fKRKkEnAWzI88lH7f8%3D   &Expires=1391425438   &AWSAccessKeyId=AKIAIRBKBJ3ZAYAXFC2Q
  74. 74. >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  conn.generate_url(3600,  'GET',  bucket='aws-­‐ianm-­‐s3-­‐masterclass',   key=‘aws_uki.txt')   'https://aws-­‐ianm-­‐s3-­‐masterclass.s3.amazonaws.com/aws_uki.txt? Signature=hEBUPczy8DXCyqTz1JHgEaihvMo %3D&Expires=1431697820&AWSAccessKeyId=AKIAI65L23YDGKGQTRFA'   >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  conn.generate_url(30,  'GET',  bucket='aws-­‐ianm-­‐s3-­‐masterclass',   key=‘aws_uki.txt’,  force_http=True)   'http://aws-­‐ianm-­‐s3-­‐masterclass.s3.amazonaws.com/aws_uki.txt? Signature=yIYPyn0DMXk2cOcZkWPRuSHoKPA %3D&Expires=1431694649&AWSAccessKeyId=AKIAI65L23YDGKGQTRFA' PythonGenerating time-limited signed links
  75. 75. >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  conn.generate_url(3600,  'GET',  bucket='aws-­‐ianm-­‐s3-­‐masterclass',   key=‘aws_uki.txt')   'https://aws-­‐ianm-­‐s3-­‐masterclass.s3.amazonaws.com/aws_uki.txt? Signature=hEBUPczy8DXCyqTz1JHgEaihvMo %3D&Expires=1431697820&AWSAccessKeyId=AKIAI65L23YDGKGQTRFA'   >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  conn.generate_url(30,  'GET',  bucket='aws-­‐ianm-­‐s3-­‐masterclass',   key=‘aws_uki.txt’,  force_http=True)   'http://aws-­‐ianm-­‐s3-­‐masterclass.s3.amazonaws.com/aws_uki.txt? Signature=yIYPyn0DMXk2cOcZkWPRuSHoKPA %3D&Expires=1431694649&AWSAccessKeyId=AKIAI65L23YDGKGQTRFA' PythonGenerating time-limited signed links 1st parameter is link lifetime in seconds Force a non-SSL link
  76. 76. Generating time-limited signed links Error response: link expired
  77. 77. VERSIONING & CROSS REGION REPLICATION
  78. 78. VERSIONING Bucket level Automatically preserves all copies of objects Persistent Even deleted object history is held
  79. 79. AWS Console
  80. 80. AWS Console
  81. 81. AWS CLI uses the ‘aws s3api’ CLI command, which has additional functionality over ‘aws s3’ $  aws  s3api  list-­‐object-­‐versions  -­‐-­‐bucket  aws-­‐ianm-­‐s3-­‐masterclass Working with versioned objects via the CLI http://docs.aws.amazon.com/cli/latest/reference/s3api/index.html#cli-aws-s3api
  82. 82. AWS CLI $  aws  s3api  list-­‐object-­‐versions  -­‐-­‐bucket  aws-­‐ianm-­‐s3-­‐masterclass   None   None   VERSIONS   "36bc67941830bb388c9bf201440683a4"   True   s3-­‐masterclass-­‐logo.txt   2015-­‐05-­‐18T15:47:38.000Z   337   STANDARD   q2FcbXFAgU7P82Hh6y81hYkrbGYtOCCc   OWNER   ianm   4ee381d180ee58aa815e7d4a3a5f739b20bb8980a568947384e59c8d0ff8379b   VERSIONS   "e0253c9354f61097cbf6ce239afd0464"   False   s3-­‐masterclass-­‐logo.txt   2015-­‐05-­‐18T15:47:30.000Z   337   STANDARD   p_4oF4eG7Be.0aNXqUYFlZL7Q9OuK9nU   OWNER   ianm   4ee381d180ee58aa815e7d4a3a5f739b20bb8980a568947384e59c8d0ff8379b   VERSIONS   "84defb05031845e8b0616a9b70b2ae93"   False   s3-­‐masterclass-­‐logo.txt   2015-­‐05-­‐18T15:47:10.000Z   328   STANDARD   BykgMQ6bRY02Y6krxvMaMvwrL2Ep2e6X   OWNER   ianm   4ee381d180ee58aa815e7d4a3a5f739b20bb8980a568947384e59c8d0ff8379b
  83. 83. AWS CLI $  aws  s3api  get-­‐object  -­‐-­‐bucket  aws-­‐ianm-­‐s3-­‐masterclass  -­‐-­‐key  s3-­‐ masterclass-­‐logo.txt  -­‐-­‐version-­‐id   q2FcbXFAgU7P82Hh6y81hYkrbGYtOCCc  version.txt   bytes   337   text/plain   "36bc67941830bb388c9bf201440683a4"   Mon,  18  May  2015  15:47:38  GMT   q2FcbXFAgU7P82Hh6y81hYkrbGYtOCCc   $  more  version.txt    ____  _____      __    __                      _                                _   /  ___|___  /    |    /    |  __  _  ___|  |_  ___  _  __  ___|  |  __  _  ___  ___   ___    |_      |  |/|  |/  _`  /  __|  __/  _    '__/  __|  |/  _`  /  __/  __|    ___)  |__)  |  |  |    |  |  (_|  __    ||    __/  |  |  (__|  |  (_|  __  __     |____/____/    |_|    |_|__,_|___/_____|_|    ___|_|__,_|___/___/   Version  3 http://docs.aws.amazon.com/cli/latest/reference/s3api/index.html#cli-aws-s3api
  84. 84. >>>  import  boto   >>>  conn  =  boto.connect_s3()   >>>  bucket=conn.get_bucket('aws-­‐ianm-­‐s3-­‐masterclass')   >>>  versions  =  bucket.list_versions()   >>>  for  version  in  versions:   ...      print  version.name  +  '  '  +  version.version_id   ...   s3-­‐masterclass-­‐logo.txt  q2FcbXFAgU7P82Hh6y81hYkrbGYtOCCc   s3-­‐masterclass-­‐logo.txt  p_4oF4eG7Be.0aNXqUYFlZL7Q9OuK9nU   s3-­‐masterclass-­‐logo.txt  BykgMQ6bRY02Y6krxvMaMvwrL2Ep2e6X PythonListing object versions
  85. 85. >>>  key  =  bucket.get_key('s3-­‐masterclass-­‐logo.txt',   version_id='p_4oF4eG7Be.0aNXqUYFlZL7Q9OuK9nU')   >>>  key.get_contents_as_string()   "n  ____  _____      __    __                      _                                _                              n/  ___|___  /    |     /    |  __  _  ___|  |_  ___  _  __  ___|  |  __  _  ___  ___  n___    |_      |  |/|  |/  _`  /   __|  __/  _    '__/  __|  |/  _`  /  __/  __|n  ___)  |__)  |  |  |    |  |  (_|  __    ||    __/  |  |   (__|  |  (_|  __  __  n|____/____/    |_|    |_|__,_|___/_____|_|    ___|_| __,_|___/___/nnVersion  2n”   >>>  key.generate_url(300)   'https://aws-­‐ianm-­‐s3-­‐masterclass.s3.amazonaws.com/s3-­‐masterclass-­‐logo.txt? Signature=c%2BjgGY5EZ4tDuI0xcKg572qL%2B9Y %3D&Expires=1431965853&AWSAccessKeyId=AKIAI65L23YDGKGQTRFA&versionId=p_4oF4eG7Be. 0aNXqUYFlZL7Q9OuK9nU'   >>> PythonGetting a specific object version
  86. 86. CROSS REGION REPLICATION
  87. 87. AWS Console
  88. 88. AWS Console
  89. 89. AWS Console
  90. 90. AWS Console
  91. 91. AWS Console
  92. 92. AWS Console
  93. 93. AWS CLI $  aws  s3  cp  s3-­‐masterclass-­‐logo.txt  s3://aws-­‐ianm-­‐s3-­‐masterclass   upload:  ./s3-­‐masterclass-­‐logo.txt  to  s3://aws-­‐ianm-­‐s3-­‐masterclass/s3-­‐masterclass-­‐logo.txt   $  aws  s3  ls  s3://aws-­‐ianm-­‐s3-­‐masterclass-­‐fra  -­‐-­‐region=eu-­‐central-­‐1   2015-­‐05-­‐18  17:27:47                337  s3-­‐masterclass-­‐logo.txt
  94. 94. LIFECYCLE RULES
  95. 95. LIFECYCLE RULES Object Deletion Permanently delete objects from S3 Object Archiving Move objects from S3 to Glacier
  96. 96. Amazon Glacier
  97. 97. Amazon Glacier Durable Designed for 99.999999999% durability of archives Cost Effective Write-once, read-never. Cost effective for long term storage. Pay for accessing data
  98. 98. Logs accessible from S3 logs Expiry time
  99. 99. logs ✗ Objects expire and are deleted Logs accessible from S3 Expiry time
  100. 100. Txns Object transition to Glacier invoked Logs logs ✗ Objects expire and are deleted accessible from S3 accessible from S3 ExpiryTransition time
  101. 101. Restoration of object requested for x hrs Logs logs ✗ Objects expire and are deleted accessible from S3 accessible from S3 Txns ExpiryTransition Object transition to Glacier invoked time
  102. 102. time 3-5hrs Object held in S3 RRS for x hrs ExpiryTransition Logs logs ✗ Objects expire and are deleted accessible from S3 accessible from S3 Txns Object transition to Glacier invoked Restoration of object requested for x hrs
  103. 103. Configuring Lifecycle Rules AWS Console
  104. 104. Configuring Lifecycle Rules AWS Console
  105. 105. Configuring Lifecycle Rules AWS Console
  106. 106. Configuring Lifecycle Rules AWS Console
  107. 107. Configuring Lifecycle Rules AWS Console
  108. 108. Configuring Lifecycle Rules AWS Console
  109. 109. Configuring Lifecycle Rules AWS Console
  110. 110. using  (client  =  new  AmazonS3Client()){      var  lifeCycleConfiguration  =  new  LifecycleConfiguration()      {          Rules  =  new  List<LifecycleRule>          {                  new  LifecycleRule                  {                            Id  =  "Archive  and  delete  rule",                            Prefix  =  "projectdocs/",                            Status  =  LifecycleRuleStatus.Enabled,                              Transition  =  new  LifecycleTransition()                              {                                        Days  =  365,                                        StorageClass  =  S3StorageClass.Glacier                              },                              Expiration  =  new  LifecycleRuleExpiration()                              {                                        Days  =  3650                              }                  }          }      }; .NetConfiguring Lifecycle Rules
  111. 111. using  (client  =  new  AmazonS3Client()){      var  lifeCycleConfiguration  =  new  LifecycleConfiguration()      {          Rules  =  new  List<LifecycleRule>          {                  new  LifecycleRule                  {                            Id  =  "Archive  and  delete  rule",                            Prefix  =  "projectdocs/",                            Status  =  LifecycleRuleStatus.Enabled,                              Transition  =  new  LifecycleTransition()                              {                                        Days  =  365,                                        StorageClass  =  S3StorageClass.Glacier                              },                              Expiration  =  new  LifecycleRuleExpiration()                              {                                        Days  =  3650                              }                  }          }      }; .NetConfiguring Lifecycle Rules Transition to Glacier after 1 year
  112. 112. using  (client  =  new  AmazonS3Client()){      var  lifeCycleConfiguration  =  new  LifecycleConfiguration()      {          Rules  =  new  List<LifecycleRule>          {                  new  LifecycleRule                  {                            Id  =  "Archive  and  delete  rule",                            Prefix  =  "projectdocs/",                            Status  =  LifecycleRuleStatus.Enabled,                              Transition  =  new  LifecycleTransition()                              {                                        Days  =  365,                                        StorageClass  =  S3StorageClass.Glacier                              },                              Expiration  =  new  LifecycleRuleExpiration()                              {                                        Days  =  3650                              }                  }          }      }; .NetConfiguring Lifecycle Rules Delete object after 10 years
  113. 113. Restoring from Amazon Glacier AWS Console
  114. 114. Restoring from Amazon Glacier AWS Console
  115. 115. Restoring from Amazon Glacier AWS Console
  116. 116. Restoring from Amazon Glacier AWS Console
  117. 117. Restoring from Amazon Glacier AWS Console
  118. 118. WEBSITE HOSTING
  119. 119. Static Website Hosting with Amazon S3 You can host your entire static website on Amazon S3 for a low-cost, highly available hosting solution that can scale automatically to meet traffic demands With Amazon S3, you can reliably serve your traffic and handle unexpected peaks without worrying about scaling your infrastructure docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html
  120. 120. Static Website Hosting Bucket Properties
  121. 121. Setting Default Documents
  122. 122. Redirecting Requests
  123. 123. {      "Version":"2008-­‐10-­‐17",      "Statement":[{     "Sid":"PublicReadGetObject",                  "Effect":"Allow",        "Principal":  {                          "AWS":  "*"                    },              "Action":["s3:GetObject"],              "Resource":["arn:aws:s3:::example-­‐bucket/*"              ]          }      ]   }   Bucket Policy
  124. 124. {bucket-­‐name}.s3-­‐website-­‐{region}.amazonaws.com   e.g.  mybucket.s3-­‐website-­‐eu-­‐west-­‐1.amazonaws.com Website Addressing s3-­‐{region}.amazonaws.com/{bucket-­‐name}/{object-­‐key}   e.g.  s3-­‐eu-­‐west-­‐1.amazonaws.com/mybucket/img.png   {bucket-­‐name}.s3-­‐{region}.amazonaws.com/{object-­‐key}   e.g.  mybucket.s3-­‐eu-­‐west-­‐1.amazonaws.com/img.png   Normal Addressing
  125. 125. DNS Record set for: aws-­‐exampl.es Route 53
  126. 126. DNS Record set for: aws-­‐exampl.es Route 53 bucket bucket Website bucket name: Website bucket name: www.aws-­‐exampl.es aws-­‐exampl.es Error .html Index .html
  127. 127. DNS Record set for: aws-­‐exampl.es Route 53 bucket bucket Website bucket name: Website bucket name: www.aws-­‐exampl.es aws-­‐exampl.es Error .html Index .html Website redirect to: aws-­‐exampl.es
  128. 128. DNS Record set for: aws-­‐exampl.es Route 53 bucket Website bucket name: Website bucket name: www.aws-­‐exampl.es aws-­‐exampl.es Error .html Index .html Website redirect to: aws-­‐exampl.es A Record ‘Alias’ to S3 website: aws-­‐exampl.es  @   s3website-­‐eu-­‐west1-­‐amazonaws.com   bucket
  129. 129. DNS Record set for: aws-­‐exampl.es Route 53 bucket Website bucket name: Website bucket name: www.aws-­‐exampl.es aws-­‐exampl.es Error .html Index .html Website redirect to: aws-­‐exampl.es A Record ‘Alias’ to S3 website: aws-­‐exampl.es  @   s3website-­‐eu-­‐west1-­‐amazonaws.com   bucketbucket CNAME for www. to: www.aws-­‐exampl.es.s3-­‐website-­‐ eu-­‐west-­‐1.amazonaws.com  
  130. 130. docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html
  131. 131. SUMMARY
  132. 132. S3 provides developers with secure, durable & highly scalable object storage1 S3 can be used alone with other AWS services or 3rd party tools & services2 Cost effective for a wide variety of use-cases from cloud applications, content distribution, backup, archiving & disaster recovery to analytics 3
  133. 133. THINGS WE DIDN’T COVER
  134. 134. Amazon CloudFront aws.amazon.com/cloudfront
  135. 135. S3 EVENT NOTIFICATIONS http://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
  136. 136. AWS Lambda S3 event notifications AutomaticCloud Functions aws.amazon.com/lambda
  137. 137. RESOURCES YOU CAN USE TO LEARN MORE
  138. 138. aws.amazon.com/s3
  139. 139. Getting Started with Amazon S3: docs.aws.amazon.com/AmazonS3/latest/gsg/GetStartedWithS3.html Amazon S3 Deep Dive & Best Practices session from AWS re:Invent 2014 https://youtu.be/2DpOS0zu8O0 Amazon S3 Documentation: aws.amazon.com/documentation/s3/
  140. 140. Certification aws.amazon.com/certification Self-Paced Labs aws.amazon.com/training/
 self-paced-labs Try products, gain new skills, and get hands-on practice working with AWS technologies aws.amazon.com/training Training Validate your proven skills and expertise with the AWS platform Build technical expertise to design and operate scalable, efficient applications on AWS AWS Training & Certification
  141. 141. Follow us for m ore events & w ebinars @AWScloud for Global AWS News & Announcements @AWS_UKI for local AWS events & news @IanMmmm Ian Massingham — Technical Evangelist

×