How do I provision infrastructure and applications, manage systems, and operate and monitor a Hybrid Cloud on AWS is one of the first questions I get from enterprise customers as they start their cloud adoption journey. This presentations covers the tools, technologies, and AWS Services that can be used to manage, operate, and monitor a hybrid cloud. It also covers CI/CD in a hybrid cloud environment.
Good morning, good afternoon, evening.
Today we are discussing hybrid cloud customer use cases and also cover AWS landing zone and hybrid cloud landing zones as well as a couple of AWS services that are new and help you configure and run a hybrid cloud environment.
Assumes knowledge of cloud and basics of AWS
Tom Laszewski NA enterprise architecture leader.
We have come a long way by listening to our customs. When I joined 6 years ago you cloud not say hybrid..then hybrid architecture…now hybrid cloud. Went from 16 services to over 130 services
Let’s go…
Level 300 | Solutions Best Practices
Operating in a hybrid architecture is a step in the cloud adoption journey for many organizations that have on-premises technology investments. Migrating legacy IT systems takes time, and can be disruptive to current processes, organizational structure, and culture. AWS has developed a broad set of hybrid cloud capabilities across storage, networking, security, application deployment, and management tools to help you build and operate a secure, performant, reliable, and scalable hybrid cloud. Join this tech talk to learn how customers are leveraging AWS hybrid cloud capabilities for cloud bursting and integrating devices and edge systems. The webinar will start with a review of customer success stories for datacenter capacity extension, delivery of new services and applications, and ensuring business continuity and disaster recovery, as well as covering the configuration of a hybrid cloud landing zone. Missed part one? Watch it on-demand.
Learning Objectives: • Hear about customer AWS Hybrid Cloud success stories• Learn the best practices of how customers are building hybrid cloud landing zones• Learn the best practices of hybrid cloud for cloud bursting, and integrated devices and edge systems
Who Should Attend: Technical Decision Makers, IT Architects, Cloud Architects, Application DevelopersSpeaker(s): Tom Laszewski, Enterprise Technologist, AWS
On premise storage integration with AWS data storage services.
Business continuity with hot standby on AWS
DR as a Service with VMWare Cloud on AWS
On premise storage integration with AWS data storage services.
Business continuity with hot standby on AWS
DR as a Service with VMWare Cloud on AWS
Networking is foundational to all hybrid cloud use cases.
1. Amazon Virtual Privat Cloud (VPC) lets you provision a logically isolated section of the AWS Cloud where you can launch AWS resources. Can contain public subenets (accessible from internet) and private subnets (accessible from within AWS or through a VPN).
2. DirectConnect, is a Private connection, Separate from the Internet that provides Port speeds of 1 Gbps, 10 Gbps or sub-1 Gbps. If you have bandwidth-heavy workloads that you wish to run in AWS, AWS Direct Connect can reduce your network costs into and out of AWS.
3. VPN - IPsec authentication and encryption through IPSec or SSL through third parties
Three options :AWS Managed VPN, Software VPN (EC2) – Cisco CSR on marketplace, openswan, openvpn
Amazon VPC
Amazon Virtual Private Cloud (Amazon VPC) lets you provision a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define. You have complete control over your virtual networking environment, including selection of your own IP address range, creation of subnets, and configuration of route tables and network gateways. Additionally, you can create a Hardware VPN connection between your corporate data center and your VPC to leverage the AWS Cloud as an extension of your corporate datacenter.
Learn more »
AWS Direct Connect
AWS Direct Connect makes it easy to establish a dedicated network connection from your premises to AWS. Using AWS Direct Connect, you can establish private connectivity between AWS and your data center, office, or colocation environment, which in many cases can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections. This dedicated connection can be partitioned into multiple virtual interfaces to maintain network separation between public and private environments.
Learn more »
Integrated Networking
The next layer of hybrid architecture involves connecting on-premises and cloud resources through a common network to facilitate the creation of a single enterprise environment. AWS can extend your on-premises network configuration into your virtual private networks on the AWS Cloud so that AWS resources operate as if they are part of your existing corporate network. You can also extend your physical connectivity to provide dedicated, consistent, private networking between your data centers and the AWS regions of your choice.
4. IAM - Users for authentation using password plus MFA accessing from AWS console, or have access keys when using the AWS APIs. Groups to combine ’like’ users – developers, finance, operators etc.
5. AWS SSO - AWS Single Sign-On (SSO) is a cloud SSO service that makes it easy to centrally manage SSO access to multiple AWS accounts and business applications., you can create Security Assertion Markup Language (SAML) 2.0 integrations to third party apps.
6. AWS Microsoft AD Connector - Active .
Directory Connector gives you an easy way to establish a trusted relationship between your Active Directory and AWS. You continue to run MS AD on-premises.
7. AWS Directory Service for Microsoft Active Directory, also known as AWS Microsoft AD, enables your directory-aware workloads and AWS resources to use managed Active Directory in the AWS Cloud. AWS Microsoft AD is built on actual Microsoft Active Directory and does not require you to synchronize
----------------------------------
Extra notes…..
Integrated Identity and Access
Establishing a single identity and access strategy often goes hand-in-hand with integrating networks. You can create and manage AWS users, groups, and permissions to allow and deny access to AWS resources at extremely fine level of detail. Additionally, AWS offers managed services that allow you to connect your AWS resources with an existing on-premises Microsoft Active Directory and manage policies with existing tools.
Public Internet – public IPs assigned to compute by AWS or Elastic Ips that are generated by AWS can be moved to different VMs
AWS using Customer gateway and virtual private gateway or Software managed using OpenVPN, Cisco CSR on AWS Marketplace (prepacked AWS Machine Image)
Private connection through 67 locations offering speeds of up to 10Gbps, does not use internet, consistent performance. Can be a lower overall cost because of low data transfer out costs.
Describe the services -
AWS Storage Gateway – NFS, ISCSI, SMB. Hybrid Storage Integration, on premises Virtual gateway appliance that can be utilize for backup and restore, pilot light, standby DR, or active/active. AWS. VTL support as well. VM import/export to create Windows, Vmware or Citrix Xe
Amazon S3 – Scalable Storage in the Cloud, as indicated used to store files, EBS snapshots which can be restore storage on AWS and attach to EC2 volumes on AWS.
Amazon Glacier – Low-cost archive storage in the cloud. Used to archive on-premises data on AWS much like tapes.
Amazon EBS Snapshots - Protect your data by creating point-in-time snapshots of EBS volumes, which are backed up to Amazon S3 for long-term durability. Amazon Machine Images stored in S3 that can be instantiated as EC2 instances.
Snowball - Snowball is a petabyte-scale data transport solution that uses devices designed to be secure to transfer large amounts of data into and out of the AWS Cloud.
Amazon RDS (relational databse service) – Run a DR Aurora, PostgreSQL, MySQL, MariaDB, Oracle, and Microsoft SQL Server. database in the cloud.
Route 53 and ELB are used for hot standby (active – active)
Amazon Route53 – Scalable Domain Name System for routing traffic between AWS and on premises.
Elastic Load Balancer – High Scale Load Balancing
Use route53 DNS failover with DNS weighting to failover to a hot standby site on AWS, the failover will occur using health checks on the load balancer and reverse proxy.
--------------------------more material -------------------------------
AWS Storage Gateway
The AWS Storage Gateway service seamlessly enables hybrid cloud storage between on-premises environments and the AWS Cloud. It combines a multi-protocol storage appliance with highly efficient network connectivity to deliver local performance with virtually unlimited scale.
Customers use it in remote offices and datacenters for hybrid cloud workloads involving migration, bursting and storage tiering. The Storage Gateway virtual appliance connects directly to your local infrastructure as a file server, as a local disk volume, or as a virtual tape library (VTL). This seamless connection makes it simple for organizations to augment existing on-premises storage investments with the high scalability, extreme durability and low cost of AWS cloud storage.
Integrated resources and deployment management is all about DevOps and management tools.
1. Systems Manager is a service to help manage your Amazon EC2 and on-premises instances to automatically apply patches, updates, and configuration changes across any resource group acrpss cloud and aws.
2. AWS OpsWorks is a configuration management service that helps you configure and operate applications, both on-premises and in the AWS Cloud, using AWS managed Chef or Puppet.
3. Amazon Elastic Container Service for Kubernetes (Amazon EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to install and operate your own Kubernetes clusters. Kuberneters is a popular open source, on-premises microservices open source orchestration system.
4. AWS CodeDeploy .
AWS CodeDeploy automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises.
5. Vmware cloud on AWS for management seamless integration - fully managed VMware environment on the AWS Cloud that can be accessed on an hourly, on-demand basis or by subscription. It allows you to continue to leverage your investments in VMware without continuing to buy and maintain hardware
-------------------------more information-------------------
SSM
operational data for monitoring and troubleshooting, and take action on your groups of resources to shorten time to detect problems.
automatically apply patches, updates, and configuration changes across any resource group. This ensure consistent configurations of firewall policies, anti-virus definitions, logging software across your fleet of compute
Using the EC2 run command no need to SSH into servers to apply patches and reduces security blast radius by reducing need to SSH into instances.
https://www.youtube.com/watch?v=zwS8lssaY_k
Amazon EC2 Run Command
Amazon EC2 Run Command lets you remotely and securely manage servers or virtual machines running in your data center or on a cloud platform. Amazon EC2 Run Command provides a simple way of automating common administrative tasks such as executing Shell scripts and commands on Linux, running PowerShell commands on Windows, installing software or patches across multiple instances and provides visibility into the results, making it easy to manage configuration change across large fleets of instances.
Capabilities:
Automation
Inventory
Maintenance windows
Parameter store
Patch management
State management
Run command
AWS OpsWorks helps you automate operational tasks like code deployment, software configurations, package installations, and database setups on any server including existing EC2 instances or servers running in your own data center. You can use a single application management service to deploy and operate applications across your hybrid architecture.
Supports any application
Configuration as code
Automation to run at scale
Resource organization
Supports any server
2. AWS OpsWorks supports a wide variety of architectures, from simple web applications to highly complex custom applications, and any software that has a scripted installation. Since AWS OpsWorks supports Chef recipes and Bash scripts, you can leverage community-built configurations such as MongoDB and Elasticsearch. You start by modeling and visualizing your application with layers that define resource and software configuration. You control every aspect of your application's configuration to match your needs, processes, and tools. You can extend and adapt the built-in layers or create your own.
AWS OpsWorks
AWS OpsWorks is a configuration management service that helps you configure and operate applications, both on-premises and in the AWS Cloud, of all shapes and sizes using Chef. You can define the application’s architecture and the specification of each component including package installation, software configuration, and resources such as storage. Start from templates for common technologies like application servers and databases or build your own to perform any task that can be scripted. AWS OpsWorks includes automation to scale your application based on time or load and dynamic configuration to orchestrate changes as your environment scales.
3. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. Popular in enterprise data centers as companies move to microservices - loosely coupled services, which implement business capabilities in small pieces of code/services
2. Kubernetes gives you the orchestration and management capabilities required to deploy containers, at scale, for these workloads.
4. AWS CodeDeploy
AWS CodeDeploy automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. You can use AWS CodeDeploy to automate software deployments, eliminating the need for error-prone manual operations, and the service scales with your infrastructure so you can easily deploy to one instance or thousands.
In order to assist with running your workloads on aws you can utilize….
1. AWS CloudFormation to allows you to model your entire infrastructure in a text file – Infrastrucutre as Code). This template becomes the single source of truth for your infrastructure – your virtual data center in a box (well, actually a JSON or YAML)
2. Amazon CloudWatch – To monitor services for running on AWS resources
3. AWS CloudTrail enables governance, compliance, operational auditing, and risk auditing of your AWS account.
Now that we are familiar with the use cases and are knowledgable about the AWS services related to these uses let’s dive deep into some customer success stories. I specifically used customer successes that have youtube videos, are on slide share, or public case study and white papers so you can find more information after this session.
Integrated resources and deployment management is all about DevOps and management tools.
1. Systems Manager is a service to help manage your Amazon EC2 and on-premises instances to automatically apply patches, updates, and configuration changes across any resource group acrpss cloud and aws.
2. AWS OpsWorks is a configuration management service that helps you configure and operate applications, both on-premises and in the AWS Cloud, using AWS managed Chef or Puppet.
3. Amazon Elastic Container Service for Kubernetes (Amazon EKS) is a managed service that makes it easy for you to run Kubernetes on AWS without needing to install and operate your own Kubernetes clusters. Kuberneters is a popular open source, on-premises microservices open source orchestration system.
4. AWS CodeDeploy .
AWS CodeDeploy automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises.
5. Vmware cloud on AWS for management seamless integration - fully managed VMware environment on the AWS Cloud that can be accessed on an hourly, on-demand basis or by subscription. It allows you to continue to leverage your investments in VMware without continuing to buy and maintain hardware
-------------------------more information-------------------
SSM
operational data for monitoring and troubleshooting, and take action on your groups of resources to shorten time to detect problems.
automatically apply patches, updates, and configuration changes across any resource group. This ensure consistent configurations of firewall policies, anti-virus definitions, logging software across your fleet of compute
Using the EC2 run command no need to SSH into servers to apply patches and reduces security blast radius by reducing need to SSH into instances.
https://www.youtube.com/watch?v=zwS8lssaY_k
Amazon EC2 Run Command
Amazon EC2 Run Command lets you remotely and securely manage servers or virtual machines running in your data center or on a cloud platform. Amazon EC2 Run Command provides a simple way of automating common administrative tasks such as executing Shell scripts and commands on Linux, running PowerShell commands on Windows, installing software or patches across multiple instances and provides visibility into the results, making it easy to manage configuration change across large fleets of instances.
Capabilities:
Automation
Inventory
Maintenance windows
Parameter store
Patch management
State management
Run command
AWS OpsWorks helps you automate operational tasks like code deployment, software configurations, package installations, and database setups on any server including existing EC2 instances or servers running in your own data center. You can use a single application management service to deploy and operate applications across your hybrid architecture.
Supports any application
Configuration as code
Automation to run at scale
Resource organization
Supports any server
2. AWS OpsWorks supports a wide variety of architectures, from simple web applications to highly complex custom applications, and any software that has a scripted installation. Since AWS OpsWorks supports Chef recipes and Bash scripts, you can leverage community-built configurations such as MongoDB and Elasticsearch. You start by modeling and visualizing your application with layers that define resource and software configuration. You control every aspect of your application's configuration to match your needs, processes, and tools. You can extend and adapt the built-in layers or create your own.
AWS OpsWorks
AWS OpsWorks is a configuration management service that helps you configure and operate applications, both on-premises and in the AWS Cloud, of all shapes and sizes using Chef. You can define the application’s architecture and the specification of each component including package installation, software configuration, and resources such as storage. Start from templates for common technologies like application servers and databases or build your own to perform any task that can be scripted. AWS OpsWorks includes automation to scale your application based on time or load and dynamic configuration to orchestrate changes as your environment scales.
3. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. Popular in enterprise data centers as companies move to microservices - loosely coupled services, which implement business capabilities in small pieces of code/services
2. Kubernetes gives you the orchestration and management capabilities required to deploy containers, at scale, for these workloads.
4. AWS CodeDeploy
AWS CodeDeploy automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. You can use AWS CodeDeploy to automate software deployments, eliminating the need for error-prone manual operations, and the service scales with your infrastructure so you can easily deploy to one instance or thousands.
In order to assist with running your workloads on aws you can utilize….
1. AWS CloudFormation to allows you to model your entire infrastructure in a text file – Infrastrucutre as Code). This template becomes the single source of truth for your infrastructure – your virtual data center in a box (well, actually a JSON or YAML)
2. Amazon CloudWatch – To monitor services for running on AWS resources
3. AWS CloudTrail enables governance, compliance, operational auditing, and risk auditing of your AWS account.
Now that we are familiar with the use cases and are knowledgable about the AWS services related to these uses let’s dive deep into some customer success stories. I specifically used customer successes that have youtube videos, are on slide share, or public case study and white papers so you can find more information after this session.
On premise storage integration with AWS data storage services.
Business continuity with hot standby on AWS
DR as a Service with VMWare Cloud on AWS
Application platforms, such as the Red Hat OpenShift Container Platform and the Pivotal Cloud Foundry, help accelerate the adoption of cloud native principles which will accelerate application deployment, enable faster application iteration and provide a unified experience. These platforms simplify the deploying of application code, automating software release processes, and monitoring your application and infrastructure performance. Additionally, application platforms provide you with an out of the box cloud native experience across multiple environments. Leveraging these technologies, you can extend your applications into the cloud without significantly changing your existing application code base or development procedures and avoid costly refactoring processes.
On premise storage integration with AWS data storage services.
Business continuity with hot standby on AWS
DR as a Service with VMWare Cloud on AWS
Application platforms, such as the Red Hat OpenShift Container Platform and the Pivotal Cloud Foundry, help accelerate the adoption of cloud native principles which will accelerate application deployment, enable faster application iteration and provide a unified experience. These platforms simplify the deploying of application code, automating software release processes, and monitoring your application and infrastructure performance. Additionally, application platforms provide you with an out of the box cloud native experience across multiple environments. Leveraging these technologies, you can extend your applications into the cloud without significantly changing your existing application code base or development procedures and avoid costly refactoring processes.
AWS Service Broker is an implementation of the Open Service Broker API. On the Red Hat OpenShift platform, the Kubernetes Service Catalog provides an intermediate layer that allows users to deploy services using native manifests and the OpenShift graphical UI.
AWS Service Broker supports a subset of AWS services, including Amazon Relational Database Service (Amazon RDS), Amazon EMR, Amazon DynamoDB, Amazon Simple Storage Service (Amazon S3), and Amazon Simple Queue Service (Amazon SQS); for a full list, see the AWS Service Broker documentation. The broker includes AWS CloudFormation templates that manage infrastructure, resources, and build logic. These templates contain both prescriptive and customizable parameter sets that provide best-practice implementations for production, test, and development environments. Applications can consume or interact with these resources by using a set of values such as endpoints and credentials. Binding allows developers to create microservices that consume AWS services without knowledge or insight into the underlying resources.
On premise storage integration with AWS data storage services.
Business continuity with hot standby on AWS
DR as a Service with VMWare Cloud on AWS
automate maintenance and deployment tasks on Amazon EC2 and on-premises instances
applications spanning AWS and onpremise
DevOps spanning AWS and on premise
Integrated Resource and Deployment Management
The most robust form of hybrid architecture involves integrating application deployment and management across on-premises and cloud environments. AWS and VMware have developed a deep, unique relationship to enable VMware-based workloads to be run on the AWS Cloud.
Additionally, all AWS services are driven by robust APIs that allow for a wide variety of monitoring and management tools to integrate easily with your AWS Cloud resources. Common tools from vendors such as Microsoft, VMware, BMC Software, Okta, RightScale, Eucalyptus, CA, Xceedium, Symantec, Racemi, and Dell already support AWS, and that’s just naming a few.
Ancestry, founded in 1983, is a family-history company with petabytes of historical records. They use DNA analysis to identify the people who share your DNA . Ancestry is currently moving all in on AWS, but had challenges with their current compute deployment management platform.
Prior to AWS, Ancestry was manually racking and stacking hardware, and using Ansible, Bash, and Python scripts to provision VMs with no error handling in place, and it was a slow and error-prone processes. With limited automation, It required someone to be up at 3 a.m. for an emergency. The limitations of user data made it not an option for Ancestry…
Systems Manager (SSM) became the answer as it…
automates Enterprise IT operations safely and securely
integrates with AWS services such as IAM, CloudTrail, CloudWatch Events, and AWS Config to provide automation and visibility
SSM is SOC and HIPAA certified
------END
DEV306_Embrace DevOps and Learn How to Automate Operations_NoNotes.pptx
https://www.youtube.com/watch?v=vS8cuSLXNi4
SSM is used to boot strap all windows. Linux instances that need to join a domain are booted with SSM., AMI patching and creation monthly, auditing patch levels by the DevOps teams when required – self service, patching existing servers on AWS,
Running commands against an instance so IT personnel don’t need to log directly into a server or stand up a Bastion host. No more need for administrates to access machines directly with the EC2 run command.
The results are:
Provisioning servers went from 2-3 days to 30-45 minutes
Automation reliability went from 60% of time to 95% of time which means less human intervention which means less change of error
Patch can not only be done at an instance by instance level or at a patch-group level which is condusive to better fleet management/consistency across all servers.
Self service - More flexibility at the line of business to make configuration changes instead of waiting for Operations team to make centralized script changes and deploy these to AMIs or individual servers.
We saw customers also experienced the same challenges of deployment automation
So we decided to make Apollo available to external customers through CodeDeploy