Reimagining Data Management in the Cloud at NASA and CMS

P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Reimagine the Public Cloud
Experience with AWS
Governance@Scale
Mark McInerney
Deputy Project Manager,
Earth Science Data &
Information System
NASA
3 1 9 0 0 5
Brett Miller
CMS Senior Solutions Architect &
Technical Program Manager
AWS
Brian Price
President & CEO
cloudtamer.io

S U M M I T
Agenda
Achieving the common goals of unique missions
Reimagining data in the cloud at NASA
Reimagining user onboarding and orchestration in the
cloud at Centers for Medicare & Medicaid Services (CMS)
AWS Governance@Scale 101
Automating Governance@Scale

S U M M I T

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
5
There’s more to cloud
transformation than this…
S U M M I T

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6
Successful cloud
transformation
looks like this…
S U M M I T

S U M M I T
The typical AWS adoption reality
Amazon Simple
Storage Service
(Amazon
S3)
Project 1 AWS Account
Amazon
Elastic
Compute
Cloud
(Amazon EC2)
Amazon
S3
Amazon EC2 Amazon Relational
Database Service
(Amazon RDS)
Stage 1
Specific Systems
Limited Accounts
Minimal Services
Stage 2
Numerous Systems
Multiple Accounts
Many Services
Amazon
S3
Amazon
EC2
Amazon Virtual
Private Cloud
(Amazon VPC)
Amazon
S3
Amazon
EC2
Amazon
VPC
Amazon
EMR
Amazon
Kinesis
Amazon
Redshift
Amazon
S3
Project 4 AWS
Account
Amazon
EC2
Project 5 AWS
Account
Amazon API
Gateway
Amazon Simple
Queue Service
(Amazon SQS)
Amazon
WorkSpaces
Amazon Elastic
Container Service
(Amazon ECS)
AWS Elastic
BeanstalkAmazon
S3
Amazon
EC2
Amazon
EMR

S U M M I T
Common governance challenges
Organizational
Support
Business Unit
Autonomy
Multi-Account
Strategy
Integration
Delegation

S U M M I T
Goals for NASA and CMS
• Accelerate cloud adoption
• Deliver a controlled, but flexible, cloud environment to staff
• Make it easier to migrate applications and cost effectively deploy new
solutions with greater agility than ever before

S U M M I T
Earth Science Data Collections of the Earth Observing
System Data and Information System (EOSDIS)
11NASA Earthdata Cloud (EDC) – http://earthdata.nasa.gov
• Land
• Cover & Usage
• Surface temperature
• Soil moisture
• Surface topography
• Ocean Dynamics
• Surface temperature
• Surface wind fields & Heat
flux
• Surface topography
• Ocean color
NASA Operating Missions
• Airborne
• International Space Station
• Field Campaigns
• International Partners
March 2017
• Atmosphere
• Winds & Precipitation
• Aerosols & Clouds
• Temperature & Humidity
• Solar radiation
• Cryosphere
• Sea/Land Ice & Snow Cover
• Human Dimensions
• socioeconomic data (e.g.,
population, infrastructure, ..)

S U M M I T
EOSDIS Organization
12
NASA Earthdata Cloud (EDC) – http://earthdata.nasa.gov

S U M M I T
NASA’s EOSDIS provides end-
to-end capabilities for
managing NASA’s Earth
science data from satellites,
aircraft, field measurements,
and various other programs.
EOSDIS is responsible for a
data collection that is large in
volume and projected to grow
rapidly over the next several
years.
High-Level EOSDIS End-to-End
EOSDIS
Applications
capture
and
clean
data
downlink
Education
process
archive
subset
distribute
Research
Users

S U M M I T
80 TBs/day
generation
400 TBs/day
reprocessing
300 GB
Granules
150 PBs @ 50 Gbps
processing speed for months
NISAR Mission
NASA-ISRO Synthetic Aperture Radar (NISAR)

S U M M I T
EOSDIS Data System Evolution
Growth of Mission Data & Processing: Projected
rapid archive growth and the need to effectively
process significantly larger volumes of new mission
data requires rethinking existing architectures.
Data Systems: More cost-effective, flexible, and
scalable data system ingest, archive, and
distribution solutions are needed to keep pace with
new mission advancement.
Science Users: Significantly larger data volumes
requires additional ways to access and utilize this
data, with “Data Close to Compute” or Data Lake”.
15
Motivationfor Cloud Projected Data Volumes

S U M M I T
Earthdata Cloud (EDC)
● “Managed” commercial cloud for EOSDIS on AWS
● Improves the efficiency of NASA’s data systems operations –
maintaining free/open data policy
● Designed for EOSDIS applications and mission data ingest, archive,
distribution
● Increase opportunity for researchers and commercial users to
access/process petabytes of data quickly without the need for data
management

S U M M I T
07
Community
Development
Model
06
Metrics
Management
05
Budget
Controls
04
Organizational
Governance
03
Data Services
& analytics
02
Mission Data
Management
01
Cloud Platform
Infrastructure
Components of the Earthdata Cloud

S U M M I T
Cloud Platform
Infrastructure
01
Project Level Components and Core Elements
User Access /
Support
Services
Networking
Governance of
Cloud
Accounts
Common
Services and
Controls
07
Community
Development
Model
06
Metrics
Management
05
Budget
Controls
04
Organizational
Governance
18
Certification
and
Accreditation
Strategy
Security
Services

S U M M I T
Cloud Platform
Infrastructure
01
Project Level Components and Core Elements
User Access /
Support
Services
Networking
Governance of
Cloud
Accounts
Common
Services and
Controls
07
Community
Development
Model
06
Metrics
Management
05
Budget
Controls
04
Organizational
Governance
19
Certification
and
Accreditation
Strategy
Security
Services
05
Budget
Controls

S U M M I T
Earthdata Cloud Platform Infrastructure
Maximizes Flexibility
Provide projects the freedom
to implement solutions that
fit their problem domains
Maximizes Autonomy
Be a platform, not a gate; foster
experimentation/innovation and
support production needs of
application owners
Shared Services & Controls
Platform manages common
shared services & controls to
reduce duplication, system
complexity, and cost across
EOSDIS
Cloud Platform
Infrastructure
Common Services
and Controls
A multi-account, Infrastructure-as-a-Service (IaaS) cloud platform operating on AWS under
a single top-level “payer account”, providing shared cloud services and controls to EOSDIS.

S U M M I T
Common Services and Controls
Components of the EOSDIS Earthdata Cloud (EDC)
21
1. NASA-Approved Amazon Services - vetted AWS and
third-party SAAS services and process to add new. Focus is
on using AWS cloud-native services
2. Code Deployment Services - DevOps CICD Pipeline to
security scan code, build, and deploy into EDC
3. Use of Infrastructure as Code including re-useable
template to define a multi-account ecosystem
4. cloudtamer.io Identity and Access Management to:
• Rotate AWS access keys
• Apply session limits
• Provide role-based access control
Cloud Platform
Infrastructure
Common Services
and Controls

S U M M I T
User Access
Cloud Platform
Infrastructure
Governance of
Cloud Accounts
22
1. User Authentication
• Grant uniform access and experience to end users from
multiple identity management systems (NAMS (SAML) /
Internal directory / Active Directory)
• Enforce and validate minimum authentication levels via
two-factor authentication
• Ensure that users only have access commensurate to the
authentication type with cloudtamer.io
• Secure PIV/Token login, NASA NAMS account provisioning
2. User Authorization
• Control who views finances, who accesses AWS resources,
and who manages finance
• cloudtamer.io manages AWS Identity and Access
Management (IAM) roles and policies at an organizational
level

S U M M I T
Account Structure
23
• Single Payer Account using AWS
Organization and Consolidated billing
• Multi-Account structure divided into
NASA / Mission-defined organizational
units
• Isolation based on organizational units
• Isolation based on application development,
test, and production accounts
• Isolation of Management and Security
accounts from end user environment
• Track AWS expenses to NASA organizations
and funding sources
Cloud Platform
Infrastructure
Governance of
Cloud Accounts

S U M M I T
The Antideficiency Act (ADA) and Pay-as-You-Go
4Points
The ADA act prohibits federal agencies from obligations or expending federal funds in
advance or in excess of an appropriation, and from accepting voluntary services.
Federal employees who violate the Antideficiency Act are subject to two types of
sanctions: administrative and penal. Employees may be subject to appropriate
administrative discipline including, when circumstances warrant, suspension from duty
without pay or removal from office. In addition, employees may also be subject to fines,
imprisonment, or both.
Budget
Controls
24

S U M M I T
The cloudtamer.io account-level view
• Enforces individual AWS account-level budget
through “budget caps”
• Provides account-alert spend monitoring and
budget control actions
• Allows for flexible access levels:
• Top-level view for ESDIS management &
business teams
• Account view for APP-Admins & Developers
as needed
Budget
Controls

S U M M I T
Pre-Production
Near-term Application and Data Onboarding
26
Global Hydrology Resource Center (GHRC) mission
data into EDC production end of 2019
GHRC *
Alaska Satellite Facility (ASF) NASA-ISRO
Synthetic Aperture Radar (NISAR) mission
data into EDC production early 2022 (Dec
2021 launch)
NISAR *
NASA’s Goddard Earth Science Data
and Information Services Center
(GES DISC) Giovani visualization &
analysis tool in EDC to support ESD
analytics capabilities, pre-production
environment 2019
Giovani +
Alaska Satellite Facility (ASF)
Spaceborne Imaging Radar-C
(SIR-C) mission data into EDC
production early 2020
SIR-C *
ESDIS Global Imagery Browse Service in The Cloud
(GITC), NGAP2.0 pre-production environment
GITC +
Physical Oceanography (PO) Data Active
Archive Center (DAAC) Surface Water
Ocean Topography (SWOT) mission data
into EDC production 2022
SWOT *
Sentinel-1
ASF Sentinel-1 mission in production on Earthdata
Cloud (NGAP2.0)
Common Metadata Repository
ESDIS / EED Common Metadata Repository (CMR) in
production on Earthdata Cloud (NGAP1.0)
Earthdata Search Client
ESDIS / EED Earthdata Search Client (EDSC) in production on
Earthdata Cloud (NGAP1.0).
1
2
3
Production
GHRC
SIR-C
* Mission Data
+ Application
6 key projects in
queue for
production into the
Earthdata Cloud
NISAR
SWOT
GITC
Giovani
ESDIS/DAAC dataset
prioritization activity
underway to determine
mission data onboarding
Oct 1, 2019
NASAs Earthdata Cloud (EDC) – http://earthdata.nasa.gov

S U M M I T
Cloud computing at
the Centers for
Medicare & Medicaid
Services
70+ applications running in AWS
Shared Services providing a
variety of security and
operational capabilities
including VPN, Active Directory,
Gold Image AMIs, Log
Aggregation and Analytics
28
hush-naidoo-382152-unsplash

S U M M I T
Cloud computing objectives at CMS
• Reusability: productize cloud computing resources and services at CMS
• Easy-to-adopt: clear onboarding, education, and training for teams
• Developer-first: build modular components and support dev
community
• Cost-effective: reduce operating cost and create a governance structure
to support cost transparency and better business owner education
• Adaptable: build on modern software technology stack, leveraging
infrastructure as code
29

S U M M I T
With growth, comes obstacles…
● Efficiency in bringing in new users and applications in the cloud
● Flexible services for both the novice and advanced cloud user
● Cost and onboarding transparency
● Growing complexity to meet security and compliance needs and obtain
ATOs
30

S U M M I T
Taking the next steps…CMS AWS West Project
• An opportunity to build out a “greenfield” cloud environment for the
future without disruption to ongoing operations in the Legacy CMS
AWS East or the constraints of the current cloud environment
• Align with leadership’s vision for a modern cloud infrastructure and
suite of services, structured as 9 workstreams collaborating and
coordinating to get work completed
31

S U M M I T
Guiding principles
• Iterative forward progress – deliver a Minimum Viable Product (MVP) with a “greenfield”
approach over two 30/60/90 day MVP cycles
• Early stakeholder engagement – USDS serving as Product Owner(s), IUSG/OIT/CMS as System
Owner(s), and WNMG/OC/CMS as Business Owner(s)
• Full transparency – working in the open using ALM tools such as Jira, Confluence, Hipchat, and
GitHub
• Tackle big risks early – agile delivery to deliver units of value quickly, think about onboarding
and team education and satisfaction from the beginning
• Approach with humility – this is a complex effort, solicit feedback early and often, apply
lessons learned from previous cloud migration and operations; build on previous knowledge and
efforts
• Great user experience – through formal and self-serve training, comprehensive agile
onboarding, and improved governance and cost model
32

S U M M I T
AWS Governance@Scale framework
• Provides customers with the
blueprint for enterprise cloud
success
• We drink our own champagne:
these best practices were first
adopted by AWS to manage
thousands of cloud accounts

S U M M I T
Three principles of Governance@Scale
Account management
• Align AWS accounts with the organization through a common interface. Standardize
and streamline provisioning, maintenance, and access control policies for many AWS
accounts and workloads
Cost enforcement
• Ensure AWS accounts and workloads do not exceed budget
Compliance automation
• Accelerate security authorizations, provide continuous monitoring and configuration
management, and enforce security controls

S U M M I T
Embracing the AWS Governance@Scale framework
• Account automation
• Budget planning
• Identity & access automation
• Security automation
36
• Budget enforcement
• Policy enforcement
• Identity federation
• Identity & access automation

S U M M I T
• Enforces fiscal and compliance policies
with more than notifications
• Provides native access to cloud
capabilities. It’s not a cloud broker
• Easy access for technical staff to create
the resources they need
• Easy for senior leadership to enact
financial and compliance oversight as
adoption scales
cloudtamer.io enables organizations to manage
their cloud presence at scale

S U M M I T
Cloud@Scale: Account Management
Company X
Dept. A
Project 1
Project 2
Project 3
Dept. B
Project 4
Project 5
Dept. C
Project 6
• Centralized management of all cloud accounts
• Federated single sign-on and 2-factor authentication (MFA)
• Automated, self-service account creation with native
Console, CLI, and API access
Account Management

S U M M I T
Cloud@Scale: Budget Enforcement
Company X
Dept. A
Project 1
Project 2
Project 3
Dept. B
Project 4
Project 5
Dept. C
Project 6
Account Management
Budget Enforcement
• Hierarchical budget alignment to projects and organizational
units with real-time spend tracking
• Configurable enforcement actions to alert, freeze spending,
and terminate cloud resources

S U M M I T
Cloud@Scale: Compliance Automation
• Inheritable rules to enforce and share policies,
configurations, and approved resources
• Cloud rule and policy exemption workflows to simplify
change management
• Integration with GRC tools to validate account
configuration and accelerate accreditation
Company X
Dept. A
Project 1
Project 2
Project 3
Dept. B
Project 4
Project 5
Dept. C
Project 6
Compliance Automation
Account Management
Budget Enforcement
• Hierarchical budget alignment to projects and organizational
units with real-time spend tracking
• Configurable enforcement actions to alert, freeze spending,
and terminate cloud resources

S U M M I T
Visit cloudtamer.io in
booth 618

S U M M I T
Related breakouts
Do Your Cloud Users Keep You Up At Night? 5 Keys to Better Rest
Wednesday, 1:50 -2:10pm, Partner Theater on Expo Floor
Brian Price, cloudtamer.io
Making Cloud Procurement Easy with AWS Marketplace,
Automation, and Governance
Wednesday, 3:50-4:40pm, Room 201
Best Friends Animal Society, University of Notre Dame, NASA, AWS

S U M M I T
Thank you!
S U M M I T
Mark McInerney
Deputy Project Manager,
Earth Science Data &
Information System
NASA
mark.mcinerney@nasa.gov
Brett Miller
CMS Senior Solutions Architect &
Technical Program Manager
AWS
brettmi@amazon.com
Brian Price
President & CEO
cloudtamer.io
bprice@cloudtamer.io

S U M M I T
S U M M I T

Reimagining Data Management in the Cloud at NASA and CMS

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Reimagining Data Management in the Cloud at NASA and CMS

Semelhante a Reimagining Data Management in the Cloud at NASA and CMS (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Reimagining Data Management in the Cloud at NASA and CMS