Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
DVC303: Technological Accelerants
for Organizational Transformation

Ant Stanley
Organizer Serverless London
User Group, co-founder JeffConf
Paul Duvall
CTO Stelligent, AWS Community
Hero
Ben Kehoe
Chief Cloud Robotics Research
Scientist, AWS Community Hero

“There is no such thing as a new idea.”
Mark Twain

Ephemeralization - “more and more with less and
less until eventually you can do everything with
nothing”
Buckminster Fuller

Everything is getting smaller

Unit of Deployment
Entire Machine Operating System Virtual
Machine
Container Function

Time to Deploy
Years Months Days Hours Minutes Seconds

Average Lifespan
> 5 Years < 5 Years < 3 Years Days Seconds
Entire Machine Operating System Virtual
Machine
Container Function

Frequency of Deployment
Years Months Days Hours Minutes

Scope of Change
Entire Platform Operating System Major
Version
Minor
Version
Commit

Jidoka – Automation with a human element
Invented by Sakichi Toyoda circa 1902

Change Advisory Board
Network Security
Storage
Sysadmin

Network Security
Storage
Sysadmin

Teams replaced by services
Consistent standardized interface
API driven
Idempotent

Large array of services automated by a
small team
Focus on delivering value
Takes full responsibility for service

“AWS will release a better feature than your
horrific workaround”
Quinn’s Law, 2017

“Any organization that designs a system
(defined broadly) will produce a design whose
structure is a copy of the organization's
communication structure”
Mel Conway (Conway’s Law, 1968)

“Your AWS Organization structure will replicate
your company’s communication structure”
Conway’s Law for AWS (2017)

New (old) challenges

Persephone (FSV304)
Creates accounts, sets policy, defines which services are enabled in
the account, and runs Lambda functions to ensure continuous
compliance.
Cloud Custodian
Custodian can be used to manage AWS accounts by ensuring real
time compliance to security policies (like encryption and access
requirements), tag policies, and cost management via garbage
collection of unused resources and off-hours resource management.
https://github.com/capitalone/cloud-custodian

DevOps Transformation Examples
P a u l D u v a l l

About Me
@PaulDuvall AWS Community
Hero
Continuous
Integration
@Stelligent

DEPLOYMENT PIPELINE
FEEDBACK LOOPDevelopers Customers
Based on http://www.slideshare.net/AmazonWebServices/dvo202-devops-at-amazon-a-look-at-our-tools-processes
≈≈≈≈≈
DevOps is About Feedback
Build Test Release
Plan Monitor

Organizational Cultural Process Tooling
Principle: Feedback

Four DevOps
Transformation Examples
1 2 3 4
Breaking Down Silos Identifying Process
Bottlenecks
Making Work Visible Fixing Problems Early
https://stelligent.com/dvc303

App
Release Management
QA/Testers
Production
Support
DBAs
App Developers
UI/UX
Analyst
Build
Gate(s)
Gate(s)
Gate(s)
Change
Advisory Board
Security
Gate(s)
Problem: How to Breakdown Silos?

Product
Security
Tooling
Spend Management
Continuity Management
API
API
API
API
App
Developers
UI/UX
Analyst
Test/QA
Infrastructure
Build | Test | Deploy | Run | Support
Example: Organize Around
API-Enabled Services

Example: API-Enabled Services
https://github.com/stelligent/infrastructure-api
Image from https://aws.amazon.com/blogs/compute/how-to-provision-complex-on-demand-infrastructures-by-using-amazon-api-gateway-and-aws-lambda/
Amazon API
Gateway
Lambda
Function
CloudFormation
Business-Function
Template
CloudFormation
Stacks
1 2 3 4

Problem: How to Make Work Visible

Example: Amazon CloudWatch
Dashboard

Example: Amazon CloudWatch
Dashboard
CodePipeline
CloudWatch
Event Rule
Lambda
Function
CloudWatch
Metrics
CloudWatch
Dashboard
Public
Events
Target Put-Metric
Get-
Statistics
CloudWatch
Event Rule
Lambda
Function
CloudWatch
Scheduled Event
CloudWatch
Dashboard
List-Metrics Trigger
Put-
Dashoard

Community: Pipeline Dashboard
https://github.com/stelligent/pipeline-dashboard

VALUE
WASTE
5 min
15 min ½ week
2 min
2 weeks
15 min
2 weeks
2 hours
1 week
15 min
3 hrs 45 min ½ week
3 min 2 hrs 40 min
6 weeks + 4 hrs
Extra 15 minutes to fill
out request form
Only 15 minutes of 4
hours should be needed
to verify
Source: Implementing Lean Software Development: From Concept to Cash.
Boston: Addison-Wesley Professional; 2006, ISBN: 0321437381
Problem: How to Identify Process
Bottlenecks?
Request
Approve
and
Prioritize
Technical
Assessment
Code and
Test
Verify
and Fix
Depoy
Form Sent
to Queue
To
Verification
To
Operations
1%
Efficiency
Weekly review of
requests means an
average wait of ½ week
Wait an average of 2
weeks for an architect
Wait an average of 2
weeks for developers
Biweekly releases means
a wait of an average of 1
week for verification
Form Sent
to Queue
Form Sent
to Queue

Example: Value-Stream Map
as a Pipeline

Example: CodePipeline in
CloudFormation
My SNS Tpoic
Topic
CodeBuild
Role
Lambda
Role
CodePipeline
Role
Bucket
Bucket
CodeBuild
Project
CodeBuild
Role
CodePipeline
Parameter
EventRule
Rule
CloudFormati
on
Role
CodePipeline
Pipeline

Community: CodePipeline Mocks
https://github.com/stelligent/mu
https://github.com/stelligent/mock-pipeline

Problem: How to Fix Problems Early?

Solution: Stop the Line

Community: CodePipeline SNS
Notifications
https://github.com/stelligent/cfn_nag
https://github.com/stelligent/devops-essentials/tree/master/samples/ec2

Additional Resources
https://stelligent.com/dvc303
1 2 3 4
Breaking Down Silos Identifying Process
Bottlenecks
Making Work Visible Fixing Problems Early

iRobot
B e n K e h o e

Ben Kehoe
• Cloud Robotics Research Scientist
at iRobot
• Serverless evangelist
• AWS Community Hero
About me
@ben11kehoe

Cloud Robotics:
Connecting robots to the
internet to help them do
more and better things

2015

Then

Then Now

Then Now Future

Then Now Future
2015

• Event-driven
• Scalable
• Lean for device makers
• Reverse: AWS Greengrass
• Focus
IoT + serverless: a natural fit

AWS IoT + serverless: an even naturaler fit

• Serverless itself
• Event-driven
• Scalable
• Integrates with AWS ecosystem
• Device shadows
• Integrates with your process
AWS IoT

• Fully serverless production cloud
• 2 million connected robots by 2018
• Mostly serverless analytics
platform
• Basis for future data-powered
platform
Long story short: success!

• No provider is immune to
problems
• Small effects are more common
than big outages
• More services = blips could be
encountered more frequently
• This comes with the territory
• Set expectations internally
• Architecting robustly is key
The cloud has weather

• You only know what the provider tells you
• Architecture
• Security
• Operations
• How do they actually do all of the stuff
they do?
• Many known unknowns and
unknown unknowns
• Unknown unknown unknowns: what you
don’t know that they don’t know they don’t
know
Visibility

• Heed the warning
• Information flow is different in
serverless architecture
• Organization must change for
architecture to succeed
Conway's Law
https://commons.wikimedia.org/wiki/File:Gospers_glider_gun.gif

• Developers can be platform testers and
canaries
• Same environment in the platform as
production
• Orders of magnitude more churn
• Exercises the account limits
• Tests metrics to determine relevance
and meaning
• Bonus – Developer activity provides
additional visibility into how the platform is
currently behaving
• Higher volume of deployments in
many different AWS accounts means
problems found quickly
• This can alert us prior to problems
hitting prod
DevOps

• Servers → serverless is like
on-prem → cloud
• Easier overall and in most
respects
• But also includes new
challenges
• Outsourcing doesn’t mean you
do zero work
• Being clear about this
organizationally is critical
DiffOps

• Everything is getting smaller
• How DevOps and Serverless affect and
improve organizational transformation
• Try some the examples:
stelligent.com/dvc303
• Become part of the AWS Community
Summary and Next Steps

Q&A

Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (19)

Semelhante a Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017

Semelhante a Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017 (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017