This document discusses continuous delivery decision points and environments. It notes that as DevOps shifts focus from build/deploy automation to continuous delivery, test environments are proliferating and need to be provisioned and managed efficiently. It also emphasizes that collaboration around testing is important, and teams must make strategic decisions about promoting code changes to subsequent stages or iterating further. Maintaining properly designed automated and manual tests is key to making these decisions. Organizational culture and leadership must also support empowering developers and teams to deliver changes incrementally.

1. Continuous Delivery Decision Points
Kelly Looney
DevOps Strategy

2. Kelly Looney - Who am I?
I’ve spent too much time with computers…
• 30 years in Software Development
• Smalltalker in the 90s – Kent Beck, RPG, Adele
Goldberg…
• Long-time Agilist - Larman, Rubin, Leffingwell,…
• Most recently a DevOps “helper”
• ThoughtWorks – Jez Humble, Martin Fowler
• Now a part of the Agile Infratructure startup Skytap
helping with DevOps strategy
Worked with all sorts of organizations on all types of software

3. What I’d like to talk about
Collaboration and Decision making about software
• Scaling Agile development is a topic of keen interest
• Scaling any kind of software development still very difficult
• DevOps is evolving from focusing on build/deploy automation to
full-on Continuous Delivery
• Culture and Organizational issues are still there and still hard
• Both require collaboration and decision-making
• Good news: for CD you can typically automate the results
• Not-as-good news: for Scaling Agile you need a process
• Environments are important for both and a next area of innovation

5. Scaling Agile Development
Best advice: Don’t do it!
• Lots of agreement on Scrum and technical practices
• Lack of agreement on how to scale:
• SAFe, LeSS, Disciplined Agile, Scrum at Scale. … at odds over small
things
• Something I find missing:
• Multiple teams all working on software that needs to interoperate
• Old method: Design Interfaces (ICDs…), then argue, at some point test
• New method: Collaborate, Demonstrate, Document
• Meta-Scrum, Scrum-of-Scrums
• I find there is a need for visualization and planning

6. Team 2
Team 3
Team 1
Team 4
Streamsof Development
1.0
1.5
2.0
2.2
3.02.92.9
Demonstrations of
coordinated
functionality
IntegrationPoints

7. How is DevOps changing?
We have shown really remarkable success in a short time…
• Initial focus was build and deployment automation
• Lots of nice technology and a bounded problem
• DevOps Enterprise Summit 2015 in SF
• “This is becoming a Continuous Delivery conference”
• Good news IMHO – broad base of examples
• In a more difficult phase now
• Automation is hard work
• State of tests and test environments is pretty bad out there
• Are we thinking about test at the right level?

8. Delivery Pipelines

9. Environments Proliferate with Modern Techniques
• Each box ideally represents a fresh environment
• Unrealistic to have this many physical labs
• Change and refresh can be costly and error-prone

10. Today’s Delivery Pipelines cont…
Ideas Code Revise Build Provision Test Deploy
Epics/Story/Tasks Editors/IDEs Version
Control
Continuous
Integration
Test Environments Pre-Production Production
Plans/Roadmap Code Entry Change tracking Executable Testable System Tested System Working System
With common Tools

11. What DevOps and CD mean for the organization
• The whole idea of holding off changes to retain stability gets
turned on its head
• Change all the time and stay stable!
• Changes get smaller and smaller, but are constantly being deployed
• With small changes integration issues become fairly simple
• Environments must proliferate along with associated infrastructure
• Ideally you need a new test environment to test every change
• Create/Destroy quickly and efficiently
• Are your environments captured as code?
• Use Cloud services here, even if you don’t want to for production
11

12. Decision Points for Software
Promote to the next stage, or iterate again?
• Classic automation mistake – find every test you can automate
and put it into the “suite”
• Results: lots of crappy new code to maintain! All night runs…
• Remember: Tests are now as important as production code
• Goal: Carefully designed tests that answer the “promote” Q.
• Manual tests need to get off the critical path to production
• Full regressions (manual or automated) as well – just run them all the time
• You want a fast way to move changes foreward with the sum of the tests
being something that gives you “enough” confidence.

14. The common thread
Collaboration around Test – it’s a Strategic thing
• Integration and Delivery Pipeline
Stages need a place
• Physical or Virtual lab – (virtual is where
things are headed)
• Collaboration capabilities matter here
• Meetings, Chats, Video, …
• Always the same pattern – move
foward or iterate further

15. Thanks

16. Organization and Culture

17. Assessing Organization and Culture
Organizational issues:
• Can teams be empowered to deliver?
• Are they be capable?
• Where are the “wait states”?
• What are the alternatives?
Culture:
• What should you value to meet your goals?
• What Leadership signals need to change?

18. Cultural changes needed
• Old style Developers
• Responsibilities: Write code
• Focus: Know ONE THING really really well.
• Deep expertise = respect
• What we want now is Developers that:
• Understand our company goals
• Understand requirements and tests
• Write, build, integrate, and test code incrementally
• Can demonstrate and explain working systems
• Maintains his/her code in production
• Understands operations
Deep expertise is great, but varied knowledge is just as important
18

19. Wow, you want developers to do everything…
• First the right attitude…then
• Todays Tools and Processes:
1. Scrum provides continuous “customer” access
2. Distributed versioning (typically Git) puts full source control
into individual developers hands
3. Continuous Integration isolates mistakes
4. Jenkins-Vagrant-Puppet-Chef-Saltstack pipelines make
infrastructure and deployment mostly automatic regardless
of complexity
5. Monitoring lets you see your running service
How is that possible?
19

21. Containers are changing hosting
• Virtualization efficiency and cost savings are obvious
• The most interesting issue is the separation of concerns presented
• “developer-land” vs infrastructure

22. What do Containers/Microservices mean for me?
You will use them, but as usual the transition will be arduous
• A lot of the excitement involves startups doing green field apps
• Fully transitioning a legacy app involves a lot of work and risk
• An incremental approach involving some microservices carved
out of a legacy app makes more sense in most cases
• You need a virtual environment with both VMs and Containers working
together
• Skytap provides this to help with the transition

23. Environments as a Service
Environment:
Core Client Applications
Config1: Shared Resources
Config2: Application Under Test
On-Premise
Resources
Test Mgmt
Server
Database
Hot
Database
Standby
Active
Directory
Load
Generator
Load
Balancer
DMZ Network
Web Network
DB Network
Corporate Network
Published
Service On-Premise
Resources
Web
Server
Web
Server

24. Skytap Enables Hybrid Apps
Accelerate and Modernize Enterprise Development
Accessible from any tool chain
Designed for Dev/QA/IT
Collaboration
Environments that mix traditional
and modern frameworks
Infrastructure choiceCompute Storage Network Compute Storage Network Compute Storage Network