What makes Continuous Delivery easy and what makes it hard? Should it be all Scala + Docker + microservices or is .Net + Windows + monoliths a safer bet? This session compares and contrasts the successful continuous delivery journeys of two completely different cultures. Both achieved weekly releases to Production, but one was a .Net monolith, the other a set of Scala microservices. We’ll explore the lessons learnt by looking at the blockers and accelerators each faced.
SLOW and CLEAR speach
PLAIN language
Breathe!
Introduce myself (lyndsayp Ltd and EE)
Roles
Context
Average time between production releases
Establish:
- what roles are present
- how many deliver to web
Show of hands - production deployment capability:
one or more per year,
per quarter,
per month,
per week,
per day
Private sector .Net Monolith
Government Scala microservices
Both teams are doing a great job of getting their doughnuts (of differing sizes and shapes) out to Production.
This is an experience reportwill compare and contrast
What best practices help these team go faster, faster
What areas of pain slow them down
TODO - show / hide russian / lithuanian logo
Emphasise:
Fast feedback
Risk reduction
Valuable software
in short cycles
reliably released at any time
Fast feedback - shortest time from concept to cash
Risk reduction - smallest value possible
2 hours East of here there’s an IT company that makes an awesome product, Target Process.
Last month, their CEO published an insightful post on how they’ve evolved over the last 90 months.
It’s a fascinating read, and covers the gory details of their journey through various agile best practices.
Over the last three years they’ve improved their delivery frequency, but still have some way to go. They’ve made lots of improvements in other areas though.
The article reminds us that whenimproving one’s Continuous Delivery, it’s important not to do so at the expense of other practices.
The overarching goal has to be Continuous Improvement, not Continuous Delivery.
HMRC Digital (Public sector)
Gov.uk tax interfaces for citizens and businesses (70% of all gov transactions, > 1 billion transactions online)
Traffic varies across the year, with peaks at key tax calendar events
30+ Product Teams
Tech Stack
Scala
Microservices
Docker
Mongo DB
Deployment frequency
You build it, you run it -Amazon CTO Werner Vogels
CI that works
Culture - Mind share
CI that works - knowing what to automate - a story about NVM
Cross functional teams
Picture of team mix
Picture of testing all the time
Story about Elateral
Cross functional teams
Picture of team mix
Picture of testing all the time
NVM and HMRC
The test automation pyramid
API and/or integration tests
Picture different test mixes
Common automated testing problems
Picture different test mixes
Common automated testing problems
Picture different test mixes, highlighting problems
Story about NVM
Tear drop shape
Behavioural focus
Reference to TDD - where did it all go wrong?
Story about Elateral, then NVM
Story about HMRC (rollback)
Eye of Sauron, looking at the whole stack (user behaviour, down to disk space)
Highlight:
What to focus on (behaviour instead of environment)
Leading vs. trailing indicators
Detect
Alert
Respond
Display
Analyse
Diagram of right and wrong mix of the above
Detect
Alert
Respond
Display
Analyse
Diagram of right and wrong mix of the above
Tools
Logos and Pictures of tools
New Relic
Graphana
KIbana
Line of logging code
Elastic Search
Graphite
Sensu
Papertrail
Pagerduty
Splunk
Tell a story
Mongo query defect at HMRC
Kibana
Change in success rate
Kibana / splunk
Tell a story
Mongo query defect at HMRC
Kibana
Change in success rate
Kibana / splunk
Making changes to Running machine / aeroplane
Stories and Diagram of
Cookie change
Database change
Automated release note
Spot the difference (automated release note)
Diagram of how it works
Diagram of layout
Microservice architecture
Amazon quote - Build It, Run It, Own it
Separation of features and concerns
Prefer service dependencies over library dependencies
Principle of least surprise
Service API's must always be backwardly compatible
Story of when we were young, teams wanted to test against fixed versions of other services
Promote Build Quality In
Excellent chapter in ILSD on VSM’s