Today, more teams are adopting continuous integration (CI) techniques to enable collaboration, increase agility, and deliver a high-quality product faster. Cloud-based development tools such as AWS CodeCommit and AWS CodeBuild can enable teams to easily adopt CI practices without the need to manage infrastructure. In this session, we showcase best practices for code reviews and continuous integration, drawing on practices used by Amazon engineering teams. We’ll incorporate demos to not just explain the practices but show you how.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous Integration
Best Practices
Nick Brandaleone Joseph Vusich
Senior Solutions Architect Senior Software Development Engineer
AWS AWS
D E V 3 1 9

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Introduction to Continuous Integration
Continuous Integration Tooling
CI best practices and techniques
Demo
Review

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Breakout repeats
DEV319 - Continuous Integration Best Practices
Monday, Nov 26, 4:00 PM - 5:00 PM
Venetian, Level 2, Titian 2204
DEV319 - Continuous Integration Best Practices
Tuesday, Nov 27, 3:15 PM - 4:15 PM
Venetian, Level 5, Palazzo O

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Related breakouts
DEV317 - Advanced Continuous Delivery Best Practices
Monday, Nov 26, 1:45 PM - 2:45 PM Tuesday, Nov 27, 2:30 PM - 3:30 PM
Venetian, Level 5, Palazzo O. Aria East, Level 1, Joshua 9
DEV309 - CI/CD for Serverless and Containerized Applications
Tuesday, Nov 27, 7:00 PM - 8:00 PM. Thursday, Nov 29, 2:30 PM - 3:30 PM
Venetian, Level 2, Venetian E. Bellagio, Level 1, Grand Ballroom 2
Thursday, November 29
Leadership Session: Using DevOps, Microservices, and Serverless
to Accelerate Innovation (SRV325)
12:15 – 1:15 PM | Venetian Theatre (Level 2)

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

7. “Continuous Integration is a software
development practice where members
of a team integrate frequently …
verified by an automatic build and test
process…”
Martin Fowler

8. “Continuous Integration produces and
validates artifacts, while Continuous
Deployment deploys them.”

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is continuous integration?
1. Developers regularly check in code to a
shared, central repository
2. Automated checks run regularly against
the shared codebase
3. Developers get feedback from the
automated checks about their code
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is continuous integration?
1. Developers regularly check in code to a
shared, central repository
2. Automated checks run regularly against
the shared codebase
3. Developers get feedback from the
automated checks about their code
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is continuous integration?
1. Developers regularly check in code to a
shared, central repository
2. Automated checks run regularly against
the shared codebase
3. Developers get feedback from the
automated checks about their code
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is continuous integration?
1. Developers regularly check in code to a
shared, central repository
2. Automated checks run regularly against
the shared codebase
3. Developers get feedback from the
automated checks about their code
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Typical Developer workflow
1) Write code - Cloud9
2) Compile and run unit tests locally - CodeBuild Local
3) Push code to a Git repository - CodeCommit
4) Build the updated images - CodeBuild
5) Run integration tests against new code - CodeBuild
6) Push new images to registry - CodeBuild and Elastic Container Registry
7) Trigger a deployment - CodeBuild or CloudWatch Events + Lambda

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why does continuous integration
matter?
• Find bugs earlier
• Fix bugs faster
• Deliver faster
• Deliver more often
• Unblock developers
• Grow developers

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why does continuous integration
matter?
• Find bugs earlier
• Fix bugs faster
• Deliver faster
• Deliver more often
• Unblock developers
• Grow developers
Quality:
5x
lower change
failure rate
Source: 2017 State of DevOps Report (Puppet)

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why does continuous integration
matter?
• Find bugs earlier
• Fix bugs faster
• Deliver faster
• Deliver more often
• Unblock developers
• Grow developers
Delivery:
440x
faster from
commit to deploy
Source: 2017 State of DevOps Report (Puppet)

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why does continuous integration
matter?
• Find bugs earlier
• Fix bugs faster
• Deliver faster
• Deliver more often
• Unblock developers
• Grow developers
Delivery:
46x
more frequent
deployments
Source: 2017 State of DevOps Report (Puppet)

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why does continuous integration
matter?
• Find bugs earlier
• Fix bugs faster
• Deliver faster
• Deliver more often
• Unblock developers
• Grow skills faster
Happy teams:
44%
more time spent
on new features
and code
Source: 2017 State of DevOps Report (Puppet)

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Teams that practice CI/CD ship more code faster,
and with more confidence
5x
Lower change
failure rate
440x
Faster from
commit to deploy
46x
More frequent
deployments
44%
More time spent
on new features
Source: Puppet 2017 State of DevOps Report

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Summary

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous integration tools
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks
Source Code
• AWS CodeCommit
• GitHub
• GitHub Enterprise
• Bitbucket
• S3

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous integration tools
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks
Build & Test
• AWS CodeBuild
• Jenkins w/ CodeBuild

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continuous integration tools
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks
Glue
• Amazon CloudWatch Events
• AWS Lambda
• Slack integration
• SNS (E-mail / Texts)
• Any AWS Service

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Fully managed Git service
• Hosts private Git repositories
• Backed by S3
• Scale with ease
• Store anything, anytime
AWS CodeCommit

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Code* Console - NEW LOOK

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeCommit
Repository
- Code

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit - Visualizer

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit – Pull Request

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit
PR comments:
• Pull Request
• Line

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit – notifications

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit – notification/trigger

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Fully managed build service
• Continuous scaling
• Pay as you go
• Extensible
AWS CodeBuild

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Build source from CodeCommit, S3,
Bitbucket and GitHub/GitHub Enterprise
• Webhook support
• Multiple input repos and output artifacts
• Build inside your VPC
• Use secrets from Amazon Parameter Store
• S3 build cache
• Debug locally with CodeBuild Agent
AWS CodeBuild: Features

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• Ubuntu
o .NET Core
o Android
o Docker
o Golang
o Java
o Node
o PHP
o Python
o Ruby
AWS CodeBuild: Supported Environments
• Windows Server 2016
o .NET Framework
o .NET Core
o C#
o F#
o Visual Basic

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Project

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Project Details

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Project Environment

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Build history

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Build status

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Phase details

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Logs

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Logs – Streamed to CB & CW

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
version: 0.2
phases:
install:
commands:
- npm install
build:
commands:
- npm test
artifacts:
files:
- '**/*'
AWS CodeBuild build spec

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Phase Description Examples
install Installation of packages into
the environment
Install testing
frameworks e.g.
RSpec, Mocha
pre_build Commands to run before the
build such as login steps or
installation of dependencies
Log in to Amazon
ECR. run Ruby
bundler or npm
build Sequence to run the build such
as compilation and/or running
tests
Run go build, sbt,
Mocha, RSpec
post_build Commands to run after a build
on success or failure
Build a JAR via Maven
or push a Docker
image to Amazon ECR

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeBuild build spec for Docker
version: 0.2
phases:
pre_build:
commands:
- $(aws ecr get-login --no-include-email)
- TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)"
- IMAGE_URI="${REPOSITORY_URI}:${TAG}"
build:
commands:
- docker build --tag "$IMAGE_URI" .
post_build:
commands:
- docker push "$IMAGE_URI"

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild Metrics

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeCommit – CW Events

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The continuous integration journey
10 mph 65 mph 150 mph

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The continuous integration journey
Nightly
checks
Branch
checks
Pull request
checks

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technique #1: Nightly checks
1. Nightly checks
2. Branch checks
3. Pull request checks
Run a full build
and unit tests
every night to
make sure that
application still
compiles and that
tests still pass.

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technique #2: Branch checks
1. Nightly checks
2. Branch checks
3. Pull request checks
Run a full build
and unit tests
every time
someone pushes a
new change to a
branch in the
source code repo.

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technique #3: Pull request checks
1. Nightly checks
2. Branch checks
3. Pull request checks
Run a full build
and unit tests
every time
someone creates a
pull request to get
code reviewed by
the team.

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly checks: How to implement
CloudWatch
Event
Schedule
Trigger

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CodeBuild triggers

60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Automate codebase maintenance
2. Email notifications
Nightly checks: Speed boosts

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly checks: Automate upgrades
CloudWatch
Event
Schedule
Trigger Push
Upgrade

64. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
- rm npm-shrinkwrap.json
- npm install
- npm test
- npm shrinkwrap
- git add npm-shrinkwrap.json
- git commit –m “Update dependencies!”
- git push
Nightly checks: Automate upgrades

65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly Checks: Update Libraries

66. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly Checks: Update Libraries

67. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly Checks: Update Libraries

68. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Automate codebase maintenance
2. Email notifications
Nightly checks: Speed boosts

69. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Speed boost: Email notifications
CloudWatch
Event
SendNotify

70. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

71. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

72. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nightly checks summary
Implement: Run a build every night at midnight
Feedback loop: 16-24 hours
Team impact: Broken code can block team for entire days
Speed boosts: Automate codebase maintenance; Email notifications

73. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technique #2: Branch checks
1. Nightly checks
2. Branch checks
3. Pull request checks
Run a full build
and unit tests
every time
someone pushes a
new change to a
branch in the
source code repo.

74. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch checks: AWS CodeCommit
Push Notify Start
CloudWatch
Event
Team Members

75. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: CodeCommit Events

76. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: CodeCommit Events

77. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: CodeCommit Events

78. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: CodeCommit Events

79. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: CodeCommit Events

80. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Slack notifications
2. Build badges
3. Caching
Branch checks: Speed boosts

81. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Speed boost: Slack notifications
Notify
CloudWatch
Event
PostTrigger
Slack
Channel

82. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: Slack Notifications

83. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch Checks: Slack Notifications

84. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Slack notifications
2. Build badges
3. Caching
Branch checks: Speed boosts

85. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

86. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

87. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Slack notifications
2. Build badges
3. Caching
Branch checks: Speed boosts

88. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Speed boost: Caching
Update
Install
Amazon S3
Bucket
Download

89. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
cache:
paths:
- 'node_modules/**/*'
Speed boost: Caching

90. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

91. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild timing w/o cache

92. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CodeBuild timing with cache

93. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Branch checks summary
Implement: Run a build every time new code is pushed to a branch
Feedback loop: Time it takes to build the code (minimum)
Team impact: Much shorter cycles; broken code still blocks team
Speed boosts: Slack notifications; build badges; caching

94. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Technique #3: Pull request checks
1. Nightly checks
2. Branch checks
3. Pull request checks
Run a full build
and unit tests
every time
someone creates a
pull request to get
code reviewed by
the team.

95. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
PR checks: AWS CodeCommit
Propose
Notify Start
CloudWatch
Event
NotifyComment
Team Members

96. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

97. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

98. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

99. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

100. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

101. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

102. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Integration tests
2. Parallel builds
PR checks: Speed boosts

103. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Speed boost: Integration tests
Test
ElasticCache
Cluster

104. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

105. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

106. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Integration tests
2. Parallel builds
PR checks: Speed boosts

107. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Speed boost: Parallel builds
StartPropose
Multiple
CodeBuilds
Team Members

108. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

109. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pull request checks summary
Implement: Run a build while code is still in review
Feedback loop: Time it takes to build the code
Team impact: Broken code doesn’t block the team
Speed boosts: Integration tests; parallel builds

110. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

111. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

112. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1.Automate the boring stuff
Library upgrades or automatic tagging
2.Communicate status
Email, Slack, build badges
3.Build faster
Caching, parallel builds
4.Check more
Unit and Integration tests
Continuous Integration

113. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The continuous integration journey
Developer
Shared codebaseAutomated checks
Check
in code
Get
feedback
Run
checks

114. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Samples
https://github.com/aws-samples/aws-codebuild-
samples

115. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Nick Brandaleone Joseph Vusich
nbrand@amazon.com

116. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.