Building, testing, and deploying AWS Lambda-based, serverless applications introduces new challenges to developers whose development workflows are optimized for traditional VM-based applications. In this webinar, we will introduce one method for automating the deployment of serverless applications running on AWS Lambda. We will first cover how you can model and express serverless applications using the open source AWS Serverless Application Model (AWS SAM). Then, we will discuss how you can use CI/CD tooling from AWS CodePipeline and AWS CodeBuild to build an automated development workflow for your serverless app.
Learning Objectives:
1. Understand the fundamentals of the microservices architectural approach
2. Learn best practices for designing microservices on AWS
3. Learn the basics of Amazon EC2 Container Service, Amazon API Gateway, AWS Lambda, and AWS X-Ray"
2. About me:
Chris Munns - munns@amazon.com, @chrismunns
• Senior Developer Advocate - Serverless
• New Yorker
• Previously:
• Business Development Manager – DevOps, July ’15 - Feb ‘17
• AWS Solutions Architect Nov, 2011- Dec 2014
• Formerly on operations teams @Etsy and @Meetup
• Little time at a hedge fund, Xerox and a few other startups
• Rochester Institute of Technology: Applied Networking and
Systems Administration ’05
• Internet infrastructure geek
6. Common Lambda use cases
Web
Applications
• Static
websites
• Complex web
apps
• Packages for
Flask and
Express
Data
Processing
• Real time
• MapReduce
• Batch
Chatbots
• Powering
chatbot logic
Backends
• Apps &
services
• Mobile
• IoT
</></>
Amazon
Alexa
• Powering
voice-enabled
apps
• Alexa Skills
Kit
IT
Automation
• Policy engines
• Extending
AWS services
• Infrastructure
management
7. Amazon S3 Amazon
DynamoDB
Amazon
Kinesis
AWS
CloudFormation
AWS CloudTrail Amazon
CloudWatch
Amazon
Cognito
Amazon SNSAmazon
SES
Cron events
DATA STORES ENDPOINTS
DEVELOPMENT AND MANAGEMENT TOOLS EVENT/MESSAGE SERVICES
Event sources that trigger AWS Lambda
… and a few more with more on the way!
AWS
CodeCommit
Amazon
API Gateway
Amazon
Alexa
AWS IoT AWS Step
Functions
8. Development Workflow Considerations
If we combined all of our use-cases with all of our event
sources, we end up with A LOT of possible options.
• What AWS resources do we need to provision and
configure and how should we do that?
• How do we establish independent environments?
• How can we ensure that we are testing and validating
our architecture and application along the way?
• How can we best automate all of the above?
9. Development Workflow Checklist
Model your application and infrastructure
resources
Configure multiple environments
Establish your testing/validation model
Automate your delivery process
10. Model our application and infrastructure resources
An example of services for building serverless applications:
Best practice: Manage these AWS resources with
“Infrastructure as Code” practices/tools!
Amazon
API Gateway
AWS Step
Functions
Amazon S3 Amazon
DynamoDB
Amazon
Kinesis
AWS
Lambda
Amazon SNS
11. Create templates of your infrastructure
CloudFormation provisions AWS resources
based on dependency needs
Version control/replicate/update templates like
code
Integrates with development, CI/CD,
management tools
JSON and YAML supported
AWS CloudFormation
17. SAM template
AWSTemplateFormatVersion: '2010-09-09’
Transform: AWS::Serverless-2016-10-31
Resources:
GetHtmlFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: s3://sam-demo-bucket/todo_list.zip
Handler: index.gethtml
Runtime: nodejs4.3
Policies: AmazonDynamoDBReadOnlyAccess
Events:
GetHtml:
Type: Api
Properties:
Path: /{proxy+}
Method: ANY
ListTable:
Type: AWS::Serverless::SimpleTable
Tells CloudFormation this is a SAM
template it needs to “transform”
Creates a Lambda function with the
referenced managed IAM policy,
runtime, code at the referenced zip
location, and handler as defined.
Also creates an API Gateway and
takes care of all
mapping/permissions necessary
Creates a DynamoDB table with 5
Read & Write units
23. SAM Template Capabilities
• Can mix in other non-SAM CloudFormation
resources in the same template
• i.e. S3, Kinesis, Step Functions
• Supports use of Parameters, Mappings,
Outputs, etc
• Supports Intrinsic Functions
• Can use ImportValue
(exceptions for RestApiId, Policies, StageName attributes)
• YAML or JSON
24. AWS commands – Package & Deploy
Package
•Creates a deployment package (.zip file)
•Uploads deployment package to an Amazon S3 bucket
•Adds a CodeUri property with S3 URI
Deploy
•Calls CloudFormation ‘CreateChangeSet’ API
•Calls CloudFormation ‘ExecuteChangeSet’ API
25. Development Workflow Checklist
Model your application and infrastructure
resources
Configure multiple environments
Establish your testing/validation model
Automate your delivery process
26. Configure multiple environments
A good developer knows they need different environments for building,
testing, and running their application!
Why?
• Avoid overlapping usage of resources
• Safely test new code without impacting your customers
• Safely test infrastructure changes
How?
• AWS Account strategies
• Using Infrastructure as Code tools
• Automating application delivery/testing
27. Two popular ways to do this:
Same account, different stacks:
+ Easier management of
resources
+ Easier visibility via
management/monitoring tools
- Can be harder to create
permission/access separation
Better for smaller teams/individuals
Configure multiple environments
Multiple accounts:
+ Assured separation of permissions
and access
+ Resource limits per account to
control usage
- Overhead of managing multiple
accounts and controls between them
Better for larger teams/companies
!! Check out AWS Organizations
28. Template File
Defining Stack
Source
Control
Dev
Test
Prod
Use the version
control system of
your choice to
store and track
changes to this
template
Build out multiple
environments, such
as for Development,
Test, Production and
even DR using the
same template,
even across
accounts
Many Environments from One Template
29. Development Workflow Checklist
Model your application and infrastructure
resources
Configure multiple environments
Establish your testing/validation model
Automate your delivery process
30. Establish our testing/validation model
We want to make sure our code:
• is without syntax issues
• meets company standards for format
• compiles
• is sufficiently tested at the code level via unit tests
We want to make sure our serverless service:
• functions as it is supposed to in relation to other components
• has appropriate mechanisms to handle failures up or down stream
We want to make sure our entire application/infrastructure:
• functions end to end
• follows security best practices
• handles scaling demands
31. Building a deployment package
Node.js & Python
• .zip file consisting of
your code and any
dependencies
• Use npm/pip to
install libraries
• All dependencies
must be at root level
Java
• Either .zip file with all
code/dependencies,
or standalone .jar
• Use Maven / Eclipse
IDE plugins
• Compiled class &
resource files at root
level, required jars in
/lib directory
C# (.NET Core)
• Either .zip file with all
code/dependencies,
or a standalone .dll
• Use NuGet /
VisualStudio plugins
• All assemblies (.dll)
at root level
32. Fully managed build service that compiles source code,
runs tests, and produces software packages
Scales continuously and processes multiple builds
concurrently
You can provide custom build environments suited to
your needs via Docker images
Only pay by the minute for the compute resources you
use
Launched with CodePipeline and Jenkins integration
New: Can be used as a “Test” action in CodePipeline
AWS CodeBuild
34. version: 0.1
environment_variables:
plaintext:
"INPUT_FILE": "saml.yaml”
"S3_BUCKET": ""
phases:
install:
commands:
- npm install
pre_build:
commands:
- eslint *.js
build:
commands:
- npm test
post_build:
commands:
- aws cloudformation package --template $INPUT_FILE --s3-
bucket $S3_BUCKET --output-template post-saml.yaml
artifacts:
type: zip
files:
- post-saml.yaml
- beta.json
• Variables to be used by phases of
build
• Examples for what you can do in
the phases of a build:
• You can install packages or run
commands to prepare your
environment in ”install”.
• Run syntax checking,
commands in “pre_build”.
• Execute your build
tool/command in “build”
• Test your app further or ship a
container image to a repository
in post_build
• Create and store an artifact in S3
buildspec.yml Example
35. Development Workflow Checklist
Model your application and infrastructure
resources
Configure multiple environments
Establish your testing/validation model
Automate your delivery process
37. Continuous delivery service for fast and
reliable application updates
Model and visualize your software release
process
Builds, tests, and deploys your code every time
there is a code change
Integrates with third-party tools and AWS
AWS CodePipeline
38. Delivery via CodePipeline
Pipeline flow:
1. Commit your code to a source code repository
2. Package/Test in CodeBuild
3. Use CloudFormation actions in CodePipeline to
create or update stacks via SAM templates
Optional: Make use of ChangeSets
4. Make use of specific stage/environment
parameter files to pass in Lambda variables
5. Test our application between stages/environments
Optional: Make use of Manual Approvals
39. Testing tools
Code Inspection/Test Coverage:
• Landscape - https://landscape.io/ (only for Python)
• CodeClimate - https://codeclimate.com/
• Coveralls.io - https://coveralls.io/
Mocking/stubbing tools:
• https://github.com/atlassian/localstack - “A fully functional local AWS cloud stack. Develop and test
your cloud apps offline!”
• Includes:
• https://github.com/spulec/moto - boto mock tool
• https://github.com/mhart/dynalite - DynamoDB testing tool
• https://github.com/mhart/kinesalite - Kinesis testing tool
• more!
API Interface/UI testing:
• Runscope - https://www.runscope.com/ - API Monitoring/Testing
• Ghost Inspector - https://ghostinspector.com/ - Web interface testing
40. Source
Source
CodeCommit
MyApplication
An example minimal pipeline:
Build
test-build-source
CodeBuild
Deploy Testing
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-stubs
AWS Lambda
Deploy Staging
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-API-test
Runscope
QA-Sign-off
Manual Approval
Review
Deploy Prod
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Post-Deploy-Slack
AWS Lambda
This pipeline:
• Five Stages
• Builds code artifact
• Three deployed to “Environments”
• Uses CloudFormation to deploy
artifact and other AWS resources
• Has Lambda custom actions for
running my own testing functions
• Integrates with a 3rd party
tool/service
• Has a manual approval before
deploying to production
41. Development Workflow Checklist
Model your application and infrastructure
resources
Configure multiple environments
Establish your testing/validation model
Automate your delivery process