Conall Bennett from CME Group Belfast gave this talk at DevOpsDays Galway 2017: What's all the fuss with serverless architecture? To some, it is the future of architecture implementation and is disrupting DevOps, traditional implementation models, enterprise approaches to IT, and entry to and time to market. This talk is an experience report on how our team fell into a serverless implementation without any real experience of the technology and how we learned about it along the way. I will give a brief overview of what serverless is and its benefits and how we went from sceptics of it initially to advocates along our journey of discovery. I will describe our team test first mentality and the multiple functional and non-functional approaches and patterns we applied to serverless, as well as the existing challenges of testing it. That includes how we ran exploratory performance tests to determine and quantify language deficiencies on the platform in order to help drive our implementation approach.

1. Reference https://serverless.com/

2. •
•
•
•
•
•
•
•
•
•

3. •
•
•
•
•

4. •
•
•
•
•
Source: https://www.loggly.com/blog/infographic-it-
operations-serverless-world/

5. •
•
•
•
•
•

6. Amazon
S3
corporate data
center
web app
server
web app
server
Data consumers
Apache
Kafka
Microservice
A
Microservice
B
Microservice
C
Microservice
D
Microservice
E
Many simplifications aside, esp. on the consumer side, the data pipeline & flow broadly looks like this. At this point in the
project, the micro service architecture is high level, we know they will be restful micro services.
Main AWS VPC
Message
decoding
Message
enrichment
Real time
message sink

7. Of course its not simples, that would be boring!
Its not really “lift and shift” either!
More like “lift – architect – re-architect – shift – re-
architect again – lift some more – re-architect, then
shift and deliver before further re-architecting

9. Amazon
S3
corporate data
center
web app
server
Consumer
service
Data consumers
Apache
Kafka
So AWS Lambda’s have moved in as the micro service technology, with API Gateways exposing the endpoints.
Main AWS VPC
Message
decoding
Message
enrichment
Real time
message sink

10. • How do we automate the deployment?
• How do we know our consumers can integrate?
• Does serverless really scale & perform?
• Is it secure enough?
•How do Lambdas work with an API Gateway?
•Does it function the way we would expect a RestFul
service to function?
•What could possibly go wrong with a Lambda?

11. Func
Unit tests in CI
Mocked http tests in CI
Deployment tests in CD
Consumer driven contract tests
Manual exploratory (lots of this)
Perf
Load
Soak
Exploratory
Security
Automated Fuzzing
Manual exploratory
Static scanning
Reliabili
ty
Manual monkeying!
• Can we integrate with the
consumer(s)?
• Can we run our code locally, without
being on the platform, to see if it
works?
• Does it run properly on the platform?
• Are we getting the right responses for
the right data?
• When do lambda containers go cold
after being warm?
• How much concurrent load can a single
lambda container handle?
• What’s the initialization time for a java
lambda?
• Is the lambda responsive? What is the
error rate and does memory size matter?
• Can we inject instability into the
platform in any unexpected ways?
• Our we using secure libraries with no
known vulnerabilities?
• Can we fuzz our way into the system
and past security?
• Can we find dynamic ways of getting
past the platform security and
injecting malicious code through
requests?

12. Post-deploy - run on demand or in a CD pipelinePre-deploy - these live in CI
Functional Unit Mocked http
Deployment
test
Exploratory
Performanc
e
Load Soak Exploratory
Security
Static
scanning
Automated
fuzzing
Exploratory
Reliability
Manual
monkeying
Predeployteststhatit“can”
work
Postdeployteststhatthe
functionality“does”work

14. •
•
•
•
•
•

15. •
•
•
•
•
•
•

17. • HTTPS://12FACTOR.NET/
• HTTPS://MARTINFOWLER.COM/ARTICLES/MICROSERVICES.HTML
• HTTPS://MARTINFOWLER.COM/ARTICLES/SERVERLESS.HTML
• HTTP://DOCS.AWS.AMAZON.COM/LAMBDA/LATEST/DG/DEPLOYING-LAMBDA-APPS.HTML
• HTTPS://SERVERLESS.COM/BLOG/KEEP-YOUR-LAMBDAS-WARM/
• HTTPS://SERVERLESS.COM/FRAMEWORK/DOCS/PROVIDERS/AWS/GUIDE/TESTING/#TESTING

18. • HTTPS://SERVERLESS.COM/
• HTTPS://GITHUB.COM/ATLASSIAN/LOCALSTACK
• HTTPS://GITHUB.COM/AWSLABS/SERVERLESS-APPLICATION-MODEL/BLOB/MASTER/HOWTO.MD
• HTTPS://GITHUB.COM/BBC/CHAOS-LAMBDA/BLOB/MASTER/README.MD