The document discusses lessons learned from building a serverless company. It introduces Planet 9 Energy and their use of AWS Lambda and the Serverless framework. Key topics covered include security, quality assurance, developer experience, costs, and lessons learned. Some challenges discussed are debugging, API Gateway custom domains, and Lambda limitations. The document emphasizes that serverless architectures provide infinite scalability at low cost but also have some limitations that require management.

1. AWS Lambda &
Serverless framework
Lessons learned while building a serverless company
26/01/2017@Podgeypoos79 - @loige

2. Luciano Mammino
● @loige on Twitter
● Lmammino on GitHub
● Blog at loige.co
● Co-author of:
Node.js Design Patterns(Second Edition)

3. Padraig O’Brien (a.k.a “Podge”)
● @podgeypoos79 on Twitter
● Does not have a book (yet…)
● Organiser of Nodeschool Dublin and Dun Laoghaire
● Organiser of Dublin Serverless meetup
● He is writing his own database, unicorndb

4. Agenda
● Planet 9 Energy
● Serverless
● Security
● Quality
● Developer Experience
● Costs
● Lessons learned

5. planet9energy.com

7. 17.520 * * 24
Meter readings Customer size Data versions
≈4 billion data pts / year (tot.)
SMALL customer ≈3 mln data pts / year MEDIUM customer ≈20 mln data pts / year BIG customer ≈110 mln data pts / year
( Per customer / year )

8. ● Limited number of “Full stack” engineers
● Write & deploy quality code as fast as possible
● Adopt hot and relevant technologies
● No servers... or more to the point, NO CALLS AT 2 AM!
Our requirements

9. Early experiments

10. Current Technology

11. “Serverless”
Meaning
Serverless framework
AWS Lambda

12. What does
“serverless” mean?

13. Why Serverless
framework?

14. Anatomy of Serverless.yml
Serverless.yml header

15. Defining functions and events

16. Why?

17. What is a lambda?
● Function as a service (FAAS)
● Pay for invocation / processing time
● Virtually “infinite” auto-scaling
● Focus on business logic, not on servers
daaaa!

18. Lambda as micro-services
● Events are first-class citizens
● Every lambda scales independently
● Agility (develop features quick and in an isolated fashion)
Classic micro-services concerns:
● Granularity (how to separate features? BDD? Bounded Contexts?)
● Orchestration (dependencies between lambdas, service discovery…)

19. Anatomy of a Lambda in Node.js

20. Some use cases
● REST over HTTP (API Gateway)
● SNS messages
● Schedule/Cron
● DynamoDB data changes
● S3 files changes
● IoT

21. HTTP REQUEST - API Call
POST /hello/Elvis?foo=bar
{
“test”: “body”
}

22. Security
Authentication
Authorization
Sensitive data

23. Authentication
(JWT tokens) Custom
“authorizer lambda”
“Who is the current user?”

24. Users
API 1
Authentication
Authorizer
API 2
API 3
Credentials
API
Request
JWT token
JWT tokenVerify credentials
Validate token
& extract userId
User: Podge
Pass: Unicorns

25. Authorization
“Can Podge trade for Account23 ?”
User Action Resource

26. Authorization
● Custom ACL library imported by every lambda
● Built on top of:
○ node-acl
○ Knex.js
● Persistence in Postgres
User Action Resource
Podge trade Account23
Podge changeSettings Account23
Luciano delete *
... ... ...

27. ACL Lib Example

28. Environment variables

30. Quality
Testing
Continuous Integration
Deployment

31. Testing
JEST Node-TAP

32. Unit Testing
● Split business logic into small testable modules
● Use dependency injection for external resources
(DB, Filesystem, etc.)
● Mock dependencies with Sinon
● Aim for 100% coverage

33. ● Use child_process.exec to launch “serverless invoke
local”
● Use node-tap to make assertions on the output
● Test environment recreated locally with docker (e.g.
Postgres test DB).
● Cannot always test locally (Redshift, SNS, SQS…)
Functional Testing

34. Functional test with sls invoke local

35. Building the project

36. Babel custom preset (transpile to Node 4.3.2)

37. Build process
JSdoc
Up to 90% code size reduction
Using Webpack 2 “tree-shaking”

38. Continuous Integration
Git-Flow
● Check code style (ESLint)
● Run unit tests
● Build the project
● Run functional tests
● If commit is on “master”:
Create deployable artifact
● Develop features/fixes locally
using git branches
● Push to GitHub

39. Deployments
local test dev qa production
Deploy lambdas and React apps

42. Downloading the artifact from CircleCI

43. Deploying the code using Serverless

44. Developer Experience
Local development
Debugging
Monitoring

45. ● Develop locally - Invoke local.
● Deploy to AWS.
● Invoke on AWS and stream the logs.
● This is a SLOW feedback loop.
● Lots of requests to improve local dev.
● Plugins are helping to improve this.
Local development

46. Serverless Plugins

47. Serverless Offline

48. Serverless Api Gateway Logs

49. Serverless Mocha

50. Monitoring

62. Cost
The formula
Cost forecasts

63. $ = Time * Memory * Invocation

64. http://serverlesscalc.com

65. Our current Lambda costs

66. Lessons learned
( we found out…)

67. DEBUGGING

68. How do we do debug then...
● console.log …
● Using debug module
● Enabling detailed logs only when needed (per feature)

70. Learn Cloudformation

73. Api Gateway
Custom domain mapping
It is a manual step…

74. With Api Gateway...
You have 10 30 seconds!

75. AWS Lambda limitations
● 512 MB of TMP space
● Lambdas can only execute for 300 seconds
● Request Response size of 6 MB
● Event size of 128 Kb
● Max 50 MB for deployment package

76. AWS Soft limits
http://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html

77. Cold starts...
☃

78. https://www.iopipe.com/2016/09/understanding-aws-lambda-coldstarts/

79. serverlessbeer.com
is a real thing!
Well… it’s not, but it’s a good idea!
(read the tutorial)

80. Recap
● Serverless architectures are COOL!
○ Infinite scalability at low cost
○ Managed service
○ Still has some limitations
● Managing a project might be hard but:
○ Technology progress and open source projects (Serverless.com) will make
things easier

81. Planet 9 Energy tech-team
We are
hiring :)
Peter Saŝa Podge Luciano YOU?
Alberto Joe DomagojHugh Gus Ruth
Check out
Loki.js

82. THANK YOU
Planet9energy.com - tech@planet9energy.com
Special Thanks to @johnbrett_
Use the ,
Luke!