Scaling API-first – The story of a global engineering organization
Serverless, The Middy Way - Workshop
1. ServerlessServerless
The "Middy" wayThe "Middy" way
—
Workshop
loige.link/middy-way
Community Day Dublin
October , 1st 2019
Luciano Mammino
@loige
Wi-Fi:
DC_Conference
October-2019
1
13. What is serverlessWhat is serverless
Compute as functions (FaaS)Compute as functions (FaaS)
Event-based modelEvent-based model
@loige 5
14. Why serverless is goodWhy serverless is good
Focus on business logicFocus on business logic
ScalabilityScalability
Pay per usage model (compute time * memory)Pay per usage model (compute time * memory)
Managed infrastructureManaged infrastructure
Forces you to think micro-servicesForces you to think micro-services
@loige 6
15. My Serverless experienceMy Serverless experience
Open source
A semi-automated weekly newsletter ( )
A middleware framework for AWS Lambda ( )
Enterprise
Trading, Billing engine, Market data aggregator solutions ( )
Big data pipeline to make network metadata searchable ( )
Organised various workshops around Europe ( )
Fullstack bulletin
middy.js.org
ESB
Vectra.ai
Serverlesslab.com
@loige 7
16. Let's get this party startedLet's get this party started
@loige 8
30. ⏱ 20 mins warm up ♀ ⏱ 20 mins warm up ♀
1. Use the APIs to create a dashboard that contains:1. Use the APIs to create a dashboard that contains:
An IrishRail station widget
A LUAS stop widget
A DublinBus stop widget
(there are utility scripts to list all the stops and stations, check the README)
2. Have a look at the code2. Have a look at the code
If you find bugs, please send a or report anPR issue
@loige 21
33. exports.myLambda = function (
event,
context,
callback
) {
// get input from event and context
// use callback to return output or errors
}
Anatomy of a Node.js lambda on AWSAnatomy of a Node.js lambda on AWS
@loige 24
34. (event, context, callback) => {
// decrypt environment variables with KMS
// deserialize the content of the event
// validate input, authentication, authorization
// REAL BUSINESS LOGIC
// (process input, generate output)
// validate output
// serialize response
// handle errors
}
A typical "REAL" Lambda functionA typical "REAL" Lambda function
@loige 25
35. (event, context, callback) => {
// decrypt environment variables with KMS
// deserialize the content of the event
// validate input, authentication, authorization
// REAL BUSINESS LOGIC
// (process input, generate output)
// validate output
// serialize response
// handle errors
}
A typical "REAL" Lambda functionA typical "REAL" Lambda function
@loige 25
36. (event, context, callback) => {
// decrypt environment variables with KMS
// deserialize the content of the event
// validate input, authentication, authorization
// REAL BUSINESS LOGIC
// (process input, generate output)
// validate output
// serialize response
// handle errors
}
A typical "REAL" Lambda functionA typical "REAL" Lambda function
@loige 25
37. (event, context, callback) => {
// decrypt environment variables with KMS
// deserialize the content of the event
// validate input, authentication, authorization
// REAL BUSINESS LOGIC
// (process input, generate output)
// validate output
// serialize response
// handle errors
}
A typical "REAL" Lambda functionA typical "REAL" Lambda function
LOTS of BOILERPLATE
@loige 25
65. When an error happens...When an error happens...
Flow is stopped
First middleware implementing `onError` gets control
It can choose to handle the error, or delegate it to the next handler
If the error is handler a response is returned
If the error is not handled the execution fails reporting the unhandled error
@loige 33
66. Writing a middlewareWriting a middleware
const myMiddleware = (config) => {
// might set default options in config
return ({
before: (handler, next) => {
// might read options from `config`
},
after: (handler, next) => {
// might read options from `config`
},
onError: (handler, next) => {
// might read options from `config`
}
})
}
module.exports = myMiddleware
@loige
34
67. Inline middlewaresInline middlewares
const middy = require('@middy/core')
const handler = middy((event, context, callback) => {
// do stuff
})
handler.before((handler, next) => {
// do something in the before phase
next()
})
handler.after((handler, next) => {
// do something in the after phase
next()
})
handler.onError((handler, next) => {
// do something in the on error phase
next()
})
module.exports = { handler }
@loige
35
69. Exercise 0Exercise 0: let's middify!: let's middify!
In the next exercises we will need the full power of middy, so to get started let's
just "middify" all our handlers
—
Note: handlers are defined in "src/handlers", and they get used in
"src/handler.js". You can either middify every single handler file or middify them
when used in src/handler.js. In the first case, make sure you middify the actual
handler function and not the "handler factory".
( )a solution
@loige 37
70. Exercise 1Exercise 1: body parsing: body parsing
We are manually deserializing the JSON input in our APIs.
We can simplify the code a bit by using the middleware.
—
Install @middy/http-json-body-parser and remove all the instances of
JSON.parse() in our handlers.
( )
http-json-body-parser
a solution
@loige 38
71. What happens when one of our handlers crashes? We are actually not even
dealing with errors right now... by using the middleware we
will be able to handle all HTTP errors and return codes consistently.
—
Install @middy/http-error-handler and apply that to all our handlers.
( )
http-error-handler
a solution
Exercise 2Exercise 2: what if we fail?: what if we fail?
@loige 39
72. We want to make sure that data is validated before being written to the
database. We can use the middleware for this task!
—
Install @middy/validator and apply that to all our POST API handlers. You can
use to define your schemas or, if you are struggling, you can
find some schemas ready-made for you :)
Note: The validator should be added after the json-body-parser, so you can
validate individual fields of the input data.
( )
validator
jsonschema.net
here
a solution
Exercise 3Exercise 3: validate!: validate!
@loige 40
73. We have our dashboards fully working, but consuming them in JSON is definitely
not the friendliest experience! It would be much better to do in a properly
rendered HTML so that we could easily visualize them on our mobile device!
—
You can implement this in (at least) 2 ways :
Using the middleware and building a frontend somewhere like
or
Using the middleware and implementing a
specific rule for accept "text/html"
( - )
http-cors
codesandbox codepen
http-content-negotiation
a CORS solution a CodeSandbox React UI
Exercise 4Exercise 4: mobile rendering: mobile rendering
@loige 41
74. Did you notice that our APIs are fully open? Knowing the endpoint and the id,
anyone can basically change or delete the dashboards! That's unacceptable!
—
Find a way to secure the APIs.
One easy way could be to create a secret random token when a dashboard is
created the first time. Then you can create a custom middleware that checks
the secret every time someone is trying to modify or delete that dashboard!
Note: if you feel fancy you can use JWT tokens, in such case you could take
inspiration from the middleware!jwt-auth
Exercise 5Exercise 5: better safe than sorry: better safe than sorry
@loige 42
75. Sometimes our APIs are very slow. They take literally seconds to answer. This is
because of a common serverless issue known as " ". If you are
running for your train every morning, this is unacceptable! We should fix this...
—
Use the middleware to mitigate the cold start issue
Note: make sure to also setup the proper event for every API.
cold start problem
warmup
schedule
2
@loige
Exercise 6Exercise 6: stop the cold!: stop the cold!
43
76. Some of your Java friends would love to use these APIs to build a shiny SWING
UI... The problem is that they don't really like JSON and they would rather have a
good old XML-based API. You might think that this is actually their problem, but
also, you don't want to lose your friends...
—
Use the middleware to provide responses in XML for
accept "application/xml".
Note: You could use a library like to automate the conversion of
JavaScript objects to XML.
http-content-negotiation
json2xml
@loige
Exercise 7Exercise 7: legacy friendship: legacy friendship
44
77. In summaryIn summary
Serverless is cool, it helps you to build apps quickly and with a greater
focus on business logic, rather than on infrastructure!
Middy helps you to keep focusing on your business logic first
You can add extra behaviours with very minimal changes to your core
logic by introducing dedicated middlewares
You can easily share common functionality through middlewares
45
78. If you like MiddyIf you like Middy
Use itUse it
Give feedbackGive feedback
Contribute (I am looking for co-maintainers)Contribute (I am looking for co-maintainers)
Version 1.0 (stable) coming soonVersion 1.0 (stable) coming soon
46
80. CreditsCredits
Cover Image by from
A special thank you to all the amazing Middy users
and !
Thanks to for reviewing these
slides and to for finding lots of bugs!
TotumRevolutum Pixabay
contributors
@StefanoAbalsamo
@organicdelight
@loige 48