Serverless computing allows you to build and run applications and services without thinking about servers. Serverless applications don't require you to provision, scale, and manage any servers. However, under the hood, there is a sophisticated architecture that takes care of all the undifferentiated heavy lifting for the developer. Join Holly Mesrobian, Director of Engineering, and Marc Brooker, Senior Principal of Engineering, to learn how AWS architected one of the fastest-growing AWS services. In this session, we show you how Lambda takes care of everything required to run and scale your code with high availability

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A Serverless Journey:
Under the Hood of AWS Lambda
Marc Brooker
Senior Principal Engineer
Serverless
S R V 4 0 9
Holly Mesrobian
Director of Engineering
AWS Lambda

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
processes 4,000 requests
per second
ingests, analyzes and
stores 17+ petabytes of
data per season
processes half a trillion
validations of stock
trades daily
executes 16 million
requests a month
processes tens of
billions of data
points monthly
SERVERLESS AT SCALE IS THE NEW NORM
API traffic to register and license
more than 47 million driver
records in Great Britain,

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Load
Balancing

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Scaling
Up and
Down

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling
Failures

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lambda Handles
Load Balancing
Auto Scaling
Handling Failures
Security Isolation
Managing Utilization
(and many other things) for you

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Front End Invoke
Orchestrate both synchronous and
asynchronous Invokes

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Counting Service
Provides a region wide view of customer
concurrency to help enforce set limits

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Worker Manager
Tracks container idle and busy state and
schedules incoming invoke requests to
available containers

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Worker
Provisions a secure environment for customer
code execution

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Placement Service
Places sandboxes on workers to maximize
packing density without impacting customer
experience or cold-path latency

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Routing function traffic across hosts distributed
across Availability Zone

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Lambda customer
(Existing Worker,
New Sandbox)
Availability zone 2
Availability zone 1
Invoke
Front End
Invoke
Front End
Worker Mgr
Worker Mgr
Reserve Sandbox
Invoke
Worker
Worker
Worker
Init

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Lambda customer
(Existing Worker,
Existing Sandbox)
Availability zone 2
Availability zone 1
Invoke
Front End
Invoke
Front End
Worker Mgr
Worker Mgr
Reserve Sandbox
Invoke
Worker
Worker
Worker

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SERVERLESS CUSTOMERS

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“What took us just a few days to build using a serverless
solution based on AWS Lambda would have taken us six
months to build from scratch.Our CTO and the rest of the
project stakeholders were really happy with how much money
and time we saved.”
Nitin Mahajan
Executive Director for Service Engineering

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Provision function capacity when needed and
releasing when not needed

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Lambda customer
(New Function or
Scaling Up)
Availability zone 2
Availability zone 1
Invoke
Front End
Invoke
Front End
Worker Mgr
Worker Mgr
Reserve Sandbox
Invoke
Init
Placement
Claim Worker
Worker
Worker
Worker

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Availability zone 2Availability zone 1
Placement
Claim Worker
Worker Worker Worker Worker Worker
Placement
Claim Worker
Worker Worker Worker Worker Worker

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Availability zone 2Availability zone 1
Placement
Return Worker
Worker Worker Worker Worker Worker
Placement
Return Worker
Worker Worker Worker Worker Worker

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Cloud
Region
Lambda customer
Availability zone 2
Availability zone 1
Front End
Front End
Worker Mgr
Worker Mgr
Placement
Return Worker
Worker
Worker
Worker

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
SERVERLESS CUSTOMERS

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling Host and Availability Zone failure

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
With Lambda:
Always have a healthy host

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instance Instance Instance InstanceInstance Instance
AWS Cloud
Region
Availability zone 2Availability zone 1

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Instance Instance Instance InstanceInstance Instance
AWS Cloud
Region
Availability zone 2Availability zone 1

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Sandbox
Lambda Runtime
Your Code
One Function
Many
Accounts

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Virtual Devices
Device Emulation
Physical
Devices

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
virtio drivers
virtio host in Firecracker
Physical
Devices

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hardware
Host OS
Hypervisor
Guest OS
Virtual Devices
Device Emulation
Physical
Devices

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
% of Resources
Doing Useful Work
(vs. idle or waste)

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
With Lambda:
Pay only for useful work.

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Inside Lambda:
Optimize To Keep Servers
Busy

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bad:
60% 60% 60% 60% 60% 60% 60%

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Good:
99% 99% 99% 99% 0% 0% 0%
Cache Locality
Ability to Autoscale

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Server
Bad: Pack Server With One Workload
Workload
Workload
Workload
Workload
Workload
Workload

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Server
Better: Pack With Many Loads
Workload
Workload
Workload
Workload
Workload
Workload

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Server
Best: Placement Optimization
Workload
Workload
Workload
Workload
Workload
Workload

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Worker
Lambda
Function
ENI in
your VPC
Your VPC
Local
NAT

59. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Worker
Lambda
Function
ENI in
your VPC
Your VPC
Remote
NAT

60. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

61. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Firecracker Hypervisor vs. Others
↓
↓
↑

62. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Firecracker Unlocks
Higher Utilization and Scale

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

64. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Marc Brooker – Senior Principal Engineer, Amazon Serverless
Holly Mesrobian – Director of Engineering, Amazon Lambda

65. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.