One of the most common performance issues in serverless architectures is elevated latencies from external services, such as DynamoDB, ElasticSearch or Stripe. In this webinar, we will focus on how to monitor, detect and fix latency issues that arise when our Lambda functions need to talk to other services.

1. Debugging AWS Lambda
Performance Issues

2. Yan Cui
http://theburningmonk.com
@theburningmonk
Developer Advocate @
Independent Consultant
AWS user since 2009
since 2018
yan@lumigo.io

3. Aviad Mor
@aviadmor
Co-Founder & CTO @
aviad@lumigo.io

7. observation
majority of performance problems originates from a function’s integration points

8. macro
how well is this service performing in general?
micro
why did this transaction perform poorly?

9. macro
how well is this service performing in general?
micro
why did this transaction perform poorly?
identify systemic
issues

10. macro
how well is this service performing in general?
micro
why did this transaction perform poorly?
why did this user
get a bad exp?

11. In control theory, observability is a measure of how well
internal states of a system can be inferred from knowledge
of its external outputs.

12. In control theory, observability is a measure of how well
internal states of a system can be inferred from knowledge
of its external outputs.
what do we need
to collect?

13. API Gateway Lambda DynamoDB

14. API Gateway Lambda DynamoDBhow long did this
req take?

15. what is the state
of the world?

16. In control theory, observability is a measure of how well
internal states of a system can be inferred from knowledge
of its external outputs.
what are the most
important outputs to
collect?

17. macro
how well is this service performing in general?
micro
why did this transaction perform poorly?

18. API Gateway API GatewayLambda Lambda DynamoDB
Service A Service B

19. API Gateway API GatewayLambda Lambda DynamoDB
Service A Service B
how long did service
B took to respond?

20. API Gateway API GatewayLambda Lambda DynamoDB
Service A Service B
how long did service
B took to respond?
was DynamoDB slow?
was it a cold start?
could it be API
Gateway?

21. How do we collect these external output to help us infer the
application’s internal state to debug performance issues?

22. log them manually

23. log them manually
turn these into metrics
using Metric Filter

24. Embedded Metric Format (EMF)

25. intercept all HTTP requests

26. X-Ray

27. Lumigo

28. DEMO!

29. API Gateway Lambda
GET /
Lambda
POST
/restaurants/search
API Gateway Lambda
GET /
get-index get-restaurants
search-restaurants
DynamoDB
EventBridge Lambda
notify-restaurant
Lambda
place-order
POST /order
SNS

30. github.com/theburningmonk/debugging-lambda-perf-issues-demo

31. get 15% off

33. @theburningmonk
theburningmonk.com
github.com/theburningmonk
yan@lumigo.io