Serverless is the new hotness. It works great for parallelized workflows and event-driven systems. But what happens when your boss comes along and needs you to build a serverless system that works with a SQL database? Or you need to scrape a website, but you need to rate limit your scraping? For that matter, how can you build a serverless state machine?

1. Square Peg Round Hole
Chase Douglas, CTO @ Stackery.io
SERVERLESS SOLUTIONS FOR NON-SERVERLESS PROBLEMS

2. Serverless Solutions For Non-Serverless Problems
• Working With SQL Databases
• Rate Limiting
• State Machines

3. Working With SQL
Databases
INTERFACING WITH NON-SERVERLESS TECHNOLOGIES

4. Working With SQL Databases
Scenario: PHB says you need to build an API
to access data in a MySQL database

6. Working With SQL Databases
1. Add my fav ORM and DB driver
2. Connect and query
3. Profit!

7. Working With SQL Databases
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host = "rds-instance-endpoint"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
sys.exit()
logger.info("SUCCESS: Connection to RDS mysql instance succeeded")
def handler(event, context):
"""
This function fetches content from mysql RDS instance
"""
...
HTTPS://DOCS.AWS.AMAZON.COM/LAMBDA/LATEST/DG/VPC-RDS-DEPLOYMENT-PKG.HTML

8. HERE BE DRAGONS!

9. Working With SQL Databases
HTTPS://WWW.DRUPAL.ORG/PROJECT/DRUPAL/ISSUES/930876

10. Working With SQL Databases

11. Working With SQL Databases

12. Working With SQL Databases
What’s going on?

13. Working With SQL Databases
Most SQL databases are not serverless-
friendly
They do not scale horizontally

14. Working With SQL Databases
Each DB connection is handled on an independent thread of
execution
Each thread of execution requires memory set aside for it
Switching execution between threads adds overhead
Most SQL databases limit concurrent clients

15. Working With SQL Databases
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host = "rds-instance-endpoint"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
sys.exit()
logger.info("SUCCESS: Connection to RDS mysql instance succeeded")
def handler(event, context):
"""
This function fetches content from mysql RDS instance
"""
...
HTTPS://DOCS.AWS.AMAZON.COM/LAMBDA/LATEST/DG/VPC-RDS-DEPLOYMENT-PKG.HTML
This is never closed until the
function instance is terminated

16. Working With SQL Databases
Each function has a set
of idle instances with
open connections
20/20
20/20
20/20
20/20
20/20
Total Connections: 100

17. Working With SQL Databases
40/40
15/20
15/20
15/20
15/20
Total Connections: 120

18. Working With SQL Databases
15/40
15/20
40/40
15/20
15/20
Total Connections: 140!

20. Working With SQL Databases
Solution I: Wait for AWS, Azure, GCP to solve the problem
HTTPS://WWW.FLICKR.COM/PHOTOS/MANDJ98/5079617428
Sir Aurora Serverless
d’AWS has no limits

21. Working With SQL Databases
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host = "rds-instance-endpoint"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def handler(event, context):
"""
This function fetches content from mysql RDS instance
"""
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
sys.exit()
logger.info("SUCCESS: Connection to RDS mysql instance succeeded”)
...
Solution II

22. Working With SQL Databases
Wait!

23. Working With SQL Databases
Doesn’t this add overhead on every invocation?

24. Working With SQL Databases
Yes, it does.
Measure the overhead.
It is likely within reasonable limits for your needs.

25. Rate Limiting
SERVERLESS SOLUTION IN ABSENCE OF A MANAGED SERVICE

26. Rate Limiting
Scenario:
• Marketing has syndicated content out
• Need to page scrape websites to find out where
the content is ranking
• Need to rate limit web scraping

27. Rate Limiting
Rate: Quantity / Time

28. Rate Limiting
Example
Desired Limit: 60,000 requests per hour
Per-Minute Limit: 1,000 requests per minute
Implementation: Each minute scrape 1000 pages

29. Rate Limiting
Time: Heartbeat Generator
• Consistent interval to ensure steady rate
• Providers
• AWS: CloudWatch Events
• Azure: Timer Triggers
• GCP: App Engine Cron + App Engine + Pub/Sub

30. Rate Limiting
Wait! How are we going to make 1,000
requests each minute?
Concurrency!

31. Rate Limiting
After measuring, we can safely do 20 requests
per minute per invocation
We need 1,000 / 20 = 50 concurrent functions

32. Rate Limiting
Solution I: Trigger functions manually
• Each minute, grab 1,000 pages to scrape
• Invoke 50 scraper functions concurrently
with 20 pages each

33. Rate Limiting
Solution I: Trigger functions manually
Pros
• Very configurable (batch size & concurrency)
• Pay-per-use cost

34. Rate Limiting
Solution I: Trigger functions manually
Cons
• Poor retry capabilities

35. Rate Limiting
Fillable Queue
• Contains an amount of work to perform
• Retryable
• Providers
• Queuing services
• Databases

36. Rate Limiting
Solution II: Use a partitioned queue (AWS Kinesis Streams or
Azure Event Hubs)
• Provision queue with 50 partitions for concurrency
• Each minute, add 1,000 pages to queue
• 50 Scraper functions will be invoked with a batch of ~20
pages each

37. Rate Limiting
Solution II: Use a partitioned queue
Pros
• Retryable
• Configurable (batching & concurrency)

38. Rate Limiting
Solution II: Use a partitioned queue
Cons
• Expensive (pay per partition)

39. Rate Limiting
Solution III: Use a Key-Value store with a stream (AWS
DynamoDB or Azure Cosmos DB)
• Each minute add 1,000 pages to KV store
• Each page record is batched into 20 page messages that
invoke the Scraper function
• Concurrency is possible, but not configurable

40. Rate Limiting
Solution III: Use a Key-Value store with a stream
Pros
• Retryable
• Somewhat configurable (batching)
• Queue visibility

41. Rate Limiting
Solution III: Use a Key-Value store with a stream
Cons
• Concurrency not configurable
• A little expensive (pay per KV store
throughput)

42. Rate Limiting
What should you choose?
If you’re ok with errors, use a spawner function
If you need precision control over retries and concurrency,
use a partitioned queue
If you are cost conscious, investigate a Key-Value store with
a stream, or use a spawner function with extra error handling

43. State Management
BRINGING NON-SERVERLESS CAPABILITIES TO SERVERLESS

44. State Management

45. State Management
Concurrency Coordination
State Transitions
Any serverless state machine can be built
with a combination of the above

46. State Management
Concurrency Coordination Requires External
State

47. State Management
Concurrency Coordination Example: Map / Reduce
• Add a record to a database with count of concurrent invocations
• Perform invocations
• At completion, invocations atomically reduce concurrent count
• If no more concurrent invocations, transition to next state

48. State Management
State Transitions Require Local State

49. State Management
State Transitions Example: Custom Retry Logic
• Attempt to execute function
• On failure, check “retries” property of the input message
• If retries > MAX_RETRIES, then report failure
• Otherwise, increment “retries” property in message, then reinvoke
self

50. State Management
Can we make this more general?

51. State Management
AWS Step Functions and Azure Logic Apps
But, they’re expensive for high-throughput
apps

52. State Management

53. Serverless Solutions For Non-Serverless Problems
• Working With SQL Databases
• Rate Limiting
• State Machines

54. Go Forth And Do
More With Serverless!

55. Thank you!
Chase Douglas, CTO @ Stackery.io
chase@stackery.io
@txase