Try it now ! : https://welovedevs.com/app/companies Learn more : Serverless CPH : https://serverlesscph.dk/ WeLoveDevs.com : https://welovedevs.com/ Spread the love <span class="emoji-outer emoji-sizer"><span class="emoji-inner" style="background: url(chrome-extension://immhpnclomdloikkpcefncmfgjbkojmh/emoji-data/sheet_apple_32.png);background-position:50% 28.025851938895418%;background-size:5418.75% 5418.75%" data-codepoints="1f499"></span></span>

1. Serverless CPH
15-04-18
Deploying machine learning in
production without servers
Damien Cavaillès
@TheDamfr

2. Serverless CPH
15-04-18
About
We built a 10K MAU WebApp using Firebase, some Javascript and as less
servers as possible.
Why ?
How ?
What are the anti-patterns ? Good surprises ?

3. Serverless CPH
15-04-18
App’s "Business" Case

4. Serverless CPH
15-04-18
What is the app about ?
8000+ Content pages
Companies, jobs, landing pages : developer’s "god mode" on job market to find a company he will loves to work at

5. Serverless CPH
15-04-18
Dev likes companies he would like to work at
App is learning what companies you like
and predict companies you will like
You can also use a search engine,
a map and some editorial content
to find companies
1

6. Serverless CPH
15-04-18
When ready, a developer a can fill a profile
Developer's profiles are anonymous
and confidentials
It's a safe place where a developer can
explain her career path, what she has
learned and built.
But also what are her expectations
about is future positions.
2

7. Serverless CPH
15-04-18
Conversations with real people working at thoses companies
Conversations are real time and can
support multiple agents
You get one-click link by SMS, mobile
layout is available and the green dot shows
who is online
3

8. Serverless CPH
15-04-18
250 Hires in 2018 !
Mostly in France and with French companies
more great things to come

9. Serverless CPH
15-04-18
Context

10. Serverless CPH
15-04-18
We founded the company as two developers
Content, users,
marketing, sales,
operations
More SaaS less work
We wanted to spend time
with users and ship features
on a daily basis.
Not spend hours
maintaining a large app and
systems we have built
ourselves.
It's okay to have vendor-
locking (tight coupling)
and more expensives
services if it reduces the
human time needed at
operation.

11. Serverless CPH
15-04-18
From idea to production in a day
● Developer’s journey…
Talk with a customer over livechat in morning
Build a PoC during day
● Deploy in production and give feedback to user
● Go Home !
Whether there is 2 or 20 of us

12. Serverless CPH
15-04-18
Search
First production app
Web
Process
Worker
Processes
Jobbing

13. Serverless CPH
15-04-18
What was key ?
Services we relied
on early
ElasticSearch
Allowed us to provide a good search experience thanks to
SearchKit. Bonsai provided a free shared instance to startup.
At launch we only had one index and 20 documents.
Firebase
Allowed us to quickly setup authentication and be realtime at
day0.
Users testified : "this looks like MeteorJS"
Redux was key to solve the callback spaghetti
Clever Cloud
Does more than Hosting
It also operate continuous integration by building and deploying
every commit on Github's master. #WebpackInTheCloud

14. Serverless CPH
15-04-18
What was hell ?
Requiring
monitoring and fixing
Redis
Redis instance as too many clients.
Provided Redis instance is now too small. Migration to a larger
one is required
Node Worker Processes
Redis instance refused connection, process is stalled.
Process crashed, PM2 rebooted.
Flashlight was crashed, updates has not been pushed to
ElasticSearch
COMMAND Jobbing over HTTP
Client create jobs via an express API.
Consistency hell : "Didn't got an HTTP reply, should I resend ?"
Authentication hell : "Okay got your request, need to check your
credential"
XHR

15. Serverless CPH
15-04-18
How is Firebase Working
Large JSON
Tree
Websocket
Listen for a node in the tree
Ref : root/ref/is/a/path
Downstream updates
Path : root/ref/is/a/path
Event : updated/created/deleted

16. Serverless CPH
15-04-18
How is Firebase Working
Also Functions SDK provide authentication out of the box
Triggers
Node Updated
Node created
Node Deleted
User registerd
User deleted
…

17. Serverless CPH
15-04-18
The first Cloud Function
That was easy to deploy ! From scratch tooks 2h on a saturday

18. Serverless CPH
15-04-18
"That job is broken"
Rewrote it as a Firebase Cloud Function
Easier and faster to deploy than fix production server

19. Serverless CPH
15-04-18
Hey Firebase Functions are still Beta
Okay !
They are going to fail.
Make sure our system is resilient enough.

20. Serverless CPH
15-04-18
There is no more jobs useful
Shutting down redis
Shutting down PM2 and worker process on node app
Removing API endpoints on express server
Few month later...

21. Serverless CPH
15-04-18
New services !
New Kids On The
Block
Algolia : Search as a Service
"Hey ! I tried Algolia and reproduced our Elastic Search setup in
a 4h"
Shutting down Elastic Search and Bonsai
Sendgrid : Transactional Emails
Templates using a Wysiwyg.
Locked-in for the best. Nobody likes to code emails anyway.
Twilio for SMS
"Wouldn't be great to send a transactionnal SMS at this moment
?
Let me try Twilio ? … `firebase deploy`...
Done, works great "
""

22. Serverless CPH
15-04-18
Search
New Service Map

23. Serverless CPH
15-04-18
Machine Learning Usecase
● From Local to Functions

24. Serverless CPH
15-04-18
What are we looking to predict ?
a Developer
a Company
a Company
a Company
a Company
[
…
Prediction :
=>

25. Serverless CPH
15-04-18
This kind of algorithm are known as :
Association Rules Mining
What are we looking to predict ?

26. Serverless CPH
15-04-18
Association Rules Mining
What are we looking to predict ?

27. Serverless CPH
15-04-18
How to
implement ?
Train LSTM with Watson 0,50€ / 1000 predictions
Training : $0.50/h
Train LSTM with Tensorflow
Easy to deploy on Google Cloud AI
But, at the time, only in python
Implement FP Growth in Node
Run it as a Cloud Function
It took me two weeks to get
access to documentation.
Training : $0.49/h
Batch Prediction :
$0.09262/h
Online Predictions :
$0.3/h
Compute time :
$0,1/h

28. Serverless CPH
15-04-18
Building the tree
Machine Learning + In-production learning
Pre-flight Checklist
1. 1. Preprocess (SPSS, Node, Python)
2. Train a model
3. Produce a batch of predictions
4. Control and report training
2. On the fly : Predict

29. Serverless CPH
15-04-18
FP Growth in detail
Building this locally
1. Preprocess => Node Script => Transactions as JSON
2. Train a model => Node Script => FPtree as JSON
3. Produce a batch of predictions => Node Script => JSON
4. Control and report training => Node Script => Console output
On the fly : Predict => Traverse FPTree with a Node Script =>
Console Output

30. Serverless CPH
15-04-18
FP Growth in detail

31. Serverless CPH
15-04-18
FP Growth in detail

32. Serverless CPH
15-04-18
Function-based FPGrowth Training
Training
Function
Fetch all likes
Preprocess
Make tree
Store Tree
Batch
Predictions Store Predictions
Control
Store Distribution
Remote Cron Trigger
Distribution
update
Trigger
Reporting
email
Boostrap
DataStore Bootstrap

33. Serverless CPH
15-04-18
Function-based On the fly prediction
On the fly
function
Fetch Tree
Traverse Tree
Store Predictions
Like written trigger
Update Client

34. Serverless CPH
15-04-18
Function-based On the fly prediction
Démo (not live)

35. Serverless CPH
15-04-18
Performance ?
Execution time : 30 sec by call
Memory : 800Mo on average
0,02$ a day – 144 000 prediction a day

36. Serverless CPH
15-04-18
Back to the Big Picture

37. Serverless CPH
15-04-18
Some Scale
numbers
Firebase functions
400K invocation by month
98 Functions in production
Longest 25min on HTTP Trigger
Firebase Realtime Database
60 Gb Bandwidth / Month
Max simultaneous connection : 450
Algolia
18,547 Records
9,805 Queries over 24h - 90% Under 3 ms
570+ Indexes
Cost 60$ / Month
Still free tier
Cost 35$ / Month

38. Serverless CPH
15-04-18
Some Scale
numbers
Sendgrid
1 Dedicated IP
30K email a month
Twilio
400 SMS a week (peak)
400 SMS a month (average)
Clever Cloud
1 XS instance + auto scalability
+ 4 apps & 9 addons for marketing mainly
Cost 50$ / Month
Cost App : 12€ / Month | Overall 37€
Cost 80$ / Month

39. Serverless CPH
15-04-18
Think bigger picture
There is many other "Organisational" services

40. Serverless CPH
15-04-18
Staging ?
Every service should have a staging version you can test a
staging function against

41. Serverless CPH
15-04-18
Learnings : Do and Don’t
● Patterns and Anti-patterns we have seen in production

42. Serverless CPH
15-04-18
Do : Use HTTP Triggers with an external Cron Service
We are using SetCronJob.com
Warning : Optimization !
This data fetch used to consume 6Gb of
bandwidth every day
Once optimized it wasn’t significant anymore
(need an .indexOn in database rules)
Firebase SDK will show a warning in logs
Firebase CLI offers an handy profiler
1

43. Serverless CPH
15-04-18
The Cold Start is going to give you a very bad experience !
Better run a Express server somewhere on a PaaS.
2
Don’t : Use HTTP Triggers as a WebServer

44. Serverless CPH
15-04-18
Hard to reproduce and debug
Easy to have side effects
Check events and data before executing the core of the function
3
Don’t : Triggers Cloud Functions recursively

45. Serverless CPH
15-04-18
If you can reproduce each jobs it’s okay.
Make sure the writing of the job is not triggering an other functions
Pub/Sub is also a good option
4
Do : Use Firebase RDB as a Queue

46. Serverless CPH
15-04-18
It’s easier to manage side effects if it’s all contained in one function
5
Don’t : Two functions aren’t better than one

47. Serverless CPH
15-04-18
Do : Realtime feedback in a function
It’s easy to push an update to firebase
6
Acknowledge : ”received”
Fetch data from external
Parse data
Persist “pushing”
Done

48. Serverless CPH
15-04-18
Do : Have an Express process listen to Firebase DB
Very efficient for response time
(No need to query a SGBD)
Need to be carefull about process
memory
Pretty convenient for Server Side
Rendering
Warning : React ServerSide rendering
mean you are going to
Babel/Webpack your server
7

49. Serverless CPH
15-04-18
Don’t : Create Race Condition
For concurrency purpose :
Functions can be executed multiple
time in the same runtime or in the
same container (you shouldn't rely on
it).
Because of cold start and resource
sharing, concurrent functions can
show race condition
If you have to :
Look for Transactions on RDB
8

50. Serverless CPH
15-04-18
Do : Use RDB for Uncoupling & Function as Couplers
Get out of the Client/Server logic : Stripe Checkout use case
9
listen CustomerId
write customerId:tbd
No
CustomerId
Write Trigger Function
(hold Stripe
SK)
Fetch
write
customerId
Render
Get Payment
Source write Source
Confirmation Write Request
Subscription
Function
(hold Stripe
SK)
Write Trigger Create
Sub
write
currentPlan
Render

51. Serverless CPH
15-04-18
Do : Functions are great for ETL
Multiposting Usecase : Extract / Load / Transform
10
Extract (Parse) Transform
Extract and Transform
Load
Load
Trigger
Email
Email
Email
Load

52. Serverless CPH
15-04-18
Do and Don’t : Summary
1. Do : Use HTTP Triggers with an external Cron Service
2. Don’t : Use HTTP Triggers as Express servers
3. Don’t : Triggers Cloud Functions in a recursive way
4. Do : Use Firebase RDB as a Queue
5. Don’t : Two functions aren’t better than one
6. Do : Realtime feedback in a function
7. Do : Have an Express process listen to Firebase DB
8. Don’t : Create Race Condition
9. Do : Use RDB for Uncoupling & Function as Couplers
10. Do : Use Functions as ETL

53. Serverless CPH
15-04-18
Glad to help you further !
Damien Cavaillès
@TheDamfr
damien@welovedevs.com