In HelNode November meetup, Tuomas Rinta (Development Director, Unity Technologies Helsinki) shares their five-year journey with Node.js, from a startup to a several hundred machine deployment one point was the biggest known Node.js deployment. This talk is a 100 minute war story on Node.js featuring culture and practices it takes to run operations on scale. The talks will not limit to Node.js only, but the giant machinery as a whole. More information on the event and the group: http://www.meetup.com/Helsinki-Node-js/

1. Five years of Node.js
Unity <3 Node.js, or do we…?

2. The Beginning

4. The Beginning

5. PHP + MySQL

6. PHP + MongoDB

7. Node.js + MongoDB

8. Late 2010:
Node.js 0.4
700M ads served per day

9. Thus began our 5-year
journey with Node.js

10. Initially, Node was used only
for APIs for ad delivery

11. New products and features
were an easy way to
transition

13. Backbone + Node.js +
MongoDB

14. But it was not always a
smooth ride…

15. Updating between Node
versions was horrible

16. We ended up having
different services run on
different Node versions

17. A bleeding-edge tech stack
will give you new challenges

19. Things we learned:
MongoDB shards well…
on paper

20. Things we learned:
It was easier to optimize code
and data layout than shard
MongoDB

21. Things we learned:
Handling memory leaks in
Node.js is a pain

22. Things we learned:
External dependencies,
especially NPM, can make your
deplos unreliable

23. While on the subject of
NPM(JS.org)…

24. One of the best parts of
Node.js is the community

25. One of the best worst parts of
Node.js is the community

26. NPM, like actually most
package managers, can be
a security issue

27. Are you doing this?
~1.2.3
or
^1.2.3

28. Are you doing this?
If you are, stop right now.

29. But you have no control over
dependencies of used
packages!

30. Worst case scenario
Your
application
Dependency
A
Dependency
B
Sub-
dependency
A.1
Sub-
dependency
A.2

31. Worst case scenario
Your
application
Dependency
A
Dependency
B
Sub-
dependency
A.1
Compromise
d package

32. Worst case scenario
Your
application
Dependency
A
Dependency
B
Sub-
dependency
A.1
Compromise
d package

33. npm shrinkwrap

34. The Present

35. Tech Stack
Node.js
MongoDB + Cassandra +
Redis
Angular.js + Bootstrap

36. Infrastructure
Everything on AWS
Docker for deployment

37. Teams
2010: 8 people in Helsinki
2015: 70+ people in Helsinki

38. Requirements
99.9% uptime
Continuous deployment
Handle massive scale

39. How are we handling that?

40. #1: Testing

42. For continuous deployment,
we must aim for ~100% test
coverage

43. Tests are our compiler type-
safety check

44. #2: Automated Deployment

45. Developers must be able to
handle deployment and
rollback independently

46. Auto-scaling must be able to
set up new instances in a few
minutes

47. Developer
commit
No.fies
of
commit
Clones
repository
Tests
run
Docker
container
pushed
to
private
registery
Deploy
Happiness

48. #3: Understanding the
weaknesses of Node.js

49. Leaking memory is easy, and
debugging memory leaks is
insanely difficult

50. Callback-hell makes
debugging painful

51. Node.js is single-threaded

52. Let’s talk about that for a
while…

57. Making sure your code yields
adds complexity you don’t
want

58. Measuring and monitoring
these spikes is a nightmare

59. How do you measure?

60. Single-threaded also means
that you cannot cache data
efficiently on the local
machine

61. We run 40 Redis instances to
act as data caches

62. … and we still run
Memcached locally in
addition to that

63. Writing CPU-optimized code
with Node.js is extremely
difficult

64. The Future

65. Would we choose Node.js
again?

66. Probably not.

67. Node.js is useful for
prototyping.

68. Node.js does create burden
in sustainable development.

69. Node.js requires a lot of
discipline from the
development team.

70. In the end: right tool for the
right task.

71. Thank you!
http://unity3d.com/helsinki