SlideShare uma empresa Scribd logo
1 de 52
Baixar para ler offline
PLANNING FOR THE
                                  HORIZONTAL
                                     SCALING NODE.JS APPLICATIONS




                        TheReddest           Brandon Cannaday       brandon@modulus.io

Thursday, April 4, 13
ME




                        HOSTING, DATA, STATS FOR NODE.JS

                                   modulus.io



Thursday, April 4, 13
WHEN TO SCALE?


                        1. RESPONSE TIMES
                        2. CPU USAGE
                        3. CONCURRENT CONNECTIONS




Thursday, April 4, 13
NODEFLY.COM




Thursday, April 4, 13
STARTING POINT
                           mydomain.com




                               > NODE


                              SERVER




Thursday, April 4, 13
NODE TWEAKS

                                     CONCURRENT OUTGOING
                                       CONNECTION LIMIT

                        http.globalAgent.maxSockets = Number.MAX_VALUE;




Thursday, April 4, 13
HURTLE: THE SERVER

                        LINUX CONFIGURATION
                          1. FILE-MAX
                          2. SOMAXCONN
                          3. ULIMIT



Thursday, April 4, 13
FILE-MAX

                        SYSTEM FILE DESCRIPTOR LIMIT

                        1. Run sysctl -w fs.file-max=65535
                        2. Run sysctl -p




Thursday, April 4, 13
SOMAXCONN

                             SOCKET LISTEN QUEUE LENGTH

                        1. Run sysctl -w net.core.somaxconn=65535
                        2. Run sysctl -p




Thursday, April 4, 13
ULIMIT
                        PER PROCESS FILE DESCRIPTOR LIMIT
                          1. Edit /etc/security/limits.conf
                          2. Add the following:
                          *      soft   nofile    65535
                          *      hard   nofile    65535
                          root   soft   nofile    65535
                          root   hard   nofile    65535
Thursday, April 4, 13
RUNNING SMOOTH

                            mydomain.com




                                > NODE


                               SERVER




Thursday, April 4, 13
HURTLE: THE CPU

                           BUY A BIGGER BOX



                         > NODE         > NODE


                        SERVER
                                        SERVER
                         1 CORE
                                        4 CORES

Thursday, April 4, 13
MULTICORE NODE

                           100%




                        USAGE




                                  1   2      3   4
                                      CORE


Thursday, April 4, 13
CLUSTER MODULE
                           mydomain.com




                            > NODE    > NODE



                            > NODE    > NODE



                                 SERVER




Thursday, April 4, 13
CLUSTER EXAMPLE

             var cluster = require('cluster');            The Cluster Module
             var http = require('http');
             var numCPUs = require('os').cpus().length;

             if(cluster.isMaster) {
               for(var i = 0; i < numCPUs; i++) {
                 cluster.fork();
               }
             }
             else {
               http.createServer(function(req, res) {
                 res.writeHead(200);
                 res.end('Hello World.');
               }).listen(80);
             }


Thursday, April 4, 13
CLUSTER EXAMPLE

             var cluster = require('cluster');
             var http = require('http');
             var numCPUs = require('os').cpus().length;

             if(cluster.isMaster) {
               for(var i = 0; i < numCPUs; i++) {
                 cluster.fork();                          Fork Children
               }
             }
             else {
               http.createServer(function(req, res) {
                 res.writeHead(200);
                 res.end('Hello World.');
               }).listen(80);
             }


Thursday, April 4, 13
CLUSTER EXAMPLE

             var cluster = require('cluster');
             var http = require('http');
             var numCPUs = require('os').cpus().length;

             if(cluster.isMaster) {
               for(var i = 0; i < numCPUs; i++) {
                 cluster.fork();
               }
             }
             else {
               http.createServer(function(req, res) {     Handle Requests
                 res.writeHead(200);
                 res.end('Hello World.');
               }).listen(80);
             }


Thursday, April 4, 13
CLUSTER LISTEN


                                 listen(...)

                        WORKER                 MASTER

                                 Handle




Thursday, April 4, 13
ROLLING UPDATES

                        1. UPDATE SCRIPT
                        2. WORKER -> STOP LISTENING
                        3. KILL WORKER
                        4. CALL FORK() AGAIN


Thursday, April 4, 13
CLUSTER MODULE
                           mydomain.com




                            > NODE    > NODE



                            > NODE    > NODE



                                 SERVER




Thursday, April 4, 13
HURTLE: SHARED STATE


                                          > NODE    > NODE

                        NO SHARED STATE
                                          > NODE    > NODE



                                               SERVER




Thursday, April 4, 13
INSTALL REDIS


                           > NODE    > NODE



                           > NODE    > NODE


                                REDIS


                                SERVER




Thursday, April 4, 13
EXAMPLE 1: SESSION
                                     MEMORY STORE

                          var express = require('express'),
                              app = express();


                          app.use(express.cookieParser());
                          app.use(express.session({
                            secret: 'My Cookie Signing Secret'
                          }));

                          app.get('/', function(req, res) {
                            req.session.somekey = 'some value';
                          });




Thursday, April 4, 13
EXAMPLE 1: SESSION
                                               REDIS STORE

                        var express = require('express'),
                            RedisStore = require('connect-redis')(express),
                            app = express();


                        app.use(express.cookieParser());
                        app.use(express.session({
                          store: new RedisStore({ host: 'localhost', port: 6379 }),
                          secret: 'My Cookie Signing Secret'
                        }));

                        app.get('/', function(req, res) {
                          req.session.somekey = 'some value';
                        });



Thursday, April 4, 13
EXAMPLE 2: SOCKET.IO

                        var   RedisStore = require('socket.io/lib/stores/redis')
                          ,   redis = require('socket.io/node_modules/redis')
                          ,   pub    = redis.createClient()
                          ,   sub    = redis.createClient()
                          ,   client = redis.createClient();

                        io.set('store', new RedisStore({
                          redisPub : pub
                        , redisSub : sub
                        , redisClient : client
                        }));


                        https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO


Thursday, April 4, 13
RUNNING SMOOTH
                            mydomain.com




                            > NODE    > NODE



                            > NODE    > NODE


                                 REDIS


                                 SERVER


Thursday, April 4, 13
LAST HURTLE: HORIZONTAL


                           > NODE    > NODE   > NODE    > NODE



                           > NODE    > NODE   > NODE    > NODE


                                REDIS              REDIS


                             APP SERVER A       APP SERVER B




Thursday, April 4, 13
SEPARATE REDIS

                        > NODE    > NODE            > NODE    > NODE



                        > NODE    > NODE            > NODE    > NODE


                          APP SERVER A                APP SERVER B




                                           REDIS

                                           SERVER

Thursday, April 4, 13
LOAD BALANCING

                   mydomain.com              LOAD BALANCER

                                                     SERVER



                                  > NODE    > NODE             > NODE    > NODE



                                  > NODE    > NODE             > NODE    > NODE


                                    APP SERVER A                 APP SERVER B




                                                      REDIS

                                                      SERVER



Thursday, April 4, 13
LOAD BALANCING


                         1. MANAGED
                         2. INSTALL ONE
                         3. WRITE YOUR OWN




Thursday, April 4, 13
NGINX

                        http {
                          upstream mydomain_com {
                            server host1.mydomain.com:80;
                            server host2.mydomain.com:80;
                          }                                       LOAD BALANCER

                            server {
                                                                     SERVER
                              listen 80;
                              server_name www.mydomain.com;
                              location / {
                                proxy_pass http://mydomain_com;
                              }
                            }
                        }



Thursday, April 4, 13
WRITE ONE




                        https://github.com/substack/bouncy


Thursday, April 4, 13
BOUNCY
              var bouncy = require('bouncy');                    bouncy module
              var hosts = [
                 'host1.mydomain.com',
                 'host2.mydomain.com'
              ];

              var count = 0;

              var server = bouncy(function(req, res, bounce) {

                   count++;
                   var host = hosts[count % hosts.length];

                   bounce(host, 80);

              });

              server.listen(80);


Thursday, April 4, 13
BOUNCY
              var bouncy = require('bouncy');

              var hosts = [                                      Server collection
                 'host1.mydomain.com',
                 'host2.mydomain.com'
              ];

              var count = 0;

              var server = bouncy(function(req, res, bounce) {

                   count++;
                   var host = hosts[count % hosts.length];

                   bounce(host, 80);

              });

              server.listen(80);


Thursday, April 4, 13
BOUNCY
              var bouncy = require('bouncy');

              var hosts = [
                 'host1.mydomain.com',
                 'host2.mydomain.com'
              ];

              var count = 0;

              var server = bouncy(function(req, res, bounce) {   Create server
                   count++;
                   var host = hosts[count % hosts.length];

                   bounce(host, 80);

              });

              server.listen(80);


Thursday, April 4, 13
BOUNCY
              var bouncy = require('bouncy');

              var hosts = [
                 'host1.mydomain.com',
                 'host2.mydomain.com'
              ];

              var count = 0;

              var server = bouncy(function(req, res, bounce) {

                   count++;
                   var host = hosts[count % hosts.length];

                   bounce(host, 80);                             Bounce request
              });

              server.listen(80);


Thursday, April 4, 13
AFFINITY


                        SESSION AFFINITY
                        STICKY SESSIONS
                          SEND THE SAME PERSON
                         BACK TO THE SAME SERVER



Thursday, April 4, 13
NGINX AFFINITY

                        http {
                          upstream mydomain_com {
                            sticky;
                            server host1.mydomain.com:80;
                            server host2.mydomain.com:80;
                          }

                            server {
                              listen 80;
                              server_name www.mydomain.com;
                              location / {
                                proxy_pass http://mydomain_com;
                              }
                            }
                        }



Thursday, April 4, 13
CUSTOM AFFINITY


                        req.headers['x-forwarded-for']
                        req.connection.remoteAddress




Thursday, April 4, 13
RUNNING SMOOTH

                   mydomain.com              LOAD BALANCER

                                                     SERVER



                                  > NODE    > NODE             > NODE    > NODE



                                  > NODE    > NODE             > NODE    > NODE


                                    APP SERVER A                 APP SERVER B




                                                      REDIS

                                                      SERVER



Thursday, April 4, 13
ROLLING UPDATES


                    1. REMOVE APP SERVER FROM LOAD BALANCER
                    2. UPGRADE APP SERVER
                    3. ADD BACK
                    4. REPEAT


Thursday, April 4, 13
SSL


                        TERMINATE EARLY




Thursday, April 4, 13
SSL

                                    LB               SSL                SSL TERMINATOR

                                           SERVER



                        > NODE    > NODE             > NODE    > NODE



                        > NODE    > NODE             > NODE    > NODE


                          APP SERVER A                 APP SERVER B




                                            REDIS

                                            SERVER



Thursday, April 4, 13
SSL

                        mydomain.com
                              80   443




                         LB             SSL



                               SERVER




Thursday, April 4, 13
STUD
                           EXAMPLE CONFIG FILE


                        frontend = [*]:443
                        backend = [127.0.0.1]:80
                        ssl = on
                        pem-file = "myCert.pem"



                            https://github.com/bumptech/stud

Thursday, April 4, 13
RUNNING SMOOTH W/SSL

                   mydomain.com               LB               SSL

                                                     SERVER



                                  > NODE    > NODE             > NODE    > NODE



                                  > NODE    > NODE             > NODE    > NODE


                                    APP SERVER A                 APP SERVER B




                                                      REDIS

                                                      SERVER



Thursday, April 4, 13
HUGE

                        LB            SSL            LB            SSL

                             SERVER                       SERVER




                                            REDIS

                                            SERVER




Thursday, April 4, 13
DNS


                        ROUND-ROBIN DNS
                         MULTIPLE RECORDS,
                           ONE DOMAIN




Thursday, April 4, 13
ROUND-ROBIN DNS


                        CLIENT 1   1. xxx.xxx.xxx.x
                                   2. xxx.xxx.xxx.y


                        CLIENT 2   1. xxx.xxx.xxx.y
                                   2. xxx.xxx.xxx.x


Thursday, April 4, 13
RUNNING SMOOTH

                          LB            SSL            LB            SSL

                               SERVER                       SERVER




                                              REDIS

                                              SERVER




Thursday, April 4, 13
BIG ENOUGH



                        > NODE


                        SERVER




Thursday, April 4, 13
BIG ENOUGH



                        > NODE


                        SERVER




Thursday, April 4, 13

Mais conteúdo relacionado

Mais procurados

FUSE Developing Fillesystems in userspace
FUSE Developing Fillesystems in userspaceFUSE Developing Fillesystems in userspace
FUSE Developing Fillesystems in userspace
elliando dias
 
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
康志強 大人
 
Running hadoop on ubuntu linux
Running hadoop on ubuntu linuxRunning hadoop on ubuntu linux
Running hadoop on ubuntu linux
TRCK
 
Domino9on centos6
Domino9on centos6Domino9on centos6
Domino9on centos6
a8us
 

Mais procurados (19)

PythonFuse (PyCon4)
PythonFuse (PyCon4)PythonFuse (PyCon4)
PythonFuse (PyCon4)
 
FUSE Developing Fillesystems in userspace
FUSE Developing Fillesystems in userspaceFUSE Developing Fillesystems in userspace
FUSE Developing Fillesystems in userspace
 
[Python] Quick book for dell switch_os10
[Python] Quick book for dell switch_os10[Python] Quick book for dell switch_os10
[Python] Quick book for dell switch_os10
 
Installing and Configuring Domino 10 on CentOS 7
Installing and Configuring Domino 10 on CentOS 7Installing and Configuring Domino 10 on CentOS 7
Installing and Configuring Domino 10 on CentOS 7
 
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
Hadoop 2.2.0 Multi-node cluster Installation on Ubuntu
 
Andresen 8 21 02
Andresen 8 21 02Andresen 8 21 02
Andresen 8 21 02
 
도커 없이 컨테이너 만들기 5편 마운트 네임스페이스와 오버레이 파일시스템
도커 없이 컨테이너 만들기 5편 마운트 네임스페이스와 오버레이 파일시스템도커 없이 컨테이너 만들기 5편 마운트 네임스페이스와 오버레이 파일시스템
도커 없이 컨테이너 만들기 5편 마운트 네임스페이스와 오버레이 파일시스템
 
D space manual
D space manualD space manual
D space manual
 
Solaris
SolarisSolaris
Solaris
 
Sun raysetup
Sun raysetupSun raysetup
Sun raysetup
 
The Ring programming language version 1.10 book - Part 92 of 212
The Ring programming language version 1.10 book - Part 92 of 212The Ring programming language version 1.10 book - Part 92 of 212
The Ring programming language version 1.10 book - Part 92 of 212
 
Guide to clone_sles_instances
Guide to clone_sles_instancesGuide to clone_sles_instances
Guide to clone_sles_instances
 
Most frequently used unix commands for database administrator
Most frequently used unix commands for database administratorMost frequently used unix commands for database administrator
Most frequently used unix commands for database administrator
 
SSD based storage tuning for databases
SSD based storage tuning for databasesSSD based storage tuning for databases
SSD based storage tuning for databases
 
Running hadoop on ubuntu linux
Running hadoop on ubuntu linuxRunning hadoop on ubuntu linux
Running hadoop on ubuntu linux
 
The State of Puppet
The State of PuppetThe State of Puppet
The State of Puppet
 
Web Server Free Bsd
Web Server Free BsdWeb Server Free Bsd
Web Server Free Bsd
 
ubunturef
ubunturefubunturef
ubunturef
 
Domino9on centos6
Domino9on centos6Domino9on centos6
Domino9on centos6
 

Destaque

A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
Tom Croucher
 
An Overview of HTML5 Storage
An Overview of HTML5 StorageAn Overview of HTML5 Storage
An Overview of HTML5 Storage
Paul Irish
 
Building servers with Node.js
Building servers with Node.jsBuilding servers with Node.js
Building servers with Node.js
ConFoo
 
Экономика продуктов и метрики (Илья Краинский, Magic Ink)
Экономика продуктов и метрики (Илья Краинский, Magic Ink)Экономика продуктов и метрики (Илья Краинский, Magic Ink)
Экономика продуктов и метрики (Илья Краинский, Magic Ink)
PCampRussia
 

Destaque (15)

Horizontally Scaling Node.js and WebSockets
Horizontally Scaling Node.js and WebSocketsHorizontally Scaling Node.js and WebSockets
Horizontally Scaling Node.js and WebSockets
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
Anatomy of a Modern Node.js Application Architecture
Anatomy of a Modern Node.js Application Architecture Anatomy of a Modern Node.js Application Architecture
Anatomy of a Modern Node.js Application Architecture
 
Progressive Advancement in Web8
Progressive Advancement in Web8Progressive Advancement in Web8
Progressive Advancement in Web8
 
Building and Scaling Node.js Applications
Building and Scaling Node.js ApplicationsBuilding and Scaling Node.js Applications
Building and Scaling Node.js Applications
 
Why i hate node js
Why i hate node jsWhy i hate node js
Why i hate node js
 
An Overview of HTML5 Storage
An Overview of HTML5 StorageAn Overview of HTML5 Storage
An Overview of HTML5 Storage
 
Dynamic Application Development by NodeJS ,AngularJS with OrientDB
Dynamic Application Development by NodeJS ,AngularJS with OrientDBDynamic Application Development by NodeJS ,AngularJS with OrientDB
Dynamic Application Development by NodeJS ,AngularJS with OrientDB
 
Building servers with Node.js
Building servers with Node.jsBuilding servers with Node.js
Building servers with Node.js
 
Scalability using Node.js
Scalability using Node.jsScalability using Node.js
Scalability using Node.js
 
Экономика продуктов и метрики (Илья Краинский, Magic Ink)
Экономика продуктов и метрики (Илья Краинский, Magic Ink)Экономика продуктов и метрики (Илья Краинский, Magic Ink)
Экономика продуктов и метрики (Илья Краинский, Magic Ink)
 
Architecting large Node.js applications
Architecting large Node.js applicationsArchitecting large Node.js applications
Architecting large Node.js applications
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
Clustering with Node.js
Clustering with Node.jsClustering with Node.js
Clustering with Node.js
 
Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...
Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...
Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...
 

Semelhante a Planning for the Horizontal: Scaling Node.js Applications (7)

Rails Intro & Tutorial
Rails Intro & TutorialRails Intro & Tutorial
Rails Intro & Tutorial
 
Node.js - A Quick Tour II
Node.js - A Quick Tour IINode.js - A Quick Tour II
Node.js - A Quick Tour II
 
Put on Your Asynchronous Hat and Node
Put on Your Asynchronous Hat and NodePut on Your Asynchronous Hat and Node
Put on Your Asynchronous Hat and Node
 
Arp security by_abdimuna_sep_28
Arp security by_abdimuna_sep_28Arp security by_abdimuna_sep_28
Arp security by_abdimuna_sep_28
 
Building Reusable Puppet Modules
Building Reusable Puppet ModulesBuilding Reusable Puppet Modules
Building Reusable Puppet Modules
 
Intravert atx meetup_condensed
Intravert atx meetup_condensedIntravert atx meetup_condensed
Intravert atx meetup_condensed
 
Cors
CorsCors
Cors
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Último (20)

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 

Planning for the Horizontal: Scaling Node.js Applications

  • 1. PLANNING FOR THE HORIZONTAL SCALING NODE.JS APPLICATIONS TheReddest Brandon Cannaday brandon@modulus.io Thursday, April 4, 13
  • 2. ME HOSTING, DATA, STATS FOR NODE.JS modulus.io Thursday, April 4, 13
  • 3. WHEN TO SCALE? 1. RESPONSE TIMES 2. CPU USAGE 3. CONCURRENT CONNECTIONS Thursday, April 4, 13
  • 5. STARTING POINT mydomain.com > NODE SERVER Thursday, April 4, 13
  • 6. NODE TWEAKS CONCURRENT OUTGOING CONNECTION LIMIT http.globalAgent.maxSockets = Number.MAX_VALUE; Thursday, April 4, 13
  • 7. HURTLE: THE SERVER LINUX CONFIGURATION 1. FILE-MAX 2. SOMAXCONN 3. ULIMIT Thursday, April 4, 13
  • 8. FILE-MAX SYSTEM FILE DESCRIPTOR LIMIT 1. Run sysctl -w fs.file-max=65535 2. Run sysctl -p Thursday, April 4, 13
  • 9. SOMAXCONN SOCKET LISTEN QUEUE LENGTH 1. Run sysctl -w net.core.somaxconn=65535 2. Run sysctl -p Thursday, April 4, 13
  • 10. ULIMIT PER PROCESS FILE DESCRIPTOR LIMIT 1. Edit /etc/security/limits.conf 2. Add the following: * soft nofile 65535 * hard nofile 65535 root soft nofile 65535 root hard nofile 65535 Thursday, April 4, 13
  • 11. RUNNING SMOOTH mydomain.com > NODE SERVER Thursday, April 4, 13
  • 12. HURTLE: THE CPU BUY A BIGGER BOX > NODE > NODE SERVER SERVER 1 CORE 4 CORES Thursday, April 4, 13
  • 13. MULTICORE NODE 100% USAGE 1 2 3 4 CORE Thursday, April 4, 13
  • 14. CLUSTER MODULE mydomain.com > NODE > NODE > NODE > NODE SERVER Thursday, April 4, 13
  • 15. CLUSTER EXAMPLE var cluster = require('cluster'); The Cluster Module var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Thursday, April 4, 13
  • 16. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); Fork Children } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Thursday, April 4, 13
  • 17. CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { Handle Requests res.writeHead(200); res.end('Hello World.'); }).listen(80); } Thursday, April 4, 13
  • 18. CLUSTER LISTEN listen(...) WORKER MASTER Handle Thursday, April 4, 13
  • 19. ROLLING UPDATES 1. UPDATE SCRIPT 2. WORKER -> STOP LISTENING 3. KILL WORKER 4. CALL FORK() AGAIN Thursday, April 4, 13
  • 20. CLUSTER MODULE mydomain.com > NODE > NODE > NODE > NODE SERVER Thursday, April 4, 13
  • 21. HURTLE: SHARED STATE > NODE > NODE NO SHARED STATE > NODE > NODE SERVER Thursday, April 4, 13
  • 22. INSTALL REDIS > NODE > NODE > NODE > NODE REDIS SERVER Thursday, April 4, 13
  • 23. EXAMPLE 1: SESSION MEMORY STORE var express = require('express'), app = express(); app.use(express.cookieParser()); app.use(express.session({ secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); Thursday, April 4, 13
  • 24. EXAMPLE 1: SESSION REDIS STORE var express = require('express'), RedisStore = require('connect-redis')(express), app = express(); app.use(express.cookieParser()); app.use(express.session({ store: new RedisStore({ host: 'localhost', port: 6379 }), secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); Thursday, April 4, 13
  • 25. EXAMPLE 2: SOCKET.IO var RedisStore = require('socket.io/lib/stores/redis') , redis = require('socket.io/node_modules/redis') , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient(); io.set('store', new RedisStore({ redisPub : pub , redisSub : sub , redisClient : client })); https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO Thursday, April 4, 13
  • 26. RUNNING SMOOTH mydomain.com > NODE > NODE > NODE > NODE REDIS SERVER Thursday, April 4, 13
  • 27. LAST HURTLE: HORIZONTAL > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE REDIS REDIS APP SERVER A APP SERVER B Thursday, April 4, 13
  • 28. SEPARATE REDIS > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVER Thursday, April 4, 13
  • 29. LOAD BALANCING mydomain.com LOAD BALANCER SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVER Thursday, April 4, 13
  • 30. LOAD BALANCING 1. MANAGED 2. INSTALL ONE 3. WRITE YOUR OWN Thursday, April 4, 13
  • 31. NGINX http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80; } LOAD BALANCER server { SERVER listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } Thursday, April 4, 13
  • 32. WRITE ONE https://github.com/substack/bouncy Thursday, April 4, 13
  • 33. BOUNCY var bouncy = require('bouncy'); bouncy module var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  • 34. BOUNCY var bouncy = require('bouncy'); var hosts = [ Server collection 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  • 35. BOUNCY var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { Create server count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13
  • 36. BOUNCY var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); Bounce request }); server.listen(80); Thursday, April 4, 13
  • 37. AFFINITY SESSION AFFINITY STICKY SESSIONS SEND THE SAME PERSON BACK TO THE SAME SERVER Thursday, April 4, 13
  • 38. NGINX AFFINITY http { upstream mydomain_com { sticky; server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } Thursday, April 4, 13
  • 39. CUSTOM AFFINITY req.headers['x-forwarded-for'] req.connection.remoteAddress Thursday, April 4, 13
  • 40. RUNNING SMOOTH mydomain.com LOAD BALANCER SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVER Thursday, April 4, 13
  • 41. ROLLING UPDATES 1. REMOVE APP SERVER FROM LOAD BALANCER 2. UPGRADE APP SERVER 3. ADD BACK 4. REPEAT Thursday, April 4, 13
  • 42. SSL TERMINATE EARLY Thursday, April 4, 13
  • 43. SSL LB SSL SSL TERMINATOR SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVER Thursday, April 4, 13
  • 44. SSL mydomain.com 80 443 LB SSL SERVER Thursday, April 4, 13
  • 45. STUD EXAMPLE CONFIG FILE frontend = [*]:443 backend = [127.0.0.1]:80 ssl = on pem-file = "myCert.pem" https://github.com/bumptech/stud Thursday, April 4, 13
  • 46. RUNNING SMOOTH W/SSL mydomain.com LB SSL SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVER Thursday, April 4, 13
  • 47. HUGE LB SSL LB SSL SERVER SERVER REDIS SERVER Thursday, April 4, 13
  • 48. DNS ROUND-ROBIN DNS MULTIPLE RECORDS, ONE DOMAIN Thursday, April 4, 13
  • 49. ROUND-ROBIN DNS CLIENT 1 1. xxx.xxx.xxx.x 2. xxx.xxx.xxx.y CLIENT 2 1. xxx.xxx.xxx.y 2. xxx.xxx.xxx.x Thursday, April 4, 13
  • 50. RUNNING SMOOTH LB SSL LB SSL SERVER SERVER REDIS SERVER Thursday, April 4, 13
  • 51. BIG ENOUGH > NODE SERVER Thursday, April 4, 13
  • 52. BIG ENOUGH > NODE SERVER Thursday, April 4, 13