2600Hz - Tuning Kazoo to 10,000 Handsets - KazooCon 2015
•Transferir como PPTX, PDF•
6 gostaram•5,649 visualizações
People love to talk about scale. Some vendors pitch that their systems easily support 100,000 simultaneous calls, or 500 calls per second, etc. The reality is, in the real world, people’s behaviors vary and the feature sets they use can cut these numbers down quickly. For example, ask that same vendor claiming 100,000 simultaneous calls if it can be done while call recording, call statistics and other features are turned on at the same time, and you’ll usually get a very different, cautious, qualified response. In this presentation, we'll show you how to set up your infrastructure to support 100,000 simultaneous calls.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a 2600Hz - Tuning Kazoo to 10,000 Handsets - KazooCon 2015
Semelhante a 2600Hz - Tuning Kazoo to 10,000 Handsets - KazooCon 2015 (20)
Mais de 2600Hz
Mais de 2600Hz (20)
Último
Último (20)
2600Hz - Tuning Kazoo to 10,000 Handsets - KazooCon 2015
- 1. PRESENTED BY: Tuning Kazoo to 10,000+ handsets Luis Azedo
- 2. @kazoocon@kazoocon About the Topic • People love to talk about scale. • Customer’s usage patterns are different. • No cookie-cutter solution.
- 4. @kazoocon@kazoocon General linux considerations • File Descriptors • Sockets • Processes • Limits • Logs
- 5. #kazoocon14 Let’s Talk About Each Component
- 6. @kazoocon@kazoocon Couch • Overprovisioning is good • Plan for # of accounts you’ll have in 1-2 years • Z, N, W parameters • Set these when you first start • Z = Number of Datacenters / Zones • N = Number of copies • W = Number of writes before it’s “good enough” to move on • Db tool can be used later to redo
- 7. @kazoocon@kazoocon Freeswitch • Bare metal vs VMs / docker • 5 x 200 calls vs 1 x 1000 calls • Sqlite vs mysql (memory locks) • mod_kazoo filters
- 8. @kazoocon@kazoocon Kamailio • TCP Workers • Shared Memory • db_text vs mysql (in memory) • Configure Zones • AMQP workers / consumers • Use ASYNC when possible
- 9. @kazoocon@kazoocon Kamailio #!substdef "!MY_AMQP_URL_1!kazoo://user:pass@amqp-01.lx1.com:5672!g" #!substdef "!MY_AMQP_URL_2! kazoo://user:pass@amqp-02.lx1.com:5672!g” #!substdef "!MY_AMQP_URL_3! kazoo://user:pass@amqp-03.lx1.com:5672!g” #!substdef "!MY_AMQP_URL_4!zone=lx2;kazoo://user:pass@amqp-01.lx2.com:5672!g" #!substdef "!MY_AMQP_URL_5! zone=lx2;kazoo://user:pass@amqp-02.lx2.com:5672!g” #!substdef "!MY_AMQP_URL_6! zone=lx2; kazoo://user:pass@amqp-03.lx2.com:5672!g” #!substdef "!MY_AMQP_URL_7!zone=lx3;kazoo://user:pass@amqp.lx3.com:5672!g” modparam("kazoo", "amqp_consumer_processes", 8); modparam("kazoo", "amqp_consumer_workers", 32); Zone Configuration
- 10. @kazoocon@kazoocon Kamailio route[KAZOO_AUTHORIZATION] { xlog("L_INFO", "$ci|log|check $adn"); $var(amqp_payload_request) = '{"Event-Category" : "directory" , "Event-Name" : "authn_req", "Method" : "REGISTER", "Auth-Nonce" : "$adn", "Auth-Realm" : "$fd", "Auth-User" : "$fU", "From" : "$fu", "To" : "$tu" }'; $var(amqp_routing_key) = "authn.req." + $(fd{kz.encode}); $avp(kz_timeout) = 2500; if(kazoo_async_query("callmgr", $var(amqp_routing_key), $var(amqp_payload_request), "KAZOO_AUTHORIZATION_OK", "KAZOO_AUTHORIZATION_ERROR") != 1) { xlog("L_INFO", "$ci|log|failed to send Kazoo query for authentication credentials for $Au $si:$sp"); append_to_reply("Retry-After: 60rn"); send_reply("500", "Retry Later"); exit; } } route[KAZOO_AUTHORIZATION_OK] { $var(password) = $(kzR{kz.json,Auth-Password}); $var(nonce) = $adn; $xavp(ulattrs=>custom_channel_vars) = $(kzR{kz.json,Custom-Channel-Vars}); xlog("L_INFO", "$ci|log|authenticating $Au via Kazoo query response"); route(CHECK_AUTHORIZATION); } route[KAZOO_AUTHORIZATION_ERROR] { xlog("L_INFO", "$ci|log|failed to query Kazoo for authentication credentials for $Au $si:$sp"); append_to_reply("Retry-After: 60rn"); send_reply("500", "Retry Later"); exit; }
- 11. @kazoocon@kazoocon Whapps 3 Servers 2 Servers One Server crossbar, registrar, callflow, fax crossbar, callflow, fax crossbar callflow, fax registrar registrar
- 13. @kazoocon@kazoocon Rabbit • Thresholds • Granular statistics (10% gain) • Cluster rabbit • Federation & Zones
- 14. @kazoocon@kazoocon Rabbit Clustering • High available queues • Publish & consume from any node • Policies for replication • LAN Only
- 15. @kazoocon@kazoocon Rabbit Federation • Federation plugin primary Zone a Zone b • Kazoo Federation • Granular control
- 16. @kazoocon@kazoocon REGISTER 401 REGISTER AUTHN_REQ COUCH AUTHN_RESP USRLOC REG_SUCCESS ECALLMGR registration scenario 2 kamailio , 2 x kazoo registrar instances with 4 listeners, 2 x rabbit (clustered) In this scenario, we are showing 5,800+ registrations per second. Most phones register every 360 seconds, so: 360 x 5800 = 2,088,000 phones (THEORETICAL!!!)
- 18. @kazoocon@kazoocon PROGRESSING CLUSTER Kamailio FreeSWITCH Couch Rabbit apps HAProxy Couch Couch
- 23. @kazoocon@kazoocon PROGRESSING CLUSTER Kamailio FreeSWITCH Couch Rabbit apps HAProxy Couch Couch FreeSWITCH apps Rabbit Kamailio FreeSWITCH apps apps apps apps Rabbit
- 24. @kazoocon@kazoocon PROGRESSING CLUSTER Kamailio FreeSWITCH Couch Rabbit apps HAProxy Couch Couch FreeSWITCH apps Rabbit Kamailio FreeSWITCH apps apps apps apps Rabbit Kamailio FreeSWITCH Rabbit apps HAProxy Start a New Zone
- 25. @kazoocon@kazoocon The World & Beyond “A thing called open-source kazoo platform actually make it possible.”
Notas do Editor
- People love to talk about scale. Some vendors pitch that their systems easily support 100,000 simultaneous calls, or 500 calls per second, etc. The reality is, in the real world, people’s behaviors vary and the feature sets they use can cut these numbers down quickly. For example, ask that same vendor claiming 100,000 simultaneous calls if it can be done while call recording, call statistics and other features are turned on at the same time, and you’ll usually get a very different, cautious, qualified response. The reality is scaling an open-source cluster to 10,000 phones and beyond can be a challenge because every customer’s usage patterns are different. There is no cookie-cutter solution if one customer has heavy dialer traffic, another heavy API traffic and yet another has business VoIP traffic that’s feature-rich. Hardware variatioms, cloud based, vms, docker, hardware size, network capacity Transcoding
- Kazoo platform works with several components, each with its own tuning and scaling particularities We will go through these componenst and see how to optimize them for scale.
- All components work on top of a operating system. And the OS also has some setting we should check/configure so that the components will work better when scaling up your ckuster File Descriptors Limits -> sockets -> processes Logs -> syslog, drive capacity , SSD -> network if your shipping your log to another server Network capacity
- Overprovisioning Db tool to reshard File descriptors Views Crawlers Auth db puts pressure Balancing
- Bare metal vs scale up 4 x 250 calls vs 1 x 1000 calls Network capacity Cpu if transcoding Sqlite vs mysql (sofia tracks dialogs in db) Mod_kazoo filters Do not cross connect among zones
- TCP workers - udp listeners - tcp listeners Worker counts Buffer parameters Amqp workers / consumers Memory (lots of) Shared memory where we keep cached credentials, presence information, anti flood, dispatcher rules, acls Db_text vs mysql Mysql in memory – hard to maintain
- Example of 3 zone configuration 1 has haproxy
- Async example kazoo_async_query KAZOO_AUTHORIZATION_OK KAZOO_AUTHORIZATION_ERROR NO BLOCKING HERE
- Separate servers for api / call process / utils Package whapps independently to easily start new instances Enough registrars Enough registrar listeners Handle the logs Limits on lager / syslog
- Balance the components of kazoo platform Cheap load balancer Rabbit websockets File descriptors Limits Balance to other zones , backup Terminate ssl
- Clustering Granular statistics 10% Cluster policies, not for private queues Limits on rabbit Thresholds that rabbit stops
- Shared queues need ha policy
- Federation plugin doesn’t have granular support for what you can downstream