SlideShare uma empresa Scribd logo
1 de 58
Baixar para ler offline
You Shall NotGet
Excited
Ivan Ribeiro Rocha
ivan.ribeiro@gmail.com
@irr
Agenda
● Why not?
●
”History” about getting excited...
●
What should I use...?
●
My choice: Erlang
●
Learning from
experience...
Why not?
”Once upon a time...”
”History” about getting excited...
(what is a ”technical”
manager?)
…a ”technical”
manager meets...
… a ”smart guy”
(a.k.a salesman)
Deal! You deserveit!
”no problem... we can fixit!”
”relax... the guys will change
it in the upcoming version...”
sometimes is not so easy to
fix all problems
”Ignorance more frequently begets
confidence than does
knowledge.”
Charles Darwin
What should I use...?
Programming Languages
and tools...
● Languages
– Java, Ruby, Python, Perl, ...
– C/C++, Lisp, Haskell, Ocaml, Erlang, ...
● Web Servers
– Apache, nginx, ...
– Jetty/Tomcat, ...
– Yaws, inets, ...
Anyone using it?
Is it stable?
Does it scale?
(think about...)
You're not:
what means
scalable?
- don’t design to scale infinitely
- consider 5X - 50X growth
- but > 100X requires redesign
- break large systems into smaller services
Jeff Dean
http://bit.ly/clIJfL
You shall not get excited
You shall not get excited
You shall not get excited
no documentation...?
● Good APIs?
● Available books?
● % Comments...?
● Source code?
● Please! No
”magazines”...
still in doubt...?
You shall not get excited
Joe Armstrong
Robert Virding
Mike Williams
Claes “Klacke” Wikström
and more...
My choice: Erlang
You shall not get excited
Learning from experience...
● > 3 years and learning...
● back end development
– REST API
– memcached API
– syslog API
– MySQL and Mnesia (data storage)
● functional + concurrent programming
● fault-tolerant and highly scalable
● very light-weight concurrency (processes)
●
running massive systems for 20 years
● bit syntax and binaries (for protocol programming)
● links (encourages “let it crash” programming)
●
”concurrency belongs to the language and not the
operating system”
Erlang
(Why you should get excited...)
Concurrent Programming
–lots of processes
●
the only way for processes to interact is
through message passing
– event
based (epoll)
Erlang
(Why you should get excited...)
No shared memory
– ”sharing is the property that prevents
fault tolerance”
– destructive shared data modifications
do not occur!
Erlang
(Why you should get excited...)
It's easyto test
and update your code
Erlang
(Why you should get excited...)
HTTP/JSON support
– yaws (embedded)
– mochiweb (json)
– misultin
–inets
– httpc
Erlang
(Why you should get excited...)
many Libraries/tools for distributed
programs...
– net_[adm|kernel]
net_kernel:connect_node(N).
net_adm:ping(N).
– rpc
{R, E} = rpc:multicall(nodes(), Mod, Fun, [N], T),
lists:foldl(fun(L, A) ->
merge(A, [X || X <- L, not member(X, A)], N)
end, C, R),
– epmd...
Erlang
(Why you should get excited...)
”Mnesia is a distributed DataBase Management System
(DBMS), appropriate for telecommunications applications and
other Erlang applications which require continuous operation
and exhibit soft real-time properties”
Mnesia is great
but you should really know how to use it
and you MUST architecture your application
to get the best results...
Erlang
(Why you should get excited...)
OTP (Unix <-> C <==> OTP <-> Erlang)
”It’s an application operating system and a set of
libraries and procedures used for building
large-scale, fault-tolerant, distributed applications”
– supervisor
– applications
– gen_server
– gen_fsm
– gen_event
”We should forget about small efficiencies, say
about 97% of the time: premature
optimization is the root of all evil.”
Donald Knuth
Erlang
(caveats: You must be careful...)
Mnesia
– can't handle very large data
– 2 Gb
● ETS
● DETS
● MNESIA
Erlang
(caveats: You must be careful...)
Mnesia
–2 Gb fragments (mod (2^X))
– avoid rehash: create more fragments...
● add
● move
● del*
Erlang
(caveats: You must be careful...)
Mnesia
–CAP theorem
–replicax partitioned
networks
Erlang
(caveats: You must be careful...)
Mnesia
– need to check replica consistency?
●
vector clocks... (avoid dependencies)
●
should try timestamps
Erlang
(caveats: You must be careful...)
Mnesia
– event "running partitioned network"
● mnesia:set_master_nodes(L).
●
must restart other nodes
– if some node will not recover
soon...
● mnesia:force_load_table(T).
Erlang
(caveats: You must be careful...)
Mnesia (QLC)
●
avoid retrieve large data sets
●
use cursors inside
transactions
mnesia:activity(sync_transaction,
fun(X) ->
QC = qlc:cursor(X),
QR = qlc:next_answers(QC, N),
qlc:delete_cursor(QC),
QR
end,
[qlc:q([E || E <- mnesia:table(Tab)])],
mnesia_frag).
Erlang
(caveats: You must be careful...)
Mnesia
–load balance (read)
– avoid ”overload” one instance
mnesia_lib:set({T, where_to_read}, Node)
Erlang
(caveats: You must be careful...)
Logging
– avoid overhead
● yaws
● error_logger
– syslog
● udp
● tcp*
● gen_server
+ handle_cast
Erlang
(caveats: You must be careful...)
Messages
– like a mailbox
– per process (don't forget to read your messages)
Erlang
(caveats: You must be careful...)
OTP
–only API
– poor docs,
books,
articles...
Erlang
(caveats: You must be careful...)
OTP
– avoid ”synchronize” long calls
test(X) ->
gen_server:call(?MODULE, {test, X}, 5000).
handle_call({test, X}, From, State) →
spawn(fun() →
%% do some long task
gen_server:reply(From, Reply)
end),
{noreply, State}.
Erlang
(caveats: You must be careful...)
Security
– must be treated externally
● firewall
● private networks
– cookie based
-kernel inet_dist_use_interface {127,0,0,1}
-kernel inet_dist_listen_min <min>
-kernel inet_dist_listen_max <max>
> erl -kernel inet_dist_listen_min 9001 inet_dist_listen_max 9005
(4369 TCP port, as it is used by epmd, ERL_EPMD_PORT environment variable)
Erlang
(caveats: You must be careful...)
Code swapping/replacement
– Be careful with spawn inside old (replaced) module... it will die!
– The code of a module can exist in two variants in a system: current
and old (fully qualified function calls always refer to current code)
– If a third instance of the module is loaded, the code server will remove
(purge) the old code and any processes
lingering in it will be terminated
-module(m).
-export([loop/0]).
loop() ->
receive
code_switch ->
m:loop();
Msg ->
% ...
loop()
end.
Undocumented features...
you must be careful!
Erlang
(caveats: You must be careful...)
Using async acceptors (prim_inet)
– gen_tcp:accept(Listen)
Usages: https://github.com/irr/erl-tutorials/blob/master/ts/ts1/src/ts.erl
– prim_inet:async_accept(Socket, -1)
Usages:
https://github.com/irr/erl-tutorials/blob/master/ts/ts2/src/ts.erl
http://svn.apache.org/viewvc/incubator/thrift/trunk/thrift_server.erl
http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles
”It is undocumented because it is an internal module
that is not ment to be called from applications. Its interface may
change without warning in even the smallest patch.”
Erlang
(caveats: You must be careful...)
Using socket in http mode
case gen_tcp:listen(Port, [binary,
{packet, http},
{reuseaddr, true},
{active, false},
{backlog, 30}]) of ...
case gen_tcp:recv(C#c.sock, 0, ?server_idle_timeout) of
{ok, {http_header, _, 'Content-Length', _, Val}} ->
...
{error, {http_error, "rn"}} ->
...
{ok, http_eoh} ->
...
Usage:
http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features
Erlang
(caveats: You must be careful...)
1> inet:ifget("eth0", [addr]).
{ok,[{addr,{192,168,1,101}}]}
2> inet:getiflist().
{ok,["lo","eth0"]}
3> inet_parse:ntoa({192,168,1,101}).
"192.168.1.101"
4> inet_parse:address("192.168.1.101").
{ok,{192,168,1,101}}
You shall not get excited
there is no
silver
bullet!
Any
doubts?

Mais conteúdo relacionado

Semelhante a You shall not get excited

Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdminsPuppet
 
Erlang Message Passing Concurrency, For The Win
Erlang  Message  Passing  Concurrency,  For  The  WinErlang  Message  Passing  Concurrency,  For  The  Win
Erlang Message Passing Concurrency, For The Winl xf
 
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...siouxhotornot
 
Play with elm - Choucri fahed, Finstack - Lambadays
Play with elm - Choucri fahed, Finstack - LambadaysPlay with elm - Choucri fahed, Finstack - Lambadays
Play with elm - Choucri fahed, Finstack - LambadaysFinstack
 
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018Holden Karau
 
Dart the Better JavaScript
Dart the Better JavaScriptDart the Better JavaScript
Dart the Better JavaScriptJorg Janke
 
Beyond the Style Guides
Beyond the Style GuidesBeyond the Style Guides
Beyond the Style GuidesMosky Liu
 
Node.js Course 1 of 2 - Introduction and first steps
Node.js Course 1 of 2 - Introduction and first stepsNode.js Course 1 of 2 - Introduction and first steps
Node.js Course 1 of 2 - Introduction and first stepsManuel Eusebio de Paz Carmona
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with ErlangMaxim Kharchenko
 
Максим Харченко. Erlang lincx
Максим Харченко. Erlang lincxМаксим Харченко. Erlang lincx
Максим Харченко. Erlang lincxAlina Dolgikh
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with ErlangMaxim Kharchenko
 
Keynote joearmstrong
Keynote joearmstrongKeynote joearmstrong
Keynote joearmstrongSentifi
 
Building modern web apps with html5, javascript, and java
Building modern web apps with html5, javascript, and javaBuilding modern web apps with html5, javascript, and java
Building modern web apps with html5, javascript, and javaAlexander Gyoshev
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeDmitri Nesteruk
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys AdminsPuppet
 

Semelhante a You shall not get excited (20)

Elm dev front-end
Elm   dev front-endElm   dev front-end
Elm dev front-end
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
 
Erlang Message Passing Concurrency, For The Win
Erlang  Message  Passing  Concurrency,  For  The  WinErlang  Message  Passing  Concurrency,  For  The  Win
Erlang Message Passing Concurrency, For The Win
 
Erlang, an overview
Erlang, an overviewErlang, an overview
Erlang, an overview
 
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
 
Play with elm - Choucri fahed, Finstack - Lambadays
Play with elm - Choucri fahed, Finstack - LambadaysPlay with elm - Choucri fahed, Finstack - Lambadays
Play with elm - Choucri fahed, Finstack - Lambadays
 
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018
Apache spark as a gateway drug to FP concepts taught and broken - Curry On 2018
 
Dart the Better JavaScript
Dart the Better JavaScriptDart the Better JavaScript
Dart the Better JavaScript
 
Beyond the Style Guides
Beyond the Style GuidesBeyond the Style Guides
Beyond the Style Guides
 
Node.js Course 1 of 2 - Introduction and first steps
Node.js Course 1 of 2 - Introduction and first stepsNode.js Course 1 of 2 - Introduction and first steps
Node.js Course 1 of 2 - Introduction and first steps
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
Максим Харченко. Erlang lincx
Максим Харченко. Erlang lincxМаксим Харченко. Erlang lincx
Максим Харченко. Erlang lincx
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
IN4308 1
IN4308 1IN4308 1
IN4308 1
 
Keynote joearmstrong
Keynote joearmstrongKeynote joearmstrong
Keynote joearmstrong
 
Play with Elm!
Play with Elm!Play with Elm!
Play with Elm!
 
Building modern web apps with html5, javascript, and java
Building modern web apps with html5, javascript, and javaBuilding modern web apps with html5, javascript, and java
Building modern web apps with html5, javascript, and java
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/Invoke
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 

Último

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 

Último (20)

AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 

You shall not get excited

  • 1. You Shall NotGet Excited Ivan Ribeiro Rocha ivan.ribeiro@gmail.com @irr
  • 2. Agenda ● Why not? ● ”History” about getting excited... ● What should I use...? ● My choice: Erlang ● Learning from experience...
  • 4. ”Once upon a time...” ”History” about getting excited...
  • 5. (what is a ”technical” manager?)
  • 7. … a ”smart guy” (a.k.a salesman)
  • 9. ”no problem... we can fixit!”
  • 10. ”relax... the guys will change it in the upcoming version...”
  • 11. sometimes is not so easy to fix all problems
  • 12. ”Ignorance more frequently begets confidence than does knowledge.” Charles Darwin
  • 13. What should I use...?
  • 14. Programming Languages and tools... ● Languages – Java, Ruby, Python, Perl, ... – C/C++, Lisp, Haskell, Ocaml, Erlang, ... ● Web Servers – Apache, nginx, ... – Jetty/Tomcat, ... – Yaws, inets, ...
  • 19. what means scalable? - don’t design to scale infinitely - consider 5X - 50X growth - but > 100X requires redesign - break large systems into smaller services Jeff Dean http://bit.ly/clIJfL
  • 23. no documentation...? ● Good APIs? ● Available books? ● % Comments...? ● Source code? ● Please! No ”magazines”...
  • 26. Joe Armstrong Robert Virding Mike Williams Claes “Klacke” Wikström and more... My choice: Erlang
  • 29. ● > 3 years and learning... ● back end development – REST API – memcached API – syslog API – MySQL and Mnesia (data storage)
  • 30. ● functional + concurrent programming ● fault-tolerant and highly scalable ● very light-weight concurrency (processes) ● running massive systems for 20 years ● bit syntax and binaries (for protocol programming) ● links (encourages “let it crash” programming) ● ”concurrency belongs to the language and not the operating system”
  • 31. Erlang (Why you should get excited...) Concurrent Programming –lots of processes ● the only way for processes to interact is through message passing – event based (epoll)
  • 32. Erlang (Why you should get excited...) No shared memory – ”sharing is the property that prevents fault tolerance” – destructive shared data modifications do not occur!
  • 33. Erlang (Why you should get excited...) It's easyto test and update your code
  • 34. Erlang (Why you should get excited...) HTTP/JSON support – yaws (embedded) – mochiweb (json) – misultin –inets – httpc
  • 35. Erlang (Why you should get excited...) many Libraries/tools for distributed programs... – net_[adm|kernel] net_kernel:connect_node(N). net_adm:ping(N). – rpc {R, E} = rpc:multicall(nodes(), Mod, Fun, [N], T), lists:foldl(fun(L, A) -> merge(A, [X || X <- L, not member(X, A)], N) end, C, R), – epmd...
  • 36. Erlang (Why you should get excited...) ”Mnesia is a distributed DataBase Management System (DBMS), appropriate for telecommunications applications and other Erlang applications which require continuous operation and exhibit soft real-time properties” Mnesia is great but you should really know how to use it and you MUST architecture your application to get the best results...
  • 37. Erlang (Why you should get excited...) OTP (Unix <-> C <==> OTP <-> Erlang) ”It’s an application operating system and a set of libraries and procedures used for building large-scale, fault-tolerant, distributed applications” – supervisor – applications – gen_server – gen_fsm – gen_event
  • 38. ”We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.” Donald Knuth
  • 39. Erlang (caveats: You must be careful...) Mnesia – can't handle very large data – 2 Gb ● ETS ● DETS ● MNESIA
  • 40. Erlang (caveats: You must be careful...) Mnesia –2 Gb fragments (mod (2^X)) – avoid rehash: create more fragments... ● add ● move ● del*
  • 41. Erlang (caveats: You must be careful...) Mnesia –CAP theorem –replicax partitioned networks
  • 42. Erlang (caveats: You must be careful...) Mnesia – need to check replica consistency? ● vector clocks... (avoid dependencies) ● should try timestamps
  • 43. Erlang (caveats: You must be careful...) Mnesia – event "running partitioned network" ● mnesia:set_master_nodes(L). ● must restart other nodes – if some node will not recover soon... ● mnesia:force_load_table(T).
  • 44. Erlang (caveats: You must be careful...) Mnesia (QLC) ● avoid retrieve large data sets ● use cursors inside transactions mnesia:activity(sync_transaction, fun(X) -> QC = qlc:cursor(X), QR = qlc:next_answers(QC, N), qlc:delete_cursor(QC), QR end, [qlc:q([E || E <- mnesia:table(Tab)])], mnesia_frag).
  • 45. Erlang (caveats: You must be careful...) Mnesia –load balance (read) – avoid ”overload” one instance mnesia_lib:set({T, where_to_read}, Node)
  • 46. Erlang (caveats: You must be careful...) Logging – avoid overhead ● yaws ● error_logger – syslog ● udp ● tcp* ● gen_server + handle_cast
  • 47. Erlang (caveats: You must be careful...) Messages – like a mailbox – per process (don't forget to read your messages)
  • 48. Erlang (caveats: You must be careful...) OTP –only API – poor docs, books, articles...
  • 49. Erlang (caveats: You must be careful...) OTP – avoid ”synchronize” long calls test(X) -> gen_server:call(?MODULE, {test, X}, 5000). handle_call({test, X}, From, State) → spawn(fun() → %% do some long task gen_server:reply(From, Reply) end), {noreply, State}.
  • 50. Erlang (caveats: You must be careful...) Security – must be treated externally ● firewall ● private networks – cookie based -kernel inet_dist_use_interface {127,0,0,1} -kernel inet_dist_listen_min <min> -kernel inet_dist_listen_max <max> > erl -kernel inet_dist_listen_min 9001 inet_dist_listen_max 9005 (4369 TCP port, as it is used by epmd, ERL_EPMD_PORT environment variable)
  • 51. Erlang (caveats: You must be careful...) Code swapping/replacement – Be careful with spawn inside old (replaced) module... it will die! – The code of a module can exist in two variants in a system: current and old (fully qualified function calls always refer to current code) – If a third instance of the module is loaded, the code server will remove (purge) the old code and any processes lingering in it will be terminated -module(m). -export([loop/0]). loop() -> receive code_switch -> m:loop(); Msg -> % ... loop() end.
  • 53. Erlang (caveats: You must be careful...) Using async acceptors (prim_inet) – gen_tcp:accept(Listen) Usages: https://github.com/irr/erl-tutorials/blob/master/ts/ts1/src/ts.erl – prim_inet:async_accept(Socket, -1) Usages: https://github.com/irr/erl-tutorials/blob/master/ts/ts2/src/ts.erl http://svn.apache.org/viewvc/incubator/thrift/trunk/thrift_server.erl http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles ”It is undocumented because it is an internal module that is not ment to be called from applications. Its interface may change without warning in even the smallest patch.”
  • 54. Erlang (caveats: You must be careful...) Using socket in http mode case gen_tcp:listen(Port, [binary, {packet, http}, {reuseaddr, true}, {active, false}, {backlog, 30}]) of ... case gen_tcp:recv(C#c.sock, 0, ?server_idle_timeout) of {ok, {http_header, _, 'Content-Length', _, Val}} -> ... {error, {http_error, "rn"}} -> ... {ok, http_eoh} -> ... Usage: http://www.trapexit.org/A_fast_web_server_demonstrating_some_undocumented_Erlang_features
  • 55. Erlang (caveats: You must be careful...) 1> inet:ifget("eth0", [addr]). {ok,[{addr,{192,168,1,101}}]} 2> inet:getiflist(). {ok,["lo","eth0"]} 3> inet_parse:ntoa({192,168,1,101}). "192.168.1.101" 4> inet_parse:address("192.168.1.101"). {ok,{192,168,1,101}}