TDC2016POA | Trilha Infraestrutura - Apache Mesos & Marathon: gerenciando remotamente recursos e aplicações on premises

Apache Mesos &
Marathon Framework
Gerenciando remotamente recursos
e aplicações on-premises
Pedro Arthur P. R. Duarte
pedroarthur.jedi@gmail.com

Alguém lembra das camadas?
Physical
Link
Network
3

Physical
Link
Network
{Media Layers
3

Physical
Link
Network
{Media Layers
Transport
Session
Application
3

Physical
Link
Network
{Media Layers
Transport
Session
Application
}Host Layers
3

Inovação e Evolução das Redes
Physical
Link
Network
Transport
Session
Application
4

Physical
Link
Network
Transport
Session
Application
Inovation/EvolutionNumber of Protocols
4

Physical
Link
Network
Transport
Session
Application
Adaptions to
new technologies
Addressing new
use cases
Signaling techniques
Physics evlotution
Addressing scalabilty
requirements
4

Physical
Link
Network
Transport
Session
Application
Adaptions to
new technologies
Addressing new
use cases
Signaling techniques
Physics evlotution
Addressing scalabilty
requirements
Minor adjustments
in techniques
Mostly impossible
to avoid TCP/IP
4

Hardware Ossiﬁcado
Packet Switching Hardware
Hardware Abstraction Layer
Feature 0 Feature N
5

Feature 0 Feature N
{Mostly bloated to deal with
protocols' requirements
5

Feature 0 Feature N
{Vendorized without
APIs or any kind of
customization facility
5

Feature 0 Feature N
{Vendorized without
APIs or any kind of
customization facility
{
Thousand of
lines of code
More than 6k standards
(IETF, ITU-T, IEEE, etc)
5

Rede Ossiﬁcada
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Feature 0 Feature N
Packet Switching HardwarePacket Switching Hardware
6

Separação de Planos
Networking Operating System
App 0 App 1 App N
Simple Packet
Switching Hardware
Simple Packet
Switching Hardware
Simple Packet
Switching Hardware
7

Separação de Planos
Networking Operating System
App 0 App 1 App N
Simple Packet
Switching Hardware
Simple Packet
Switching Hardware
Simple Packet
Switching Hardware
Open Hardware
Interfaces
Vendor Agnostic
Standardized APIs
7

Open Networking Foundation Architecture
8

Infrastructure
Layer
Networking Device Networking Device
Networking Device
8

Infrastructure
Layer
Networking Device
Control Layer SDN Control
Software
Network Service A ... Network Service X
8

Infrastructure
Layer
Networking Device
Software
Control/Dataplane
Interfaces
8

Infrastructure
Layer
Networking Device
Software
Control/Dataplane
Interfaces
Application Layer
Business Application Business Application
Business Application
8

Infrastructure
Layer
Networking Device
Software
Control/Dataplane
Interfaces
Application Layer
Business Application Business Application
High level APIs
8

Software Deﬁned Networking (aka SDN)
9

9

Feature 0 Feature N
Feature 0 Feature N
9

Hardware Middleboxes
Public
Network
Service Provider
Router
Firewall
10

Network Function Virtualization (aka NFV)
Public
Network
Service Provider
Point of Presence
Mostly COTS Hardware
11

Network Function Virtualization (aka NFV)
Public
Network
Service Provider
Point of Presence
Mostly COTS Hardware
Virtual Appliances
11

Service Function Chaining (aka SFC)
Service Provider
Public
Network
12

Service Provider
Public
Network
1
2
3
4
5
6
7
12

Service Provider
Public
Network
1
2
3
4
5
6
7
App 0 App N
SDN Control Software
12

Em resumo...
Sotware Deﬁned Networking desacopla o controle da rede dos
elementos que realizam o encaminhamento de pacotes e os
abstraem para que possam ser utilizados por aplicações e serviços.
Network Function Virtualization transforma os tradicionais
middleboxes dedicados em appliances de rede de propósito geral
que podem ser especializados de acordo com as necessiades das
aplicações e serviços
13

Bem vindo ao Deserto do Real!
SDN + NFV
14

SDN + NFV
Redução de Custos
14

SDN + NFV
Redução de Custos
Versatilidade
& Flexibilidade
14

SDN + NFV
Redução de Custos
Versatilidade
& Flexibilidade
Escalabilidade
14

Como entregar novas aplicações/appliances?
Service Provider
Public
Network
App 0 App N
16

Service Provider
Public
Network
App 0 App N App (N+1)
16

Service Provider
Public
Network
App 0 App N App (N+1)App (N+2)
16

Publicação e Alocação de Recursos no Mesos
18

cpus(...) mem(...) ports(...) gpus(...)
cpus(...) mem(...) ppms(...)
cpus(...) m
em
(...)cpus(...) m
em
(...)
18

cpus(...) mem(...) ports(...) gpus(...)
cpus(...) mem(...) ppms(...)
cpus(...) m
em
(...)cpus(...) m
em
(...)
oﬀer([(s1, ...), ..., (s5, ...)])
oﬀer([(s1, ...), ..., (s5, ...)])
18

Mesos: Publicando Recursos
[
{ ... },
{
"name": "ppms",
"type": "SCALAR",
"scalar": {
"value": 10000000
}
},
{
"name": "processors ",
"type": "SCALAR",
"set": {
"item": 24
}
}
]
19

Mesos: Publicando Atributos
$ cat /etc/mesos-slave/attributes
ixgbe:1,pf_ring_dna:1,so_busy_poll:1,flowdirector:1
20

Framework Marathon
Escalabilidade
21

Framework Marathon
Escalabilidade Fácil set-up
21

Framework Marathon
Health Check
21

Framework Marathon
Health CheckAPIs & Event Bus
21

Marathon Example: Application Payload
{
"id": "snort -ips",
"instances": 1,
"cpus": 1,
"mem": 1024 ,
"ppms": 133334 ,
"processors ": 1,
" constraints ": [
["ixgbe", "CLUSTER", "true"],
[" flowdirector ", "CLUSTER", "true"]],
"container": { ... }
}
22

Marathon Example: HTTP API
Instanciar o novo serviço:
$ curl -XPOST http://marathon:8080/v2/apps -d @service.json
HTTP/1.1 201 Created
Content-Type: application/json
Location: http://marathon:8080/v2/apps/snort-ips
Server: Jetty(8.y.z-SNAPSHOT)
Transfer-Encoding: chunked
{ ... }
Atualizar um serviço existente:
$ curl -XPUT http://marathon:8080/v2/apps/snort-ips -d @service.json
23

Marathon Example: Health Checking
" healthChecks ": [
{
"protocol": "HTTP",
"path": "/v1/ping",
"portIndex": 0,
" gracePeriodSeconds ": 3,
" intervalSeconds ": 5,
" maxConsecutiveFailures ": 3
}
]
24

Em resumo...
Apache Mesos abstrai os recursos de hardware e provê interfaces
para gerência e alocação desses recursos.
Marathon gerencia o ciclo de vida de long-lived processes através
do recursos oferecidos por um cluster Mesos.
25

Mas nem tudo são ﬂores...
26

Overhead de Infraestrutura
Service Provider
Public
Network
App 0 App N
27

Overhead de Infraestrutura
Service Provider
Public
Network
App 0 App N
App 0 App N
App 0 App N
27

Movendo o máximo para a cloud
Service Provider
App 0 App N
Cloud
Provider
29

Premises: vazão de dados
Cloud
Provider
Client's
Network
ISP-to-ISP
L2/L3 Network
30

Cloud
Provider
Client's
Network
ISP-to-ISP
L2/L3 Network
1Gbps < tx/rx < 10Gbps 1Gbps < tx/rx < 10Gbps
30

Cloud
Provider
Client's
Network
ISP-to-ISP
L2/L3 Network
1Gbps < tx/rx < 10Gbps 1Gbps < tx/rx < 10Gbps
max(tx/rx) = 1Gbps
30

Premises: latência
Cloud
Provider
Client's
Network
ISP-to-ISP
L2/L3 Network
avg(rtt) ~ 1ms avg(rtt) ~ 1ms
avg(rtt) > 30ms
31

Tolerância a Falhas do Canal de Controle
Service Provider
App 0 App N
Cloud
Provider
33

Service Provider
App 0 App N
Cloud
Provider
X
33

Service Provider
App 0 App N
Cloud
Provider
X
X
X
X
33

Health Check "complexo"
Service Provider
App 0 App N
Public
Network
34

Service Provider
App 0 App N
Public
Network
1
2
3
4 5
34

Service Provider
App 0 App N
Public
Network
1
2
3
4 5
1
4
5
34

Alocação de recursos arbitrários no Marathon
Support for arbitrary resources #375
https://github.com/mesosphere/marathon/issues/375
35

These aren’t the droids you’re looking for
$ # yank 6 lines from the empty line
37

TDC2016POA | Trilha Infraestrutura - Apache Mesos & Marathon: gerenciando remotamente recursos e aplicações on premises

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a TDC2016POA | Trilha Infraestrutura - Apache Mesos & Marathon: gerenciando remotamente recursos e aplicações on premises

Semelhante a TDC2016POA | Trilha Infraestrutura - Apache Mesos & Marathon: gerenciando remotamente recursos e aplicações on premises (20)

Mais de tdc-globalcode

Mais de tdc-globalcode (20)

Último

Último (20)

TDC2016POA | Trilha Infraestrutura - Apache Mesos & Marathon: gerenciando remotamente recursos e aplicações on premises