Sistemas com requisitos de alta escalabilidade e desempenho, demandam atenção especial no que diz respeito a Arquitetura e Operação. A sustentação de um cluster de dezenas/centenas servidores se torna inevitavelmente complexa. Da mesma forma a Arquitetura que precisa ser projetada de forma a usar recursos do cluster de forma eficiente. Apache Mesos atua em ambas as dimensões facilitando atividades de gerencimento e uso otimizado de recursos de cluster.

1. Managing Computational resources with Apache Mesos
Jackson Oliveira
@cyber_jso
Cesar Mesquita
@cmesquita00

2. Software Architect, more than 13 years working with I.T
Currently Working at ilegra as consultant.
SOA specialist.
Believe on agile, devops principles, people transformation and
open source community ideas behind.
Like videogames and watch series.
Football fun.
Blog: http://jackson-s-oliveira.blogspot.com.br/
Linkedin: https://www.linkedin.com/in/jacksonsoliveira
facebook:https://www.facebook.com/jackson.dossantos.5

4. Storymap traditional data centers resource usage ….
Storymap traditional data centers resource usage...

5. Siloed cluster (static partition,low granularity ) no shared resource ...
Siloed cluster (static partitioning, low granularity) no shared resource

6. Siloed cluster (static partition,low granularity ) no shared resource ...
Siloed cluster (static partitioning, low granularity) no shared resource
Siloed data center Data center managed by mesos
mesos

7. Management complexity (technology , hardware facts ) ...
Management complexity (technology, hardware, facts) ..

8. Resilience ...
Resilience

9. Scalability problems ( tech limits , license cost ) ...
Scalability problems (tech limits, licence costs) ..

10. Distributed systems added complexity

11. Releases demand more effort

12. Orchestration complexity

13. Heterogeneous Archs on the same datacenter

14. Failures

15. Service Discovery

16. Big Data - distribute is needed

17. Mesos - The datacenter operational system

18. Resources as abstractions

19. Mesos Architecture

20. Mesos Architecture

21. Mesos Architecture

22. Mesos Architecture

23. Mesos Architecture

24. Slave node anatomy

25. Master node responsibilities

26. Master nodes High Availability

27. Frameworks responsibility

28. Frameworks Ecosystem
Long Running jobs
Big Data Processing
Batch Scheduling
Data Storage

29. Frameworks can coexist on the same datacenter

30. Resource Offering process

31. Resource Offering process

32. Resource Offering process

33. Resource Offering process

34. Resource Isolation

35. Native isolation using Linux containers

36. Isolation Mechanisms

37. CPU Share, Disk quotas and Bandwith limits
CPU Disk Network
- Core Isolation - Enforce maximum space usage
limits
- Limit I/O bandwidth usage
- Ports

38. Resource allocation

39. How to prevent frameworks starving?
Before workload After workload

40. How to prevent frameworks starving?

41. Static reservation - Good for Stateful services

42. Dynamic reservation - Good for Stateful Scheduled tasks

43. Resource preemption

44. Resource preemption

45. Frameworks: Marathon!

46. Built to support long running jobs

47. Specificing contranstraints
Good to ensure tasks are:
● Running all slaves
● Running At least one per
datacenter
● Running on specific slaves
● Running at least one per
cluster group

48. Scaling applications out
New tasks can
be allocated
dinamically

49. Handling failures

50. Handling failures
Timeout!

51. Handling failures
Report the
frameworks!

52. Handling failures
Reschedule the tasks!

53. ● Automation is heavily needed on this environment
● Troubleshooting can be tricky
● Monolithic systems may not take full advantage from this solution
● Ops: supporting the platform rather than specific products
● Better resources utilization => TCO decreasing.
Considerations

54. Managing Computational resources with Apache Mesos
Jackson Oliveira
@cyber_jso
Cesar Mesquita
@cmesquita00
Thank you!