Slides from talk I gave at Denver Docker Meetup about a Federated Graphite Cluster in Docker using Consul, Consul-template, Registrator, Mesos and Marathon.

1. Containing Graphite
(And Herding Cats)

2. Trying to Make the World at TWC a Better Place
*Drinker of Good Beer
Phil Zimmerman*
@phil_zimmerman

3. Graphite: Overview
Legos and Herding Cats
Show Me Some Cool Shit
Meet The Players

4. Registrator

8. A Self-Serve Interface to Your Cluster
Distributed init For Long-running Services

9. You Know The Company Behind Consul
(Think Vagrant)

10. Key/Value Store
Service Discovery (HTTP and DNS)
Failure Detection
Multi Datacenter

11. Queries Consul and Updates Specified
Templates on the Filesystem

12. Registrator
https://github.com/gliderlabs/
registrator
Auto Register/De-register Docker Containers

13. Graphite
Overview

14. Graphite
Carbon-cache
Whisper
Graphite Web
Carbon-relay

15. Carbon-cache
Receives and Writes Incoming Metrics
Caches and flushes data
2 Receiver ports, 1 Query Port

16. Whisper Database
Time-series Data

17. Graphite Web
Queries carbon-cache
Reads whisper data
AND

18. Carbon-relay
Load Balances and Proxies Metrics
Can Write To Multiple carbon-caches
For High Volume of Metrics

19. Carbon Relay
[:2003]
Metrics Metrics Metrics
Carbon Cache
[:2003]
[:2004]
[:7002]
Whisper Data
Carbon Webapp
[:80]
postgresql
(dashboards)
[:5432]
Typical Single-node Setup

20. Does Anyone See
Containers??

21. Master Webapp
[:80]
pgsql[:5432]
Clustered Setup
Carbon Relay
[:2003]
Metrics Metrics Metrics
Carbon Relay
[:2003]
Carbon Relay
[:2003]
Carbon Relay
[:2003]
Carbon Cache
[:2003]
[:2004]
[:7002]
Whisper Data
Carbon Cache
[:2003]
[:2004]
[:7002]
Carbon Webapp
[:80]
MemcachedMemcachedMemcached
[:11211]
Carbon Relay
[:2003]
Carbon Cache
[:2003]
[:2004]
[:7002]
Whisper Data
Carbon Cache
[:2003]
[:2004]
[:7002]
Carbon Webapp
[:80]

22. And You Thought
You Were Too Old
To Play With Legos

23. How Do We Orchestrate
All Of This??

24. Without Service Discovery

25. Containers
Service Announcement
Service Discovery
Configuration

26. What We Need
When Containers Start/Stop, They Register/
De-register Their Services
When Dependent Service Endpoints Change,
Config Files Auto-Update and Trigger
Service Restart

27. Registrator

28. The Service Containers
Carbon-cache: Dockerfile

29. The Service Containers
Carbon-relay: Dockerfile

30. The Service Containers
Carbon-webapp: Dockerfile

31. The Service Containers
postgresql: Dockerfile

32. The Service Containers
memcached

33. The Data Volume Containers
Whisper
Create a carbon-cache container without
running the cmd (use entrypoint)

34. The Data Volume Containers
graphite-db
Create a postgresql container without
running the cmd (use entrypoint)

35. The Data Volume Containers
consul-template
Create a consul-template container
without running the cmd (use
entrypoint)

36. Service Announcement/
Registration
Registrator
Runs in a container
Pass consul url as argument for docker run

37. Service Announcement/
Registration
docker run -d -P -e
SERVICE_5432_NAME=graphite-db
--volumes-from dashboard-data twcable/
postgresql93:0.2
Start & Register postgresql

38. Service Discovery
graphite-master-webapp
local_settings.ctmpl (consul-template)

39. Service Discovery
graphite-master-webapp
tmpl.json - config for consul-template

40. Services
Registered In
Consul

41. Great, Now How Do You
Launch All of These
Containers??

42. Registrator

43. Mesos Master
!
Mesos
Marathon
Zookeeper
Mesos Master
!
Mesos
Marathon
Zookeeper
Mesos Master
!
Mesos
Marathon
Zookeeper
Mesos Slave
!
Mesos
Docker
Consul
Mesos Slave
!
Mesos
Docker
Consul
Mesos Cluster

44. Marathon JSON Deploy
Definition

46. Post JSON To Running
Marathon Instance

49. Show Me Something Cool

50. Helpful Links
http://mesos.apache.org/
https://mesosphere.github.io/marathon/
https://consul.io/
https://github.com/hashicorp/consul-template
https://github.com/gliderlabs/registrator
https://grey-boundary.io/the-architecture-of-clustering-graphite/
http://philzim.com/2014/11/12/service-discovery-orchestration-
with-mesos-and-consul/