Ariel Moskovich Presentation at Docker Tlv

1. Production with Consul & Docker
And a few more friends...

2. Challenges
● High Traffic: 1.7 Billion
requests per day
● Maintaining Lots of data
● Data integrity
● Fast growing user base
● Need to deploy new features
fast
● Getting out of the garage: be
scalable & redundant
● Bottom line: We need tools
that can help us grow fast!

3. What we will cover
How It's all connected

4. Our Dockerfiles Principles
● Create a base image and
template for each application
type: Clojure, Python...
● Then auto generate from
project name
● Infrastructure Dockerfiles
examples are all over, just
download and modify.
● Keep it as minimized as
possible.
● But not at all costs.

5. Docker private registry
● Build & push only once, pull many
● Manage tags
● Centralized repository
● Simple usage via Docker cmd / API
● Supported by Mesos / Marathon
● Use Local / S3 as storage
● Enable multiple environments: test,
prod
You can also USE
External payed hubs with web user interface and Github /
Bitbucket integration.

6. let me grab it from Bitbucket
and build it for you
I need x
application jar
You know that's a legacy code,
right? ;-)
Done, just take it from
the target directory
Now, Docker Daemon, Please
build, tag and push the image to
the repository

7. Consul
“Service Discovery and configuration”

8. Principles
 Agent: Long running Daemon, can be client or server (same executable)
 Client: agent that forwards all RPC to server
 Server: responsible for Raft quorum, cluster state, RPC queries
 Running as client:
consul agent -dc=$DC -join $CONSULMASTER -pid-file=$PIDFILE
-data-dir $DATADIR -config-dir $CONFDIR -bind $IP
 Running as server:
consul agent -server -bootstrap -pid-file=$PIDFILE -dc=$DC -ui-dir $UIDIR
-data-dir $DATADIR -config-dir $CONFDIR -bind $IP

9. Forwarding queries to consul DNS
ariel@ariels:~/present$ host neo4j.eu1.appsflyer.com
neo4j.eu1.appsflyer.com is an alias for neo4j.service.consul.
neo4j.service.consul has address 10.12.6.71
neo4j.service.consul has address 10.12.5.25
zone "service.consul" IN {
type forward;
forward only;
forwarders { 10.12.4.88 port 8600; 10.12.6.88 port 8600; 10.12.8.88 port 8600;};
};

11. Apache Mesos & Mesosphere Marathon
Mesos: Abstracts compute resources away from machines,
enable to build clusters upon frameworks
Marathon: Framework for long-running applications (like
docker), provide a rest API
c

12. Provisioning & Deploy PPrroovviissiioonniinngg && DDeeppllooyy FFFlllooowww wwwiiittthhh MMMeeesssooosss ––– SSStttaaagggeee 111
I need x docker instances in AWS I can do that
Now we have Mesos cluster of X instances, each with Chef, Docker Daemon,
Mesos slave and Consul agent which serves as discovery source

13. Provisioning & Deploy Flow with Mesos – Stage 2
Please deploy an application to the
cluster from the registry
That's fine by me
Deploy, deploy, deploy
Also, I will need you to provide
me the servers and ports that
serve this application, I would
like to update my buddy consul
Sure
Here's what
Marathon
knows, go
gossip about it

14. Up and running

15. Connecting to Dockers
● Using ssh?
● Nsenter?
● 1.3 exec?
● Other methods?
● Avoid connecting?

16. Questions? ariel@appsflyer.com
@relmos