Jon Hammant, Head of Cloud & DevOps for UK & EU for Epam Systems, presented an overview of using the ELK stack together with the Beats Plugin data shippers to provide detailed system metrics, network traffic, file analysis, and more. In addition, he provided an overview of how to monitor multiple Docker containers in a cloud native environment, with logs sent back to a central host.
4. PROBLEM
Too many syste ms an d n ot
e n ou gh visib ility
Massive ly d istrib u te d
In cre asin g n u mb e r of
microser vices
Fu ll d e -centralization
Painfu l p roce ss
We need log ging & metrics
5. WHY DO WE NEED METRICS?
Bloodletting
Starte d arou n d 100BCE
C ontin u e d u ntil 19 th C e ntu r y
H u n d re d s of Th ou san d s h ave
d ie d
It was d on e b e cau se p e op le
cared
Th ey ju st d id n ’t h ave right
th e information
6. WHY ELK?
Easy to setu p
Massive ly Powe rfu l
Scale s ve r y we ll
Op e n sou rce
Availab le as a se r vice
10 min u te setu p
8. • WHO WE SERVEWE NEED A WAY OF GETTING LOGS IN
We don’t want to run Syslog everywhere
Increasingly the applications are running on
cloud native systems
For a lightweight process we can’t add
heavyweight logging
No point writing loads of logging code
9. • WHO WE SERVEWHAT ARE BEATS?
Beats are the Elasticsearch platform for
single purpose, lightweight data shippers.
Designed to be small & portable
Logstash is still important for data
enrichment, reformatting
Replaces Logstash Forwarder & more
15. WHEN LOGGING & METRICS WORK
“Every th in g we kn ow in
aviation , eve r y ru le in th e
ru le b ook, eve r y p roce d u re
we h ave , we kn ow b e cau se
some on e somewh e re d ie d …
We h ave p u rch ase d at gre at
cost, lesson s literally
b rou ght with b lood ”
-
" Su lly" Su lle n b e rge r
17. • WHO WE SERVEOPENSOURCE HIGHLIGHTS
h t t p b e a t
Po l l a h t t p e n d p o i n t
my s q l b e a t
R u n a s c h e d u l e d q u e r y o n a my S q l
s e r v e r
M a n y m o r e u s e f u l B e a t s a v a i l a b l e
o r w r i t e y o u r o w n
C l o u d t ra i l b e a t , P i n g b e a t ,
C o n s u l b e a t e t c . .
exe c b e a t
Pe r i o d i c a l l y r u n c o m m a n d s a n d
s e n d o u t p u t a n d e r r o r
18. • WHO WE SERVEdockbeat
git clone clone https://github.com/Ingensi/dockbeat.git
wget https://github.com/Ingensi/dockbeat/releases/download/v1.0.0/dockbeat-v1.0.0-x86_64
chmod +x dockbeat-v1.0.0-x86_64
vi dockbeat/dockbeat.yml
Replace Docker_Socket & Elasticsearch or Logstash host
./dockbeat-v1.0.0-x86_64 -c dockbeat/dockbeat.yml -v –e
(can also be started in a container or swarm and permissioned)
20. • WHO WE SERVEUSE!
D i s c o v e r
L i s t h i s t o r i c C P U u s a g e
F i n d o u t w h i c h c o n t a i n e r s w e r e r u n
A n a l y ze fo r i n s e c u r e c o n t a i n e r s
M e t r i c s
S h o w r e a l t i m e m e t r i c s o f sy s t e m u s e
D i s p l ay b u s i n e s s v a l u e
V i e w t h e w h o l e sy s t e m a t o n e
V i s u a l i z e
L o o k b a c k a t p e r fo r m a n c e s t a t s
C o r r e l a t e c o s t / p e r fo r m a n c e a n d r e v e n u e
S h o w l o n g t e r m t r e n d s
A l e r t
U s e E l a s t A l e r t o n c o n t a i n e r s
B e i n fo r m e d w h e n t h i n g s s t o p
K n o w w h e n c a p a c i t y i s a n i s s u e