SlideShare a Scribd company logo
1 of 30
Contents
 Why continuous security monitoring
 Intro to ELK Stack
 Install Elasticsearch and Cerebro
 Install Kibana and create dashboards
 Install Logstash and create config files
 Install filebeat agent and forward logs
CONTINUOUS
SECURITY
MONITORING
INTRODUCTION TO
Security Systems In Use
 Firewall
 Antivirus software
 Web application firewall (waf)
SIEM
"In the field of computer security, security information and event
management (SIEM) software products and services combine security
information management (SIM) and security event management (SEM).They
provide real-time analysis of security alerts generated by applications and
network hardware."
Wikipedia
SIEM
The general perception is that setting up
SIEM is a very expensive exercise,
however with the right knowledge and
skill it can be done at a fraction of the
cost.
Functions of a SIEM
Logs
Log Aggregator
Log Broker
Correlation rules
Logs
Logs
Logs
Storage Visualization
Continuous Security Monitoring
 End point security monitoring
 Network security monitoring
ELK STACK
PRACTICAL
SECURITY
MONITORING WITH
Functions of a SIEM
Logs
Log Aggregator
Log Broker
Correlation rules
Logs
Logs
Logs
Storage Visualization
Elastic Stack vs Commercial SIEM
 Elastic Stack
 Free, Paid support features
 Outstanding visualizations
 Advanced log enrichments
 Capable handling high volume
• Commercial SIEM
• Licensed on volume, log
sources, events per second
• Limited log enrichment
• High volume = High cost
Minimum Hardware Requirements
 Ram: 8GB
 Storage: 40GB
 2 Network interfaces
 CPU: 64-bit 2.0+ GHz processor or higher
ELASTICSEARC
H
Install Elasticsearch
#sudo dpkg -i elasticsearch-6.0.0.deb
Config files locations:-
Elasticsearch has three configuration files:
 elasticsearch.yml for configuring Elasticsearch
 jvm.options for configuring Elasticsearch JVM settings
 log4j2.properties for configuring Elasticsearch logging
Elasticsearch
 Distributed, real-time data store, real-time analysis full text search
engine
 Opensource
 Highly scalable
Indices, Shards and Replicas
 An index is stored on a node, which is a part of a cluster
 Indices are broken into shards
 Each shard is either a primary or replica
 Each log item is a document that contains fields and values
CEREBRO
Cerebro
 Cerebro is an opensource Elasticsearh web admin tool
 Displays cluster health
 Makes index managements easy
Install Cerebro
#sudo unzip cerebro-0.7.1.zip -d /opt
#sudo mv /opt/cerebro-0.7.1/ /opt/cerebro/
Create a user for cerebro
#sudo useradd cerebro
Give permissions for the user
#sudo chown -R cerebro: /opt/cerebro/
Create a service for cerebro
#sudo cp cerebro.service /etc/systemd/system
#sudo systemctl daemon-reload
#sudo systemctl enable cerebro.service
#sudo service cerebro start
KIBANA
Install Kibana
#sudo dpkg -i kibana-6.0.0-amd64.deb
Enable kibana service
#sudo systemctl enable kibana.service
Start kibana service
#sudo service kibana start
LOGSTASH
Install Logstash
#sudo dpkg -I logstash-6.2.1.deb
Config file
jvm.options
Logstash Config File Format
input{
}
filter{
}
output{
}
Logstash Config File Format
input {
stdin { codec => "json" }
}
filter {
if [event_id] == 123 {
drop { }
}
}
output {
stdout { codec => rubydebug }
}
THANK
YOU
FOLLOW US ON
/econIntconference @econ_int @int.econ

More Related Content

Similar to Practical Security Monitoring with ELKStack

Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019
Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019
Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019Janith Malinga
 
Présentation ELK/SIEM et démo Wazuh
Présentation ELK/SIEM et démo WazuhPrésentation ELK/SIEM et démo Wazuh
Présentation ELK/SIEM et démo WazuhAurélie Henriot
 
Automate threat detections and avoid false positives
  Automate threat detections and avoid false positives  Automate threat detections and avoid false positives
Automate threat detections and avoid false positivesElasticsearch
 
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...Nikhil Jain
 
Automate threat detections and avoid false positives
Automate threat detections and avoid false positivesAutomate threat detections and avoid false positives
Automate threat detections and avoid false positivesElasticsearch
 
Automatiza las detecciones de amenazas y evita los falsos positivos
Automatiza las detecciones de amenazas y evita los falsos positivosAutomatiza las detecciones de amenazas y evita los falsos positivos
Automatiza las detecciones de amenazas y evita los falsos positivosElasticsearch
 
Automatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosAutomatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosImma Valls Bernaus
 
Automatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosAutomatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosImma Valls Bernaus
 
Logging, tracing and metrics: Instrumentation in .NET 5 and Azure
Logging, tracing and metrics: Instrumentation in .NET 5 and AzureLogging, tracing and metrics: Instrumentation in .NET 5 and Azure
Logging, tracing and metrics: Instrumentation in .NET 5 and AzureAlex Thissen
 
Microservices, Continuous Delivery, and Elasticsearch at Capital One
Microservices, Continuous Delivery, and Elasticsearch at Capital OneMicroservices, Continuous Delivery, and Elasticsearch at Capital One
Microservices, Continuous Delivery, and Elasticsearch at Capital OneNoriaki Tatsumi
 
Launch .NET Applications in the Cloud
Launch .NET Applications in the CloudLaunch .NET Applications in the Cloud
Launch .NET Applications in the CloudRightScale
 
Using Tetration for application security and policy enforcement in multi-vend...
Using Tetration for application security and policy enforcement in multi-vend...Using Tetration for application security and policy enforcement in multi-vend...
Using Tetration for application security and policy enforcement in multi-vend...Joel W. King
 
Linux Assignment 3
Linux Assignment 3Linux Assignment 3
Linux Assignment 3Diane Allen
 
Apache Eagle: Architecture Evolvement and New Features
Apache Eagle: Architecture Evolvement and New FeaturesApache Eagle: Architecture Evolvement and New Features
Apache Eagle: Architecture Evolvement and New FeaturesHao Chen
 
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Ajeet Singh Raina
 
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...Create useful data center health visualizations with Dell iDRAC Telemetry Ref...
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...Principled Technologies
 
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...IBM India Smarter Computing
 
Java Performance & Profiling
Java Performance & ProfilingJava Performance & Profiling
Java Performance & ProfilingIsuru Perera
 
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)Elasticsearch
 

Similar to Practical Security Monitoring with ELKStack (20)

Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019
Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019
Practical security monitoring with ELASTIC STACK by Janith Malinga econ2019
 
Présentation ELK/SIEM et démo Wazuh
Présentation ELK/SIEM et démo WazuhPrésentation ELK/SIEM et démo Wazuh
Présentation ELK/SIEM et démo Wazuh
 
Automate threat detections and avoid false positives
  Automate threat detections and avoid false positives  Automate threat detections and avoid false positives
Automate threat detections and avoid false positives
 
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...
Second phase report on "ANALYZING THE EFFECTIVENESS OF THE ADVANCED ENCRYPTIO...
 
Automate threat detections and avoid false positives
Automate threat detections and avoid false positivesAutomate threat detections and avoid false positives
Automate threat detections and avoid false positives
 
Automatiza las detecciones de amenazas y evita los falsos positivos
Automatiza las detecciones de amenazas y evita los falsos positivosAutomatiza las detecciones de amenazas y evita los falsos positivos
Automatiza las detecciones de amenazas y evita los falsos positivos
 
Automatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosAutomatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivos
 
Automatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivosAutomatiza las detecciones de amenazas y evita falsos positivos
Automatiza las detecciones de amenazas y evita falsos positivos
 
Logging, tracing and metrics: Instrumentation in .NET 5 and Azure
Logging, tracing and metrics: Instrumentation in .NET 5 and AzureLogging, tracing and metrics: Instrumentation in .NET 5 and Azure
Logging, tracing and metrics: Instrumentation in .NET 5 and Azure
 
Microservices, Continuous Delivery, and Elasticsearch at Capital One
Microservices, Continuous Delivery, and Elasticsearch at Capital OneMicroservices, Continuous Delivery, and Elasticsearch at Capital One
Microservices, Continuous Delivery, and Elasticsearch at Capital One
 
Launch .NET Applications in the Cloud
Launch .NET Applications in the CloudLaunch .NET Applications in the Cloud
Launch .NET Applications in the Cloud
 
Using Tetration for application security and policy enforcement in multi-vend...
Using Tetration for application security and policy enforcement in multi-vend...Using Tetration for application security and policy enforcement in multi-vend...
Using Tetration for application security and policy enforcement in multi-vend...
 
AI on the Edge
AI on the EdgeAI on the Edge
AI on the Edge
 
Linux Assignment 3
Linux Assignment 3Linux Assignment 3
Linux Assignment 3
 
Apache Eagle: Architecture Evolvement and New Features
Apache Eagle: Architecture Evolvement and New FeaturesApache Eagle: Architecture Evolvement and New Features
Apache Eagle: Architecture Evolvement and New Features
 
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
Collabnix Online Webinar: Integrated Log Analytics & Monitoring using Docker ...
 
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...Create useful data center health visualizations with Dell iDRAC Telemetry Ref...
Create useful data center health visualizations with Dell iDRAC Telemetry Ref...
 
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...
Protecting the IBM Storwize V7000 Unified system with Symantec AntiVirus for ...
 
Java Performance & Profiling
Java Performance & ProfilingJava Performance & Profiling
Java Performance & Profiling
 
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)
Replicate Elasticsearch Data with Cross-Cluster Replication (CCR)
 

Recently uploaded

Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 

Recently uploaded (20)

Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 

Practical Security Monitoring with ELKStack

  • 1.
  • 2.
  • 3. Contents  Why continuous security monitoring  Intro to ELK Stack  Install Elasticsearch and Cerebro  Install Kibana and create dashboards  Install Logstash and create config files  Install filebeat agent and forward logs
  • 5. Security Systems In Use  Firewall  Antivirus software  Web application firewall (waf)
  • 7. "In the field of computer security, security information and event management (SIEM) software products and services combine security information management (SIM) and security event management (SEM).They provide real-time analysis of security alerts generated by applications and network hardware." Wikipedia SIEM
  • 8. The general perception is that setting up SIEM is a very expensive exercise, however with the right knowledge and skill it can be done at a fraction of the cost.
  • 9. Functions of a SIEM Logs Log Aggregator Log Broker Correlation rules Logs Logs Logs Storage Visualization
  • 10. Continuous Security Monitoring  End point security monitoring  Network security monitoring
  • 12.
  • 13. Functions of a SIEM Logs Log Aggregator Log Broker Correlation rules Logs Logs Logs Storage Visualization
  • 14. Elastic Stack vs Commercial SIEM  Elastic Stack  Free, Paid support features  Outstanding visualizations  Advanced log enrichments  Capable handling high volume • Commercial SIEM • Licensed on volume, log sources, events per second • Limited log enrichment • High volume = High cost
  • 15. Minimum Hardware Requirements  Ram: 8GB  Storage: 40GB  2 Network interfaces  CPU: 64-bit 2.0+ GHz processor or higher
  • 17. Install Elasticsearch #sudo dpkg -i elasticsearch-6.0.0.deb Config files locations:- Elasticsearch has three configuration files:  elasticsearch.yml for configuring Elasticsearch  jvm.options for configuring Elasticsearch JVM settings  log4j2.properties for configuring Elasticsearch logging
  • 18. Elasticsearch  Distributed, real-time data store, real-time analysis full text search engine  Opensource  Highly scalable
  • 19. Indices, Shards and Replicas  An index is stored on a node, which is a part of a cluster  Indices are broken into shards  Each shard is either a primary or replica  Each log item is a document that contains fields and values
  • 21. Cerebro  Cerebro is an opensource Elasticsearh web admin tool  Displays cluster health  Makes index managements easy
  • 22. Install Cerebro #sudo unzip cerebro-0.7.1.zip -d /opt #sudo mv /opt/cerebro-0.7.1/ /opt/cerebro/ Create a user for cerebro #sudo useradd cerebro Give permissions for the user #sudo chown -R cerebro: /opt/cerebro/ Create a service for cerebro #sudo cp cerebro.service /etc/systemd/system #sudo systemctl daemon-reload #sudo systemctl enable cerebro.service #sudo service cerebro start
  • 24. Install Kibana #sudo dpkg -i kibana-6.0.0-amd64.deb Enable kibana service #sudo systemctl enable kibana.service Start kibana service #sudo service kibana start
  • 26. Install Logstash #sudo dpkg -I logstash-6.2.1.deb Config file jvm.options
  • 27. Logstash Config File Format input{ } filter{ } output{ }
  • 28. Logstash Config File Format input { stdin { codec => "json" } } filter { if [event_id] == 123 { drop { } } } output { stdout { codec => rubydebug } }
  • 30. FOLLOW US ON /econIntconference @econ_int @int.econ