4. Agenda
• Housekeeping: Overview & House Rules
• Presentation: Deployment Best Practices
• Group Discussion: Deployment Challenges & Solutions
• Presentation: Security Best Practices
• Group Discussion: Security Challenges & Solutions
• Group Discussion: Favourite Use Cases [Optional]
4
5. [Splunk Official] User Group
“The overall goal is to create an authentic, ongoing
user group experience for our users, where
they contribute and get involved”
● User Lead
● Technical Discussions
● Sharing Environment
● Build Trust (With Community & Splunk)
● No Sales!
5
9. Planning & Design
9
● High Level Design & Environment Diagram
● High Availability / Load Balancing
– Minimum Number of Nodes (SHC x3 / IXC x2-3)
– Forwarder Based (AutoLB), Search Heads (Persistent Sessions via Load Balancer)
● Hardware & Storage Requirements
– Availability / Retention / Archiving
● Development / Staging Environment
● Environment Orchestration & Configuration
– Version Control, Configuration Management, Access Management, Packaging
10. Pre-Implementation
10
● Raise Required Changes (Network, Identity, Architecture)
● Validate Connectivity & System Access
● Download Binaries / Licences / Apps
– Splunk Software & Splunk Licenses
● Ensure DNS Records Function
– IP Addresses Should Be Avoided In Config (Use DNS Records)
● Forwarder Deployment
– Engage with Platform Teams
– Develop Automation Script (Requires deploymentsclient.conf with DNS Entry)
11. Implementation
11
● Build Sequence
– Management Layer > Indexer Layer > Search Layer
● Data Source On-boarding Process
– Use Case Identification, Data Source Profiling, Develop, Test & Deliver (RTL)
● Utilise Splunk Apps & Add-ons (Free & Premium)
– Unix App, Windows Infrastructure App, VM Ware App, Apache App, Etc.
● Bundle Search Objects Into Custom Apps
– Breakdown by Business Unit, Grouped Use Cases, Etc.
● Use Splunk Documentation & Splunk Answers for Guidelines
12. Post-Implementation
12
● Update Designs / Diagrams (Delivered Implementation)
● Training & Knowledge Sharing
– Education Courses (Free / Paid), Community Support & Partner Training
● Identify Splunk Champions
– Technical & Business
● Build Business Value
– Identify Secondary User Cases
● Build Entitlement Framework
– Cost Centre Clawback, Shared Financial Burden, Shared Responsibility
17. Pre-Install Hardening & Validation
17
● Secure Operating System Pre-Installation
● Industry Standard Guidelines
– Centre For Internet Security (CIS) - Security Benchmarks
● Create Splunk Specific User/Group with Relevant Permissions
– Ensure Splunk Doesn’t Run as ‘Administrator’ or ‘Root’
● Verify Integrity of Binaries (Checksum Hash / Signature)
18. Implementation Hardening
18
● User Authentication & Role-Based Access Control
● Transport Encryption & Authentication (TLS)
● Secure Password Deployment
– Shared splunk.secret / Hashed Passwords in Deployment Apps
● Access Control Lists
– Simple IP/DNS Whitelisting or Blacklisting
● Disable Unnecessary Splunk Components (Splunk Web / REST Port)
● Configuration Change Monitoring via Splunk
19. Monitoring Environment (Security & IT Ops)
19
● Collect Local Operating System Hosts Logs / Report on Anomalies
– Security, Access, Application, Configuration, Patching & Performance
● Forward All Splunk’s Internal Logs into Indexers
● Splunk Crafted Reporting for ‘Splunk’ (Previously: Splunk on Splunk)
– Indexing Performance, Search Performance, Search Activity, Missing Forwarders
● Report On Users Attempting to Search Restricted Indexes
● Use Data Integrity Checking & Monitor Exceptions
22. Security Challenges & Solutions
22
● Example Security Challenges:
– Easier Implementation of Transport Encryption (TLS)?
‣ Scripted Certification Generation & Deployment via App
– How to Segment Data?
‣ According to Business Unit or Use Case (via Indexes)
● Discussion Time Limit: 15mins
24. Favourite Use Cases
24
● Example Use Cases:
– Self Healing with ServiceNow Integration with Ansible
– IT Operational Monitoring with IT Service Intelligence (Glass Tables)
– Malicious Behaviour Detection with Entropy Analysis on DNS Logs
● Discussion Time Limit: 15mins
25. Updates Announced at .conf 2016
● Introducing Splunk Enterprise 6.5 - Available Now
‣ Splunk ML Toolkit – a guided workbench and SPL extensions to help you create
and operationalize your own custom analytics based on your choice of algorithms.
‣ Tables, a new feature that lets you create and analyse tabular data views without
using SPL.
‣ Hadoop Data Roll give you another way to reduce historical data storage costs
while keeping full search capability.
● New Releases (General Availability October 2016):
– Splunk Enterprise Security [Minor Release]
– Splunk IT Service Intelligence [Major Release]
– Splunk User Behaviour Analytics [Major Release]
25
26. Get Involved!
● Splunk User Group Edinburgh
– https://usergroups.splunk.com/group/splunk-user-group-edinburgh.html
● Splunk’s Slack Group
– Register via www.splunk402.com/chat
– Channel: #edinburgh
● Present & Share at the User Group?
Connect:
‣ Harry McLaren | harry.mclaren@ecs.co.uk | @cyberharibu | harrymclaren.co.uk
‣ ECS | enquiries@ecs.co.uk | @ECS_IT | ecs.co.uk
26
High availability deployment - Indexer cluster: http://docs.splunk.com/Documentation/Splunk/6.5.0/Deploy/Indexercluster
Design considerations: http://docs.splunk.com/Documentation/Splunk/6.5.0/Deploy/Deploymentcharacteristics#Design_considerations
Set up load balancing: http://docs.splunk.com/Documentation/Splunk/6.5.0/Forwarding/Setuploadbalancingd
Use a load balancer with search head clustering: http://docs.splunk.com/Documentation/Splunk/6.5.0/DistSearch/UseSHCwithloadbalancers
Reference hardware: http://docs.splunk.com/Documentation/Splunk/6.5.0/Capacity/Referencehardware
Estimate your storage requirements: http://docs.splunk.com/Documentation/Splunk/6.5.0/Capacity/Estimateyourstoragerequirements
How Splunk Enterprise calculates disk storage: http://docs.splunk.com/Documentation/Splunk/6.5.0/Capacity/HowSplunkcalculatesdiskstorage
Universal forwarder system requirements: http://docs.splunk.com/Documentation/Forwarder/6.5.0/Forwarder/Systemrequirements
Install a Windows universal forwarder remotely with a static configuration: http://docs.splunk.com/Documentation/Forwarder/6.5.0/Forwarder/InstallaWindowsuniversalforwarderremotelywithastaticconfiguration
Install a *nix universal forwarder remotely with a static configuration: http://docs.splunk.com/Documentation/Forwarder/6.5.0/Forwarder/Installanixuniversalforwarderremotelywithastaticconfiguration
Splunk Education: http://www.splunk.com/view/education/SP-CAAAAH9
Use Cases: https://www.splunk.com/en_us/solutions/solution-areas.html
Centre For Internet Security: https://benchmarks.cisecurity.org/
Run Splunk Enterprise as a different or non-root user: https://docs.splunk.com/Documentation/Splunk/6.5.0/Installation/RunSplunkasadifferentornon-rootuser
Securing Splunk Enterprise: http://docs.splunk.com/Documentation/Splunk/6.5.0/Security/WhatyoucansecurewithSplunk
Use access control to secure Splunk data: http://docs.splunk.com/Documentation/Splunk/6.5.0/Security/UseaccesscontroltosecureSplunkdata
About securing Splunk Enterprise with SSL: http://docs.splunk.com/Documentation/Splunk/6.5.0/Security/AboutsecuringyourSplunkconfigurationwithSSL
Deploy secure passwords across multiple servers: http://docs.splunk.com/Documentation/Splunk/6.5.0/Security/Deploysecurepasswordsacrossmultipleservers
Use Access Control Lists: http://docs.splunk.com/Documentation/Splunk/6.5.0/Security/Useaccesscontrollists
What Splunk software logs about itself: http://docs.splunk.com/Documentation/Splunk/6.5.0/Troubleshooting/WhatSplunklogsaboutitself