3. 3
The Climate Corporation
Started in 2006 based out of San Francisco, CA as an insurance company
Changed focus to agriculture in 2010
2015 moved exclusively to data analytics for farmers
Integrated Climate FieldView™ digital agriculture platform provides
farmers with field data collection, advanced agronomic modeling and
local weather monitoring
The Climate Corporation has separate networks and infrastructure but
can leverage our Parent company’s security teams and information
4. 4
My Background and Role
Joined The Climate Corporation in September 2015
Transitioned from a role with very similar responsibilities in Security
Analytics, Vulnerability Management, and Security Architecture.
Previously helped build out a Splunk infrastructure from a 30 GB/day
server to a global infrastructure that was licensed for 150 GB/day and
was built with scalability in mind
Splunk certifications as Certified Knowledge Manager and Certified
Admin
Splunk Certified Architect test later this year
5. 5
Security Team and Splunk
Startup environment – building security team and architecture
Splunk Enterprise was the first building block
First initiative was to onboard as much data as possible (50GB License)
Local Forwarders at each site with Indexers and a Search Head in AWS
Built a platform that could scale outside of IT Security
Ansible to centrally manage, configure, and build Linux Splunk systems
7. 7
Data Integrity and Continuity
All onboarding of data goes through our team
Alerting on incorrect timestamps
|metadata type=hosts index=* | eval diff=recentTime-lastTime | where diff > 1800 or diff < -1800 |
convert ctime(lastTime) AS last_log_date | table host diff last_log_date | where NOT [inputlookup
decom_maint_systems.csv | rename Host as host | table host]
Alerting on hosts that quit reporting
|metadata type=hosts index=aerohive | eval timenow=now() | eval lastseen=timenow-recentTime
| where lastseen > 3600 | eval last_seen=tostring(lastseen, “duration”) | table host last_seen |
where NOT [inputlookup decom_maint_systems.csv | rename Host as host | table host ]
8. 8
Data Integrity and Continuity
Ability to decommission or maintenance a system or site so it wasn’t
alerted on
9. 9
Application Visibility
API access allowed us to get data from our cloud providers
Splunk apps allowed us easy ways to get data from cloud applications
(AWS, Box, Google, Cloudlock, Qualys)
Much of the time we gain more customizable reporting, dash-boarding,
and alerting then the cloud apps give us out of the box
With the right API we have the ability to take action on this data without
giving users access to the cloud app
10. 10
Alert First Mentality
Searches and dashboards find Indicators of Compromise or Problems
Alerts to email, Jira, Slack, PagerDuty let us know when Indicator is
found
11. 11
What’s Next
Onboard more users into Splunk
More data sources
– Windows and Linux servers
– Okta and Sophos
Building our own Apps
Clustered Search heads and/or Clustered Indexers
Splunk IT Service Intelligence or Splunk Enterprise Security
3rd party integrations with Splunk
15. 15
Splunk Icons
Search Bar chart Lock Cloud Cloud – alt Folder Envelope
Storage - 3Storage
Server Indexer Forwarder Search head
Datacenters
Splunk serverFirewall
Desktop Laptop
Failed server
Hadoop storage Datacenter
16. 16
Splunk Icons
Application - alt Virtual machine Virtual server Network www or Global Tools
Log file Router Load balancer Script
Pie chart Gears/Settings Gear Messaging Tag/ticket
Document
Application
Analyze
Network Switch
Shield
Active Directory
17. 17
Splunk Icons
Checkmark InfoAlert StopiPhoneiPadAndroid
Twitter Facebook LinkedIn RSS You Tube Shopping cartGPS Tower
Healthcare Hospital Office building VoIP Phone Support POS Card Reader RFID
18. 18
Splunk Icons
Forwarder - AIX Forwarder- Datastore Forwarder- Free BSD Forwarder- Linux Forwarder- Windows Forwarder- Web Forwarder- OSX
Forwarder- Solaris Splunk server - AIX Splunk server -
Datastore
Splunk server -
Gear
Splunk server -
Linux
Splunk server -
Network
Splunk server -
Web
Splunk server - OSX Splunk server –
Free BSD
Splunk server –
Solaris Splunk search Failed Splunk server
24. 24
Splunk Object Style and Color
Hardware Product
Business/Corporat
e
Highlight OnlyGenericVirtualization
Generic
These are suggested
uses for colors only.
25. 25
Applying Splunk Object Style
To apply the Splunk object style to any shape:
1. Select the shape (Object A) with the desired style
2. Click on Format Painter (paintbrush) tool in toolbar
3. Click on new shape (Object B) to apply style
30. 30
Splunk Enterprise Overview
AlertsMessages Metrics ChangesScriptsConfiguration
s
Log Files
Indexes Any Data from Any Source
DatabasesNetworks Servers
Virtual
Machines
Smartphones
and Devices
Custom
Applications
Security
Tickets
Web
Server
Sensors
31. 31
Splunk Enterprise Scalability
Enterprise-class Scale, Resilience and Interoperability
Send data from thousands of servers using any combination of Splunk forwarders
Auto load-balanced forwarding to Splunk Indexers
Offload search load to Splunk Search Heads
32. 32
Integrated Analytics Platform for Hadoop Data
3
Full-featured,
Integrated
Product
Insights for
Everyone
Works with
What You
Have Today
Explore Visualize Dashboards ShareAnalyze
Hadoop
(MapReduce &
HDFS)
NoSQL
Data Stores
33. 33
Industry Leading Platform for Machine Data
Any Machine Data Operational Intelligence
HA Indexes
and Storage
Search and
Investigation
Proactive
Monitoring
Operational
Visibility
Real-time
Business
Insights
Commodity
Servers
Online
Services Web
Services
Servers
Security GPS
Location
Storage
Desktops
Networks
Packaged
Applications
Custom
ApplicationsMessaging
Telecoms
Online
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
34. 34
Table Example
Column Title Column Title Column Title Column Title
Text Text Text Text
Text Text Text Text
Text Text Text Text
Text Text Text Text
Text Text Text Text
Notas do Editor
Splunk is the leading platform for machine data analytics with over 5,600 organizations using Splunk (as of 8/1/13) – for data volumes ranging from tens of GBs to tens of TBs to over 100 TBs of data PER DAY.
Splunk software reliably collects and indexes all the streaming data from IT systems, technology devices and the Internet of Things in real-time - tens of thousands of sources in unpredictable formats and types. Splunk software is optimized for real-time, low latency and interactivity.
Organizations use Splunk software and their data the following ways:
1. Find and fix problems dramatically faster
2. Automatically monitor to identify issues, problems and attacks
3. Gain end-to-end visibility to track and deliver on IT KPIs and make better-informed IT decisions
4. Gain real-time insight from operational data to make better-informed business decisions
This is described as Operational Intelligence: visibility, insights and intelligence from operational data.
Hunk (Splunk Analytics for Hadoop) is a full-featured, integrated product offering – that delivers interactive data exploration, analysis and visualization for Hadoop.
Full-featured, integrated product:
Insights for everyone:
Works with what you have today:
Splunk is the leading platform for machine data analytics with over 5,600 organizations using Splunk (as of 8/1/13) – for data volumes ranging from tens of GBs to tens of TBs to over 100 TBs of data PER DAY.
Splunk software reliably collects and indexes all the streaming data from IT systems, technology devices and the Internet of Things in real-time - tens of thousands of sources in unpredictable formats and types. Splunk software is optimized for real-time, low latency and interactivity.
Organizations use Splunk software and their data the following ways:
1. Find and fix problems dramatically faster
2. Automatically monitor to identify issues, problems and attacks
3. Gain end-to-end visibility to track and deliver on IT KPIs and make better-informed IT decisions
4. Gain real-time insight from operational data to make better-informed business decisions
This is described as Operational Intelligence: visibility, insights and intelligence from operational data.