Between indexers, search heads, and forwarders, there’s a lot of configuration to manage in an Enterprise Splunk installation. In this session we’ll cover how to leverage Puppet to manage these configurations easily and efficiently. We’ll also touch on using Foreman to gain greater visibility into your deployment.

1. Using
Puppet
To
Manage
Splunk
Carl
Schwenk,
Senior
Systems
Administrator,
Citrix
Systems

2. Introduc@on
ü Carl
Schwenk
ü Senior
Systems
Administrator
ü Citrix
Online
ü Santa
Barbara,
CA
ü Carl.Schwenk@Citrix.com
The 2nd Annual Splunk Worldwide Users Conference 2
© Copyright Splunk 2011

3. The 2nd Annual Splunk Worldwide Users Conference 3
© Copyright Splunk 2011

4. Splunk
@
Citrix
Splunk Index
100+ Sourcetypes
10000+ Sources
Custom Config Files
NetScaller Load Balancer
Scripted input data
Host & Product status data
The 2nd Annual Splunk Worldwide Users Conference 4
© Copyright Splunk 2011

5. What
is
Puppet?
Puppet
…is
not..
…is…
A
config
file
transport
system
State
Enforcement
DriK
Management
A
means
of
remotely
execu@ng
arbitrary
Rapid
deployment
and
configura@on
Automated
commands
management.
Provisioning
A
replacement
for
good
administra@on
Only
as
good
as
the
developer
that
runs
it.
prac@ces.
Adop@on
The 2nd Annual Splunk Worldwide Users Conference 5
© Copyright Splunk 2011

6. Why
Manage
Splunk
with
Puppet?
Deployment
Manager
Puppet
• Manage
forwarders
by
classes
of
• Uses
exis@ng
host
classifica@ons.
Forwarder
Config
servers
• Automa@cally
provisioned
for
new
hosts
Management
• Manage
Indexer
and
Search
head
• Manage
Splunk
servers
in
one
place.
inputs
in
one
place.
• Rapid
Splunk
scaling.
Indexer
&
Search
• Configura@ons
are
backed
up
and
load
Head
Management
balanced
• U@lizes
the
Splunk
Deployment
• Maintains
forwarder
running
state
Monitor
applica@on
to
alert
status
of
Forwarder
Running
forwarders
• Keeps
forwarder
updated
with
current
config
Management
• No
code
to
learn.
Easy
to
use
interface.
The 2nd Annual Splunk Worldwide Users Conference 6
© Copyright Splunk 2011

7. class splunk::forwarder {
Puppet
Code
File {
owner => ‘splunk',
group => ‘splunk',
require => Package['splunkforwarder'],
notify => Exec['splunk_first_time_run', 'splunk_restart'],
}
$splunk_home = "/opt/splunkforwarder"
Package { "splunkforwarder":}
ensure => latest
service { "splunkforwarder":
enable => true,
ensure => running,
require => [File['splunkforwarder-init'],Package['splunkforwarder']],
}
file { "${splunk_home}/etc/apps/${outputs}":
ensure => directory,
recurse => true,
alias => 'outputs',
source => "puppet:///modules/splunk/${outputs}",
}
file { "${splunk_home}/etc/apps/base_inputs":
ensure => directory,
recurse => true,
source => "puppet:///modules/splunk/base_inputs",
alias => 'base_inputs',
}
if $splunk_profile {
$inputs = split($splunk_profile,",")
define install_class_apps {
file { "${splunk_home}/etc/apps/${name}":
ensure => directory,
recurse => true,
source => "puppet:///modules/splunk/${name}",
}
}
install_class_apps { $inputs:; }
}
exec { "${splunk_home}/bin/splunk start --accept-license":
alias => "splunk_first_time_run",
onlyif => "/usr/bin/test -e ${splunk_home}/ftr",
require => Package["splunkforwarder"],
}
exec { "${splunk_home}/bin/splunk restart":
alias => "splunk_restart",
onlyif => "/usr/bin/test ! -e ${splunk_home}/ftr",
refreshonly => true;
}
}
The 2nd Annual Splunk Worldwide Users Conference 7
© Copyright Splunk 2011

8. The
Foreman
–
Configura@on
Inheritance
Global Configurations
All hosts get packages splunkforwarder
WWW MySQL
Host Host
Group Group
$splunk_app = apache $splunkapp = mysql
Secure MySQL
www-backup Host
Host
$splunk_app = apache, backup $splunk_app = mysql, audit
The 2nd Annual Splunk Worldwide Users Conference 8
© Copyright Splunk 2011

9. The
Foreman
–
Dashboard
The 2nd Annual Splunk Worldwide Users Conference 9
© Copyright Splunk 2011

10. Lessons
Learned
ü Splunk
+
Puppet
=
BFF
ü Start
simply
ü Grow
slowly
ü Document
ü User
adop@on
may
be
your
hardest
challenge
The 2nd Annual Splunk Worldwide Users Conference 10
© Copyright Splunk 2011

11. Puppet
Code
for
Splunk
4.2
hdp://forge.puppetlabs.com
ü Universal
Forwarder
code
coming
soon
ü Indexer
and
Search
head
code
to
come
soon
The 2nd Annual Splunk Worldwide Users Conference 11
© Copyright Splunk 2011

12. Using
Puppet
To
Manage
Splunk
Carl
Schwenk,
Senior
Systems
Administrator,
Citrix
Systems