Virtual Puppet Ecosystem Workshop - March 18,2020

Puppet Ecosystem
Technical Workshop
Virtual
18 MARCH 2020

Housekeeping
&
Introductions
2

Meet our Presenters
BOLT WORKSHOP3
Dan Shauver
Senior Sales Engineer, TOLA
shauver@puppet.com
● 20+ years of Dabbling in Automation & Orchestration
● Enthusiast of Music, Martial Arts, Fishing, and Big Data
Jerry Mozes
Senior Sales Engineer, West
jerry@puppet.com
● Lifelong Learner
● 25 years of Enterprise IT Insight
● Lover of hiking, biking, fishing, and camping
John Laffey
Senior Sales Engineer, Independent Llama Empire (ILE)
john.laffey@puppet.com
● World Champion Mime, Furry Division
● Published Bloggist - Top Secret Website
● K8S-phile

Continuous Compliance
Reduce financial risk, simplify audits
5
Configuration Management
Scale hybrid/multi-cloud management
Incident Remediation
Reduce time spent in a vulnerable state
Continuous Delivery
Accelerate customer time-to-value
Where does Puppet Fit?

Evergreen Automation Needs Key Workflows Products + Ecosystem
Continuous delivery: “We need to provide our
users with infrastructure and applications in an
automated self-service way, quickly and reliably.”
● Self-service infrastructure
● Monolithic application delivery
● Cloud-native application delivery
● Puppet Enterprise + vmWare
● Puppet Enterprise + Terraform
● Puppet Enterprise + Various
Incident remediation: “We need to fix issues in
our environment that have been raised through
security, break-fix, or monitoring triggers in an
automated, repeatable way.”
● Vulnerability management
● Event-based
● Break/fix
● Puppet Remediate + scan tools
● Puppet Enterprise + Splunk
● Puppet Enterprise + various
Continuous compliance: “We need to ensure our
systems and applications continuously meet
operational, regulatory and security requirements
and we’re flagged when they don’t.”
● Policy definition
● Policy enforcement
● Compliance reporting
● Puppet Enterprise
Configuration management: “We need to ensure
consistency by configuring and managing our
systems in an automated, predictable and
scalable way.”
● Self-service infrastructure
● Patch management
● OS baseline management
● Puppet Enterprise
● Puppet Enterprise + Terraform
● Puppet Enterprise + patch tool6

All About Bolt
• Bolt provides a simple way to execute agentless automation against remote hosts
• Zero requirements to the remote host. No agents, no python, no nothing
• Authenticate via SSH, WinRM, PCP
• Execute arbitrary commands, scripts, Bolt Tasks and Bolt Plans
• Use scripts in any language the remote host can execute
• Mature at your own pace from scripts → tasks → plans → puppet code
• If you have Puppet Enterprise, leverage PE from Bolt
BOLT WORKSHOP8

BOLT WORKSHOP10
Lab One:
Environment
Setup

Environment Setup
• Create a Bolt playground directory (i.e. ~/boltworkshop or c:usersyouboltworkshop)
• Create a Boltdir within your playground directory (i.e. ~/boltworkshop/Boltdir)
• Grab the Linux cert:
• Web Browser Method
• Visit https://hou-0318.classroom.puppet.com/download/student.pem
• Log in as client<X>@puppet.com
• <X> refers to your student number
• Store the contents in your Bolt playground directory as student.pem.
• i.e. ~/boltworkshop/Boltdir/student.pem
• c:usersyouboltworkshopBoltdir/student.pem
BOLT WORKSHOP11

Using Bolt
• Bolt command line syntax:
bolt [command|script|task|plan] run <name> --targets <targets> [options]
• To run a simple Bash command on a remote SSH host:
bolt command run 'echo Hello World!' --targets 10.0.0.1,10.0.0.2
--user root --private-key /path/to/key --transport ssh --no-host-key-check
• To run a simple PowerShell command on a remote WinRM host:
bolt command run 'write-host Hello World!' --targets 10.0.0.1,10.0.0.2
--user Administrator --password ‘Puppetlabs!' --transport winrm --no-ssl
BOLT WORKSHOP12

BOLT WORKSHOP13
Lab One:
Bolt Command

Lab One Instructions (A Long Command For A Ping!)
• Student Bolt Instances
Linux: hou-0318nix#.classroom.puppet.com
Windows: hou-0318#win.classroom.puppet.com
• Credentials
Linux: centos / student.pem
Windows: Administrator / Puppetlabs!
• Run these from the command line
bolt command run 'ping 8.8.8.8 -c2' --targets <linux_node>
--user centos --private-key ./Boltdir/student.pem --no-host-key-
check
bolt command run 'ping 8.8.8.8 –n 2’ --targets <win_node> --user
Administrator --password Puppetlabs! --transport winrm --no-ssl
BOLT WORKSHOP14

Easing Bolt Configuration
http://www.puppet.com/docs/bolt
• Bolt provides ways to make common activities more efficient
• Use a bolt.yaml file to store generic settings like modulepath or PE integration
• Use an inventory.yaml file to prevent typing in connection info every time
• Use a Boltdir to bundle all the files you need and have Bolt automatically use it
BOLT WORKSHOP15

Bolt Configuration File
• Bolt supports a configuration file to manage default configuration settings
• The configuration file is YAML and can have any name you want
• If unspecified, Bolt will look in these locations for an configuration file
• ./Boltdir/bolt.yaml
• ~/.puppetlabs/bolt/bolt.yaml (~ = %HOMEPATH%)
• A custom configuration file can be specified at runtime with --configfile [full path]
BOLT WORKSHOP16

Bolt Configuration File Syntax
http://www.puppet.com/docs/bolt/latest/bolt_configuration_options.html
modulepath: "/path/one:/path/two:/path/three“
inventoryfile: "~/.puppetlabs/bolt/inventory.yaml“
ssh:
host-key-check: false
winrm:
ssl: false
pcp:
[options]
log:
console: # or /path/to.log
level: info
BOLT WORKSHOP17

BOLT WORKSHOP18
Lab Two:
Use Bolt with
bolt.yaml

Lab Two Instructions (Making some Defaults)
1. Create a Boltdir directory in your playground folder
2. Create Boltdir/bolt.yaml in your bolt playground folder.
3. add host-key-check: false to SSH section of bolt.yaml and ssl: false to WinRM
section of bolt.yaml
ssh:
host-key-check: false
winrm:
ssl: false
3. Run commands to targets without specifying these 2 options
bolt command run 'ping 8.8.8.8 -c2' --targets <linux_node>
--user centos --private-key ./Boltdir/student.pem
bolt command run 'ping 8.8.8.8 –n 2’ --targets <win_node> --user Administrator
--password Puppetlabs! --transport winrm
BOLT WORKSHOP19

Bolt Inventory
• Bolt supports an inventory file to maintain a list of known targets
• The inventory file is YAML and can have any name you want
• If unspecified, Bolt will look in these locations for an inventory file:
• ./Boltdir/inventory.yaml
• ~/.puppetlabs/bolt/inventory.yaml (~ = %HOMEPATH%)
• A custom inventory file can be specified on the command line with --inventoryfile
[full path]
• A custom inventory file can be specified in bolt.yaml with the inventoryfile keyword.
BOLT WORKSHOP20

Bolt Inventory
groups:
- name: group_name
targets:
- IP_address_or_name_of_node1
- IP_address_or_name_of_node2
config:
transport: [ ssh | winrm ]
ssh:
user: user_name
run-as: root_name
private-key: /path/to/key
host-key-check: [ true | false ]
winrm:
user: user_name
password: password
ssl: [ true | false ]
BOLT WORKSHOP21
Nesting of groups is allowed:
groups:
- name: top_group
groups:
- name: sub_group
targets:
- …

BOLT WORKSHOP22
Lab Three:
Build an Inventory
File

Lab Three Reference
1. Create an inventory.yaml in your workshop folder
2. One group for your Linux node, connecting over SSH
3. One group for your Windows node, connecting over WinRM
Reference:
http://bit.ly/hou-0318inventory
Note:
● You’ll need to use your student number in the provided file. Replace #
BOLT WORKSHOP23

BOLT WORKSHOP24
Lab Four:
Use Bolt with
Inventory

Lab Four Reference (Using our Inventory)
1. Run bolt command run 'ping 8.8.8.8 -c2’ --targets linux
1. Run bolt command run 'ping 8.8.8.8 -n 2’ --targets windows
1. Run bolt command run 'hostname’ --targets linux,windows
BOLT WORKSHOP25

The Boltdir
To assist in packaging Bolt with source code, Bolt supports a Boltdir
When Bolt sees a directory called ./Boltdir it overrides all other configuration
The Boltdir has the following structure:
./Boltdir/bolt.yaml # Configuration settings
./Boltdir/inventory.yaml # Node inventory
./Boltdir/Puppetfile # Additional Forge modules
./Boltdir/modules # Path where modules are installed via Puppetfile
./Boltdir/site # Another modulepath, safe from Puppetfile
./Boltdir/modules/mymod/tasks # Bolt Tasks in module ‘mymod’
./Boltdir/modules/mymod/plans # Bolt Task Plans in module ‘mymod’
BOLT WORKSHOP26

Running Scripts
• Bolt will copy the script file to the remote host and run it in the native shell
• Linux = Bash
• Powershell = Windows
• Bolt expects the shell to execute the correct parser (based on file extension)
• You can pass arguments, but Bolt doesn’t do input validation for scripts
bolt script run <script> [[arg1] ... [argN]] [options]
BOLT WORKSHOP27

BOLT WORKSHOP28
Lab Five:
Run Scripts with
Bolt

Lab Five Instructions (Running a Script)
1. On your laptop, recreate the timesync.ps1 script at http://bit.ly/vbolttimesync
• Place this file above your Boltdir, in our ~/boltworkshop directory
2. From our boltworkshop directory: Use Bolt to run the script on your Windows node
bolt script run timesync.ps1 --targets windows
BOLT WORKSHOP29

Scripts into Tasks!
• Make your scripts more useful in Bolt by turning them into Puppet Tasks
• Any script file in a tasks directory of a module becomes a Task
• Tasks are name spaced automatically, using familiar Puppet syntax:
site/mymod/tasks/script1.ps1 # mymod::script1
site/aws/tasks/show_vpc.sh # aws::show_vpc
site/mysql/tasks/sql.rb # mysql::sql
site/yum/tasks/init.rb # yum
BOLT WORKSHOP30

BOLT WORKSHOP31
Lab Six:
Convert a Script
to a Task

Lab Six Instructions (Turning Scripts into Tasks)
1. Create Boltdir/site/tools/tasks
2. Move the timesync.ps1 script into the tasks directory
3. Run bolt task show to verify the new task is available
4. Run bolt task run tools::timesync --targets windows to execute the task.
BOLT WORKSHOP32

Bolt Task Metadata
• Make your Tasks more useful and robust by writing metadata files for them
• A metadata file has the same name as the script file, but with a .json extension
• Metadata files using the following (JSON) syntax:
{
"description": "Description of your Puppet Task",
"input_method": "environment | stdin | powershell",
"parameters": {
"param1": {
"description": "Description of the parameter usage",
"type": "String | Enum | Pattern | Integer | Array | Hash | Boolean“
}
}
}
BOLT WORKSHOP33

Bolt Task Input Methods
• The chosen input method determines how variables are accessible in the script
"input_method": "environment | stdin | powershell“
• environment: creates environment variable for each parameter as $PT_<variable>
• stdin: creates a JSON hash of all parameters and passes it via stdin
• powershell: creates a PowerShell named argument for each parameter
• The default for Linux is environment and stdin
• The default for Windows is powershell
BOLT WORKSHOP34

BOLT WORKSHOP35
Lab Seven:
Create and Run
Bolt Task with
Metadata

Lab Seven Instructions (Parameterizing Tasks)
1. Retrieve timesync.json from http://bit.ly/vbolttimesyncjson
2. Retrieve upgraded timesync.ps1 from http://bit.ly/vbolttimesyncrestart
• Adds a “Restart” Parameter
• Adds an if statement restarting W32Time if Restart is passed
3. Copy timesync.json and timesync.ps1 to ./Boltdir/site/tools/tasks
4. Run bolt task show (Look, we have a description now!)
5. Run bolt task show tools::timesync
6. Run bolt task run tools::timesync -t windows restart=true
BOLT WORKSHOP36

Writing Bolt Plans
Bolt Plans can use all the previously covered capabilities, and more, in a single plan. It’s
ideally suited to:
• Orchestrate multiple tasks
• Perform more complex logic & error handling, or interact with Puppet Enterprise
• Combine command/scripts/Tasks with applying desired-state Puppet code
• Plans are stored in a plans directory of a module and have a .pp extension
• Plans must be name spaced according to their module & plan name
BOLT WORKSHOP37

Writing Bolt Plans
located in modules/my_mod/plans/my_plan.pp
plan my_mod::my_plan(
String[1] $load_balancer,
TargetSpec $frontends,
TargetSpec $backends
) {
# process frontends
run_task('my_mod::lb_remove', $load_balancer, frontends => $frontends)
run_task('my_mod::update_frontend_app', $frontends, version => '1.2.3’)
run_task('my_mod::lb_add', $load_balancer, frontends => $frontends)
}
BOLT WORKSHOP38

Bolt Functions
Puppet Task Plans are written in Puppet DSL, with extra plan-specific functions:
BOLT WORKSHOP39
● add_facts: Add Facts
● add_to_group: Grouping
● apply_prep: Install Agent
● facts: Gather Facts
● fail_plan: Fail Condition
● get_targets: Target Node
● puppetdb_fact: Facts
● puppetdb_query: PQL Query
● run_command: Run Shell
● run_plan: Run a Plan
● run_script: Run a Script
● run_task: Run a Task
● set_feature: Shell/PS/Agent
● set_var: Set a Variable
● upload_file: Upload a File
● vars: Returns Variables
● wait_until_available: Wait
● without_default_logging: Slim
Logs
And More: https://puppet.com/docs/bolt/latest/plan_functions.html

Bolt Plan with Functions
plan loop(
TargetSpec $targets
) {
$targets = get_targets($targets)
$certnames = $targets.map |$target| { $target.host }
$targets.each |$target| {
run_task('my_task', $target, certificate => $certnames[$target.host] )
}
}
BOLT WORKSHOP40

BOLT WORKSHOP41
Lab Eight:
Create and Run a
Bolt Plan

Lab 8 Instructions (Building a Plan)
1. Retrieve http://bit.ly/vbolttimesyncplan
2. Place timesync.pp in Boltdir/site/tools/plans (New Directory)
3. Run bolt plan show
4. Run bolt plan show tools::timesync
5. Run bolt plan run tools::timesync --targets windows
BOLT WORKSHOP42

Desired State What Now?
• So far, we’ve been using scripting approaches to fix time synchronization issues
• But the script only works on Windows
• If we also built a script for Linux, it wouldn’t look anything like the Windows one
• We don’t *want* to keep running scripts on systems over and over
• How would we know if we needed to run the script again? Would that even work?
• Surely *someone* has solved this issue already, right?!
BOLT WORKSHOP43

Desired State What Now?
• To ensure Puppet modules are easy to use, the attributes a module supports for
configuration often align closely to the technology the module manages.
• Time synchronization on Linux and Windows are different enough that the attributes for
one platform are difficult to understand on the other
• It does not often happen that someone builds a fully cross platform module
• A fully cross platform time synchronization module could still emerge at some point, it will
just have to use more generic attributes for configuration and translate those to each
platform as appropriate.
• ^^^ Which is exactly what desired state configuration is all about!
BOLT WORKSHOP46

BOLT WORKSHOP47
Lab Nine:
Apply a Puppet
Manifest

Lab Nine Instructions (Applying Puppet Code)
• Retrieve Plan manifest from http://bit.ly/timesyncmanifest and save it as
timesync_windows.pp in your working directory (above Boltdir)
• bolt apply timesync_windows.pp --targets windows
NOTE: This lab will fail to complete: Could not find declared class
windowstime is the proper error!
BOLT WORKSHOP48

BOLT WORKSHOP50
Lab Ten:
Apply a Puppet
Manifest with a
Puppetfile

Lab Ten Instructions (Dependencies, the Puppetfile and You!)
1. Create boltworkshop/Boltdir/Puppetfile
1. Enter in dependencies: Stdlib, Registry, Windowstime and NTP
# Modules from the Puppet Forge.
mod 'puppetlabs-stdlib', '5.1.0'
mod 'puppetlabs-registry', '2.1.0'
mod 'ncorrare-windowstime', '0.4.3'
mod 'puppetlabs-ntp', '7.3.0'
1. bolt puppetfile install
2. With the modules now installed, let’s try this again:
bolt apply timesync_windows.pp --targets windows
BOLT WORKSHOP51

BOLT WORKSHOP53
Lab Eleven:
Cross Platform
Plans

Lab Eleven Instructions (Let’s get Multi-Platform!)
1. Retrieve http://bit.ly/vboltmultiplatform and place it in
boltworkshop/Boltdir/site/tools/plans/timesync_code.pp
2. Run bolt plan run tools::timesync_code --targets windows,linux
BOLT WORKSHOP54

Recap Time!
We’ve now learned how with Puppet Bolt:
• Commands, scripts, tasks, plans and manifests can be run with Puppet Bolt
• What the natural progression of automation looks like
• Turning interactive commands into scripts
• Turning scripts into tasks
• Turning tasks into plans
• Leveraging existing desired state modules and manifests
• Incorporating desired state code into plans
BOLT WORKSHOP55

Connecting to Puppet Enterprise
• To complete the automation journey, all that’s left to do is maturing into PE
• Leverage PE to continuously & automatically enforce desired state code
• Gain auditability in PE on Bolt Tasks, Task Plans and manifests
• Use RBAC in PE to delegate permissions to other teams/coworkers
• Connect Bolt to PE to gain direct control over PE-managed targets
BOLT WORKSHOP56

BOLT WORKSHOP57
Bolt in the Wild

What are people doing with bolt?
Patching systems (Puppet Enterprise + Bolt)
Users generally need to keep their systems up to date to maintain the integrity of their systems and applications that run on
top of them, patching systems can be tricky because it involves multiple steps that need to happen in a particular order, often
times those steps are undefined and users need to collaborate with other team members to understand the necessary steps.
On top of this these patches have to be applied during certain change windows so that they don’t disrupt business critical
systems. Usually these steps involve collecting some information about where to apply the patch, running a script to apply
the patch, verifying the patch was applied and informing someone or some system that the patch was applied.
Patching Network Devices (Puppet Enterprise + Bolt)
When updating network devices very often a binary file needs to be uploaded to the device. So the process may be: take
device out of pool, back up device config locally, upload file to remote device, then run a script to update the device,
subsequently check that the device is back online with config updated and then return the device to the pool. This is typically
done during a specific change window to reduce business impact. In some cases there won’t be a pool of devices, and
instead network segments will be offline during the updates.
Deploying new certificates
Making updates to systems
BOLT WORKSHOP58

Installing packages
To meet the needs of internal customers and unblock others in the organization generally folks in IT teams are asked to install
packages onto systems for various stakeholders. This typically means finding where the package is stored, understanding the version
that needs to be installed, logging into the system with the right credentials, installing the package with the right credentials that may
or may not be the same as the ones used to login, in some cases, creating a directory for the packages to be installed in and verifying
they were installed. This is typically done once but across a fleet of servers so it is difficult to login to each and install.
Clearing out log files when disks are full (Puppet Enterprise + Splunk + Bolt)
A very common problem in IT organizations is when logs start filling up due to some error or poor log rotation and end up filling the
disk space, this can lead to the application running on the system to fail or even sometimes the server itself becomes unusable.
Typically this means an ops person find out this happened either via a ticket or an alert and will run a script to clear out the space or
rotate and compress the logs.
Gathering information about systems
Administrators often get asked about their fleet for reporting, troubleshooting, or decision making and historically have been very
manual having to remote into each machine to gather details like status of a specific service/daemon, versions of installed packages
or hotfixes, free disk space, specific parameter, registry key or settings
59

Provisioning servers
Servers need to be provisioned so that different team members can use them. Provisioning can be a multistep process where users
need to understand what they are building, how large the different attributes should be, what type of server they are provisioning, from
where are they provisioning it from. This is tricky because these steps can sometimes take days. Due to virtualization provisioning
VMs is easier than provisioning bare metal however it still takes most of the same steps. Multiple “gold images” with different
applications/settings specific to a business group or team. Building instances on popular cloud providers also takes multiple steps
and users have to learn the right steps on the right platform to provision the servers, this can be tedious when it is clicking things in a
web gui or running multiple separate commands.
Restarting servers
In order to keep systems working properly, often times they need to be rebooted. This could be because a new application was
installed, this could be because a process is hung, or other administrative tasks have lead to a reboot being needed. This involves
multiple steps, logging into systems, knowing the right credentials for those systems that have the right permissions to reboot, and
then doing this across several servers.
Troubleshooting
When an application or service fails, we need to check status of services, verify settings, restart services, check connectivity status, or
reboot systems as troubleshooting steps
60

Automating runbooks
Many applications have maintenance steps that need to be performed periodically and generally are just documented as a list of
manual procedures to follow. Tasks and plans can be used to gradually mix in automation for more and more portions of the runbook
to make them easier, faster and more reliable, while still letting the hard-to-automate steps stay manual
Puppet Code development
When developing puppet code, practitioners need to set up a master server, install an agent on their target server, deploy code to the
master server, and then do a noop run on the target server to see if the code made the correct changes, if they want to iterate on that
they need to make a change to the code, commit to git, push, then deploy again to their master/agent, now they can run a noop run
again to test and see the behavior. Bolt lets them just do a “bolt apply manifest” to a target system, and avoid a lot of process in the
early stages of code development.
Stopping services
Services that run on machines have processes that are long running, in order for those servers to work they need to be started and
running, however there are some cases where an update is happening or something is broken that a service needs to stop. This
involves logging into servers with the right credentials, having permissions to stop or start services, know when is an appropriate time
to stop a service and communicate to people and monitoring tools that the services were stopped for the right reason so that
unnecessary firefighting doesn’t occur.
61

Editing Registry Settings
Some Windows configuration require setting registry key values to enable/disable a particular capability/feature/protocol (eg
TLS). Often this is done via the GUI on each machine by launching registry editor, navigating to the specified key and
inspecting the value Bolt can be used to gather specific registry key values to determine if its enabled/disabled as well as
setting the value without having to log onto each host.
Checking to see if a particular feature/capability is enabled/disabled.
Some vulnerabilities can be exploited if a particular Windows features is enabled (eg in the case of Wannacry SMBv1).
Usually requires an admin check a particular registry key value or server feature is enabled either by logging in and checking
via the gui or running a script on a list of machines. Bolt can be used to check if a particular feature is enabled across a
number of Windows hosts quickly and disabled/enable them if needed.
Checking to see which hotfixes have been applied (Puppet Enterprise + Splunk + Bolt)
Hotfixes are Windows patches that address a bug or security issue on Windows or other Microsoft product. Often when a
vulnerability is announced, they way to check to see if a Windows host is vulnerable is to check if a Hotfix(es) has been
installed or not. This may be done via the gui by navigating to Programs and viewing the list of installed updates. Bolt allows
a Windows admin to run powershell scripts to determine if a hotfix is applied across their fleet versus having to login to each
host.
62

Making Windows firewall changes/updates -
Windows Server has a built in host firewall that allows/blocks inbound and outbound traffic. When new applications are
deployed and need to talk/access resources on the Windows host, firewall policies need to be created/modified. This may be
done on each host by going to server manager, Windows Firewall, Advanced Security and creating a new policy or editing an
existing policy. Bolt allows this to be done across a number of Windows Server host concurrently versus having to login and
create/modify a firewall rule on each server host.
Create new AD Users
Before a user can login to their machines, their user accounts need to be created in Active Directory and can be done via the
Add User Account wizard but is slow and tedious if there are multiple users that need to be added at once. Bolt and
Powershell can be used to automate this as well as ensure that the user is created and added to the proper domain group.
63

BOLT WORKSHOP65
To Automation, and Beyond!
Self Paced Learning
https://bolt.guide
https://puppet.com/try-puppet/puppet-learning-vm/
Documentation and Help
https://docs.puppet.com/bolt/
https://slack.puppet.com/
Additional Resources
https://forge.puppet.com
https://learn.puppet.com
Puppet Engineering
shauver@puppet.com
jerry@puppet.com
john.laffey@puppet.com

Virtual Puppet Ecosystem Workshop - March 18,2020

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Virtual Puppet Ecosystem Workshop - March 18,2020

Semelhante a Virtual Puppet Ecosystem Workshop - March 18,2020 (20)

Mais de Puppet

Mais de Puppet (20)

Último

Último (20)

Virtual Puppet Ecosystem Workshop - March 18,2020