SlideShare uma empresa Scribd logo
1 de 43
Baixar para ler offline
© betadots GmbH 2023
© betadots GmbH 2023
Puppet is YAML
or
The power of Hiera 5
ConfigManagementCamp 2023
Martin Alfke
ma@betadots.de
© betadots GmbH 2023
Martin Alfke
CEO/Consultant/Trainer at betadots GmbH
Berlin, Germany
• Puppet Trainer and Puppet Solution Engineer
• Platform Engineering, Consulting and Training
• Agile methods, Scrum
• tuxmea (Twitter, GitHub, Slack)
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Puppet is YAML - Topics
- Starting with Puppet is hard
- YAML is simple
- Hiera is YAML
- YAML node classification
- YAML resource declaration
- YAML limitations of resource declaration
- Usage of Library Modules in YAML
- Puppet Plans in YAML
- Summary
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Why Puppet is hard
© betadots GmbH 2023
Starting with Puppet is hard
People new to Puppet IT Automation must learn many new things:
- GIT and Control-Repo
- Facter
- Node classification
- Puppet DSL
- Hiera
- Modules
- PDK and Onceover
How much can you learn in 3 days?
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Starting with Puppet is hard using YAML
How can we reduce learning time needed?
Is there something we can skip in the beginning?
- GIT and Control-Repo
- Facter
- Node classification (partly)
- Puppet DSL
- Hiera (YAML)
- Modules (partly)
- PDK and Onceover
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Hiera is YAML
- Hiera was introduced to separate code and data
- Layers of hierarchies provide different options for different configs
- Hierarchies are based on facts
- OS
- Datacenter/Network zone
- Application and Service
- Stage (Dev, Test, Prod)
- Most people use YAML in Hiera
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Puppet uses Hiera
- Puppet can query Hiera for data
- explicit lookup
- automatic data binding (from classes)
- Puppet can query ANY data from Hiera
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Node classification in YAML
© betadots GmbH 2023
Node classification
- Many people still pray the roles and profile pattern for node classification
- Profiles use Library Modules to implement technical settings
- Roles reflect the system business use case
But:
- Roles make sense only if one has many similar systems or
- If you insist in static node classification
- Roles and Profiles need an understanding of Puppet DSL and Modules
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Array
Query Hiera for classes to add to the nodes catalog:
# manifests/site.pp
lookup(
{
'name' => 'classes',
'value_type' => Array,
'default_value' => [],
}
).include
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Array
Hierarchy structure allows one to separate base from OS and from application
classes:
# Classes which are needed on all systems
data/common.yaml
# OS specific classes
data/os/%{facts.os.name}-%{facts.os.release.major}.yaml
# Application specific classes
data/app/%{trusted.extension.pp_application}-%{trusted.ext
ension.pp_service}-%{trusted.extension.pp_stage}.yaml
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Array
Configure Hiera to lookup classes from ALL hierarchies:
# data/common.yaml
---
lookup_options:
'classes':
merge: 'unique'
classes:
- 'class_a'
- 'class_b'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Array
Downside of Hiera Array node classification
- Classes can only be added
- Classes can not be overwritten
- There is no possibility to remove a class in higher hierarchy
- One only can set the merge behavior on a higher Hierarchy (first), omitting
all other classes arrays
Solution: Hiera Hash node classification
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Hash
Query Hiera for classes to add to the nodes catalog:
# manifests/site.pp
1 lookup( 'classes_hash',
2 {
3 'value_type' => Hash,
4 'default_value' => {},
5 }
6 ).each |$name, $c| {
7 unless $c.empty {
8 contain $c
9 } else {
10 # needs ipcrm/echo module
11 echo { "Class ${name} on ${facts['networking']['fqdn']} is disabled":
12 withpath => false,
13 }
14 }
15 }
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Hash
Configure Hiera to lookup classes Hash from ALL hierarchies:
# data/common.yaml
---
lookup_options:
'classes_hash':
merge: 'deep'
classes_hash:
'description of class_a': 'class_a'
'description of class_b': 'class_b'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Hash
Disabling a class on a higher Hierarchy:
# data/node/rz12pw5jz.domain.tld.yaml
---
classes_hash:
'description of class_a': '' # ← An empty string
'description of class_c': 'class_c'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Hash
This is a flexible solution where you can even query OS based classes:
# manifests/site.pp
lookup( "${facts['kernel'].downcase}_classes_hash",
{
'value_type' => Hash,
'default_value' => {}
}
).each |$name, $c| {
unless $c.empty {
contain $c
} else {
echo { "Class ${name} on ${facts['networking']['fqdn']} is
disabled": } # ← needs ipcrm/echo module
}
}
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Node classification using Hiera Hash
Configure Hiera to lookup classes Hash from ALL hierarchies:
# data/common.yaml
---
lookup_options:
"(.*)_classes_hash":
merge: 'deep'
linux_classes_hash:
'description of class_a': 'class_a'
'description of class_b': 'class_b'
windows_classes_hash:
'description of class_a': 'class_a'
'description of class_c': 'class_c'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Resource declaration in
YAML
© betadots GmbH 2023
Resource declaration in YAML
Resource types are a core concept of Puppet.
Every resource type describes a small portion to be configured on a system.
Core resource types are part of Puppet Agent installation.
Resource Type declaration via Hiera is possible using the stdlib::manage
class since version 8.2.0.
The stdlib Library Module must be added to Puppetfile
# Puppetfile
forge:
baseurl: https://forgeapi.puppetlabs.com/
mod 'ipcrm-echo', '0.1.7'
mod 'puppetlabs/stdlib', '8.5.0' # needs 8.2.0 or
newer
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Configure Hiera to use the stdlib::manage class and to fetch data from all
hierarchies
# data/common.yaml
---
lookup_options:
'classes_hash':
merge: 'deep'
'stdlib::manage::create_resources':
merge: 'deep'
classes_hash:
'hiera_yaml_resources': 'stdlib::manage'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
The stdlib::manage class has a parameter (create_resources).
Any resource will be created programmatically from this data hash.
The data hash has the following syntax:
stdlib::manage::create_resources:
'<resource type>':
'<title or name>':
'<parameters of the type>': '<value>'
stdlib::manage::create_resources:
'package':
'htop':
'ensure': 'installed'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Now we can add core resources to YAML data
# data/os/RedHat-8.yaml
---
stdlib::manage::create_resources:
'package':
'chrony':
'ensure': 'installed'
'file':
'/etc/chrony.conf':
'ensure': 'file'
'source': 'http://server/path/file'
'service':
'chrony':
'ensure': 'running'
'enable': true
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Static files should be part of the control-repo so they are also under version
control.
Files can be added to a module and Puppet must know where to find modules:
1. configure modulepath
# environment.conf
modulepath=site:modules:$basemodulepath
2. add profile module and files directory
mkdir -p site/profile/files
3. add config file
vi site/profile/files/chrony.conf
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Switch file to Puppet code location
# data/os/RedHat-8.yaml
---
stdlib::manage::create_resources:
'package':
'chrony':
'ensure': 'installed'
'file':
'/etc/chrony.conf':
'ensure': 'file'
'source': 'puppet:///modules/profile/chrony.conf'
'service':
'chrony':
'ensure': 'running'
'enable': true
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Resource defaults are possible using anchors and aliases (but only within the
same YAML file)
First we set the anchor:
# data/app/zoofoo-web-dev.yaml
---
file_defaults: @file_defaults
'owner': 'zoofoo'
'group': 'zoofoo'
'mode': '0644'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Resource declaration in YAML
Now we can use the alias:
# data/app/zoofoo-web-dev.yaml
---
stdlib::manage::create_resources:
'file':
'/etc/zoofoo':
<< : *file_defaults
'ensure': 'directory'
'/etc/zoofoo/app.cfg':
<< : *file_defaults
'ensure': 'file'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Limitations of resource declaration in YAML
1. it is not possible to execute Puppet functions within Hiera data
e.g.
content: "%{epp('profile/chrony.conf.epp')}"
https://tickets.puppetlabs.com/browse/HI-638
https://github.com/voxpupuli/hiera-eyaml/issues/336
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Limitations of resource declaration in YAML
2. it is not possible to set a hash key to an array in Hiera data (yes, this is
valid YAML)
e.g.
stdlib::manage::create_resources:
'package':
['htop', 'less', vim']:
'ensure': 'installed'
https://tickets.puppetlabs.com/browse/HI-637
https://github.com/voxpupuli/hiera-eyaml/issues/337
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Library modules in YAML
© betadots GmbH 2023
Usage of Library Modules in YAML
Library Modules offer the possibility to configure technical components and are
made available on Puppet Forge.
Carefully check which module you want to use (badges, author reputation, last
release date, open issues, active development, code review).
Modern modules allow settings to be configured via Hiera data
# data/node/rz12pw5jz.domain.tld.yaml
---
classes_hash:
'webserver': 'nginx'
nginx::port: 8080
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Usage of Library Modules in YAML
Defined types (based on existing resource types) are used to configure
component specific settings and can be added using stdlib::manage:
# data/node/rz12pw5jz.domain.tld.yaml
---
classes_hash:
'webserver': 'nginx'
stdlib::manage::create_resources:
'nginx::resource::server': # ← Defined Type
'www.domain.tld':
'listen_port': 80
'proxy': 'http://localhost:8088'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Bolt plans in YAML
© betadots GmbH 2023
Puppet Plans in YAML
Puppet/Bolt Plans can be written in PuppetDSL or YAML
# site/profile/plans/zoofoo/install.yaml
---
parameters:
version:
type: 'String'
description: 'Version of ZooFoo to deploy'
frontends:
type: 'TargetSpec'
description: 'The frontend web servers'
backends:
type: 'TargetSpec'
description: 'The backend servers'
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Puppet Plans in YAML
Puppet/Bolt Plans can be written in PuppetDSL or YAML
# site/profile/plans/zoofoo/install.yaml - continued
---
steps:
- name: 'zoofoo_fe'
task: 'profile::zoofoo_install'
targets: $frontends
description: 'Install zoofoo frontends'
parameters:
version: $version
return: $zoofoo_fe.map |result| { result['stdout']}
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Summary
© betadots GmbH 2023
Summary
YAML hash based node classification is flexible, extensible and mature and
should be considered best practice.
Puppet YAML resources allows an easy start but has its limitations.
Keep your YAML data
- simple
- obvious
- flexible
Don't be afraid of long YAML files and use an IDE
Validate your YAML data in CI/CD
Visualize your Hiera Data in a web interface (Hiera Data Manager)
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Summary
YAML HELL
Check proper YAML syntax!
Quote Strings (especially if the string starts with digits)!!!!!!!
Quote Regexp (especially when starting with & or *)
Quote no as it is a Boolean.
Quote keys, see all of the above
Quote Version String. Unquoted can lead to unintentional numbers
https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Summary
Module Authors:
- please provide Hiera YAML data examples
- please add code documentation and use puppet-strings
Puppet and Voxpupuli
- please allow Hiera hash key as array (HI-637 and hiera-eyaml#337)
- please allow Puppet function class from Hiera (HI-638 and
hiera-eyaml#336)
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Summary
More complex configurations must be done in Puppet DSL modules or classes.
e.g.
- you need to manage an application on several OS
- there is more to do than only add some resource types
While using YAML, one can start exploring and learning on how to write Puppet
DSL code.
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
Summary
Use Hiera Data Manager to visualize your YAML data
https://github.com/betadots/hdm
Puppet is YAML - cfgmgmtcamp 2023
© betadots GmbH 2023
© betadots GmbH 2023
Puppet is YAML
CfgMgmtCamp 2023
Thank you!

Mais conteúdo relacionado

Semelhante a CfgMgmtCamp 2023 - Puppet is YAML.pdf

Abstract data types
Abstract data typesAbstract data types
Abstract data typesYoung Alista
 
Abstract data types
Abstract data typesAbstract data types
Abstract data typesTony Nguyen
 
Abstract data types
Abstract data typesAbstract data types
Abstract data typesFraboni Ec
 
Abstract data types
Abstract data typesAbstract data types
Abstract data typesJames Wong
 
Using hiera with puppet
Using hiera with puppetUsing hiera with puppet
Using hiera with puppetScott Lackey
 
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet Forge
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet ForgePuppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet Forge
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet ForgePuppet
 
Helm Security Webinar
Helm Security WebinarHelm Security Webinar
Helm Security WebinarDeep Datta
 
Writing and Sharing Great Modules with the Puppet Forge
Writing and Sharing Great Modules with the Puppet ForgeWriting and Sharing Great Modules with the Puppet Forge
Writing and Sharing Great Modules with the Puppet ForgePuppet
 
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)Jyotirmoy Sundi
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributesMohammad Alyan
 
Manageable Puppet Infrastructure - PuppetConf 2014
Manageable Puppet Infrastructure - PuppetConf 2014Manageable Puppet Infrastructure - PuppetConf 2014
Manageable Puppet Infrastructure - PuppetConf 2014Puppet
 
Foreman presentation
Foreman presentationForeman presentation
Foreman presentationGlen Ogilvie
 
Rpug - Puppet 4 Module Data
Rpug - Puppet 4 Module DataRpug - Puppet 4 Module Data
Rpug - Puppet 4 Module DataJere Julian
 
Puppet Camp Berlin 2014: Manageable puppet infrastructure
Puppet Camp Berlin 2014: Manageable puppet infrastructurePuppet Camp Berlin 2014: Manageable puppet infrastructure
Puppet Camp Berlin 2014: Manageable puppet infrastructurePuppet
 
Ben ford intro
Ben ford introBen ford intro
Ben ford introPuppet
 
Telemetry doesn't have to be scary; Ben Ford
Telemetry doesn't have to be scary; Ben FordTelemetry doesn't have to be scary; Ben Ford
Telemetry doesn't have to be scary; Ben FordPuppet
 

Semelhante a CfgMgmtCamp 2023 - Puppet is YAML.pdf (20)

Abstract data types
Abstract data typesAbstract data types
Abstract data types
 
Abstract data types
Abstract data typesAbstract data types
Abstract data types
 
Abstract data types
Abstract data typesAbstract data types
Abstract data types
 
Abstract data types
Abstract data typesAbstract data types
Abstract data types
 
Using hiera with puppet
Using hiera with puppetUsing hiera with puppet
Using hiera with puppet
 
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet Forge
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet ForgePuppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet Forge
Puppet Camp Amsterdam 2015: How To Leverage The Power of the Puppet Forge
 
ch11-1.ppt
ch11-1.pptch11-1.ppt
ch11-1.ppt
 
Helm Security Webinar
Helm Security WebinarHelm Security Webinar
Helm Security Webinar
 
R sharing 101
R sharing 101R sharing 101
R sharing 101
 
Writing and Sharing Great Modules with the Puppet Forge
Writing and Sharing Great Modules with the Puppet ForgeWriting and Sharing Great Modules with the Puppet Forge
Writing and Sharing Great Modules with the Puppet Forge
 
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)
Cascading talk in Etsy (http://www.meetup.com/cascading/events/169390262/)
 
2 second lesson- attributes
2 second lesson- attributes2 second lesson- attributes
2 second lesson- attributes
 
Manageable Puppet Infrastructure - PuppetConf 2014
Manageable Puppet Infrastructure - PuppetConf 2014Manageable Puppet Infrastructure - PuppetConf 2014
Manageable Puppet Infrastructure - PuppetConf 2014
 
Foreman presentation
Foreman presentationForeman presentation
Foreman presentation
 
Rpug - Puppet 4 Module Data
Rpug - Puppet 4 Module DataRpug - Puppet 4 Module Data
Rpug - Puppet 4 Module Data
 
Puppet Camp Berlin 2014: Manageable puppet infrastructure
Puppet Camp Berlin 2014: Manageable puppet infrastructurePuppet Camp Berlin 2014: Manageable puppet infrastructure
Puppet Camp Berlin 2014: Manageable puppet infrastructure
 
Little Gems in TYPO3 v12
Little Gems in TYPO3 v12Little Gems in TYPO3 v12
Little Gems in TYPO3 v12
 
Ben ford intro
Ben ford introBen ford intro
Ben ford intro
 
Telemetry doesn't have to be scary; Ben Ford
Telemetry doesn't have to be scary; Ben FordTelemetry doesn't have to be scary; Ben Ford
Telemetry doesn't have to be scary; Ben Ford
 
Perl 20tips
Perl 20tipsPerl 20tips
Perl 20tips
 

Mais de Martin Alfke

PuppetCamp2021-Testing Modules and ControlRepo.pdf
PuppetCamp2021-Testing Modules and ControlRepo.pdfPuppetCamp2021-Testing Modules and ControlRepo.pdf
PuppetCamp2021-Testing Modules and ControlRepo.pdfMartin Alfke
 
Puppet Camp Germany 2020 - Puppet Control Repo and GIT
Puppet Camp Germany 2020 - Puppet Control Repo and GITPuppet Camp Germany 2020 - Puppet Control Repo and GIT
Puppet Camp Germany 2020 - Puppet Control Repo and GITMartin Alfke
 
DevOps - How to get technical buy in
DevOps - How to get technical buy inDevOps - How to get technical buy in
DevOps - How to get technical buy inMartin Alfke
 
ADDO 2019 DevOps in a containerized world
ADDO 2019 DevOps in a containerized worldADDO 2019 DevOps in a containerized world
ADDO 2019 DevOps in a containerized worldMartin Alfke
 
OpenRheinRuhr 2018 - Ops hates containers! Why?
OpenRheinRuhr 2018 - Ops hates containers! Why?OpenRheinRuhr 2018 - Ops hates containers! Why?
OpenRheinRuhr 2018 - Ops hates containers! Why?Martin Alfke
 
PuppetConf 2016 Moving from Exec to Types and Provides
PuppetConf 2016 Moving from Exec to Types and ProvidesPuppetConf 2016 Moving from Exec to Types and Provides
PuppetConf 2016 Moving from Exec to Types and ProvidesMartin Alfke
 
Puppet Camp Paris 2016 Data in Modules
Puppet Camp Paris 2016 Data in ModulesPuppet Camp Paris 2016 Data in Modules
Puppet Camp Paris 2016 Data in ModulesMartin Alfke
 
Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Martin Alfke
 
Upgrading Puppet CommitterConf Essen 2014
Upgrading Puppet CommitterConf Essen 2014Upgrading Puppet CommitterConf Essen 2014
Upgrading Puppet CommitterConf Essen 2014Martin Alfke
 
GUUG Hamburg OpenNebula
GUUG Hamburg OpenNebulaGUUG Hamburg OpenNebula
GUUG Hamburg OpenNebulaMartin Alfke
 
Puppet camp london-modulerewritingsmartway
Puppet camp london-modulerewritingsmartwayPuppet camp london-modulerewritingsmartway
Puppet camp london-modulerewritingsmartwayMartin Alfke
 
Puppet future parser
Puppet future parserPuppet future parser
Puppet future parserMartin Alfke
 
developing sysadmin, sysadmining developersGuug devops puppet
developing sysadmin, sysadmining developersGuug devops puppetdeveloping sysadmin, sysadmining developersGuug devops puppet
developing sysadmin, sysadmining developersGuug devops puppetMartin Alfke
 
Gluster fs buero20_presentation
Gluster fs buero20_presentationGluster fs buero20_presentation
Gluster fs buero20_presentationMartin Alfke
 
Puppet buero20 presentation
Puppet buero20 presentationPuppet buero20 presentation
Puppet buero20 presentationMartin Alfke
 

Mais de Martin Alfke (17)

PuppetCamp2021-Testing Modules and ControlRepo.pdf
PuppetCamp2021-Testing Modules and ControlRepo.pdfPuppetCamp2021-Testing Modules and ControlRepo.pdf
PuppetCamp2021-Testing Modules and ControlRepo.pdf
 
Puppet Camp Germany 2020 - Puppet Control Repo and GIT
Puppet Camp Germany 2020 - Puppet Control Repo and GITPuppet Camp Germany 2020 - Puppet Control Repo and GIT
Puppet Camp Germany 2020 - Puppet Control Repo and GIT
 
DevOps - How to get technical buy in
DevOps - How to get technical buy inDevOps - How to get technical buy in
DevOps - How to get technical buy in
 
ADDO 2019 DevOps in a containerized world
ADDO 2019 DevOps in a containerized worldADDO 2019 DevOps in a containerized world
ADDO 2019 DevOps in a containerized world
 
OpenRheinRuhr 2018 - Ops hates containers! Why?
OpenRheinRuhr 2018 - Ops hates containers! Why?OpenRheinRuhr 2018 - Ops hates containers! Why?
OpenRheinRuhr 2018 - Ops hates containers! Why?
 
PuppetConf 2016 Moving from Exec to Types and Provides
PuppetConf 2016 Moving from Exec to Types and ProvidesPuppetConf 2016 Moving from Exec to Types and Provides
PuppetConf 2016 Moving from Exec to Types and Provides
 
Puppet Camp Paris 2016 Data in Modules
Puppet Camp Paris 2016 Data in ModulesPuppet Camp Paris 2016 Data in Modules
Puppet Camp Paris 2016 Data in Modules
 
Power of Puppet 4
Power of Puppet 4Power of Puppet 4
Power of Puppet 4
 
Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?
 
Upgrading Puppet CommitterConf Essen 2014
Upgrading Puppet CommitterConf Essen 2014Upgrading Puppet CommitterConf Essen 2014
Upgrading Puppet CommitterConf Essen 2014
 
GUUG Hamburg OpenNebula
GUUG Hamburg OpenNebulaGUUG Hamburg OpenNebula
GUUG Hamburg OpenNebula
 
Puppet camp london-modulerewritingsmartway
Puppet camp london-modulerewritingsmartwayPuppet camp london-modulerewritingsmartway
Puppet camp london-modulerewritingsmartway
 
One
OneOne
One
 
Puppet future parser
Puppet future parserPuppet future parser
Puppet future parser
 
developing sysadmin, sysadmining developersGuug devops puppet
developing sysadmin, sysadmining developersGuug devops puppetdeveloping sysadmin, sysadmining developersGuug devops puppet
developing sysadmin, sysadmining developersGuug devops puppet
 
Gluster fs buero20_presentation
Gluster fs buero20_presentationGluster fs buero20_presentation
Gluster fs buero20_presentation
 
Puppet buero20 presentation
Puppet buero20 presentationPuppet buero20 presentation
Puppet buero20 presentation
 

Último

Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Shubham Pant
 
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsVision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsRoxana Stingu
 
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSTYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSedrianrheine
 
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...APNIC
 
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024Jan Löffler
 
Zero-day Vulnerabilities
Zero-day VulnerabilitiesZero-day Vulnerabilities
Zero-day Vulnerabilitiesalihassaah1994
 
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdfIntroduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdfShreedeep Rayamajhi
 
Bio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxBio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxnaveenithkrishnan
 
Computer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteComputer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteMavein
 
Presentation2.pptx - JoyPress Wordpress
Presentation2.pptx -  JoyPress WordpressPresentation2.pptx -  JoyPress Wordpress
Presentation2.pptx - JoyPress Wordpressssuser166378
 
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfLESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfmchristianalwyn
 
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSLESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSlesteraporado16
 

Último (12)

Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024
 
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsVision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
 
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSTYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
 
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
 
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
 
Zero-day Vulnerabilities
Zero-day VulnerabilitiesZero-day Vulnerabilities
Zero-day Vulnerabilities
 
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdfIntroduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
 
Bio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxBio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptx
 
Computer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteComputer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a Website
 
Presentation2.pptx - JoyPress Wordpress
Presentation2.pptx -  JoyPress WordpressPresentation2.pptx -  JoyPress Wordpress
Presentation2.pptx - JoyPress Wordpress
 
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfLESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
 
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSLESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
 

CfgMgmtCamp 2023 - Puppet is YAML.pdf

  • 1. © betadots GmbH 2023 © betadots GmbH 2023 Puppet is YAML or The power of Hiera 5 ConfigManagementCamp 2023 Martin Alfke ma@betadots.de
  • 2. © betadots GmbH 2023 Martin Alfke CEO/Consultant/Trainer at betadots GmbH Berlin, Germany • Puppet Trainer and Puppet Solution Engineer • Platform Engineering, Consulting and Training • Agile methods, Scrum • tuxmea (Twitter, GitHub, Slack) Puppet is YAML - cfgmgmtcamp 2023
  • 3. © betadots GmbH 2023 Puppet is YAML - Topics - Starting with Puppet is hard - YAML is simple - Hiera is YAML - YAML node classification - YAML resource declaration - YAML limitations of resource declaration - Usage of Library Modules in YAML - Puppet Plans in YAML - Summary Puppet is YAML - cfgmgmtcamp 2023
  • 4. © betadots GmbH 2023 © betadots GmbH 2023 Why Puppet is hard
  • 5. © betadots GmbH 2023 Starting with Puppet is hard People new to Puppet IT Automation must learn many new things: - GIT and Control-Repo - Facter - Node classification - Puppet DSL - Hiera - Modules - PDK and Onceover How much can you learn in 3 days? Puppet is YAML - cfgmgmtcamp 2023
  • 6. © betadots GmbH 2023 Starting with Puppet is hard using YAML How can we reduce learning time needed? Is there something we can skip in the beginning? - GIT and Control-Repo - Facter - Node classification (partly) - Puppet DSL - Hiera (YAML) - Modules (partly) - PDK and Onceover Puppet is YAML - cfgmgmtcamp 2023
  • 7. © betadots GmbH 2023 Hiera is YAML - Hiera was introduced to separate code and data - Layers of hierarchies provide different options for different configs - Hierarchies are based on facts - OS - Datacenter/Network zone - Application and Service - Stage (Dev, Test, Prod) - Most people use YAML in Hiera Puppet is YAML - cfgmgmtcamp 2023
  • 8. © betadots GmbH 2023 Puppet uses Hiera - Puppet can query Hiera for data - explicit lookup - automatic data binding (from classes) - Puppet can query ANY data from Hiera Puppet is YAML - cfgmgmtcamp 2023
  • 9. © betadots GmbH 2023 © betadots GmbH 2023 Node classification in YAML
  • 10. © betadots GmbH 2023 Node classification - Many people still pray the roles and profile pattern for node classification - Profiles use Library Modules to implement technical settings - Roles reflect the system business use case But: - Roles make sense only if one has many similar systems or - If you insist in static node classification - Roles and Profiles need an understanding of Puppet DSL and Modules Puppet is YAML - cfgmgmtcamp 2023
  • 11. © betadots GmbH 2023 Node classification using Hiera Array Query Hiera for classes to add to the nodes catalog: # manifests/site.pp lookup( { 'name' => 'classes', 'value_type' => Array, 'default_value' => [], } ).include Puppet is YAML - cfgmgmtcamp 2023
  • 12. © betadots GmbH 2023 Node classification using Hiera Array Hierarchy structure allows one to separate base from OS and from application classes: # Classes which are needed on all systems data/common.yaml # OS specific classes data/os/%{facts.os.name}-%{facts.os.release.major}.yaml # Application specific classes data/app/%{trusted.extension.pp_application}-%{trusted.ext ension.pp_service}-%{trusted.extension.pp_stage}.yaml Puppet is YAML - cfgmgmtcamp 2023
  • 13. © betadots GmbH 2023 Node classification using Hiera Array Configure Hiera to lookup classes from ALL hierarchies: # data/common.yaml --- lookup_options: 'classes': merge: 'unique' classes: - 'class_a' - 'class_b' Puppet is YAML - cfgmgmtcamp 2023
  • 14. © betadots GmbH 2023 Node classification using Hiera Array Downside of Hiera Array node classification - Classes can only be added - Classes can not be overwritten - There is no possibility to remove a class in higher hierarchy - One only can set the merge behavior on a higher Hierarchy (first), omitting all other classes arrays Solution: Hiera Hash node classification Puppet is YAML - cfgmgmtcamp 2023
  • 15. © betadots GmbH 2023 Node classification using Hiera Hash Query Hiera for classes to add to the nodes catalog: # manifests/site.pp 1 lookup( 'classes_hash', 2 { 3 'value_type' => Hash, 4 'default_value' => {}, 5 } 6 ).each |$name, $c| { 7 unless $c.empty { 8 contain $c 9 } else { 10 # needs ipcrm/echo module 11 echo { "Class ${name} on ${facts['networking']['fqdn']} is disabled": 12 withpath => false, 13 } 14 } 15 } Puppet is YAML - cfgmgmtcamp 2023
  • 16. © betadots GmbH 2023 Node classification using Hiera Hash Configure Hiera to lookup classes Hash from ALL hierarchies: # data/common.yaml --- lookup_options: 'classes_hash': merge: 'deep' classes_hash: 'description of class_a': 'class_a' 'description of class_b': 'class_b' Puppet is YAML - cfgmgmtcamp 2023
  • 17. © betadots GmbH 2023 Node classification using Hiera Hash Disabling a class on a higher Hierarchy: # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'description of class_a': '' # ← An empty string 'description of class_c': 'class_c' Puppet is YAML - cfgmgmtcamp 2023
  • 18. © betadots GmbH 2023 Node classification using Hiera Hash This is a flexible solution where you can even query OS based classes: # manifests/site.pp lookup( "${facts['kernel'].downcase}_classes_hash", { 'value_type' => Hash, 'default_value' => {} } ).each |$name, $c| { unless $c.empty { contain $c } else { echo { "Class ${name} on ${facts['networking']['fqdn']} is disabled": } # ← needs ipcrm/echo module } } Puppet is YAML - cfgmgmtcamp 2023
  • 19. © betadots GmbH 2023 Node classification using Hiera Hash Configure Hiera to lookup classes Hash from ALL hierarchies: # data/common.yaml --- lookup_options: "(.*)_classes_hash": merge: 'deep' linux_classes_hash: 'description of class_a': 'class_a' 'description of class_b': 'class_b' windows_classes_hash: 'description of class_a': 'class_a' 'description of class_c': 'class_c' Puppet is YAML - cfgmgmtcamp 2023
  • 20. © betadots GmbH 2023 © betadots GmbH 2023 Resource declaration in YAML
  • 21. © betadots GmbH 2023 Resource declaration in YAML Resource types are a core concept of Puppet. Every resource type describes a small portion to be configured on a system. Core resource types are part of Puppet Agent installation. Resource Type declaration via Hiera is possible using the stdlib::manage class since version 8.2.0. The stdlib Library Module must be added to Puppetfile # Puppetfile forge: baseurl: https://forgeapi.puppetlabs.com/ mod 'ipcrm-echo', '0.1.7' mod 'puppetlabs/stdlib', '8.5.0' # needs 8.2.0 or newer Puppet is YAML - cfgmgmtcamp 2023
  • 22. © betadots GmbH 2023 Resource declaration in YAML Configure Hiera to use the stdlib::manage class and to fetch data from all hierarchies # data/common.yaml --- lookup_options: 'classes_hash': merge: 'deep' 'stdlib::manage::create_resources': merge: 'deep' classes_hash: 'hiera_yaml_resources': 'stdlib::manage' Puppet is YAML - cfgmgmtcamp 2023
  • 23. © betadots GmbH 2023 Resource declaration in YAML The stdlib::manage class has a parameter (create_resources). Any resource will be created programmatically from this data hash. The data hash has the following syntax: stdlib::manage::create_resources: '<resource type>': '<title or name>': '<parameters of the type>': '<value>' stdlib::manage::create_resources: 'package': 'htop': 'ensure': 'installed' Puppet is YAML - cfgmgmtcamp 2023
  • 24. © betadots GmbH 2023 Resource declaration in YAML Now we can add core resources to YAML data # data/os/RedHat-8.yaml --- stdlib::manage::create_resources: 'package': 'chrony': 'ensure': 'installed' 'file': '/etc/chrony.conf': 'ensure': 'file' 'source': 'http://server/path/file' 'service': 'chrony': 'ensure': 'running' 'enable': true Puppet is YAML - cfgmgmtcamp 2023
  • 25. © betadots GmbH 2023 Resource declaration in YAML Static files should be part of the control-repo so they are also under version control. Files can be added to a module and Puppet must know where to find modules: 1. configure modulepath # environment.conf modulepath=site:modules:$basemodulepath 2. add profile module and files directory mkdir -p site/profile/files 3. add config file vi site/profile/files/chrony.conf Puppet is YAML - cfgmgmtcamp 2023
  • 26. © betadots GmbH 2023 Resource declaration in YAML Switch file to Puppet code location # data/os/RedHat-8.yaml --- stdlib::manage::create_resources: 'package': 'chrony': 'ensure': 'installed' 'file': '/etc/chrony.conf': 'ensure': 'file' 'source': 'puppet:///modules/profile/chrony.conf' 'service': 'chrony': 'ensure': 'running' 'enable': true Puppet is YAML - cfgmgmtcamp 2023
  • 27. © betadots GmbH 2023 Resource declaration in YAML Resource defaults are possible using anchors and aliases (but only within the same YAML file) First we set the anchor: # data/app/zoofoo-web-dev.yaml --- file_defaults: @file_defaults 'owner': 'zoofoo' 'group': 'zoofoo' 'mode': '0644' Puppet is YAML - cfgmgmtcamp 2023
  • 28. © betadots GmbH 2023 Resource declaration in YAML Now we can use the alias: # data/app/zoofoo-web-dev.yaml --- stdlib::manage::create_resources: 'file': '/etc/zoofoo': << : *file_defaults 'ensure': 'directory' '/etc/zoofoo/app.cfg': << : *file_defaults 'ensure': 'file' Puppet is YAML - cfgmgmtcamp 2023
  • 29. © betadots GmbH 2023 Limitations of resource declaration in YAML 1. it is not possible to execute Puppet functions within Hiera data e.g. content: "%{epp('profile/chrony.conf.epp')}" https://tickets.puppetlabs.com/browse/HI-638 https://github.com/voxpupuli/hiera-eyaml/issues/336 Puppet is YAML - cfgmgmtcamp 2023
  • 30. © betadots GmbH 2023 Limitations of resource declaration in YAML 2. it is not possible to set a hash key to an array in Hiera data (yes, this is valid YAML) e.g. stdlib::manage::create_resources: 'package': ['htop', 'less', vim']: 'ensure': 'installed' https://tickets.puppetlabs.com/browse/HI-637 https://github.com/voxpupuli/hiera-eyaml/issues/337 Puppet is YAML - cfgmgmtcamp 2023
  • 31. © betadots GmbH 2023 © betadots GmbH 2023 Library modules in YAML
  • 32. © betadots GmbH 2023 Usage of Library Modules in YAML Library Modules offer the possibility to configure technical components and are made available on Puppet Forge. Carefully check which module you want to use (badges, author reputation, last release date, open issues, active development, code review). Modern modules allow settings to be configured via Hiera data # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'webserver': 'nginx' nginx::port: 8080 Puppet is YAML - cfgmgmtcamp 2023
  • 33. © betadots GmbH 2023 Usage of Library Modules in YAML Defined types (based on existing resource types) are used to configure component specific settings and can be added using stdlib::manage: # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'webserver': 'nginx' stdlib::manage::create_resources: 'nginx::resource::server': # ← Defined Type 'www.domain.tld': 'listen_port': 80 'proxy': 'http://localhost:8088' Puppet is YAML - cfgmgmtcamp 2023
  • 34. © betadots GmbH 2023 © betadots GmbH 2023 Bolt plans in YAML
  • 35. © betadots GmbH 2023 Puppet Plans in YAML Puppet/Bolt Plans can be written in PuppetDSL or YAML # site/profile/plans/zoofoo/install.yaml --- parameters: version: type: 'String' description: 'Version of ZooFoo to deploy' frontends: type: 'TargetSpec' description: 'The frontend web servers' backends: type: 'TargetSpec' description: 'The backend servers' Puppet is YAML - cfgmgmtcamp 2023
  • 36. © betadots GmbH 2023 Puppet Plans in YAML Puppet/Bolt Plans can be written in PuppetDSL or YAML # site/profile/plans/zoofoo/install.yaml - continued --- steps: - name: 'zoofoo_fe' task: 'profile::zoofoo_install' targets: $frontends description: 'Install zoofoo frontends' parameters: version: $version return: $zoofoo_fe.map |result| { result['stdout']} Puppet is YAML - cfgmgmtcamp 2023
  • 37. © betadots GmbH 2023 © betadots GmbH 2023 Summary
  • 38. © betadots GmbH 2023 Summary YAML hash based node classification is flexible, extensible and mature and should be considered best practice. Puppet YAML resources allows an easy start but has its limitations. Keep your YAML data - simple - obvious - flexible Don't be afraid of long YAML files and use an IDE Validate your YAML data in CI/CD Visualize your Hiera Data in a web interface (Hiera Data Manager) Puppet is YAML - cfgmgmtcamp 2023
  • 39. © betadots GmbH 2023 Summary YAML HELL Check proper YAML syntax! Quote Strings (especially if the string starts with digits)!!!!!!! Quote Regexp (especially when starting with & or *) Quote no as it is a Boolean. Quote keys, see all of the above Quote Version String. Unquoted can lead to unintentional numbers https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell Puppet is YAML - cfgmgmtcamp 2023
  • 40. © betadots GmbH 2023 Summary Module Authors: - please provide Hiera YAML data examples - please add code documentation and use puppet-strings Puppet and Voxpupuli - please allow Hiera hash key as array (HI-637 and hiera-eyaml#337) - please allow Puppet function class from Hiera (HI-638 and hiera-eyaml#336) Puppet is YAML - cfgmgmtcamp 2023
  • 41. © betadots GmbH 2023 Summary More complex configurations must be done in Puppet DSL modules or classes. e.g. - you need to manage an application on several OS - there is more to do than only add some resource types While using YAML, one can start exploring and learning on how to write Puppet DSL code. Puppet is YAML - cfgmgmtcamp 2023
  • 42. © betadots GmbH 2023 Summary Use Hiera Data Manager to visualize your YAML data https://github.com/betadots/hdm Puppet is YAML - cfgmgmtcamp 2023
  • 43. © betadots GmbH 2023 © betadots GmbH 2023 Puppet is YAML CfgMgmtCamp 2023 Thank you!