Puppet Camp London Fall 2015 - Service Discovery and Puppet
•
9 gostaram•1,915 visualizações
Presentation given at the London Puppet Camp in London on Nov 3rd 2015.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Puppet Camp London Fall 2015 - Service Discovery and Puppet
Semelhante a Puppet Camp London Fall 2015 - Service Discovery and Puppet (20)
Mais de Marc Cluet
Mais de Marc Cluet (18)
Último
Último (20)
Puppet Camp London Fall 2015 - Service Discovery and Puppet
- 5. Engineer based in London Co-Founder of Ukon Cherry Contractor at TrainLine 19 years of experience as a SysAdmin Founding member of Juju and MAAS while at Canonical Built a DevOps Engineering Team at Rackspace Been DevOps’in for the last 6 years WhoamI?
- 9. Cloud is hard! • Old style DNS • TTL is a b*tch • Health Checks • Metadata Storage TheProblem
- 11. Automatically define your services • Active Health Checking • Dynamically updated service lists • Can be DNS accessible (if needed) • API accessible (win!) ServiceDiscovery
- 13. ServiceDiscovery Service Publication Service Node A Service Node B Health Check Health Check Discovery AgentDiscovery Agent
- 14. ServiceDiscovery Service Publication Service Node A Service Node B Health Check Health Check Discovery AgentDiscovery Agent
- 15. ServiceDiscovery Service Publication Service Node A Service Node B Health Check Health Check Discovery AgentDiscovery Agent
- 16. ServiceDiscovery Service Publication Service Node A Service Node B Health Check Health Check Discovery AgentDiscovery Agent
- 19. ServiceDiscovery Service Node A Service Node C Service Node B Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 20. ServiceDiscovery Service Node A Service Node C Service Node B Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 21. ServiceDiscovery Service Node A Service Node C Service Node B Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 26. ServiceDiscoverySolutions API + DNS publication k/v Strongly Consistent Host + Service checks API publication k/v Consistent Container checks API publication Container checks Auto-Heal Puppet module Hiera access Puppet module Hiera access Puppet module
- 28. It's a Service Discovery System (duh!) • Service Publications (DNS + API) • k/v Storage (strongly consistent) • Health Checks • With encryption! (whaaaaaat) WhatisConsul?
- 32. ConsulArchitecture Consul Server A Consul Agent Consul Server B Consul Server C Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent
- 33. ConsulArchitecture Consul Server A Consul Agent Consul Server B Consul Server C Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent Consul Servers Another DC
- 35. service name: web zone: eu-west-1 <servicename>.service.<zone>.consul web.service.eu-west-1.consul ConsulDNSpublisher
- 37. { "check": { "id": "mem-util", "name": "Memory utilization", "script": "/usr/local/bin/check_mem.py", "interval": "10s" } } ConsulHealthChecks
- 45. Puppet+Consul You can integrate at several levels • Puppet Module • KyleAnderson-consul • Hiera • lynxman-hiera_consul
- 46. Puppet+Consul You can integrate at several levels • k/v access • venmo-consulr • Templates with Consul • ghdbaston-consul_template
- 47. PuppetModule-Install Puppet Forge makes it very easy to install $ puppet module install KyleAnderson-consul
- 48. PuppetModule-DNS $ puppet module install saz-dnsmasq include dnsmasq dnsmasq::conf { 'consul': ensure => present, content => 'server=/consul/127.0.0.1#8600', }
- 49. PuppetModule-NodeChecks consul::check { 'disk_space': script => '/usr/lib/nagios/plugins/check_disk -w 5% -c 1% -x /run/lock -x /run/shm -x /run/user -x /dev -x /run -x /sys/fs/cgroup', interval => '30s', }
- 50. PuppetModule-ServiceChecks consul::service { 'nginx': port => '80', checks => [ { script => "/usr/lib/nagios/plugins/check_http -H localhost -w 20 -c 60 -t 20", interval => '30s', }, ], }
- 52. PuppetModule-Install Puppet Forge makes it very easy to install $ puppet module install lynxman-hiera_consul
- 53. PuppetHiera-Config :backends: - yaml - consul :yaml: :datadir: /etc/puppet/hieradata :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/catalog/service - /v1/catalog/node
- 54. PuppetHiera-Config :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/catalog/service - /v1/catalog/node
- 55. PuppetHiera-Arrays notice('Generating rabbitmq cluster members based on Consul information') $consul_service_array = hiera('rabbitmq',[]) $mq_cluster_nodes = consul_info($consul_service_array, 'Address') notice("Result: ${mq_cluster_nodes}")
- 56. PuppetHiera-Arrays notice("Generating neo4j_ha cluster members based on Consul information") $consul_service_array = hiera('neo4j_ha',[]) $consul_fields = [ 'Address', 'ServicePort' ] $consul_ha_initial_hosts = consul_info($consul_service_array, $consul_fields, ':') $ha_initial_hosts = join($consul_ha_initial_hosts, ',') notice("Result: ${ha_initial_hosts}")
- 57. PuppetHiera-k/vaccess Long live to the API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/
- 58. PuppetHiera-k/vaccess Long live to the API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/web/
- 59. PuppetHiera-k/vaccess Long live to the API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - "/v1/kv/%{env}/"
- 62. PuppetSecurity
- 63. PuppetSecurity git commit -m "All my passwords" git push
- 71. Hieraeyaml Usage: eyaml <subcommand> Please use one of the following subcommands or help for more help: createkeys, decrypt, edit, encrypt, recrypt, version
- 76. EXTRATIME!
- 79. Vault$ vault write secret/foo value=bar Success! Data written to: secret/foo $ vault read secret/foo Key Value lease_id secret/foo/9c5f3cf1-1239-0160-4311-d6544fd1018c lease_duration 2592000 value bar $ vault delete secret/foo Success! Deleted 'secret/foo'
- 80. Vault $ vault read postgresql/creds/production Key Value lease_id postgresql/creds/production/8ade2cde-5081- e3b7-af1a-3b9fb070df66 lease_duration 3600 password 56b43bc3-b285-4803-abdf-662d6a105bd0 username vault-root-1430141210-1847