batou - multi(component|host|environment|.*) deployment
•
3 likes•1,669 views
Talk given at EuroPython 2013
Recommended
Recommended
More Related Content
Similar to batou - multi(component|host|environment|.*) deployment
Similar to batou - multi(component|host|environment|.*) deployment (20)
Recently uploaded
Recently uploaded (20)
batou - multi(component|host|environment|.*) deployment
- 7. HOW DOES CONVERGENCE HELP? Wednesday, 3.July 13
- 8. HOW DOES THIS WORK WITH BATOU? Wednesday, 3.July 13
- 11. IT'S NOT THAT BAD. Wednesday, 3.July 13
- 13. service deployment Fabric, Capistrano, ... system configuration Puppet, Chef, ... provisioning kickstart, Razor, imaging ... Wednesday, 3.July 13
- 16. "Everything that follows is a result of what you see here." (Dr. Alfred Lanning; I, Robot) Wednesday, 3.July 13
- 17. SIMPLE os.mkdir('foo') with open('foo/bar', 'w') as myfile: myfile.write('asdf') os.chmod('foo/bar', 0755) Wednesday, 3.July 13
- 18. •unexpected system state •can't resume •unnecessary updates os.mkdir('foo') with open('foo/bar', 'w') as myfile: myfile.write('asdf') os.chmod('foo/bar', 0755) SIMPLISTIC Wednesday, 3.July 13
- 19. CORRECT(?) if not os.path.isdir('foo'): os.unlink('foo') if not os.path.exists('foo'): os.mkdir('foo') try: os.lstat('foo/bar') except OSError: pass else: if os.path.isdir('foo/bar'): shutil.rmtree('foo/bar') else: os.unlink('foo/bar') if (os.path.exists('foo/bar') and open('foo/bar', 'r').read() != 'asdf'): open('foo/bar', 'w').write('asdf'): current = os.stat('foo/bar').st_mode if stat.S_IMODE(current) != 0755: os.chmod('foo', 0755) Wednesday, 3.July 13
- 21. class File(Component): namevar = 'path' def configure(self): self += Presence( self.path, leading=self.leading) self += Mode(self.path, self.mode) self += Content(self.path, self.content) File('foo/bar', content='asdf', mode=0755, leading=True) Wednesday, 3.July 13
- 22. class File(Component): namevar = 'path' def configure(self): self += Presence( self.path, leading=self.leading) self += Mode(self.path, self.mode) self += Content(self.path, self.content) File('foo/bar', content='asdf', mode=0755, leading=True) compute target state (no touching!) Wednesday, 3.July 13
- 23. class File(Component): namevar = 'path' def configure(self): self += Presence( self.path, leading=self.leading) self += Mode(self.path, self.mode) self += Content(self.path, self.content) File('foo/bar', content='asdf', mode=0755, leading=True) compute target state (no touching!) composition operator Wednesday, 3.July 13
- 24. class File(Component): namevar = 'path' def configure(self): self += Presence( self.path, leading=self.leading) self += Mode(self.path, self.mode) self += Content(self.path, self.content) File('foo/bar', content='asdf', mode=0755, leading=True) compute target state (no touching!) composition operator order matters Wednesday, 3.July 13
- 25. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass Wednesday, 3.July 13
- 26. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass run "anywhere" Wednesday, 3.July 13
- 27. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass run on target run "anywhere" Wednesday, 3.July 13
- 28. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass run on target run "anywhere" after all sub- components Wednesday, 3.July 13
- 29. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass run on target only if needed run "anywhere" after all sub- components Wednesday, 3.July 13
- 30. class Presence(Component): namevar = 'path' leading = False def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading) def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass run on target only if needed run "anywhere" after all sub- components keep delegating! Wednesday, 3.July 13
- 31. class Directory(Component): namevar = 'path' leading = False def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path) Wednesday, 3.July 13
- 32. class Directory(Component): namevar = 'path' leading = False def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path) could be done with recursive composition Wednesday, 3.July 13
- 33. class Directory(Component): namevar = 'path' leading = False def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path) could be done with recursive composition refactor with sub-components if too complex Wednesday, 3.July 13
- 34. class Directory(Component): namevar = 'path' leading = False def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path) all methods optional: no configure() could be done with recursive composition refactor with sub-components if too complex Wednesday, 3.July 13
- 35. class Directory(Component): namevar = 'path' leading = False def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded() def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path) all methods optional: no configure() could be done with recursive composition pattern: just wipe out what's wrong refactor with sub-components if too complex Wednesday, 3.July 13
- 36. CONVERGENCE resume where needed handle many system states transparently avoid unnecessary updates Wednesday, 3.July 13
- 37. COMPONENTS composition of simple components no magic bullet, just a lot easier to factor your code configure - verify - update Wednesday, 3.July 13
- 53. ENVIRONMENTS [environment] service_user = myservice host_domain = flyingcircus.io branch = production [hosts] multikarl00 = nginx, haproxy multikarl01 = postgres, redis, memcached, crontab multikarl12 = supervisor, logrotate, doctotext, myapp multikarl13 = supervisor, logrotate, doctotext, myapp Wednesday, 3.July 13
- 54. LOCAL $ bin/batou-local dev localhost Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) $ bin/batou-local dev localhost $ Wednesday, 3.July 13
- 55. REMOTE $ bin/batou-remote prod test02.gocept.net: connecting test01.gocept.net: connecting test01.gocept.net: bootstrapping test02.gocept.net: bootstrapping OK OK Deploying test01.gocept.net/hello Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) OK Deploying test02.gocept.net/hello Updating Hello > File(hello) > Presence(hello) Updating Hello > File(hello) > Content(hello) OK Wednesday, 3.July 13
- 56. OVERRIDES class Hello(Component): hostname = "foo" [environment] ... [component:hello] hostname = bar Wednesday, 3.July 13
- 57. SECRETS class Hello(Component): db_password = none secrets/production.cfg [hello] db_password = reallysecretstuff Wednesday, 3.July 13
- 58. SECRETS class Hello(Component): db_password = none secrets/production.cfg [hello] db_password = reallysecretstuff SciFi but close Wednesday, 3.July 13
- 59. PROVIDE/REQUIRE class MyApp(Component): def configure(self): self.provide('appserver', self.host.fqdn) class HAProxy(Component): def configure(self): self.backends = self.require('appserver') Wednesday, 3.July 13
- 60. PLATFORMS class HAProxy(Component): ... @platform('flyingcircus.io', HAProxy) class SystemWideHAProxy(Component): def configure(self): self += File('/etc/haproxy', ensure='symlink', link_to=self.parent.haproxy_cfg.path) Wednesday, 3.July 13
- 61. VFS MAPPING ./ ... ./work ./work/_/etc/haproxy.cfg class HAProxy(Component): def configure(self): self += File('/etc/haproxy') [environment] ... [vfs] sandbox = Developer Wednesday, 3.July 13
- 62. FEATURES class MyApp(Component): features = ['instance', 'jobrunner'] def configure(self): if 'instance' in self.features: ... [hosts] hosta = myapp:instance hostb = myapp:jobrunner hostc = myapp:instance, myapp:jobrunner hostd = myapp Wednesday, 3.July 13