DevOps intro, Puppet intro, Monitoring/metrics/logging

1. DevOps Introduction
Ops' job is NOT to keep the site stable and fast
Ops' job is to enable the business
(this is dev's job too)

2. Who am i ?
Systems Administrator @
DevOps
Infrastructure as a code ”Puppet”
Cloud Computing
…
@EslamElHusseiny

3. Dev or Ops ?
6 days after a successful deployment, server
experiences troubles ”high load, resources leak”

4. Dev or Ops ?
There is an issue with build server.
Code isn't building.

5. Dev or Ops ?
Software is deployed to the test environment.
You can't login to application.

6. Dev vs. Ops

7. Dev says ...

8. Ops replies ...
”It's not my server, it's your code”

9. Role of Devs and Ops

10. Role of Devs and Ops
Dev
Create changes
Add / modify features
Ops
Maintain stability
Create / enhance services

11. Role of Devs and Ops
Dev
Create changes
Add / modify features
Ops
Maintain stability
Create / enhance services

12. So ?
The problem
Disconnect between groups
Devs don't deploy consistent software
Ops motivated to resist changes
Devolpment process is Agile
Operations process is Static

13. Our Goal
We are all here to ENABLE business

14. change
Is required for business
Doesn't mean outage
Should be exploited and not feared
Shouldn't mean fire fighting

15. Rescue
DevOps To The Rescue !

16. What is DevOps then ?
”DevOps is an approach to bridge the gap between
agile software development and operations” ­
agileweboperations.com
Collaborative mindset of Devs and Ops
C.A.M.S

17. Culture

18. Culture
Relationships
Engage early, engage often
Be open
Eradicate ”last­mile syndrome”

19. Culture
Communication
Talk is cheap (get out of your chair)
Involve each other in core proccesses and decisions
Ask Questions
Don't say ”No”

20. Automation

21. Automation
What ca be automated ?
Builds
Deployments
Testing
Monitoring
Self­healing
System configuration

22. Metrics

23. Metrics
Capture, learn, improve.
Assist in:
Capacity planning
Trend analysis
Fault finding

24. Metrics

25. Sharing

26. Sharing
Share ideas
Share metrics
Ops : Give devs shell access
Devs : see what technology ca be leveraged

27. DevOps life cycle

28. Before
Talk about functional requirements
Talk about non­functional requirements
Security
Backups
Availability
Upgradeability
Configuration management
Monitoring / Metrics
Logging

29. During
Communication
Source control
Automate builds
Automate tests
Automate deployments (Dev, Test, and Prod)
Collate App and System metrics

30. After
Release
Continue to run tests
Monitor application and system
Issues (might happen)
Post meetings

31. What are we doing ?

32. Puppet

33. Puppet
Configuration management tool
Open Source
Infrastructure as a code

34. Puppet

35. Puppet

36. node default {
include ssh
include sudo
include ntp
}
node product.dev inherits default {
include nginx
include tomcat5
}
node product.test inherits default{
include nginx
include tomcat6
}

37. class sudo {
package {'sudo':
ensure => latest,
}
file {'/etc/sudoers':
owner => root,
group => root,
mode => '0440',
source => 'puppet:///files/sudo/sudoers',
require => Package['sudo'],
}
}

38. Puppet

39. Let's get hands on
It's deploymnt time

40. Monitoring

41. Monitoring

42. Problem ?
No one noticed
No one got alerted
No automatic recovery

43. Probelm ?
what is your application doing right now?
do you know when it fails?
failure means customers lose trust
failure means customers go elsewhere
failure means you lose money
application = providing value

44. Solution
Monitoring
Metrics
Logging

45. Monitoring means
Is the application available?

46. Examples

47. Monitoring
is this service currently providing value?
is this service consuming too many resources?

48. Examples

49. Monit
check process unicorn
with pidfile /var/run/unicorn/unicorn.pid
start program = "/etc/init.d/unicorn start"
stop program = "/etc/init.d/unicorn stop"
if mem is greater than 300.0 MB for 1 cycles then
restart
if cpu is greater than 50% for 2 cycles then alert
if cpu is greater than 80% for 3 cycles then restart
group unicorn

50. Metrics
measurements
historical data
graphs

51. Metrics
how many customers are on my site?
how many customers were on my site yesterday?

52. Metrics
how slow is paypal's api?
how slow was paypal's api yesterday?

53. Metrics
how much memory is available on my servers?
how much has memory usage grown over four weeks?

54. Metrics
number of open database connections
number of redis commands
number of 500 errors
rate of HTTP requests
number of HTTP connections

55. Metrics
median response time
number of failed resque jobs
number of twitter followers
99th percentile github api response time
95th percentile mysql query time

56. Examples

57. Munin

58. Graphite

59. Metrics as a service

60. Logging

61. Logstash
log inputs
process
outputs

62. Graylog2

63. Graylog2

64. Summary
own your monitoring
own your metrics
own your logging
none of them is optional