This document discusses how Atlasssian's Build Engineering team scaled their operations without expanding their team size through the use of DevOps practices like automation, ChatOps, tracking, and support. Some key points: - The team built automation around tasks like infrastructure provisioning and application deployment. This included using configuration management tools like Puppet and infrastructure as code with Terraform. - They created a "command centre" in HipChat to centralize communications, monitoring, and knowledge sharing. This included bots, alerts from monitoring tools, and access to documentation. - Extensive use of tracking tools like Grafana and custom solutions provided visibility into deployments, builds, and service usage. Contact rate tracking in particular

1. VINICIUS LINCK • SOFTWARE ARCHITECT • E-CORE • @VINNYLINCK
Scaling Without Expanding:
A DevOps Story

3. is the driving force of
DevOps

4. Automation
ChatOps
Tracking
Support
The heart Of DevOps
Build Your Command
Centre
Create an Army of
Sentinels
Protect Your Project
Work

5. Mandate
Build and maintain the platform
required by Atlasssian’s
developers to build, test and
release their products
Build Engineering’s

6. OPERATIONS
Build
Engineering
SUPPORT
DEVELOPMENT

7. 1.8x the
number of
customers

9. Automation
ChatOps
Tracking
Support
The heart Of DevOps
Build Your Command
Centre
Create an Army of
Sentinels
Protect Your Project
Work

10. Automate
facet of your work

11. Installing new infrastructure
Configuring your servers
Deploying a new release
Stuff you actually want to get done

13. Continuous Integration
The starting point
of every project
Bring visibility to
your automation

14. Know what’s running
in each environment

15. Continuous testing = more frequent
releases

16. full bleed image
- construction?
- farming?
- lego?
Configuring Servers
Configuration as Code

17. Building
Software
Building
Infrastructure

18. file { ‘/etc/cron.daily/clean_m2_local':
source => ‘puppet:///modules/bamboo_agent_support_maven
/etc/cron.daily/clean_m2_local',
owner => 'root',
group => 'root',
mode => '0755',
}
Configuration as Code
ldally@ip-172-30-97-250:~$ ls -la /etc/cron.daily/clean_m2_local
-rwxr-xr-x 1 root root 352 Sep 18 00:38 /etc/cron.daily/clean_m2_local

19. Puppet Bamboo

20. full bleed image
- measuring something (olympics)?
- science?
Test
Virtualization and
Behavioural Testing

21. Spin up an instance
Apply your configuration
management and deploy your app
Run your tests
The Testing
Process

22. Testing Tools
Vagrant Cucumber
Virtualization Behavioral Testing

23. Spin up an exact
mirror of your server
~/s/buildeng-puppet ❯❯❯ vagrant status
Current machine states:
aptly not created (aws)
consul not created (aws)
bambooagent not created (aws)
bambooserver running (aws)
efs not created (aws)
bambooserver_rds not created (aws)
dns not created (aws)
basebox not created (aws)
gnomes not created (aws)
status not created (aws)
graphite not created (aws)

24. Easy to
understand
expectations
Quick to write

25. full bleed image
- shipping crates?
- military drop from plane?
- ice road trucker?
Configuring Hardware
Infrastructure as Code

26. Building
Software
Configuring
Hardware

27. # Create an instance for our JIRA server
resource "aws_instance" “fried_fritter_jira” {
instance_type = "m4.2xlarge"
ami = "ami-dfb0e7b5"
disable_api_termination = "true"
ebs_optimized = "true"
subnet_id = “${aws_subnet.ff_us_east_1b.id}”
vpc_security_group_ids = [ “${aws_security_group.fried_fritter_jira.id}” ]
provider = "aws.{{ vpc.provider }}"
}
Infrastructure as Code

28. BambooTerraform

29. Everything = AutomatedStuff you actually want to get done

30. Automation
ChatOps
Tracking
Support
The Heart Of DevOps
Build Your Command
Centre
Create an Army of
Sentinels
Protect Your Project
Work

31. …software and IT teams evolved from email
to chat. They also replaced repetitive tasks
with automation, and replaced annual
change control meetings with devops’
continuous collaboration. And they pulled it
all together into a central tool like HipChat.
SEAN REGAN, HIPCHAT EVANGELIST, ATLASSIAN
“
”

32. full bleed image
- nasa ops room?
- ?
The Command Centre

33. Public
Our HipChat Rooms
Private
Alerts The FIGHT channel

34. Pull Request approvals
Green builds
No open tasks

35. See which builds are
(or aren’t) passing
Know what needs to be
done to get your PR
merged

36. You have an amazing knowledge
base!
…but no one knows where it is
:(

37. Inline access to a
knowledge base

40. I heard you like bots
So I put bots in your bot
So you can scale while they bot

44. ChatOps helps you
your DevOps culture

45. Automation
ChatOps
Tracking
Support
The Heart Of DevOps
Build Your Command
Centre
Create an Army of
Sentinels
Protect Your Project
Work

46. Everything you
seek to
you must

48. I have no idea
what I’m doing

49. Keep an eye on
known causes of
performance
issues…
…Alongside the
system’s resources

50. agent smith SS
The most important
details of each
instance

51. Most of the time your
deployments will
succeed!
… but sometimes they
won’t

53. Can you see the
forest for the trees?

54. What’s sending the
most alerts?

56. Expose the
critical stats to
your users

58. If you track your service…
shouldn’t you track how
people use your
service?

59. Contact Rate Tracking

60. Count and
categorise your
contacts

61. 0 12.5 25 37.5 50
Maven - Usage
External
Bamboo - Usage
Other
Agents - Capability Request
Contact Analysis

62. Case Study: M.A.S.H

63. go/mash
The most important
details about an
artefact
summarised

64. your tracking

65. Automation
ChatOps
Tracking
Support
The Heart Of DevOps
Build Your Command
Centre
Create an Army of
Sentinels
Protect Your Project
Work

66. OPERATIONS
Build
Engineering
SUPPORT
DEVELOPMENT

67. OPERATIONS
Maintenance
SUPPORT
DEVELOPMENT
Shields

68. OPERATIONS
Maintenance
SUPPORT
DEVELOPMENT
Shields

69. MaintenanceShields Development /
Operations
HipChat
Confluence
Questions
ServiceDesk
“Shields”
Infrastructure specialists
Development specialists

70. go/buildeng
full bleed of go/buildeng

71. Building
Champions
Public
Forum
User
Groups
Bootcamps

72. Getting proactive:
Pipeline Consulting

73. PC - Criteria
What makes a
pipeline hard to
work with?

74. Support your
customers

75. Tooling Summary
Atlasssian Stack Third Party Tooling

76. MonitoringHostingDeployment
Configuration Testing Dependency Analytics + Security
Something for everybody

78. Thank you!