Speaker: Arkadiusz Borucki, Senior Service Reliability Engineer (MongoDB/Automation), Amadeus Data Processing GmbH

1. RESTRICTED
Confidential
©AmadeusITGroupanditsaffiliatesandsubsidiaries
RESTRICTED
Confidential
Amadeus
Supercharge your MongoDB
Deployment with Ops Manager
Automation
Arkadiusz Borucki,
Senior Service Reliability
Engineer

2. RESTRICTED
Confidential
©AmadeusITGroupanditsaffiliatesandsubsidiaries
_ We are the leading technology company dedicated to the global travel industry
_ We are present in 190+ countries and employ 15,000+ people worldwide
_ Our solutions enrich travel for billions of people every year
_ We work together with our customers, partners and other players in the industry to improve
business performance and shape the future of travel
630+ million total bookings processed in 2017
using the Amadeus distribution platform
1.6 billion passengers boarded in 2017
with Amadeus and Navitaire solutions
1 of the world’s top 15 software companies
Forbes 2017 global rankings
Amadeus in a few words
2
6th consecutive year included in the DJSI
Recognised as world leader in the Software & Services
industry sector in the Dow Jones Sustainability Index in 2017

3. RESTRICTED
Confidential
Providing local expertise, all over the world
©AmadeusITGroupanditsaffiliatesandsubsidiaries
We work in
190+
markets
3

4. { NYC : MongoDB at Amadeus }
©AmadeusITGroupanditsaffiliatesandsubsidiaries
4
Supercharge your MongoDB
Deployment with Ops Manager
Automation

5. RESTRICTED
Confidential
RESTRICTED
Confidential
{ Amadeus : MongoDB farm overview }
4 Ops Managers versions : - 3.6.*.
MongoDB Clusters split between 4 Ops Managers.
„Master” Ops Manager.
100 MongoDB clusters: size (100GB – 107TB):
- 574 nodes, 3000 mongod.
Micro sharding (even 120 shards per cluster).
Automation: Jenkins CI/CD, Ansible, REST API, Python scripts.
Security: SSL/TLS, Kerberos, LDAP, CA, Auditing, Encryption at Rest, QRadar.
5
©AmadeusITGroupanditsaffiliatesandsubsidiaries

6. RESTRICTED
Confidential
6
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : MongoDB Architecture 2018 }
574 MongoDB nodes splitted between OM

7. { Amadeus : MongoDB use cases }
Instant Search - flight recommendations data.
- hundred billion of recommendations
MongoDB for real-time analytics – 124 shards (micro-sharding).
Apache Spark on MongoDB – fraud detection projects.
MongoDB Amadeus Payments.
MongoDB for Airlines - configuration, status, history.
Next TI – travel intelligence, BI for airline decision-making.
Dashboards: scheduling tools, cheange viewers, applications viewers, and
more.
7
©AmadeusITGroupanditsaffiliatesandsubsidiaries

8. { Amadeus : SSP Instant Search }
Search billions of recommendations in milliseconds!
Speed! - search time is in average 2 milliseconds!
In-house implementation of NoSQL.
SSP - 3 MongoDB clusters – 200TB of data.
always online!
8
©AmadeusITGroupanditsaffiliatesandsubsidiaries

9. { Amadeus : the biggest cluster in Europe ? }
9
©AmadeusITGroupanditsaffiliatesandsubsidiaries

10. { Amadeus : How to operate at huge scale ? }
10
©AmadeusITGroupanditsaffiliatesandsubsidiaries

11. 11
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Automation }
Infrastructure as Code - GIT.
CI/CD with Jenkins - orchestrate the deployment processes.
Ops Manager REST API.
Ansible module for Ops Manager / MongoDB.
Ansible playbooks / roles for MongoDB.
Framework for online database upgrade.
Framework for online rolling kernel upgrade.
MongoDB as a Service.

12. 12
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ MongoDB : No Downtime }

13. { Amadeus : Infrastructure as a code }
©AmadeusITGroupanditsaffiliatesandsubsidiaries
13
Bitbucket - branches
Reduce overhead, reduce human mistakes.
speed-up processes, provide consistency.

14. { Ops Manager : REST API }
©AmadeusITGroupanditsaffiliatesandsubsidiaries
14
Ops Manager follows REST architectural style and provides
internal resources which enable programmatic access to Ops
Manager features.
JSON entities, Digest authentication, Browsable interface
WEB UI

15. RESTRICTED
Confidential
15
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager class }
class OpsManager(object):
def __init__(self, mmsurl, user, key, verify):
self.mmsurl = mmsurl
self.user = user
self.key = key
self ._session = requests.Session()
self ._session.verify = verify
self ._session.auth = requests.auth.HTTPDigestAuth(self.user, self.key)

16. RESTRICTED
Confidential
16
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager class }
def post_maintenance_window(self, group, data):
return self._post_json(data, *self._maintenance_url(group))
def upgrade_mongodb(self, group, ver):
j = self.get_automation_config(group)
j = self.enable_version(j, ver)
j = self.compatibility_version(group, j, ver)
for p in j.get('processes'):
p['version'] = _enterprise_edition(ver)
self._put_json(j, *self._automation_config_url(group))
self.deploy_change(group)

17. RESTRICTED
Confidential
17
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager class }
def check_cluster_health(self, group):
if not self.cluster_goal_status(group):
raise OpsManInvalidState('Operation on cluster, try later')
self.mongodb_alerts(group)
self.check_sync(group)
def shutdown_db(self, group, host, option):
j = self.get_automation_config(group)
for p in j.get('processes'):
if p.get('hostname') == host and p.get('process Type') != 'mongos':
p['disabled'] = option
self.put_automation_config(group, j)

18. RESTRICTED
Confidential
18
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager methods }

19. RESTRICTED
Confidential
19
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager class }
def delete_maintenance_window(self, group, idnum):
return self._delete_json(*self._maintenance_url(group, idnum))
def cluster_goal_status(self, cluster):
j = self._get('/api/public/v1.0/groups', cluster, 'automation Status')
goal = j['goal Version']
return all([ goal == i.get('lastGoalVersionAchieved') for i in
j.get('processes')])
def get_group_by_name(self, group):
'''get group from Ops Manager via REST API call'''
return self._get('/api/public/v1.0/groups/by Name', group).get('id')

20. RESTRICTED
Confidential
20
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : OpsManager class }

21. RESTRICTED
Confidential
21
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Ansible OpsManager modules }
mongodb start / stop / sync modules
from ansible.module_utils.opsmanager import ansible_setup
if __name__ == '__main__':
module, opsmanager = ansible_setup()
group = opsmanager.get_group_by_name(module.params['cluster'])
response = opsmanager.up_db(group, module.params['host'])
module.exit_json(changed=False, meta=response)

22. RESTRICTED
Confidential
22
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : open-source }
Python class for MongoDB Ops Manager and Ansible
module
https://github.com/AmadeusITGroup/opsmancombo
The official global GitHub account of Amadeus IT Group.
Amadeus provides the technology that keeps the travel sector
moving.

23. RESTRICTED
Confidential
23
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Ansible-Galaxy }
Installation : $ ansible-galaxy install AmadeusITGroup.opsmancombo
Tags : automation, database, DevOps, MongoDB, NoSQL, Ops Manager
https://galaxy.ansible.com/AmadeusITGroup/opsmancombo/

24. RESTRICTED
Confidential
24
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Jenkins }
Jenkins is an open-source continuous integration (CI) tool that
helps orchestrate the development processes (build, test, and
deployment) with automation.
Jenkins is one of the leading tools
that would help a DevOps team orchestrate its
processes.

25. RESTRICTED
Confidential
25
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Jenkins Pipeline }
A continuous delivery pipeline is an automated expression of your
process for getting software from version control right through to your
deployment.
Setting up a Pipeline project means writing a script that will
sequentially apply some steps of the process we want to accomplish.

26. RESTRICTED
Confidential
26
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : Jenkins Pipeline }

27. RESTRICTED
Confidential
27
©AmadeusITGroupanditsaffiliatesandsubsidiaries
{ Amadeus : projects we working on }
• 200TB MongoDB cluster! – data repository at huge scale!
• MongoDB on OpenShift (Microservices, Ops Manager operator).
• Ops Manager and S3 Object Store.
• Migraton from RDBMS to MongoDB!
• Amadeus and MongoDB technical blog!
• More automation and more OpenSource!
Watch our webinar!
stay tuned!!

28. ©AmadeusITGroupanditsaffiliatesandsubsidiaries
You can follow us on:
AmadeusITgroup
amadeus.com
amadeus.com/blog
Thank you!