O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Couchbase Live Europe 2015: Couchbase in Amadeus Critical Systems

2.146 visualizações

Publicada em

Amadeus is one of the leading IT worldwide companies in the travel industry. Amadeus operates a very demanding IT environment in term of raw performance, scalability and reliability. The presentation will include a return on experience about our Couchbase usage, and two use cases, illustrating a performance/scalability oriented application, and an infrastructure supporting ultra-high availability.

Presented by: Dietmar Fauser & Dideier Spezia, Amadeus

Publicada em: Software
  • Seja o primeiro a comentar

Couchbase Live Europe 2015: Couchbase in Amadeus Critical Systems

  1. 1. Couchbase in Amadeus critical systems ©2014AmadeusITGroupSA Dietmar Fauser Amadeus VP R&D Architecture, Quality and Governance Didier Spezia Senior Expert Database London, March 2015
  2. 2. Page 2 Agenda ©2014AmadeusITGroupSA 1. Introduction 2. Shopping platform, low latency stores 3. R-Box, distribution and ultra- high scalability 4. Hotel shopping and booking, ultra-high availability 5. Conclusion
  3. 3. Introduction ©2014AmadeusITGroupSA
  4. 4. ©2014AmadeusITGroupSA Amadeus, the big picture Corporations Online agencies Airlines Hotels Cruise lines / Ferries Tour Operators Car rental companies Rail Travel providers Global Distribution System IT solutions Travel agents Airlines Direct distribution Airports, Airline sale offices, Call centres… Travel buyers Leisure Traditional agencies Business
  5. 5. Amadeus is present across the entire journey Travel booking solutions for travel distributors: • Travel agencies, traditional and online • Airlines (direct distribution) Offer management and optimization tools for: • Airlines: inventory, departure control • Travel agencies: management systems • Hotels : inventory, booking • Railways : inventory • Companies : business travel management
  6. 6. Page 6 ©2014AmadeusITGroupSA Amadeus, airlines partners 121 airlines contracted for Reservation and Inventory 800 millions passengers boarded each year by 2015(a) +93 airlines (List of airlines shown is a sample and not exhaustive. Passengers generated by airlines that are Altéa customers with Altéa Inventory and/or Departure Control modules implemented or have contracted the migration to any of these two modules. Estimated annual PBs calculated as follows: (i) annual PBs migrated to Altéa by 31 December 2010 plus (ii) annual PBs estimated to be migrated to Altéa by 31 December 2013 based on contracts signed through 31 December 2010; all annual PB volume data based on actual 2010 traffic volumes where migrated before 1 January 2011, remainder estimated using historic traffic volumes and therefore exclude any volume trend assumptions (a) 2015 estimated annual PB calculated by applying the IATA’s regional air traffic growth projections to the latest available annual PB figures to the 121 airlines contracted to Altéa as of 31 December 2012
  7. 7. Page 7 ©2014AmadeusITGroupSA Amadeus, Operational Figures _ Handles up to 30,000 customer transactions/sec _ <0.5 sec response time _ 16 Petabytes of storage _ 3.7 million net bookings/day _ 1.6 billion transactions/day _ 500,000+ database disk IO/second _ 4,800+ IT changes/month
  8. 8. Page 8 ©2014AmadeusITGroupSA Couchbase & Amadeus _ 2013 • Signed Premium Partnership • Prototyped 2 use cases with Couchbase (air shopping/inventory) • Low latency, high-throughput key/value stores _ 2014 • Couchbase went live for Air Shopping • Implementing a new use case (hotel shopping & booking) • Ultra-high availability _ 2015 _ Air Shopping ramp-up _ Couchbase will go live for Air Inventory _ Couchbase production readiness for hotel shopping & booking _ Implementing a new use case (R-Box) _ Ultra-high scalability
  9. 9. Shopping Platform High-throughput, low latency, key/value stores 2 ©2014AmadeusITGroupSA
  10. 10. Page 10 ©2014AmadeusITGroupSA The Shopping Flow Priced solutions End user Amadeus 1 Travel request 2 Route Explorer 3 Connection Builder 5 Faring 6 1A Agencies Airline.com Self-serviceCall Centers B2T / B2BAgent.com Flight selection 4 Flt Validation Availability calculation
  11. 11. Page 11 ©2014AmadeusITGroupSA Availability Cache Architecture Availability Data Services Miss: read Database Memcached Servers, 12 servers * 1TB Miss: Populate Memcached Availability Services (600 servers) MySQL Farm, 7 * 4 servers Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Shard 6 Shard 7
  12. 12. Page 12 _ Online rebalancing • Extending capacity currently takes careful operations, many days of preparation and many days to warm up the new cache _ Cold starts • Memcached (very rare) outages are (very) disruptive _ Operational costs • Leveraging local storage with PCI-E flash cards • The current infrastructure is complex and requires lot of effort for day to day operations _ The storage logic implemented in the application layer is complex ©2014AmadeusITGroupSA Problems we are trying to solve
  13. 13. Page 13 ©2014AmadeusITGroupSA Availability Cache New Architecture Availability Data Services Couchbase Servers, 30 servers * 1TB Availability Services (600 servers)
  14. 14. Page 14 ©2014AmadeusITGroupSA Availability Data Workloads GET 2.6 M/s SET 1 M/s
  15. 15. Page 15 _Couchbase is fast • Microseconds adds up quickly when exploring thousands of travel solutions • Our cache queries returns in less than 0.5ms _Couchbase is predictably fast • Long tail latencies would kill our application response time • Couchbase leverages our memory capacity _Couchbase online rebalancing works without impacting our query latency ©2014AmadeusITGroupSA Couchbase benefits
  16. 16. Page 16 _Ability to read on replica (2.1.0) • In failover scenarios, we prefer to be non consistent rather than a suffering from a few seconds brownout _Multithreaded read/write operations (2.1.0) • Our workload is write intensive • We want to get the most of our Fusion IO PCI-E SSDs _Rack awareness (2.5.0) • Our data centre has 6 firecells, we can lose one of them without losing any data ©2014AmadeusITGroupSA We asked for it, we got it 
  17. 17. Page 17 (benchmarked on a subset of 15% of our data) ©2014AmadeusITGroupSA Couchbase 3.0.1 _Improvement (compared to 2.5.1) • Automatic, optimized resource utilization _Consequences • Same volume processed with less writing threads • Increase of the write throughput + 30% write throughput - 20% CPU consumption on I/O threads
  18. 18. R-Box Distribution Ultra-high scalability 3 ©2014AmadeusITGroupSA
  19. 19. Page 19 R-Box: an extension of the shopping platform Projecting availability calculations in remote data centres
  20. 20. Page 20 What’s in the box? 265ced1609a17cf1a5979880a2ad364653895ae8 R-Box Shell Layer Core Layer Data Synchronization Live Data (Flight, Counters, Yields…) Data Flow Log/Stat Flow Polling Flow APP Data Synchronization Caches Data (ABR, RFD, Codeshare) File Distribution (bittorrent) Log & Stat Flow Log Server Administration Tool Admin API (XML) Admin Server Polling Request (XML, EDI) ESB Couchbase NoSQL APP APP APP APP APP Actually, a mini data centre Administration and management Storage for live data: Couchbase Storage for static data Middleware components (ESB) Application servers Monitoring and logs
  21. 21. Hotel shopping and booking Ultra-high availability 4 ©2014AmadeusITGroupSA
  22. 22. Page 22 _ Largest hotel group worldwide • A bit more than 4,700 properties • About 680,000 rooms • Occupancy rate: 60-70% _Expected traffic: 80 M bookings/year _Amadeus to provide • Shopping services – 10,000 TPS • Booking _Strict SLAs Intercontinental Hotels Group (IHG) Signed in March 2014
  23. 23. Page 23 (for mission critical and high-throughput applications) ©2014AmadeusITGroupSA Ultra-high availability _General trend of the market _Our target: _Complex infrastructure • Network, storage, databases, application servers, etc … _Not possible to achieve using conventional database engines 99.999 %99.95 %
  24. 24. Page 24 ©2014AmadeusITGroupSA Technical requirements . . . . . . . Service Isolation Zone Computing Silo Compute Silo Computing Silo . . . ESB . . . Storage Global DC . . . . . . . Load Balancer . . . Service Isolation Zone Computing Silo Computing Silo Computing Silo . . . ESB Storage Global Service Isolation Zone Computing Silo Computing Silo Computing Silo ESB Storage _ Document store for booking _ Key/value store for shopping _ Master/master replication between DC _ Full consistency for 1 DC _ Eventual consistency between DC _ Contain shopping traffic in isolation zone _ Redundant clusterware
  25. 25. Page 25 Cross data centre replication, allowing for creative topologies ©2014AmadeusITGroupSA Couchbase XDCR ring topology Primary CB Secondary CB Shopping Shopping Shopping Primary CB Secondary CB Shopping Shopping Shopping Data Centre 1 Data Centre 2
  26. 26. Conclusion 5 ©2014AmadeusITGroupSA
  27. 27. Page 27 Couchbase delivers on what Amadeus values: _ Scalability • Horizontal • Vertical _ Predictability of raw performance _ Operability _ Complex distributed topologies (XDCR) ©2014AmadeusITGroupSA Key takeaways
  28. 28. Thank you ©2014AmadeusITGroupSA You can follow us on: AmadeusITGroup amadeus.com/blog amadeus.com