Webinar - Delivering Enhanced Message Processing at Scale With an Always-on Data Management Solution

1. Delivering Enhanced Message Processing at Scale with DataStax Enterprise Keith Willard Bill Liu Matt Griffin Chief Software Architect Enterprise Architect Lead Database Engineer Surescripts Surescripts Surescripts Jie Wu Director, Product Marketing DataStax

2. From validation to momentum 400+ Employees $190M Funding 500+ Customers Founded in April 2010 Santa Clara • San Francisco • Austin • London • Paris • Berlin • Tokyo • Sydney (Series E – Sept. 2014) 30% + 2016 World’s Best 100 Cloud Companies Ranked #1 in multiple operational database categories © 2017 DataStax, All Rights Reserved. Company Confidential

6. Powering cloud applications © DataStax, All Rights Reserved.6 EFFORTLESS SCALE • Take advantage of every opportunity. • Focus on what matters most to you. ALWAYS-ON • Designed to handle any failure, no matter how catastrophic. INSTANT INSIGHT • Built into your application to create actionable, modern experiences.

7. DataStax Enterprise subscriptions DSE Standard DSE Max Data Platform Certified Apache Cassandra™ ✓ ✓ Advanced Security ✓ ✓ Automatic Management Services ✓ ✓ Advanced Replication ✓ ✓ Tiered Storage ✓ ✓ Multi-Instance ✓ ✓ In-Memory ✓ Search ✓ Analytics ✓ Graph Option Management and Development Solutions DataStax OpsCenter ✓ ✓ DataStax Studio ✓ ✓ DSE MAX For teams looking for an integrated data management platform with advanced features DSE STANDARD For teams that need a solution easy to build and effortless to scale

8. DataStax Managed Cloud (DMC) A Fully Managed, Secure Architecture 8 • DataStax Enterprise on AWS with Managed Provisioning and Scaling by DataStax • 24x7x365 Coverage, Lights-Out Management • System Configuration and Tuning to Meet Customer Specific Requirements • Architecture Advisory Services, Guidance and Best Practices

9. Why DMC Matters to our customers • Focus on business innovation • Offload operations to DataStax • Reduce risk with a secure, white glove service by experts • Grow easily as your business expands • Accelerate time-to-market • A single data layer for on-premises and public cloud • No re-write of applications against every cloud provider’s proprietary data services • Create APIs and micro-services to a consistent data layer • Choice and flexibility for hybrid architecture needs • Extend to fit your hybrid cloud architecture • Span or migrate across infrastructure models with 100% compatibility • Retain data autonomy without cloud vendor lock-in 9

10. Copyright © 2016 by Surescripts, LLC. All rights reserved.Copyright © 2016 by Surescripts, LLC. All rights reserved. DataStax Enterprise in the Surescripts Context Keith Willard, Bill Liu, Matt Griffin August 2017

11. Copyright © 2016 by Surescripts, LLC. All rights reserved. Surescripts network Surescripts is responsible for the largest number of clinical oriented electronic transactions in the country today • The electronic prescription that flows from your doctor’s office computer to your local or mail order pharmacy Surescripts Surescript Core Network Elig, Med Hx, SCRIPT SCRIPT Claim EHR PBM Pharmacy

13. Copyright © 2016 by Surescripts, LLC. All rights reserved. A core load of our processing pipeline  We are message centric • Each operation is encapsulated by the receipt or transmission of a message from or to a network participant  Large volume of messages • Average 50-60 million per week. ~1/2 between 10am and 2pm.  Median size message ~4k • But ranges in size upwards to 20 MB.  Write challenge of scale to write  Read problem • Frequently need to refer to a related past message to properly process and route incoming message. 13

14. Copyright © 2016 by Surescripts, LLC. All rights reserved. Constant pressure to improve SLA  External failures affecting the “fabric” of our data centers  Failures related to data center synchronization processes  Failures related to software mutation • Infrastructure software upgrades • Our own application software upgrades 14

15. Copyright © 2016 by Surescripts, LLC. All rights reserved. Architecture transitions  Past and near present • mixed Java and .NET transaction application tier heavily virtualized • Persistent tier an MQ/Oracle relational OLTP environment • Analytical/Reporting tier Oracle Exadata data warehouse • High reliability relies heavily on a dual physical data center model (Mpls and VA)  Future (cluster centric) • Similar mixed java and .NET transactions Docker or equivalent • Persistent Tier Kafka/DSE/some relational at the OLTP tier • Analytical Reporting Hadoop/Exadata • Higher reliability and scalability multiple “logical data centers” with two physically distinct Hadoop clusters. 15

16. Copyright © 2016 by Surescripts, LLC. All rights reserved. Logical data center  Restructure persistent tier around two themes • Horizontally, economically scalable persistence streaming • Eventually consistent, locally clusterable WAN extended queryable database  Transition from Oracle and Oracle RAC to alternative • Horizontally scalable with a practical synchronization/replication infrastructure Non-Audit Relational Data Application Pools Microservice Pools Kafka DSE

17. Copyright © 2016 by Surescripts, LLC. All rights reserved. End game 17 Non-Audit Relational Data Application Pools Microservice Pools Kafka DSE Non-Audit Relational Data Application Pools Microservice Pools Kafka DSE Non-Audit Relational Data Application Pools Microservice Pools Kafka DSE Non-Audit Relational Data Application Pools Microservice Pools Kafka DSE ZFS ZFS ZFS Replication Hbase/Kudu Replication Kafka Replicator OR Always

19. Copyright © 2016 by Surescripts, LLC. All rights reserved. Overall operational experience  In a word - outstanding  Read & write performance has equaled or bettered our relational OLTP tier • Now manageably scalable  One rolling upgrade since going live in November without loss of uptime  DataStax Support was very responsive • When we were affected by a base bug (CASSANDRA-12247) that caused intermittent repair failures and write timeouts  Improved efficiency • The involved DBE reports less on-going maintenance activity rather than more for equivalent Oracle infrastructure. 19

20. Copyright © 2016 by Surescripts, LLC. All rights reserved. Cluster performance summary (last 7 days) Write throughput, cluster peak 720 writes/sec peak ~9.5 million/day Read throughput, Cluster peak 305 reads/sec peak ~4.5 million/day Write Latency 99% latency 65 ms/write 50% latency 15 ms/write Read Latency 99% latency 79 ms/read 50% latency 38 ms/write Exceptions none

24. Copyright © 2016 by Surescripts, LLC. All rights reserved. Use case criteria guiding choice of DSE  Stereotyped (ie. development time) access patterns  Size thoughtfully  Replication requirements  Scale capabilities  Reliability via cluster architecture • We will require a very long baseline for full answer. 24

25. Copyright © 2016 by Surescripts, LLC. All rights reserved. Things we think we did right, and wrong  Things we did right • Kept to the well understood wheelhouse of DSE - primary key access pattern • Paid attention to the recommendation to stay under 50% disk capacity when determining sizing • Blue/green deployment option has given us precise understanding of behavior of new versus prior infrastructures.  Things we could do better • Have not yet broadly familiarized our db staff with DSE infrastructure ▪ So far only two individuals. • We have partition keys that include identifiers generated by our trading partners. These should be unique according to business rules, but we do not enforce uniqueness, so we are at risk of wide partitions.  Things we don’t yet know • Major version/ major OS upgrade events haven’t occurred yet. 25

26. Next steps Learn more about DataStax Enterprise:  eBook • The Magic Number 5 - Why Cloud Applications Require Always- on Data Platform • Why Your RDBMS Fails at Scale?  Analyst report • Gartner 2016 Critical Capabilities for Operational Database Management Systems  Webinar recording • Introducing DataStax Enterprise 5.1  Trainings • https://academy.datastax.com/courses  Download DSE and other toolkits 26 Learn more about DataStax Managed Cloud (DMC):  Dedicated website  eBook • Time to Innovate – Introducing DataStax Managed Cloud  DMC datasheet  Webinar recording • DataStax Enterprise Managed Cloud: Focus on Innovation, not Administration

Webinar - Delivering Enhanced Message Processing at Scale With an Always-on Data Management Solution

Esté a ler uma amostra.

Confira estes a seguir

Webinar - Delivering Enhanced Message Processing at Scale With an Always-on Data Management Solution

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Webinar - Delivering Enhanced Message Processing at Scale With an Always-on Data Management Solution (20)

Mais de DataStax (17)

Mais recentes (20)