Enviar pesquisa
Carregar
Postgres Sql at TripAdvisor
•
5 gostaram
•
1,758 visualizações
Don O'Neill
Seguir
Slide Deck from my recent talk at the first New England PostgreSQL User group meeting.
Leia menos
Leia mais
Tecnologia
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 10
Recomendados
BigDataInOperationsV8
BigDataInOperationsV8
Don O'Neill
Managing Large Scale Financial Time-Series Data with Graphs
Managing Large Scale Financial Time-Series Data with Graphs
Objectivity
Spark Summit Keynote by Suren Nathan
Spark Summit Keynote by Suren Nathan
Spark Summit
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
Shawn Jones
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Matt Stubbs
Memrise presentation @ London Snowplow meetup
Memrise presentation @ London Snowplow meetup
idan_by
Treasure Data From MySQL to Redshift
Treasure Data From MySQL to Redshift
Treasure Data, Inc.
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
confluent
Recomendados
BigDataInOperationsV8
BigDataInOperationsV8
Don O'Neill
Managing Large Scale Financial Time-Series Data with Graphs
Managing Large Scale Financial Time-Series Data with Graphs
Objectivity
Spark Summit Keynote by Suren Nathan
Spark Summit Keynote by Suren Nathan
Spark Summit
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
Shawn Jones
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Matt Stubbs
Memrise presentation @ London Snowplow meetup
Memrise presentation @ London Snowplow meetup
idan_by
Treasure Data From MySQL to Redshift
Treasure Data From MySQL to Redshift
Treasure Data, Inc.
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
confluent
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
DataWorks Summit/Hadoop Summit
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Streamsets Inc.
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Spark Summit
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Nascenia IT
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
✔ Eric David Benari, PMP
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Natalino Busa
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Spark Summit
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
dfilppi
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
SAP Technology
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Databricks
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
Marc Fielding
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
DataStax
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Spark Summit
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
alistair_hann
Presto summit israel 2019-04
Presto summit israel 2019-04
Ori Reshef
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Spark Summit
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
SingleStore
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Ori Reshef
Senior database administrator
Senior database administrator
Mustafa EL-Masry
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
Mais conteúdo relacionado
Mais procurados
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
DataWorks Summit/Hadoop Summit
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Streamsets Inc.
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Spark Summit
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Nascenia IT
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
✔ Eric David Benari, PMP
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Natalino Busa
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Spark Summit
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
dfilppi
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
SAP Technology
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Databricks
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
Marc Fielding
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
DataStax
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Spark Summit
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
alistair_hann
Presto summit israel 2019-04
Presto summit israel 2019-04
Ori Reshef
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Spark Summit
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
SingleStore
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Ori Reshef
Mais procurados
(20)
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
Presto summit israel 2019-04
Presto summit israel 2019-04
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Semelhante a Postgres Sql at TripAdvisor
Senior database administrator
Senior database administrator
Mustafa EL-Masry
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
Full Stack Web Developer (MERN STACK Developer.pptx
Full Stack Web Developer (MERN STACK Developer.pptx
RamudgarYadav
Top local databases for react native app development
Top local databases for react native app development
SameerShaik43
Neethu_Abraham
Neethu_Abraham
Neethu Abraham
List of Top Local Databases used for react native app developement in 2022
List of Top Local Databases used for react native app developement in 2022
Shelly Megan
Introduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
Cecile Le Pape
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Cécile Poyet
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Hortonworks
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Cécile Poyet
Metadata describes about data
Metadata describes about data
Dev Sanskriti Vishwavidyalaya (University)
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
confluent
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
Dynatrace
Whats new in Oracle Database 12c release 12.1.0.2
Whats new in Oracle Database 12c release 12.1.0.2
Connor McDonald
Introduction to Cloud Service Design
Introduction to Cloud Service Design
evancmiller
Cloud-Native Data: What data questions to ask when building cloud-native apps
Cloud-Native Data: What data questions to ask when building cloud-native apps
VMware Tanzu
React Native Local Database For Super Smart Applications.pdf
React Native Local Database For Super Smart Applications.pdf
Katy Slemon
Rahul_Resume_310116
Rahul_Resume_310116
RAHUL MEHROTRA
Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101
MongoDB
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar Slides
Hortonworks
Semelhante a Postgres Sql at TripAdvisor
(20)
Senior database administrator
Senior database administrator
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Full Stack Web Developer (MERN STACK Developer.pptx
Full Stack Web Developer (MERN STACK Developer.pptx
Top local databases for react native app development
Top local databases for react native app development
Neethu_Abraham
Neethu_Abraham
List of Top Local Databases used for react native app developement in 2022
List of Top Local Databases used for react native app developement in 2022
Introduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Metadata describes about data
Metadata describes about data
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
Whats new in Oracle Database 12c release 12.1.0.2
Whats new in Oracle Database 12c release 12.1.0.2
Introduction to Cloud Service Design
Introduction to Cloud Service Design
Cloud-Native Data: What data questions to ask when building cloud-native apps
Cloud-Native Data: What data questions to ask when building cloud-native apps
React Native Local Database For Super Smart Applications.pdf
React Native Local Database For Super Smart Applications.pdf
Rahul_Resume_310116
Rahul_Resume_310116
Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar Slides
Último
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
Nicole Novielli
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
Ingrid Airi González
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
IES VE
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
Curtis Poe
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Mydbops
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
BookNet Canada
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
MounikaPolabathina
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
Skynet Technologies
A Framework for Development in the AI Age
A Framework for Development in the AI Age
Cprime
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
LoriGlavin3
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Sergiu Bodiu
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
BookNet Canada
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
HarshalMandlekar2
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
ThousandEyes
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
LoriGlavin3
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
Neo4j
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
Raghuram Pandurangan
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
LoriGlavin3
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
Knoldus Inc.
Último
(20)
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
A Framework for Development in the AI Age
A Framework for Development in the AI Age
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
Postgres Sql at TripAdvisor
1.
No, we’re not
a travel agency www.tripadvisor.com/careers
2.
Postgres at TripAdvisor
By Don O’Neill & Boris Partensky 2 © 2011 TripAdvisor LLC All rights reserved.
3.
TripAdvisor is the
world’s largesttravel site 3 © 2011 TripAdvisor LLC All rights reserved.
4.
4 40M monthly
visitors 4M visitors/day 20+M registered members 45+M reviews & opinions © 2011 TripAdvisor LLC All rights reserved.
5.
Business runs on
Postgres Large Mission critical deployment Combination of 8.3 and 8.4 Site Technologies include Linux (CentOS), Apache Java, Tomcat, Velocity, and Memcached No commercial Software – All Open Source Web Scale Deployment Over 100 Frontend and Backend servers per Data Center 14 Postgres Database Servers per Data Center DR Data Center with Replication between the two Platform 5 © 2011 TripAdvisor LLC All rights reserved.
6.
33 Postgres Servers
across 3 Data Centers Not including Dev and QA servers SSD’s instead of Spinning disks for select servers Multiple Read Only DB Servers Load Balanced using BigIP Sync’d using DB Mirror replication Just under 1 TB of data in multiple databases Sharded Pin Table Over 1.5 billion rows Sharded to improve read and write performance Spread over 12 databases on 2 Servers Scale out to multiple servers as needed Routing logic built into our application layer By the Numbers 6 © 2011 TripAdvisor LLC All rights reserved.
7.
All Servers configured
as Active/Passive Cluster Not using Shared Disks or SAN’s DRBD for Block level replication between systems Heartbeat for automatic failover between nodes Data Replicated to DR Site TripAdvisor has 2 Data centers (MA and VA) One site “Live” at a time Equal capacity at both sites Routinely switch between sites Data Replicated between data centers Redundancy/Failover 7 © 2011 TripAdvisor LLC All rights reserved.
8.
Replication based on
DBmirror Like Slonyonly deprecated Community project once part of the core distribution Trigger based Master/Slave Replication Written in C and Perl All replicated tables must have primary keys TripAdvisor Updates include A Rewrite in Java! Support for SQL Server Lots of Hardening for large scale use Replication 8 © 2011 TripAdvisor LLC All rights reserved.
9.
20 TB Data
Warehouse today Expecting to grow significantly in 12 months 16 Node cluster expanding to 24 Based on Hadoop and Hive Reporting and Cubes built with SQL Server End users access the data via Excel and Reporting Services Critical for the growth of the business Long term data storage in Hadoop and not our SAN! Mobile Applications Award winning travel application Supported in iPhone, iPad, Android, Nokia, Windows, Palm http://www.tripadvisor.com/MobileApps Facebook Integration More and more users logging in via Facebook Better social experience on the site Major Initiatives 9 © 2011 TripAdvisor LLC All rights reserved.
10.
Q&A Questions? Did
I mention we’re hiring? 10 © 2011 TripAdvisor LLC All rights reserved.