O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Using BigQuery as a main Big Data solution

1.369 visualizações

Publicada em

My recent slide deck on BigQuery and how we use it in Wego.com

Publicada em: Dados e análise
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/ROpLZ ◀ ◀ ◀ ◀
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Seja a primeira pessoa a gostar disto

Using BigQuery as a main Big Data solution

  1. 1. Nikolay Novozhilov Wego.com Using BigQuery as a main Big Data solution
  2. 2. About Wego Wego.com is Asia Pacific and the Middle East’s leading flight/hotel metasearch engine used by millions of travelers. Wego was founded in 2005 in Singapore
  3. 3. Introducing BigQuery Service for interactive analysis of massive datasets (TBs) Query billions of rows: seconds to write, seconds to return Uses a SQL-style query syntax It's a service, accessed by a RESTful API Pay only for what you use Based on internal Google tool - Dremel Column oriented, append only…
  4. 4. Data architecture in Wego ...
  5. 5. Why did we do it? MySQL “Zoo” BigQuery
  6. 6. Why Hadoop is more popular?
  7. 7. My collection of concerns Your data goes to cloud Not open-source, Google can stop the service “Strange” pricing model Hadoop is trending, has bigger community Append only database ???
  8. 8. Costs: storage + cost per query Same fallacy again:  “I want to launch a mom@pop – let’s buy a building”  “I want to build a site – let’s by servers”  “I want big data – let’s build a data-warehouse” Usual concerns:  No realistic estimate upfront  “Fear of running a query”
  9. 9. StackOverflow support 53 minutes!
  10. 10. Append only… Slowly changing dimensions:  daily re-load from MySQL  daily upload from MySQL, keeping history Absolutely necessary updates:  do you really need it?  BigQuery allows to save query to initial table: Your table Query
  11. 11. Actually useful - “Discovery mode”
  12. 12. Actually useful Huge joins REGEXT_MATCH(), … Rich SQL - window functions Nested data
  13. 13. My answer
  14. 14. What is Big Data revolution? There is no difference between big data and small data anymore
  15. 15. Contacts Blog: www.novozhilov.co Email: nik@wego.com
  16. 16. “Yes, Sir, I tired to build an ROI case for our BI project - but I couldn’t access any reliable data!” TimoElliott.com

×