O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
How to build a SQL-based data warehouse for a trillion rows in Python
By Ville Tuulos
http://tuulos.github.io/pydata-2014/...
Próximos SlideShares
Carregando em…5
×

How to build a SQL-based data warehouse for a trillion rows in Python by Ville Tuulos PyData SV 2014

870 visualizações

Publicada em

In this talk, we show how and why AdRoll built a custom, high-performance data warehouse in Python which can handle hundreds of billions of data points with sub-minute latency on a small cluster of servers. This feat is made possible by a non-trivial combination of compressed data structures, meta-programming, and just-in-time compilation using Numba, a compiler for numerical Python. To enable smooth interoperability with existing tools, the system provides a standard SQL-interface using Multicorn and Foreign Data Wrappers in PostgreSQL.

Publicada em: Tecnologia
  • Entre para ver os comentários

  • Seja a primeira pessoa a gostar disto

How to build a SQL-based data warehouse for a trillion rows in Python by Ville Tuulos PyData SV 2014

  1. 1. How to build a SQL-based data warehouse for a trillion rows in Python By Ville Tuulos http://tuulos.github.io/pydata-2014/#/

×