SlideShare uma empresa Scribd logo
1 de 48
Baixar para ler offline
Postgres vs
Elasticsearch while
enriching data.
Vlad Somov @ Salt Edge Inc.
Unstructured Data
Enrichment
Incoming raw data
Structured identified data
Keyword1 Keyword2 Website Name
Tag
Keyword1 Keyword2 Website Name
Tag
Unstructured Data
Enrichment
Some Transaction Description Website
Incoming raw data
Keyword1 Keyword2 Website
Structured identified data
Name
Tag
Description
Keyword1
Tag
Basic Setup Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
~4mln. Records
Basic Setup Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
28.73
9.88
2.10
~4mln. Records
Basic Setup Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
1.37
0.99
0.73
28.73
9.88
2.10
~4mln. Records
B-tree index structure
3 39 68
meta
39 42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
39
B-tree index structure
3 68
meta
39 42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
39
39
B-tree index structure
3 68
meta
42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
39
39
B-tree index structure
3 68
meta
42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
39
39
B-tree index structure
3 68
meta
42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
39
39
B-tree index structure
3 68
meta
42 55 68 89 943 15 28
3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
Why it is useful?
• b-tree index sort values inside each node.

• b-tree is balanced

• Same level nodes are connected using doubly linked list.
After multicolumn index on country_id
and merchant_type Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
Postgres + multicolumn index
~4mln. Records
After multicolumn index on country_id
and merchant_type Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
Postgres + multicolumn index
28.73
9.88
2.1
~4mln. Records
After multicolumn index on country_id
and merchant_type Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
Postgres + multicolumn index
1.37
0.99
0.73
28.73
9.88
2.1
~4mln. Records
After multicolumn index on country_id
and merchant_type Performance
Min
Average
Max
Seconds
0 7.5 15 22.5 30
Postgres Elasticsearch
Postgres + multicolumn index
10.19
5.09
2.28
1.37
0.99
0.73
28.73
9.88
2.1
~4mln. Records
What is GiST
Generalized Search
Tree
• In GiST each leaf contains
logical expression and
pointer to TID, where
indexed data should
satisfy logical expression.

• Faster on insert, update
What is GIN
Generalized Inverted
Index
• It is b-tree with elements to
which is connected another
b-tree or plain list of TID's. 

• Faster and more accurate
on select.
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1 0,10,1
2,1
1,51,5 2,1 2,1
Yellow rectangle are TID’s. First number is a page number and second is
position on a page
0,1
1,52,1
1,5
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1
1,5
0,1 0,10,1
2,1
1,51,5 2,1 2,1
Yellow rectangle are TID’s. First number is a page number and second is
position on a page
2,1
1,5
ruby
rubylove
love
1,5
1,5
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1 0,1 0,10,1
2,1
1,51,5 2,1 2,1
Yellow rectangle are TID’s. First number is a page number and second is
position on a page
2,1
ruby
rubylove
love
love ruby
gin_trgm_ops
A trigram is a group of three consecutive characters
taken from a string.
We can measure the similarity of two strings by counting the
number of trigrams they share.
Performance after gin index on websites
Min
Average
Max
Seconds
0 5 10 15 20
Postgres
Elasticsearch
Postgres + multicolumn index
Postgres + gin index with trgm_ops on websites
~4mln. Records
Performance after gin index on websites
Min
Average
Max
Seconds
0 5 10 15 20
Postgres
Elasticsearch
Postgres + multicolumn index
Postgres + gin index with trgm_ops on websites
28.73
9.88
2.1
~4mln. Records
Performance after gin index on websites
Min
Average
Max
Seconds
0 5 10 15 20
Postgres
Elasticsearch
Postgres + multicolumn index
Postgres + gin index with trgm_ops on websites
1.37
0.99
0.75
28.73
9.88
2.1
~4mln. Records
Performance after gin index on websites
Min
Average
Max
Seconds
0 5 10 15 20
Postgres
Elasticsearch
Postgres + multicolumn index
Postgres + gin index with trgm_ops on websites
10.19
5.09
2.28
1.37
0.99
0.75
28.73
9.88
2.1
~4mln. Records
Performance after gin index on websites
Min
Average
Max
Seconds
0 5 10 15 20
Postgres
Elasticsearch
Postgres + multicolumn index
Postgres + gin index with trgm_ops on websites
0.55
0.34
0.26
10.19
5.09
2.28
1.37
0.99
0.75
28.73
9.88
2.1
~4mln. Records
How elasticsearch works
• It uses analyzers for all incoming data. (it could be custom
or default one)

• Each analyzer has at least one tokenizer

• Zero or more TokenFilters

• Tokenizer may be preceded by one or more CharFilters
How analyzer works?
How analyzer works?
Input
How analyzer works?
Input Char Filter
String
How analyzer works?
Input Char Filter Tokenizer
String String
How analyzer works?
Input Char Filter Tokenizer
Token
Filter
String String Tokens
How analyzer works?
Input Char Filter Tokenizer
Token
Filter
Output
String String Tokens Tokens
Example
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
html_strip
The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
html_strip
The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
standart tokenizer
The 2 QUICK Brown jumpedFoxes over
the lazy dog’s bone
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
html_strip
The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
standart tokenizer
The 2 QUICK Brown jumpedFoxes over
the lazy dog’s bone
lowercase
the 2 quick brown jumpedfoxes over
the lazy dog’s bone
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
html_strip
The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
standart tokenizer
The 2 QUICK Brown jumpedFoxes over
the lazy dog’s bone
lowercase
the 2 quick brown jumpedfoxes over
the lazy dog’s bone
stop
2 quick brown jumpedfoxes over lazy dog’s bone
the
the
Example
The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
html_strip
The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
standart tokenizer
The 2 QUICK Brown jumpedFoxes over
the lazy dog’s bone
lowercase
the 2 quick brown jumpedfoxes over
the lazy dog’s bone
stop
2 quick brown jumpedfoxes over lazy dog’s bone
snowball
2 quick brown jumpfox over lazi dog bone
the
the
jump lazi dog
Postgres full search
implementation
• We can use tsvector type to achieve almost the same
functionality. By using to_tsvector function

• To imporve perfomance we could create separate tsvector
column with to_tsvector values.

• To create a request we should use to_tsquery. & | <->

• plainto_tsquery works with plain text so you don’t need to
insert any special symbols. Inserts &

• phraseto_tsquery also works with plain text but marks that
each token should be close to each other. Inserts <->
Rum access method
• Based on GIN access method code

• Solves slow ranking

• Solves slow phrase search (tsquery with <-> operator)

• Supports index on tsquery column
122
1
5
3
2
4
4
3
3
4211
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
ruby, meditation, love
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1 0,10,1
2,1
1,51,5 2,1 2,1
The number in green rectangle is word position in the document.
0,1
1,52,1
1,5
8,4 8,4 8,4
122
1
5
3
2
4
4
3
3
4211
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
ruby, meditation, love
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1
1,5
0,1 0,10,1
2,1
1,51,5 2,1 2,1
The number in green rectangle is word position in the document.
2,1
1,5
ruby
rubylove
love
love
ruby
8,4 8,4 8,4
122
1
5
3
2
4
4
3
3
4211
1,5
1,5
Welcome to ruby meditation.

All of us love ruby.
Does everyone love meditation?
ruby, meditation, love
Everyone Of Welcome
All Does WelcomeRuby ToOfLove MeditationEveryone
0,1 0,1 0,10,1
2,1
1,51,5 2,1 2,1
The number in green rectangle is word position in the document.
2,1
ruby
rubylove
love
love ruby
love
ruby
8,4 8,4 8,4
Conclusion
• Postgres can also be fast.

• Multicolumn indexes can improve performance if your search has multicolumn
constraints.

• For fast text search prefer using Gin when table doesn’t update occasionally,
otherwise use GiST

• Use gin with trgm_ops when using full text search. If full text search is still slow
try to use tsvector data type with gin index on it.

• When you have some kind ‘inverse full-text search’ problem. Add tsquery type in
your table as a query and incoming data treat as a document. Add rum access
method on query column with tsquery_ops for fast classification.

• Before moving to other instrument make analysis of current/new instrument and
verify is it worth moving or not.
email: vlad.somov@icloud.com
twitter: @vsomov93
Questions?

Mais conteúdo relacionado

Mais procurados

Apache Auroraの始めかた
Apache Auroraの始めかたApache Auroraの始めかた
Apache Auroraの始めかたMasahito Zembutsu
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetLucian Oprea
 
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)NTT DATA Technology & Innovation
 
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)NTT DATA Technology & Innovation
 
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOTricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOAltinity Ltd
 
High Performance Systems in Go - GopherCon 2014
High Performance Systems in Go - GopherCon 2014High Performance Systems in Go - GopherCon 2014
High Performance Systems in Go - GopherCon 2014Derek Collison
 
後悔しないもんごもんごの使い方 〜アプリ編〜
後悔しないもんごもんごの使い方 〜アプリ編〜後悔しないもんごもんごの使い方 〜アプリ編〜
後悔しないもんごもんごの使い方 〜アプリ編〜Masakazu Matsushita
 
Auditing and Monitoring PostgreSQL/EPAS
Auditing and Monitoring PostgreSQL/EPASAuditing and Monitoring PostgreSQL/EPAS
Auditing and Monitoring PostgreSQL/EPASEDB
 
Ruby 3のキーワード引数について考える
Ruby 3のキーワード引数について考えるRuby 3のキーワード引数について考える
Ruby 3のキーワード引数について考えるmametter
 
PostgreSQL SQLチューニング入門 実践編(pgcon14j)
PostgreSQL SQLチューニング入門 実践編(pgcon14j)PostgreSQL SQLチューニング入門 実践編(pgcon14j)
PostgreSQL SQLチューニング入門 実践編(pgcon14j)Satoshi Yamada
 
Postgres connections at scale
Postgres connections at scalePostgres connections at scale
Postgres connections at scaleMydbops
 
Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer Sameer Kumar
 
雑なMySQLパフォーマンスチューニング
雑なMySQLパフォーマンスチューニング雑なMySQLパフォーマンスチューニング
雑なMySQLパフォーマンスチューニングyoku0825
 
Patroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyPatroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyAlexander Kukushkin
 
[Meetup] a successful migration from elastic search to clickhouse
[Meetup] a successful migration from elastic search to clickhouse[Meetup] a successful migration from elastic search to clickhouse
[Meetup] a successful migration from elastic search to clickhouseVianney FOUCAULT
 
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)NTT DATA Technology & Innovation
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HAharoonm
 
Vectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLVectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLJonathan Katz
 
Extreme Replication - Performance Tuning Oracle GoldenGate
Extreme Replication - Performance Tuning Oracle GoldenGateExtreme Replication - Performance Tuning Oracle GoldenGate
Extreme Replication - Performance Tuning Oracle GoldenGateBobby Curtis
 
Elasticsearchインデクシングのパフォーマンスを測ってみた
Elasticsearchインデクシングのパフォーマンスを測ってみたElasticsearchインデクシングのパフォーマンスを測ってみた
Elasticsearchインデクシングのパフォーマンスを測ってみたRyoji Kurosawa
 

Mais procurados (20)

Apache Auroraの始めかた
Apache Auroraの始めかたApache Auroraの始めかた
Apache Auroraの始めかた
 
PostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_CheatsheetPostgreSQL High_Performance_Cheatsheet
PostgreSQL High_Performance_Cheatsheet
 
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)
YugabyteDBを使ってみよう(NewSQL/分散SQLデータベースよろず勉強会 #1 発表資料)
 
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)
レプリケーション遅延の監視について(第40回PostgreSQLアンカンファレンス@オンライン 発表資料)
 
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOTricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
 
High Performance Systems in Go - GopherCon 2014
High Performance Systems in Go - GopherCon 2014High Performance Systems in Go - GopherCon 2014
High Performance Systems in Go - GopherCon 2014
 
後悔しないもんごもんごの使い方 〜アプリ編〜
後悔しないもんごもんごの使い方 〜アプリ編〜後悔しないもんごもんごの使い方 〜アプリ編〜
後悔しないもんごもんごの使い方 〜アプリ編〜
 
Auditing and Monitoring PostgreSQL/EPAS
Auditing and Monitoring PostgreSQL/EPASAuditing and Monitoring PostgreSQL/EPAS
Auditing and Monitoring PostgreSQL/EPAS
 
Ruby 3のキーワード引数について考える
Ruby 3のキーワード引数について考えるRuby 3のキーワード引数について考える
Ruby 3のキーワード引数について考える
 
PostgreSQL SQLチューニング入門 実践編(pgcon14j)
PostgreSQL SQLチューニング入門 実践編(pgcon14j)PostgreSQL SQLチューニング入門 実践編(pgcon14j)
PostgreSQL SQLチューニング入門 実践編(pgcon14j)
 
Postgres connections at scale
Postgres connections at scalePostgres connections at scale
Postgres connections at scale
 
Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer Connection Pooling in PostgreSQL using pgbouncer
Connection Pooling in PostgreSQL using pgbouncer
 
雑なMySQLパフォーマンスチューニング
雑なMySQLパフォーマンスチューニング雑なMySQLパフォーマンスチューニング
雑なMySQLパフォーマンスチューニング
 
Patroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easyPatroni - HA PostgreSQL made easy
Patroni - HA PostgreSQL made easy
 
[Meetup] a successful migration from elastic search to clickhouse
[Meetup] a successful migration from elastic search to clickhouse[Meetup] a successful migration from elastic search to clickhouse
[Meetup] a successful migration from elastic search to clickhouse
 
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)
pg_walinspectについて調べてみた!(第37回PostgreSQLアンカンファレンス@オンライン 発表資料)
 
PostgreSQL HA
PostgreSQL   HAPostgreSQL   HA
PostgreSQL HA
 
Vectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLVectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQL
 
Extreme Replication - Performance Tuning Oracle GoldenGate
Extreme Replication - Performance Tuning Oracle GoldenGateExtreme Replication - Performance Tuning Oracle GoldenGate
Extreme Replication - Performance Tuning Oracle GoldenGate
 
Elasticsearchインデクシングのパフォーマンスを測ってみた
Elasticsearchインデクシングのパフォーマンスを測ってみたElasticsearchインデクシングのパフォーマンスを測ってみた
Elasticsearchインデクシングのパフォーマンスを測ってみた
 

Semelhante a Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton #23

Basics in algorithms and data structure
Basics in algorithms and data structure Basics in algorithms and data structure
Basics in algorithms and data structure Eman magdy
 
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...ZFConf Conference
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
About elasticsearch
About elasticsearchAbout elasticsearch
About elasticsearchMinsoo Jun
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
Unveiling etcd: Architecture and Source Code Deep Dive
Unveiling etcd: Architecture and Source Code Deep DiveUnveiling etcd: Architecture and Source Code Deep Dive
Unveiling etcd: Architecture and Source Code Deep DiveChieh (Jack) Yu
 
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...Citus Data
 
The Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryThe Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryHanna Kelman
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
Better Full Text Search in PostgreSQL
Better Full Text Search in PostgreSQLBetter Full Text Search in PostgreSQL
Better Full Text Search in PostgreSQLArtur Zakirov
 
SequoiaDB Distributed Relational Database
SequoiaDB Distributed Relational DatabaseSequoiaDB Distributed Relational Database
SequoiaDB Distributed Relational Databasewangzhonnew
 
Beyond PHP - it's not (just) about the code
Beyond PHP - it's not (just) about the codeBeyond PHP - it's not (just) about the code
Beyond PHP - it's not (just) about the codeWim Godden
 
Slash n near real time indexing
Slash n   near real time indexingSlash n   near real time indexing
Slash n near real time indexingUmesh Prasad
 
Elasticsearch at Dailymotion
Elasticsearch at DailymotionElasticsearch at Dailymotion
Elasticsearch at DailymotionCédric Hourcade
 
SDPHP - Percona Toolkit (It's Basically Magic)
SDPHP - Percona Toolkit (It's Basically Magic)SDPHP - Percona Toolkit (It's Basically Magic)
SDPHP - Percona Toolkit (It's Basically Magic)Robert Swisher
 
A Call for Sanity in NoSQL
A Call for Sanity in NoSQLA Call for Sanity in NoSQL
A Call for Sanity in NoSQLC4Media
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsSerge Smetana
 
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance FuckupsNETFest
 
Moving Toward Deep Learning Algorithms on HPCC Systems
Moving Toward Deep Learning Algorithms on HPCC SystemsMoving Toward Deep Learning Algorithms on HPCC Systems
Moving Toward Deep Learning Algorithms on HPCC SystemsHPCC Systems
 
London devops logging
London devops loggingLondon devops logging
London devops loggingTomas Doran
 

Semelhante a Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton #23 (20)

Basics in algorithms and data structure
Basics in algorithms and data structure Basics in algorithms and data structure
Basics in algorithms and data structure
 
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...
ZFConf 2011: Что такое Sphinx, зачем он вообще нужен и как его использовать с...
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
About elasticsearch
About elasticsearchAbout elasticsearch
About elasticsearch
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
Unveiling etcd: Architecture and Source Code Deep Dive
Unveiling etcd: Architecture and Source Code Deep DiveUnveiling etcd: Architecture and Source Code Deep Dive
Unveiling etcd: Architecture and Source Code Deep Dive
 
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...
The Challenges of Distributing Postgres: A Citus Story | DataEngConf NYC 2017...
 
The Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus StoryThe Challenges of Distributing Postgres: A Citus Story
The Challenges of Distributing Postgres: A Citus Story
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
Better Full Text Search in PostgreSQL
Better Full Text Search in PostgreSQLBetter Full Text Search in PostgreSQL
Better Full Text Search in PostgreSQL
 
SequoiaDB Distributed Relational Database
SequoiaDB Distributed Relational DatabaseSequoiaDB Distributed Relational Database
SequoiaDB Distributed Relational Database
 
Beyond PHP - it's not (just) about the code
Beyond PHP - it's not (just) about the codeBeyond PHP - it's not (just) about the code
Beyond PHP - it's not (just) about the code
 
Slash n near real time indexing
Slash n   near real time indexingSlash n   near real time indexing
Slash n near real time indexing
 
Elasticsearch at Dailymotion
Elasticsearch at DailymotionElasticsearch at Dailymotion
Elasticsearch at Dailymotion
 
SDPHP - Percona Toolkit (It's Basically Magic)
SDPHP - Percona Toolkit (It's Basically Magic)SDPHP - Percona Toolkit (It's Basically Magic)
SDPHP - Percona Toolkit (It's Basically Magic)
 
A Call for Sanity in NoSQL
A Call for Sanity in NoSQLA Call for Sanity in NoSQL
A Call for Sanity in NoSQL
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
 
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups
.NET Fest 2019. Łukasz Pyrzyk. Daily Performance Fuckups
 
Moving Toward Deep Learning Algorithms on HPCC Systems
Moving Toward Deep Learning Algorithms on HPCC SystemsMoving Toward Deep Learning Algorithms on HPCC Systems
Moving Toward Deep Learning Algorithms on HPCC Systems
 
London devops logging
London devops loggingLondon devops logging
London devops logging
 

Mais de Ruby Meditation

Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30
Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30
Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30Ruby Meditation
 
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...Ruby Meditation
 
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29Ruby Meditation
 
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...Ruby Meditation
 
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 Ruby Meditation
 
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28Ruby Meditation
 
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...Ruby Meditation
 
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...Ruby Meditation
 
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Ruby Meditation
 
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...Ruby Meditation
 
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27Ruby Meditation
 
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26Ruby Meditation
 
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Ruby Meditation
 
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Ruby Meditation
 
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26Ruby Meditation
 
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25Ruby Meditation
 
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...Ruby Meditation
 
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...Ruby Meditation
 
Rails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan GusievRails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan GusievRuby Meditation
 
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23Ruby Meditation
 

Mais de Ruby Meditation (20)

Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30
Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30Is this Legacy or Revenant Code? - Sergey Sergyenko  | Ruby Meditation 30
Is this Legacy or Revenant Code? - Sergey Sergyenko | Ruby Meditation 30
 
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
Life with GraphQL API: good practices and unresolved issues - Roman Dubrovsky...
 
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
Where is your license, dude? - Viacheslav Miroshnychenko | Ruby Meditation 29
 
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
Dry-validation update. Dry-validation vs Dry-schema 1.0 - Aleksandra Stolyar ...
 
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28 How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
How to cook Rabbit on Production - Bohdan Parshentsev | Ruby Meditation 28
 
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
How to cook Rabbit on Production - Serhiy Nazarov | Ruby Meditation 28
 
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...
Reinventing the wheel - why do it and how to feel good about it - Julik Tarkh...
 
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
Performance Optimization 101 for Ruby developers - Nihad Abbasov (ENG) | Ruby...
 
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
Use cases for Serverless Technologies - Ruslan Tolstov (RUS) | Ruby Meditatio...
 
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
The Trailblazer Ride from the If Jungle into a Civilised Railway Station - Or...
 
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
What/How to do with GraphQL? - Valentyn Ostakh (ENG) | Ruby Meditation 27
 
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26New features in Rails 6 -  Nihad Abbasov (RUS) | Ruby Meditation 26
New features in Rails 6 - Nihad Abbasov (RUS) | Ruby Meditation 26
 
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
Security Scanning Overview - Tetiana Chupryna (RUS) | Ruby Meditation 26
 
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
Teach your application eloquence. Logs, metrics, traces - Dmytro Shapovalov (...
 
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
Best practices. Exploring - Ike Kurghinyan (RUS) | Ruby Meditation 26
 
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
Road to A/B testing - Alexey Vasiliev (ENG) | Ruby Meditation 25
 
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
Concurrency in production. Real life example - Dmytro Herasymuk | Ruby Medita...
 
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
Data encryption for Ruby web applications - Dmytro Shapovalov (RUS) | Ruby Me...
 
Rails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan GusievRails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan Gusiev
 
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
GDPR. Next Y2K in 2018? - Anton Tkachov | Ruby Meditation #23
 

Último

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 

Último (20)

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 

Postgres vs Elasticsearch while enriching data - Vlad Somov | Ruby Meditaiton #23

  • 1. Postgres vs Elasticsearch while enriching data. Vlad Somov @ Salt Edge Inc.
  • 2. Unstructured Data Enrichment Incoming raw data Structured identified data
  • 3. Keyword1 Keyword2 Website Name Tag Keyword1 Keyword2 Website Name Tag Unstructured Data Enrichment Some Transaction Description Website Incoming raw data Keyword1 Keyword2 Website Structured identified data Name Tag Description Keyword1 Tag
  • 4. Basic Setup Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch ~4mln. Records
  • 5. Basic Setup Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch 28.73 9.88 2.10 ~4mln. Records
  • 6. Basic Setup Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch 1.37 0.99 0.73 28.73 9.88 2.10 ~4mln. Records
  • 7. B-tree index structure 3 39 68 meta 39 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 8. 39 B-tree index structure 3 68 meta 39 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 9. 39 39 B-tree index structure 3 68 meta 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 10. 39 39 B-tree index structure 3 68 meta 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 11. 39 39 B-tree index structure 3 68 meta 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 12. 39 39 B-tree index structure 3 68 meta 42 55 68 89 943 15 28 3 9 15 21 29 32 39 42 42 48 55 68 68 77 89 93 94 98
  • 13. Why it is useful? • b-tree index sort values inside each node. • b-tree is balanced • Same level nodes are connected using doubly linked list.
  • 14. After multicolumn index on country_id and merchant_type Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch Postgres + multicolumn index ~4mln. Records
  • 15. After multicolumn index on country_id and merchant_type Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch Postgres + multicolumn index 28.73 9.88 2.1 ~4mln. Records
  • 16. After multicolumn index on country_id and merchant_type Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch Postgres + multicolumn index 1.37 0.99 0.73 28.73 9.88 2.1 ~4mln. Records
  • 17. After multicolumn index on country_id and merchant_type Performance Min Average Max Seconds 0 7.5 15 22.5 30 Postgres Elasticsearch Postgres + multicolumn index 10.19 5.09 2.28 1.37 0.99 0.73 28.73 9.88 2.1 ~4mln. Records
  • 18. What is GiST Generalized Search Tree • In GiST each leaf contains logical expression and pointer to TID, where indexed data should satisfy logical expression. • Faster on insert, update What is GIN Generalized Inverted Index • It is b-tree with elements to which is connected another b-tree or plain list of TID's. • Faster and more accurate on select.
  • 19. Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 0,10,1 2,1 1,51,5 2,1 2,1 Yellow rectangle are TID’s. First number is a page number and second is position on a page 0,1 1,52,1 1,5
  • 20. Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 1,5 0,1 0,10,1 2,1 1,51,5 2,1 2,1 Yellow rectangle are TID’s. First number is a page number and second is position on a page 2,1 1,5 ruby rubylove love
  • 21. 1,5 1,5 Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 0,1 0,10,1 2,1 1,51,5 2,1 2,1 Yellow rectangle are TID’s. First number is a page number and second is position on a page 2,1 ruby rubylove love love ruby
  • 22. gin_trgm_ops A trigram is a group of three consecutive characters taken from a string. We can measure the similarity of two strings by counting the number of trigrams they share.
  • 23. Performance after gin index on websites Min Average Max Seconds 0 5 10 15 20 Postgres Elasticsearch Postgres + multicolumn index Postgres + gin index with trgm_ops on websites ~4mln. Records
  • 24. Performance after gin index on websites Min Average Max Seconds 0 5 10 15 20 Postgres Elasticsearch Postgres + multicolumn index Postgres + gin index with trgm_ops on websites 28.73 9.88 2.1 ~4mln. Records
  • 25. Performance after gin index on websites Min Average Max Seconds 0 5 10 15 20 Postgres Elasticsearch Postgres + multicolumn index Postgres + gin index with trgm_ops on websites 1.37 0.99 0.75 28.73 9.88 2.1 ~4mln. Records
  • 26. Performance after gin index on websites Min Average Max Seconds 0 5 10 15 20 Postgres Elasticsearch Postgres + multicolumn index Postgres + gin index with trgm_ops on websites 10.19 5.09 2.28 1.37 0.99 0.75 28.73 9.88 2.1 ~4mln. Records
  • 27. Performance after gin index on websites Min Average Max Seconds 0 5 10 15 20 Postgres Elasticsearch Postgres + multicolumn index Postgres + gin index with trgm_ops on websites 0.55 0.34 0.26 10.19 5.09 2.28 1.37 0.99 0.75 28.73 9.88 2.1 ~4mln. Records
  • 28. How elasticsearch works • It uses analyzers for all incoming data. (it could be custom or default one) • Each analyzer has at least one tokenizer • Zero or more TokenFilters • Tokenizer may be preceded by one or more CharFilters
  • 31. How analyzer works? Input Char Filter String
  • 32. How analyzer works? Input Char Filter Tokenizer String String
  • 33. How analyzer works? Input Char Filter Tokenizer Token Filter String String Tokens
  • 34. How analyzer works? Input Char Filter Tokenizer Token Filter Output String String Tokens Tokens
  • 36. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone.
  • 37. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone. html_strip The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.
  • 38. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone. html_strip The 2 QUICK Brown-Foxes jumped over the lazy dog's bone. standart tokenizer The 2 QUICK Brown jumpedFoxes over the lazy dog’s bone
  • 39. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone. html_strip The 2 QUICK Brown-Foxes jumped over the lazy dog's bone. standart tokenizer The 2 QUICK Brown jumpedFoxes over the lazy dog’s bone lowercase the 2 quick brown jumpedfoxes over the lazy dog’s bone
  • 40. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone. html_strip The 2 QUICK Brown-Foxes jumped over the lazy dog's bone. standart tokenizer The 2 QUICK Brown jumpedFoxes over the lazy dog’s bone lowercase the 2 quick brown jumpedfoxes over the lazy dog’s bone stop 2 quick brown jumpedfoxes over lazy dog’s bone the the
  • 41. Example The 2 QUICK <p>Brown-Foxes</p> jumped over the lazy dog's bone. html_strip The 2 QUICK Brown-Foxes jumped over the lazy dog's bone. standart tokenizer The 2 QUICK Brown jumpedFoxes over the lazy dog’s bone lowercase the 2 quick brown jumpedfoxes over the lazy dog’s bone stop 2 quick brown jumpedfoxes over lazy dog’s bone snowball 2 quick brown jumpfox over lazi dog bone the the jump lazi dog
  • 42. Postgres full search implementation • We can use tsvector type to achieve almost the same functionality. By using to_tsvector function • To imporve perfomance we could create separate tsvector column with to_tsvector values. • To create a request we should use to_tsquery. & | <-> • plainto_tsquery works with plain text so you don’t need to insert any special symbols. Inserts & • phraseto_tsquery also works with plain text but marks that each token should be close to each other. Inserts <->
  • 43. Rum access method • Based on GIN access method code • Solves slow ranking • Solves slow phrase search (tsquery with <-> operator) • Supports index on tsquery column
  • 44. 122 1 5 3 2 4 4 3 3 4211 Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? ruby, meditation, love Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 0,10,1 2,1 1,51,5 2,1 2,1 The number in green rectangle is word position in the document. 0,1 1,52,1 1,5 8,4 8,4 8,4
  • 45. 122 1 5 3 2 4 4 3 3 4211 Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? ruby, meditation, love Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 1,5 0,1 0,10,1 2,1 1,51,5 2,1 2,1 The number in green rectangle is word position in the document. 2,1 1,5 ruby rubylove love love ruby 8,4 8,4 8,4
  • 46. 122 1 5 3 2 4 4 3 3 4211 1,5 1,5 Welcome to ruby meditation.
 All of us love ruby. Does everyone love meditation? ruby, meditation, love Everyone Of Welcome All Does WelcomeRuby ToOfLove MeditationEveryone 0,1 0,1 0,10,1 2,1 1,51,5 2,1 2,1 The number in green rectangle is word position in the document. 2,1 ruby rubylove love love ruby love ruby 8,4 8,4 8,4
  • 47. Conclusion • Postgres can also be fast. • Multicolumn indexes can improve performance if your search has multicolumn constraints. • For fast text search prefer using Gin when table doesn’t update occasionally, otherwise use GiST • Use gin with trgm_ops when using full text search. If full text search is still slow try to use tsvector data type with gin index on it. • When you have some kind ‘inverse full-text search’ problem. Add tsquery type in your table as a query and incoming data treat as a document. Add rum access method on query column with tsquery_ops for fast classification. • Before moving to other instrument make analysis of current/new instrument and verify is it worth moving or not.