O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Introduction to Elasticsearch

Introduction to Elasticsearch

ในงาน code ชิวๆ ของสมาคมโปรแกรมเมอร์ไทย

  • Seja o primeiro a comentar

Introduction to Elasticsearch

  1. 1. @somkiat SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  2. 2. About me SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance สมเกียรติ ปุ๋ยสูงเนิน สยาม๡ํานาญกิจ somkiat.cc SPRINT3R
  3. 3. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Introduction Speed by example • Search • Aggregation Agenda • Operating System and Hardware
  4. 4. SELECT SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance FROM TABLE WHERE TEXT LIKE ‘%SHIT%’ Search system
  5. 5. SELECT SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance FROM many Table GROUP BY some column Report/analytic system
  6. 6. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Saaaaaaaadddddddddd
  7. 7. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance First version 0.4 in Feb 2010 Rewrite from Compass project Add scalability Created by Shay Banon Current version 1.5.1 History
  8. 8. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance an open source, distributed, scalable, highly availability, document-oriented, RESTful Elasticsearch full text search engine with near real-time search and analytics
  9. 9. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics Apache 2.0 Licence
  10. 10. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  11. 11. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  12. 12. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  13. 13. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch { "id" : 1, "name" : "somkiat", "family_name" : "puisungnoen", "website" : "somkiat.cc" } an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  14. 14. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  15. 15. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  16. 16. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Elasticsearch an open source, distributed, scalable, highly availability, document-oriented, RESTful full text search engine with near real-time search and analytics
  17. 17. realtime, search and analytics engine SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Lucene Distributed Scalability open-source documentation JSON API RESTFul document store High-Availability JAVA Plug-ins Elasticsearch
  18. 18. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance ใครใ๡้บ้าง ?
  19. 19. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance • Search repositories, users, issues, pull request • Search sourcecode 130 พันล้านบรรทัด • Track alerts, events และ logs ใครใ๡้บ้าง ?
  20. 20. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance • ใ๡้ Full text seach + geolocation • ใ๡้ feature More-like-this ในการหาคําถาม และ คําตอบ ใครใ๡้บ้าง ?
  21. 21. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance • Java • PHP • Ruby • Python • JavaScript • Go • Scala • .Net • Clojure • Erlang • NodeJS • R http://www.elasticsearch.org/guide/en/elasticsearch/client/community/current/clients.html Clients
  22. 22. สวัสดี Elasticsearch SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  23. 23. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance สวัสดี Elasticsearch $ wget https://download.elastic.co/elasticsearch/ elasticsearch/elasticsearch-1.5.1.zip $ unzip elasticsearch-1.5.1.zip $ cd elasticsearch-1.5.1 $ bin/elasticsearch
  24. 24. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance สวัสดี Elasticsearch
  25. 25. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance สวัสดี Elasticsearch
  26. 26. Scaling SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  27. 27. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Cluster :: กลุ่มของ node node cluster
  28. 28. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Cluster :: กลุ่มของ node node cluster node
  29. 29. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Cluster :: กลุ่มของ node node cluster node
  30. 30. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Cluster :: กลุ่มของ node node cluster node node
  31. 31. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Node :: มี 4 ชนิด master data search master search data
  32. 32. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Shard :: unit of scale 0 12 3
  33. 33. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Shard :: unit of scale 0 1 2 3
  34. 34. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Replication 0 1 2 33 0 1 2
  35. 35. Search SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  36. 36. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance RDBMS vs Elasticsearch D ATA B A S E TA B L E R O W C O L U M N I N D E X T Y P E D O C U M E N T F I E L D
  37. 37. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance CRUD
  38. 38. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance CRUD :: Create PUT <index name>/<type>/<id> { … your data .. }
  39. 39. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance CRUD :: Create PUT shop/book/1 { "name" : "Elasticsearch: The Definitive Guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "pages": 724, "published_date": "2015/01/31" }
  40. 40. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance CRUD :: Read GET shop/book/1 GET shop/book/_search GET shop/book/_search?q=elasticsearch
  41. 41. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance CRUD :: Delete DELETE shop/book/1
  42. 42. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Searching with DSL GET shop/book/_search { "query": { "filtered": { "query": { "match": { "name": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }
  43. 43. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Search response in JSON
  44. 44. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Search from all fields GET shop/book/_search { "query": { "filtered": { "query": { "match": { "_all": "elasticsearch"}}, "filter": { "range": { "published_date": { "gte": "now-1y" }} } } } }
  45. 45. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Reduce search response in JSON
  46. 46. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Reduce search response in JSON GET shop/book/_search { "_source": ["name", "pages"], … }
  47. 47. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Query vs Filter
  48. 48. Aggregations SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  49. 49. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance High level
  50. 50. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations Buckets and Metrics
  51. 51. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations Documents
  52. 52. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations Query/Filter Documents
  53. 53. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations Buckets Query/Filter Documents
  54. 54. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations Documents Query/Filter Buckets Metrics 2 1 4
  55. 55. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations ผู้แต่งแต่ละคนมีหนัง๤ือกี่เล่ม ?
  56. 56. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Aggregations GET shop/book/_search?search_type=count { "aggs": { "book_by_author": { "terms": { "field": "authors" } } } }
  57. 57. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Results :: ปัญหาคืออะไร ?
  58. 58. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Results :: ปัญหาคืออะไร ?
  59. 59. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance ปัญหาจาก Inverted index Term Document ID clinton 1 gormley 1 tong 1 zachary 1
  60. 60. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance ๤ิ่งที่ต้องการจาก Inverted index Term Document ID Clinton Gormley 1 Zachary Tong 1
  61. 61. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance กําหนด mapping PUT shop { "mappings": { "book": { "properties": { "authors": { "index": "not_analyzed", "type": "string" } } } } }
  62. 62. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Results
  63. 63. Operating System & Hardware SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  64. 64. SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance • CPU • Memory Elasticsearch use • I/O • Network Indexing, searching Indexing, searching, merging Aggregation, indices Relocation, snapshot & restore
  65. 65. Summary SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  66. 66. Summary SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Speed is key …
  67. 67. Summary SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance Search Tradeoff
  68. 68. Summary SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance
  69. 69. Thank you and Questions SPRINT3R Siam Chamnankit Co., Ltd., Odd-e (Thailand) Co., Ltd. and Alliance

×