1. Real-time search in Drupal.
Meet Elasticsearch
By Alexei Gorobets
asgorobets

2. Elasticsearch
Flexible and powerful open source, distributed
real-time search and analytics engine for the
cloud

3. Elasticsearch
Flexible and powerful open source, distributed
real-time search and analytics engine for the
cloud
RESTful API on top of Lucene library

4. Why use
Elasticsearch?

5. ● RESTful API
● Open Source
● JSON over HTTP
● based on Lucene
● distributed
● highly available
● schema free
● massively scalable

6. STOP! Why not SOLR?

7. ● Solr vs Elasticsearch: Features Smackdown
● Detailed overview
● Slideshare presentation by Sematext
● Realtime Search: Solr vs Elasticsearch
Apache Solr vs Elasticsearch

9. Setup in 2 steps:
1. Extract the archive
2. > bin/elasticsearch

10. How to use it?

11. > curl -XGET localhost:9200/?pretty

12. > curl -XGET localhost:9200/?pretty
{
"ok" : true,
"status" : 200,
"name" : "Infinity",
"version" : {
"number" : "0.90.1",
"snapshot_build" : false,
"lucene_version" : "4.3"
},
"tagline" : "You Know, for Search"
}

13. > curl -XGET localhost:9200/?pretty
action

14. > curl -XGET localhost:9200/?pretty
node + port

15. > curl -XGET localhost:9200/?pretty
path

16. > curl -XGET localhost:9200/?pretty
query string

17. Let's index some data

18. > PUT /index/type/id
Where?
It's very similar to
database in SQL

19. > PUT /index/type/id
What?
Table
Content type,
Entity type,
any kind of type you decide

20. > PUT /index/type/id
Which?
Node ID,
Entity ID,
any kind of serial ID

21. > PUT /mysite/node/1 -d
{
"nid": "1",
"status": "1",
"title": "Hello elasticsearch",
"body": "First elasticsearch document"
}

22. > PUT /mysite/node/1 -d
{
"nid": "1",
"status": "1",
"title": "Hello elasticsearch",
"body": "First elasticsearch document"
}
{
"ok":true,
"_index":"mysite",
"_type":"node",
"_id":"1",
"_version":1
}

23. Let's GET some data

24. > GET /mysite/node/1
{
"_index" : "mysite",
"_type" : "node",
"_id" : "1",
"_version" : 1,
"exists" : true,
"_source" : {
"nid":"1",
"status":"1",
"title":"Hello elasticsearch",
"body":"First elasticsearch document"
}

25. > GET /mysite/node/1?fields=title,body
Get specific fields

26. > GET /mysite/node/1?fields=title,body
Get specific fields
> GET /mysite/node/1/_source
Get source only

27. Let's UPDATE some data

28. > PUT /mysite/node/1 -d
{
"status":"0"
}

29. > PUT /mysite/node/1 -d
{
"ok":true,
"_index":"mysite",
"_type":"node",
"_id":"1",
"_version":2
}
{
"status":"0"
}

30. UPDATE = DELETE + PUT

31. Let's DELETE some data

32. > DELETE /mysite/node/1

33. > DELETE /mysite/node/1
{
"ok":true,
"found":true,
"_index":"mysite",
"_type":"node",
"_id":"1",
"_version":3
}

34. Let's SEARCH for something

35. > GET /_search

36. > GET /_search
{
"took" : 32,
"timed_out" : false,
"_shards" : {
"total" : 20,
"successful" : 20,
"failed" : 0
},
"hits" : { results... }
}

37. Let's SEARCH in multiple
indices and types

38. > GET /index/_search
> GET /index/type/_search
> GET /index1,index2/_search
> GET /myapp_*/type, entity_*/_search

39. Let's PAGINATE results

40. > GET /_search?size=10&from=20
size = results per page
from = starting from

41. Let's search oldschool

42. > GET /_search?q=title:elasticsearch
> GET /_search?q=nid:60

43. +title:awesome
+status:1
+created:>1369917354

44. ?q=title:awesome%20%2Bcreated:
>1369917354%20%2Bstatus:1
+title:awesome
+status:1
+created:>1369917354
The ugly encoding =)

45. Query DSL style

46. > GET /_search -d
{
"query": {
"match": "awesome"
}
}

47. > GET /_search -d
{
"query": {
"field" : {
"title" : {
"query" : "+awesome -poor",
"boost" : 2.0,
}
}
}
}

48. Queries & Filters

49. Queries & Filters
full text search
relevance score
heavy
not cacheable
exact match
show or hide
lightning fast
cacheable

50. Combine Filters & Queries

51. > GET /_search -d
{
"query": {
"filtered": {
"query": {
"match": { "title": "awesome" }
},
"filter": {
"term": { "type": "article" }
}
}
}
}

52. and Sorting

53. > GET /_search -d
{
"query": {
"filtered": {
"query": {
"match": { "title": "awesome" }
},
"filter": {
"term": { "type": "article" }
}
}
}
"sort": {"date":"desc"}
}

54. and Facets

55. Facets on Amazon

56. > GET /_search -d
{
"facets": {
"home_team": {
"terms": {
"field": "field_home_team"
}
}
}
}

57. > GET /_search -d
{
"facets": {
"home_team": {
"terms": {
"field": "field_home_team"
}
}
}
}
Give your facet a name

58. > GET /_search -d
{
"facets": {
"home_team": {
"terms": {
"field": "field_home_team"
}
}
}
}
Your facet filter can be:
● Terms
● Range
● Histogram
● Date Histogram
● Filter
● Query
● Statistical
● Terms Stats
● Geo Distance

59. "facets" : {
"home_team" : {
"_type" : "terms",
"missing" : 203,
"total" : 100,
"other" : 42,
"terms" : [ {
"term" : "hou",
"count" : 8
}, {
"term" : "sln",
"count" : 6
}, ...

60. STOP! I want this in Drupal?

61. Available modules:
elasticsearch
Search API elasticsearch

62. Development directions:
1. Search API implementation
2. Field Storage API
Available modules:
elasticsearch
Search API elasticsearch

63. Field Storage API implementation
Elasticsearch field storage sandbox by Damien Tournoud
Started in July 2011

64. Field Storage API implementation
Elasticsearch field storage sandbox by Damien Tournoud
Started in July 2011
Elasticsearch EntityFieldQuery sandbox
Commited today! =)

65. Let's DEMO

66. Elasticsearch sandbox here:
https://drupal.org/sandbox/asgorobets/2073151

67. Let the Search be with you