Presentation given at the elasic meetup in Zurich on October, 12th.

1. By Floris Dekker – September 2nd, 2013
Click to edit Master title style
By Tobias Rein & Alex Eftimie – October 12th, 2016
Elastic @ GetYourGuide

3. 5 students wanted to build a
peer-to-peer platform for tour guides.

5. But we also learned one thing:
There is a greater problem
to be solved.

6. So the team started
from scratch . . .

7. . . . and we relaunched

8. is me e too o

9. e e
e p e
e
1

11. We built a unique culture.

13. @
GetYourGuide

14. In 2012 …
… we turned MySQL into elasticsearch

16. 0.19.8
0.90.0
1.0.0
2.3.1
Upgrading was usually
painless, thanks to
Elastica (A PHP client for elasticsearch.)

17. MySQL is still our primary storage.
We are using asynchronous messages
to update the elasticsearch index
Our Setup

18. Our architecture

19. 1. Location lookup
[elasticsearch]
2. Tour search
[elasticsearch]
3. Availability filter
[MySQL]
Multi-step

20. Our Cluster
56 indexes
for customer search
26 indexes
for intranet
4 indexes in 14 languages up to 2.5 million docs per index
3 nodes 8.1 GB
2 replicas, no sharding 10 million documents

21. Our migration strategy
• Unique index names
using timestamp
• An alias <name>_live points to the
current index
• For migration we use temporary alias
names <name>_temp

22. @
GetYourGuide
ELK

23. What do we do with logs @ GYG ?
• Debugging • Business
Intelligence

24. Motivation for a new infrastructure
• SSH to host no longer an option
• Local storage in the cloud is limited
• No one uses outdated tools
• Fault tolerant trustworthy logs are a must!
• Redundancy, Auto failover, Monitoring

25. Logging infrastructure - FLKLEK
Two m4.2xlarge AWS instances
(One tie breaker instance in Digitalocean)

26. The Numbers
120 GB 20 Mio
elasticsearch store size log lines /day
42 5.5K
nodes avg. logs /second

27. Lessons learned
• Large documents affect ES performance
• Dynamic mappings are a pain
• Beats don’t play well with file rotation
• Kibana is a powerful tool

28. 29

29. Questions?