Slides of the Talk Jodok Batlogg held at the Elasticsearch Meetup in Berlin

1. SQL on Elasticsearch?

2. How all started

3. You know, for search
querying 24 000 000 000 Records in 900ms
@jodok

4. 6 ES Master Nodes
c1.xlarge
6 Node Hadoop Cluster
+ Spot Instances
40 ES nodes per zone
m1.large
8 EBS Volumes
3 AP server / MC
c1.xlarge

6. Elastic Search as
Primary Storage?
NoSQL Roadshow 2013
Jodok Batlogg

7. • Security Model?
• Transactions?
• Data security?
• Toolsets?
• Larger Computations?
• Availability?

8. D I S T R I B U T E D D A T A S T O R E W I T H S Q L .
S I M P L E . R E L I A B L E . S C A L A B L E .

9. Open Source (Apache 2.0)
shared nothing
is high available and cheap to operate.
not NOSQL but SQL
NOFS but distributed BLOBs

11. Client
Query
Data
Aggregation
Network/
Cluster
Storage
CRATE DATA – Module overview
CRATE
Python DB-API Dashboard SQLAlchemy
Java
CRATE
Shell
ES
native
Transpo
rt
FB Presto
SQL Parser
Query
planner
Ruby
Bulk import/
export
BLOB streaming
Distributed SQL
ES Transport
protocol
ES Discovery
and state
Lucene BLOB
ES storage
CRATE
3rd
party
Open
Source
Module
s
BLOB
streaming
support
Netty
ES Scatter/
Gather
Distributed reduce Data transformation
and reindex support
ES
Sharding

12. S T A R T A
CLUSTER
I N 1 M I N
HTTPS://CRATE.IO

13. How is Crate Data different
than Elasticsearch?
BLOB Storage
Distributed Accurate Aggregations
Partitioned Tables
Import/Export
Update by Query
Insert by Query
Integrated Admin-UI

15. Thank you
Jodok Batlogg, @jodok, jodok@crate.io
github.com/crate, #crate / freenode, @cratedata

17. Demo Video

18. http://bigdatanerd.files.wordpress.com/2011/12/cap-theorem.jpg
BASE & CAP
• Basically Available -
you always get an
response
• Soft State - it’s not
consistent all the time.
• Eventually Consistent -
it becomes consistent at
a later point in time