Apache Zeppelin

1. @doanduyhai
Apache Zeppelin, the missing GUI
for your BigData eco-system
DuyHai DOAN, Technical Advocate

2. @doanduyhai
Who Am I ?
Duy Hai DOAN
Cassandra technical advocate
• talks, meetups, confs
• open-source devs (Achilles, …)
• OSS Cassandra point of contact
☞ duy_hai.doan@datastax.com
☞ @doanduyhai
2

3. @doanduyhai
Datastax
• Founded in April 2010
• We contribute a lot to Apache Cassandra™
• 400+ customers (25 of the Fortune 100), 400+ employees
• Headquarter in San Francisco Bay area
• EU headquarter in London, offices in France and Germany
• Datastax Enterprise = OSS Cassandra + extra features
3

4. What is Apache Zeppelin ?
Presentation
Architecture

5. @doanduyhai
Zeppelin Presentation
5

6. @doanduyhai
Demo
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

7. @doanduyhai
Zeppelin Architecture
Zeppelin Server
Zeppelin Engine
7
R
E
S
TWebSocket
Spark Interpreter Group
Spark SparkSQL
Zeppelin
Interpreter
Factory
Tajo Interpreter
Flink Interpreter
Cassandra Interpreter
JVM
JVM
JVM
JVM
JVM

8. @doanduyhai
What does Zeppelin provide ?
Front-end & display system for free
Generic back-end with REST APIs & WebSocket
Pluggable interpreters system
Task scheduler (à la CRON)
8

9. Zeppelin UI Layout
Notebook
Paragraph
UI elements

10. @doanduyhai
Demo
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

11. Zeppelin Display System
Raw, Table, HTML
Available graphs
View modes
Dynamic form
Iframe export

12. @doanduyhai
Demo
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

13. Interpreter system
Core interpreters
Third-parties interpreters
Interpreters conf & usage

14. @doanduyhai
Interpreter processing lifecycle
① Receive input commands/data
• as raw text
• from form data
② Process the input commands/data by the external back-end
③ Format the response using Zeppelin display system
④ Send response back to the Zeppelin engine
14

15. @doanduyhai
Core interpreters
• Spark (Spark core, SparkSQL/DataFrame, PySpark)
• Spark core = default (or %spark)
• SparkSQL = %sql
• Shell (%sh)
• Markdown (%md)
• AngularJS (%angular)
15

16. @doanduyhai
Third-parties interpreters
• Hive
• Phoenix
• Tajo
• Flink
• Ignite
• Lens
• Cassandra
• Geode
• PostgreSQL
• Kylin
16

17. @doanduyhai
Interpreter conf & usage
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

18. Writing An Interpreter
How To
Simple interpreter example (AsciiDoc)
Complex interpreter example (Cassandra)

19. @doanduyhai
Steps to write your own interpreter
• Create a class that extends Interpreter base class
• Register it in a static block
• Optionnally define default config params
19
static {
Interpreter.register("MyInterpreterName", MyClassName.class.getName());
}
static {
Interpreter.register("MyInterpreterName", MyClassName.class.getName(),
new InterpreterPropertyBuilder()
.add("property1", "default value", "Description of property1").build());
}

20. @doanduyhai
To register your interpreter as default
• Edit the enum ZeppelinConfiguration.ConfVars
• Add your interpreter FQCN in the property ZEPPELIN_INTERPRETERS
20

21. @doanduyhai
To register your interpreter in config files
• Create conf/zeppelin-site.xml from conf/zeppelin-site.xml.template
• Add your interpreter FQCN in the property zeppelin.interpreters
21
<property>
<name>zeppelin.interpreters</name>
<value>org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,
org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,
org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.shell.ShellInterpreter,
org.apache.zeppelin.hive.HiveInterpreter,com.me.MyNewInterpreter
</value>
</property>

22. @doanduyhai
Simple AsciiDoc Interpreter
22
Zeppelin Server
AsciiDoc Interpreter
JVMZeppelin Engine
Raw
Text
Block
Raw
Text
Block
Converted
To
HTML
HTML
Output
① ②
③④
JVM

23. @doanduyhai
Simple interpreter (AsciiDoc)
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

24. @doanduyhai
Cassandra Interpreter Architecture
24
Cassandra
Interpreter
JVM
Display
Results as
HTML
① ②
⑤
Zeppelin
Server
JVM
Raw
Text
Block
Raw
Text
Block
Cassandra
Cassandra
Java
Driver
③
Async CQL
statements
④Render
HTML
⑥

25. @doanduyhai
Cassandra Interpreter Commands
25
Native CQL statements
SELECT * FROM …;
INSERT INTO …;
…
Schema commands
DESCRIBE TABLE …;
DESCRIBE KEYSPACE …;
…
Prepared statements
Commands
@prepare …;
@bind …;
@remove_prepared …;
Help command
HELP;
Options Commands
@consistency …;
@retryPolicy …;
@fetchSize …;

26. @doanduyhai
Complex interpreter (Cassandra)
https://github.com/doanduyhai/incubator-zeppelin/tree/ZeppelinPresentation

27. @doanduyhai
Cassandra Online Interpreter Docs
27
• http://zeppelin.incubator.apache.org/docs/interpreter/cassandra.html

28. @doanduyhai
Cassandra Interactive Help
28
• Type HELP; in the interpreter

29. Zeppelin future
Roadmap

30. @doanduyhai
Roadmap & future
• More graph options (Map viz ZEPPELIN-157)
• Helium project, packaging Zeppelin view, logic (code) & resource into
Applications
• Interpreters packaging re-design
• ship & compile core interpreters only
• third-parties interpreters can be pulled from repository
• which interpreter is core ? Who will maintain ? Community….
• Integrate security (Apache Shiro, pull request #53 by Hayssam Saleh)
30

31. @doanduyhai
Roadmap & future
• Out of incubation state to become 1st class Apache project
31

32. @doanduyhai
Q & R
! "

33. @doanduyhai
Thank You
@doanduyhai
duy_hai.doan@datastax.com
http://zeppelin.incubator.apache.org/