Presentation slide for Altinity webcast held on February 24, 2021.

1. Building Beautiful, Fast
Dashboards with Grafana
and ClickHouse
Introduction, Tips, and Ideas
Altinity Engineering Team
1

2. Presenter Bios and Altinity Introduction
The #1 enterprise ClickHouse provider. Now offering Altinity.Cloud
Major committer and community sponsor for ClickHouse in US/EU
Robert Hodges - CEO
30+ years on DBMS plus
virtualization and security.
ClickHouse is DBMS #20
Eugene Klimov - Engineer
Analytic system integration
and management; ClickHouse
datasource plugin maintainer

3. Introducing
ClickHouse &
Grafana

4. Single binary
Understands SQL
Runs on bare metal to cloud
Stores data in columns
Parallel and vectorized execution
Scales to many petabytes
Is open source (Apache 2.0)
ClickHouse is an open source data warehouse
ClickHouse Server
a b c d
And it’s really fast!
ClickHouse Server
a b c d
ClickHouse Server
a b c d
ClickHouse Server
a b c d

5. Understands time series data
Simple installation
Supports many data sources
Lots of display plugins
Highly interactive
Great for monitoring dashboards
Is open source (Apache 2.0)
Grafana is a display tool for time series data
And it’s really beautiful!

6. 6
Connectivity to combine beauty and speed
6
ClickHouse Server
Displays the
dashboard
Stores and serves
dashboards; makes
calls to backends
Processes SQL
requests and returns
results
:8123
:8443

7. 7
Altinity.Cloud provides a test endpoint
7
URL https://github.demo.trial.altinity.cloud:8443
User demo
PW demo
airports Airport names and locations
github_events Full event history from Github (3.1B rows)
ontime Airline ontime data (196M rows)
tripdata NYC taxi commission ride data (1.3B rows)
Connection
parameters
Datasets

8. Setting up
Grafana

9. Setting up Grafana locally on Ubuntu
9
# Install Grafana server.
sudo echo 'deb https://packages.grafana.com/oss/deb stable main' >
/etc/apt/sources.list.d/grafana.list
curl https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo apt-get -y install grafana
# Install ClickHouse plugin
sudo grafana-cli plugins install vertamedia-clickhouse-datasource
sudo systemctl restart grafana-server.service

10. Setting up Grafana on Docker
10
# Run Grafana in docker with automatic plugin installation.
docker run -d
-p 3001:3000
--name=grafana
-e "GF_INSTALL_PLUGINS=vertamedia-clickhouse-datasource"
grafana/grafana

11. 11
Steps to connect to ClickHouse
11
Point browser to Grafana
(http://localhost:3000)
Login
(Change password)
Add ClickHouse
datasource

12. 12
Selecting the ClickHouse data source
12
. . .
ClickHouse
plugin

13. 13
Defining data source (1)
13
Server URL
Query via Grafana server to
avoid CORS problems
Supply credentials

14. 14
Defining data source (2)
14
Login credentials
Use POST (not GET)
Save & Test to check
connection

15. Understanding ClickHouse URLs
curl http://logos3:8123
echo 'SELECT version()' | curl
https://demo:demo@github.demo.trial.altinity.cloud:8443 -d @-
15
Unencrypted! Port
Host
Encrypted Port
Credentials Host
No credentials!
More info: https://clickhouse.tech/docs/en/interfaces/http/

16. Building your
first dashboard

17. 17
Creating a basic ClickHouse dashboard
17
Start a new dashboard
Add panel
Save
Edit Panel

18. 18
Starting a new dashboard
18
Press to
add panel
Press to add dashboard

19. 19
Creating a time series graph
19
Data preview
Query editor
Panel title
Graph selection

20. 20
Defining data source and time series
20
Database & table
Time series column
Data source
Emergency
debug escape
hatch

21. 21
Defining data source and time series
21
Database & table
Time series column
Data source
Emergency
debug escape
hatch

22. 22
Defining the query
22
Use ‘Time series’ for graphs
Query with
macros

23. 23
OMG, it’s broken! Where’s the debugger???!
23
Press ‘Query inspector’
to debug
Press to [re-]run query
Query and errors
Data

24. 24
Debug SQL on ClickHouse from browser
24
URL: https://github.demo.trial.altinity.cloud:8443/play?user=demo

25. Summary of basic tips
● For Time Series Graphs:
○ Define Column:DateTime/DateTime64/Timestamp
○ Don’t use Column:Date (it’s obsolete and will disappear)
○ Use $timeSeries and $timeFilter to get automatic handling of time ranges
○ Format as ‘Time series’
● For all visualizations:
○ Add query with macros in editor
○ Use ‘Query inspector’ to debug
○ Put misbehaving queries into your favorite ClickHouse client
○ ClickHouse has a convenient web client since version 20.11.
25

26. Advanced tricks
with Grafana
and ClickHouse

27. 27
Use function macros to generate queries
27
SELECT t, groupArray((Carrier, Flights)) AS groupArr
FROM (
SELECT (intDiv(toUInt32(toDateTime(FlightDate)),
21600) * 21600) * 1000 AS t,
Carrier,
count() Flights FROM ontime
WHERE
toDateTime(FlightDate) >= toDateTime(1598229925)
GROUP BY t, Carrier ORDER BY t, Carrier)
GROUP BY t ORDER BY t
Same result as previous query!

28. 28
List of function macros for ClickHouse
28
$rate(columns…) Computes change rate per interval on each column
$columns(key, value) Produces key-value arrays per interval
$rateColumns(key, value) Combines $rate and $columns
$perSecond(columns…) Compute change rate per interval on counters
$perSecondColumns(key,
value)
Combines $perSecond and $columns
Tip: Use function macro to generate query, then edit to suit

29. 29
Defining selection variables
29
Enter from
dashboard
settings
Query to select values
Press Enter to edit

30. Now you can use the selection in queries
SELECT
$timeSeries as t,
sum(ArrDel15)/count(*)*100.0 as "Carrier Average"
FROM $table
WHERE Carrier = '$carrier' AND $timeFilter
GROUP BY t ORDER BY t
30

31. 31
Use selection variables in titles, too
31

32. Using multiple queries for pie charts
Top 5 carriers by flights
SELECT
1 AS t, /* fake timestamp */
Carrier, count() AS Flights
FROM $table
GROUP BY Carrier
ORDER by Flights DESC
LIMIT 5
All the rest…
SELECT
1 AS t, /* fake timestamp */
sum(Flights) AS Others
FROM (
SELECT
Carrier, count() AS Flights
FROM $table
GROUP BY Carrier
ORDER by Flights DESC
LIMIT 5,1000
)
32

33. 33
A nice pie chart with 2 queries
33

34. More fun things you can do in Grafana
● Alerts
● Geographic displays
34

35. Wrap-Up

36. How to get started
● ClickHouse: https://clickhouse.tech/
● ClickHouse Plugin:
https://grafana.com/grafana/plugins/vertamedia-clickhouse-datasource
● Dashboards:
https://grafana.com/grafana/dashboards?dataSource=vertamedia-click
house-datasource
● Altinity.Cloud: https://altinity.com/cloud-database/
36

37. Summary
● ClickHouse provides fast query on very large data sets
● Grafana plugin enables rapid dashboard construction
● Altinity maintains the ClickHouse Grafana plugin
○ Active roadmap
○ We use it ourselves
● For quick prototypes use Altinity.Cloud public endpoint
○ https://github.demo.trial.altinity.cloud:8443
○ User: demo, password: demo
37

38. Thank you!
Contact us for a free
Altinity.Cloud Trial!
Altinity:
https://altinity.com
Altinity Blog:
https://altinity.com/blog
Contact:
info@altinity.com
38