Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

1. Steven Soroka and Anais Dotis-Georgiou
Machine Learning
with Telegraf Execd
Processor Plugin

2. Introduction and Overview
Telegraf

3. © 2020 InfluxData. All rights reserved. 3
Introduction to Telegraf
● Active community
● Over 200 plugins
○ 170+ Input
○ 40+ Output
○ 15+ Processors
○ 5+ Aggregators
○ 5+ Serializers
● Single Binary
● 9.2k Stars
● Extendable in any language

4. © 2020 InfluxData. All rights reserved. 4
Simple example of a Telegraf Configuration
[agent]
interval = "1s"
flush_interval = "10s"
omit_hostname = true
[[inputs.cpu]]
[[outputs.influxdb_v2]]
urls = ["http://127.0.0.1:9999"]
token = "$INFLUX_TOKEN"
organization = "$INFLUX_ORG"
bucket = "brew"

5. Overview
Execd Processor Plugin

6. © 2020 InfluxData. All rights reserved. 6
Introduction to the Execd Processor Plugin
● Runs an external program as a separate process, passes
metrics back and forth.
● Your program must read Influx Line Protocol on STDIN and
write metrics in Influx Line Protocol to STDOUT.
● Your program runs continuously in STDIN read loop, so you
only pay startup cost once.
● Can be any language, not just Go

7. © 2020 InfluxData. All rights reserved. 7
Execd Processor Plugin Diagram

8. © 2020 InfluxData. All rights reserved. 8
Influx Line Protocol

9. © 2020 InfluxData. All rights reserved. 9
Simple example of Execd Processor Plugin
# A simple processor. It reads stdin and writes it to stdout.
import sys
def main():
for line in sys.stdin:
print(line.rstrip())
sys.stdout.flush()
if __name__ == '__main__':
main()
[[processors.execd]]
command = ["python", "./processors/forecasting.py"]

10. © 2020 InfluxData. All rights reserved. 10
Context: Monitoring temperatures of two beers (haze_v5 and bv_1). The ideal
temperature for the batch of beer is between 18-21.
Objectives:
● Forecast temperatures for the next 10 minutes
● Detect anomalous spikes in temperature that may indicate attention is needed
or equipment failure.
Monitoring a home brew

11. © 2020 InfluxData. All rights reserved. 12
Brewing Setup

12. © 2020 InfluxData. All rights reserved. 13
Seven Segment OCR
ssocr -b black -f white -t 85 make_mono remove_isolated crop 30 95 460
220 shear 22 photo.png

13. © 2020 InfluxData. All rights reserved. 14
Execd Processor Plugin for Forecasting and
Anomaly Detection
Using the Execd Processor Plugin to achieve our goal and keep the beer at a steady
temperature.

14. © 2020 InfluxData. All rights reserved. 15
haze_v5

15. © 2020 InfluxData. All rights reserved. 16
bv_1

16. © 2020 InfluxData. All rights reserved. 17
● Double exponential smoothing employs a level component
and a trend component at each period. Double exponential
smoothing uses two weights, (also called smoothing
parameters), to update the components at each period.
● In other words, an exponentially weighted average of the
trend and values of the data together.
What is Exponential Smoothing?

17. © 2020 InfluxData. All rights reserved. 18
● Statsmodels is a Python package that allows users to explore
data, estimate statistical models, and perform statistical
tests.
● Dataset considerations–small and sparse
● Easy to use
● Statsmodels enables more model configuration options than
Flux holtWitners() function
● Using the damped version of Holt helps eliminate false
positives.
● Algorithm works with small, irregular, and sparse datasets
Algorithm Selection: Holt’s Exponential
Smoothing

18. Code Explained

19. © 2020 InfluxData. All rights reserved. 20
FAQ about the Execd Processor Plugin
● What are the advantages to running the Execd plugin as
opposed to a cron job?
● Can I use the Execd Processor Plugin in combination with a
Neural Net?
● When shouldn’t I use the Execd Plugin?

20. Resources
• github.com/lukebond/homebrew-webcam-temperature
• github.com/influxdata/tg-brew-anomaly
• github.com/auerswal/ssocr
• Influx Line Protocol

21. © 2020 InfluxData. All rights reserved. 22
Thanks!