Going Underground with InfluxDB

Going Underground
with InfluxDB
Tobias Braun – Software Architect
Herrenknecht AG, Germany

© 2021 InfluxData Inc. All Rights Reserved.
Overview
- Herrenknecht AG and the presenter
- Building an IIoT platform for Herrenknecht AG
- Focus on handling of sensor data
- InfluxDB as central time series storage
- Migration to InfluxDB Enterprise
- Current status of the product

Herrenknecht AG
- Founded 1977 in Germany
- World market leader for tunnel
boring machines
- For tunnels from 0.1m to 19m
- Traffic, utility and mining
- Vertical and horizontal
- All geologies, even under water
- Major share of global market
- ~ 5,300 employees
- Revenue 2020: 1,007 mio. €

Tobias Braun
Software Architect, Herrenknecht AG
Build an IIoT platform with insight into live and historic data for all tunnel
boring machines delivered by Herrenknecht AG.
Small agile team of 4 developers. Frontend development is supported by an
external partner. We have to compromise and gradually improve what we
build.
E-Mail: braun.tob@herrenknecht.com
Software Architect at Herrenknecht for 2 ½ years
- More than 20 years of experience in highly available, distributed systems
- Worked for:
1&1 IONOS – Web Hosting Core
Burda GmbH – E-Commerce
Trumpf Werkzeugmaschinen GmbH – IIoT platform for a machine tool manufacturer

Building an IIoT platform for
Herrenknecht AG

Tunnel diameter and amount of data correlate
1985
HERA
5.95 m
1996
Sydney
10.70 m
1997
Hamburg
14.20 m
2006
Madrid
15.20 m
2006
Shanghai
15.43 m
2010
Sparvo
15.62 m
2016
Santa Lucia
15.87 m
2013
Hongkong
17.6 m
Concept
St. Petersburg
19.25 m

Current, correct data is vital for TBM operator

Building an IIoT platform for Herrenknecht AG
• Has to support >2000 TBM, several hundred are working simultaneously
• Up to 5000 sensors on a single machine
• Some sensors with sample rates <= 100ms
• Limited bandwidth to the job-site
• Machines can be offline for days, weeks, even months
• Historic data from 40 years and in different formats (SQL, DBase, even CSV)
has to be made available

Building an IIoT platform for Herrenknecht AG
• Can be developed and maintained by a small team
• Start with the basics: Make sure data is recorded consistently
• Development first for TBM, cloud as a second step
• Data storage for the next 10-15 years
• Robust, flexible architecture

InfluxDB as central time
series storage

Why we chose InfluxDB
• Open Source with commercial „Enterprise Edition“
• Windows-Service available (was relevant „back-then“)
• Very space efficient storage
• Query language similar to SQL with powerful aggregations
• Stored in simple, flat files

Where is sensor data stored?
• On-site
• Control room of the TBM is equipped with
ruggedized industrial PCs
• Windows operating system with limited
computing power and SSD
• Data cleansing and storage in InfluxDB OSS
• Cloud
• InfluxDB Enterprise running on Microsoft
Azure

How is data stored?
• Basically one big measurement
• Every sensor is a „float“ field
• Only few tags are used
• Low tag cardinality
• Only write values when there
was a significant change
-> 1-5 GB / month & machine

Simple architecture for InfluxDB in the cloud
• When entering the cloud, we used a quick and pragmatic
approach based on Kubernetes
• One InfluxDB OSS container per TBM
• No clustering of InfluxDB itself
• Azure File Storage (equivalent to Windows share)

Advantages of this approach
• Architecture almost identical to the environment on the TBM,
therefore minimal development effort
• Isolated impact zones
• Cheap pay-per-use storage
• File-based backup is included with Azure File Storage
• InfluxDB OSS – no licensing cost

How is data transferred to the cloud?
• Historic data is converted to InfluxDB format in batches
• Live Data:
• Data cleansing is already done on the machine
• So we simply mirror all the data from the machine to the cloud
• We built a custom synchronization, based on a REST API
• Result: Data is identical on the TBM and in the cloud

Migration to InfluxDB
Enterprise

First issues with the pragmatic approach
• Scaled well to ~100 InfluxDB containers,
• But with >> 100 containers,
• the overall system became unreliable
• … and response times for even simple queries exploded
• … and the infrastructure became prohibitively expensive
• Azure File Storage „forgot“ to unlock files in rare cases
• with hundreds of containers, even rare cases tend to happen
regularly

Decision to migrate to InfluxDB Enterprise
• 750 GB of data in 200 databases
• At most a couple of minutes of downtime for each TBM
• Small team with limited capacity, migration had to be automated
• Executed successfully in Q1/2020, took less than two weeks
• No read-downtime, write-downtime of less than 60 minutes per TBM

Results after the migration
• Reduced TCO of Influx databases by 1/3
• Extensive monitoring and stable operation
• Very reliable and reproducable response times for all
queries

Status of the product

Successful rollout to first customers

Status of the product
• Very positive feedback from customers and internal users
• Team can concentrate on feature development, little
DevOps efforts
• InfluxDB OSS on every TBM
• InfluxDB Enterprise in the cloud just works, unnoticable in
the background

Questions?
braun.tob@herrenknecht.com

Thank You

Going Underground with InfluxDB

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Going Underground with InfluxDB

Similar to Going Underground with InfluxDB (20)

More from InfluxData

More from InfluxData (20)

Recently uploaded

Recently uploaded (20)

Going Underground with InfluxDB