Meniscus – Delivering data analytics to the connected world
The Meniscus analytics software provides high performance, flexible and scalable cloud-based tools. These tools will allow and help you to develop your bespoke applications quickly and easily. Turn your big (or small) data sets into the calculated metrics you need for your business.
2. 25 October 2016
• Overview of Meniscus and core technology
• Case Studies and value added
• Overview of the Insight Analytic project
Format for presentation
3. 25 October 2016
• Focus on Cleantech sector – Water and Electricity
• Cloud based service turning data into calculated
metrics
• Software is developed and proven
– Meniscus Calculation Engine (MCE)
– Meniscus Analytics Platform (MAP)
– Dashboard solutions built on MCE and MAP
Meniscus – background
4. 25 October 2016
Cloud based Data Analytics Platform
Data pushed to MAP
Information pulled
using RESTful API
Meniscus Analytics Platform
• Apply any calculation
• Deploy to millions of Things/Entities
• Real Time
• Generic
6. 25 October 2016
Meniscus Analytics Services – core
technology
Capability MCE MAP
Developed 2008 2014
Status Enterprise ready Cloud ready
Business Model SaaS SaaS and PaaS
Database SQL Server 2012 MongoDB
What are we monitoring? 2,000 water and
wastewater sites
60,000 km2 rainfall into
6,000 catchments in 5 mins
Raw Import
Datapoints per hour
Bulk ~ 5k points/sec
Real Time ~ 1k points/sec
Bulk ~ 700k points/sec
Real Time ~ 200k points/sec
Calculation speed Bulk ~13k calcs per sec
Real Time ~5k calcs per sec
Bulk ~ 900k calcs/sec
Real Time ~ 100k calcs/sec
Benefits Highly configurable,
Proven
Scalable, Flexible and
Performance
7. Tide range,
type & wind
direction
Real time
rainfall
1 Million data
points per day
Telemetry
15 min data from ~
650 locations
Forecast
Rainfall
100,000 data points
per day
Modelled
Impact Profiles
~15 million data
points
Predicted bathing water
quality for Beach Aware
Scheduled management
reports & email alerts
Matching rainfall with
discharge event
Bathingwater
monitoring
Other
Identifying high sewer
levels in low rainfall
Identifying CSO
discharges in low rainfall
Integrating into AW
Tactical Toolbox with API
Cloud
based
Easily
configured
Flexible
Detailed RESTful API
Broad range
of dashboards
& widgets
Performance
Import – 5,000 data points per
second
Calculation – 15,000 data points per
second
MCE Bathing Water Monitoring
MCE
8. Telemetry
15 min data from ~
350 locations
Real time
rainfall
18 Million data
points per day
Env Agency
15 min data from
176 rain gauges
Modellers
1000+ Water
Recycling
catchment polygons
36 hour rainfall
forecast
9 Million data
points per day
On demand Rainfall
Return calculator
Aggregated rainfall 1000+
WRC
On demand custom
aggregation of rainfall
ModellersShopWindow
Real time flow prediction
at pumping stations
Reducing DUoS costs
Predicting pumping
volumes
Pesticide run off, by field
by crop
MAP Rainfall Aggregation
Performance
Import – 600,000 data points per
second
Calculation – 900,000 data points
per second
Cloud
based
‘What if’ capabilityScalable
Flexible
High
performance
Generic
Big Data
9. MAP generic rainfall & data service
MAP
Raw radar data
18 million data
points per day for
60,000 km2
Grid
Calc
2D array
Polygon
calcs
Point
calcs
Satellite
Imagery
Soil Saturation (API),
Daily Rainfall etc
Telemetry application
Raw asset data
~1,000
PointNames =
96,000 data points
per day
Asset Performance
metrics
RESTful API
Corporate Historian
Sewerage
Network
Mgt
Supply
Abstraction
risk
Energy Mgt
DUOS
Flood
Prediction
Aggregating rainfall
into any polygon and
then calculating
e.g Rain Gauges –
Rain Event Period
Pesticide loading by
crop by field
Examples of
applications
Create own visualisations or
Meniscus can do
Meniscus
10. 25 October 2016
Importer
ItemFactory
Item
Properties and
Metadata
RawData
InvalidatorCalculator
MAP Overview
RawData
Collection
CalcData
Collection Item
Collection
InvalidatorCalculator
RawData
Processor
ItemFactories create
and extend properties of
an Item. Template to
create Items
Calculators use Item
properties, calculate
data and store it into
CalcData Collection
Importers process
real time data into
temporary cache
RawDataProcessors
process raw data from
temporary cache and
updates RawData
Collection
Invalidators
continually poll Items
to determine if
calculations are
required
Modules can be extended as required
11. 25 October 2016
MAP – existing analytics capability
Capability MAP
Developed 2014
Business Model SaaS and PaaS
Real Time Demonstration 60,000 km2 rainfall at 1 km2 into 6,000 catchments in 5 mins
Import capability Bulk: 2 billion points per hour. Real Time 0.75b per hour
Calculation capability Bulk: ~ 900k calcs/sec . Real Time ~ 100k calcs/sec
Scalability Multiple instances of core Modules on separate servers.
Inherent scalability of MongoDB
Flexibility Change data structure and Item metadata dynamically
All Modules and Data Types are fully extensible
Performance Lightening fast calculation of data
Separate thread for ‘On Demand’ calculations
12. 25 October 2016
• Structured solution to delivering Analytics
• Rapid development and deployment of solutions
• Flexibility to change as the solution ‘matures’
– Flexibility in the software
– Flexibility in how we deliver the service
• Ability to migrate from pilot to full scale deployment
What are the benefits to the IT sector?
Delivers lower cost and more flexible analytics
capability
13. 25 October 2016
Aggregating Rainfall into
Catchment and
calculating key metrics
Predicting sewer
flooding using
simplified models
On Demand
calculations
Calculating rainfall
volumes and ground
saturation (API) for the
whole region
Satellite and other sources of
images. Soil Type map covering
60,000km2 at 1km2 pixel size.
Used to update key attribute of
the underlying model
Alarm data from Telemetry.
Combined Storm Overflow and
sewer high level alarm values –
updated every 30 minutes
Pumping Station data.
Hours run data for
pumping stations updated
at 15 min periodicity –
not yet implemented
Radar Rainfall data.
60,000 km2 of radar
rainfall intensity data at
1km2 pixel size. Updated
every 5 minutes
Case Study – Sewerage Network Decision Support tool
Polygon region shapes. ~6,000
pumping station catchments
polygons
All processed within 5 minutes
14. 25 October 2016
Delivering much greater
accuracy than currently
possible. Ground
thruthing
Free personalised
mobile app for
residents
Delivering Vouchers
from local businesses
Prediction of Rainfall
Intensity in the next
hour based on actual rain
that is falling
Forecast rainfall. Hourly updates
of rainfall forecast to provide
predictions for later in the day
Real time rain gauge data. Rain gauge
data from outside Environment
Agency outside Peterborough to
confirm intensity of rain
Personalised data. Data
gathered from mobile app
on journey routes,
preferences, normal
patterns of use etc
Radar Rainfall data. 2,500
km2 of radar rainfall
intensity data at 1km2
pixel size. Updated every
5 minutes
Case Study – Hyperlocal rainfall prediction (InnovateUK – Smart City)
Local wind direction and rainfall.
Using real time data from 25
local rain gauges to allow ‘ground
truthing’ of radar data
Looking to increase the number of people using
sustainable transport in Cities
15. 25 October 2016
• Integrating rainfall into existing operations
• Aggregation of rainfall into polygons
• Prediction of rain intensity in the short term –
potential to include
• API so users can interact directly with the server
Looking to develop new Rainfall service?
16. 25 October 2016
• Small yet experienced team
– We understand analytics
– We are wholeheartedly a service business
– Vision to develop MAP
– Backed by a recent equity investment
• Meniscus Analytics Platform – MAP
– Specifically built to deliver flexible, scalable and
powerful analytics
– A generic system for all your analytics needs
Why select Meniscus?
17. 25 October 2016
• Why is MAP so flexible?
– Database schema does not have to be updated as model evolves
– Model structure and its Entities are defined by flexible Items
containing data and metadata
– Metadata can be extended through external control and does
not involve code or database changes
– Entities easily added using configurable Item Factories
– Internal representation of data is standardised simplifying import
of data and integration into calculations
– Data structures (data types) are highly interoperable. Custom
data types can be created for complex modelling
I.e. Create a Data Type for a Journey comprising a list of points with all routes, speeds,
distances in one array making it much simpler to use internally
Meniscus Analytics Platform (MAP)
18. 25 October 2016
Data Types key to MAP performance
t0 t1 t2 t3 t4 t5 t6 t0 tn-1 tn
Any number of
User Defined
Data Types
Data Grid. User
Defined Data
Type (UDT)
256*256 cells
Float. Standard
Data Type (SDT)
Date Time Value
pair
14/06/2016 25.8
Wind Speed &
Direction (UDT)
Date Time and two
values.
14/06/2016 25.8 325
Rain Return
period (UDT)
Start
End
Duration
Rainfall Depth
etc
19. 25 October 2016
Predicting Bathing Water
Quality for 45 Beaches
Generate alert if CSO spills
into a Shellfish sensitive
production zone from 35
discharges
Risk Monitoring at
Shellfish production
areas
Predicting flooding in
sewers
Identifying overflows
operating in dry weather
Uses real time, and
forecast rainfall data plus
pump hours run data. Uses
simplified hydraulic models
Using Current Tide conditions
to create 4 ½ day ahead
forecast of Beach Conditions
Generates alert if forecasts
that pumping station will
not be able to cope with
forecast flow
Generates alert if Beach
exceeds Bathing Water limits.
45 Beaches and 175 Coastal
Discharges
Comparing CSO alarm status
to actual rainfall data to
detect CSOs operating in dry
weather for 375 Inland
Discharges and 300 Sewers
Asset condition data from
Telemetry. Battery, wet well
level and flow data – updated
every30 minutes
Alarm data from Telemetry.
Combined Storm Overflow and
sewer high level alarm values from
Telemetry – updated every 30
minutes
Hours run data. Historic
Hours run data for
pumping stations –
updated every day at 15
min periodicity
Radar Rainfall data. 7,000
km2 of radar rainfall
intensity data at 1km2
pixel size and hourly 24
hour Forecast. Updated
every 5 minutes
Case Study – Water Company 1 hosted by Meniscus
20. 25 October 2016
Case Study – Water Company 2 installed internally
Process Energy Mgt +60
Water Treatment Plant
Calculation of 30 min values
of average energy use per Ml
flow per m head for each
pump set. Uses Weighted
Average calculation
Process Energy Mgt
+130 Water Boreholes
and Distribution sites
Process Energy Mgt
+400 Pumping stations
Process Energy Mgt +275
Wastewater Treatment
Plant
% deviation from dry
weather flow conditions.
Identifies blockages etc
Chemical monitoring and
comparison of dosing rates
to theoretical targets. Calc of
chemical dose rates
Energy monitoring against
flow based targets. Calc of
Moving average values.
Benchmarks against
average use
PAM – as per Wastewater.
Process energy benchmarks
across sites
Calc of virtual elec meters
for each process & sub
process from hours run. Calc
of flow, energy & process
benchmarks & KPI’s
Chemical monitoring and
comparison to theoretical
targets and quality limits.
Calc of chemical dose rates
Process data. Daily download of
~ 15Mb of readings at 15 min
intervals from Telemetry. Flow,
DO, pressures etc – 15 minute
periodicity
Half hour electricity data. Daily
download of HH reads (D+1) for ~
700+ sites – 30 minute periodicity
Chemical data. Daily
download of tank levels,
dose rates for 50 sites – 15
minute periodicity
Hours run and kWh from
Telemetry. Daily download
of 15 min readings from ~
4,500 pumps and blowers
– 15 minute periodicity
21. 25 October 2016
Aggregating Rainfall into
Catchment and
calculating key metrics
Mass balances to predict
when pumping stations will
flood by knowing flows into
each site and maximum
pumped flows out
Predicting sewer
flooding
On Demand
calculations
Calculating rainfall
volumes and ground
saturation (API) for the
whole region
Ability to create new
catchments and calculate
new metrics ‘on demand’
Applying a simplified
hydraulic model to each
Catchment. Calculates flows
into each pumping station
Importing 15,000 data points
per minute and calculating ~
30 million calculations per
minute
Satellite and other sources of
images. Soil Type map covering
60,000km2 at 1km2 pixel size.
Used to update key attribute of
the underlying model
Alarm data from Telemetry.
Combined Storm Overflow and
sewer high level alarm values –
updated every 30 minutes
Pumping Station data.
Hours run data for
pumping stations updated
at 15 min periodicity –
not yet implemented
Radar Rainfall data.
60,000 km2 of radar
rainfall intensity data at
1km2 pixel size. Updated
every 5 minutes
Case Study – Sewerage Network Decision Support tool
Polygon region shapes. ~6,000
pumping station catchments
polygons
All processed within 5 minutes
22. 25 October 2016
Delivering much greater
accuracy than currently
possible
Personalised web app to
capture information and
deliver prediction of rainfall
for the planned journey or
event
Free personalised
web app for residents
Delivering Vouchers
from local businesses
Prediction of Rainfall
Intensity in the next
hour
Interact with users by
delivering ‘vouchers’ from
businesses for rain related
discounts – 50p off cup of
coffee whilst its raining
Using local weather stations
to ‘ground truth’ radar data
and quantify ground wind
speed and direction
Tracking the movement of
individual pixels of rainfall
and predicting arrival in
Peterborough
Forecast rainfall. Hourly updates
of rainfall forecast to provide
predictions for later in the day
Real time rain gauge data. Rain gauge
data from outside Environment
Agency outside Peterborough to
confirm intensity of rain
Personalised data. Data
gathered from mobile app
on journey routes,
preferences, normal
patterns of use etc
Radar Rainfall data. 2,500
km2 of radar rainfall
intensity data at 1km2
pixel size. Updated every
5 minutes
Case Study – Hyperlocal rainfall prediction (InnovateUK – Smart City)
Local wind direction and rainfall.
Using real time data from 25
local rain gauges to allow ‘ground
truthing’ of radar data
Looking to increase the number of people using
sustainable transport in Cities
Notas do Editor
Set up in 1997
Primarily focused on CleanTech market
Water and Electricity Industries
Process monitoring
Smart Cities and IOT
Key USP is Meniscus Analytics Platform (MCE and MAP)
Real Time calculation component/tool
Enables the creation of any calculation from multiple data sources
Wessex Water
Process centric Energy M&T at 1,200 sites
Chemical M&T at 50 sites
Anglian Water – all real time
Identification of dry weather alarms at 600+ CSO and sewers using Nowcast radar rainfall data
Beach Alerts using modelled output
Sewer flood predictions
Shellfish warnings
Vertical = Solutions Sale
Horizontal is a Platform Sale
Looking to get across the some basic concepts and explain relationship between MCE and RTAP
MCE can deliver RT calc for limited number of Entities
RTAP developed specifically for more Entities and more frequent Raw Data