Saving Energy with Apache Spark and Toon with Stephen Galsworthy

•

0 gostou•843 visualizações

Toon, a leading European smart thermostat and energy display, enables users to control and monitor gas and electricity consumption in their homes. Using the energy data we collect from over 400,000 homes we have developed a new Energy Waste Checker app to give actionable advice to end users to ensure that they do not needlessly waste energy. We identify inefficient electrical appliances and suboptimal uses of central heating and hot water by applying disaggregation algorithms to the raw sensor data. In this talk Stephen will describe our how our novel disaggregation algorithms are implemented in Spark and will show how Toon uses cloud-based big data processing to offer data driven services to hundreds of thousands of users.

Dados e análise

Stephen Galsworthy, Toon
Saving energy with
Apache Spark and Toon
#EUds10 #ToonTalks

Data management
Solar solution
Network
Mobility
Control by demand
Heat pump
Services
Storage
The beating heart
of your smart life
Smart Platforms & Energy Services

#EUds10 #ToonTalks
Boiler adapter
Z-Wave
Meter adaptor
Gas sensor
Electricity sensor
Philips Hue
Z-Wave
Toon data sources
Central heating system
Solar panels

#EUds10 #ToonTalks
Scale of Toon data
100 MB data
per user
collected per
month
350,000 thermostats
installed
300 types of
sensor and
user
interaction
data

#EUds10 #ToonTalks
Data Privacy
Data from all Toons is collected and stored
anonymously. Data is tagged with an anonymous ID,
generated locally on Toon.
The end user maintains control over this key and we
only use data in a non-anonymous way when a
customer explicitly opts-in.

#EUds10 #ToonTalks
Toon Innovators
Innovators agree to share their data for research
and product development.
In return they they are first in line to try out our
new services!

#EUds10 #ToonTalks
Data Collection setup
data
collector
data
storage
Processed
data
Data science
platform

#EUds10 #ToonTalks
Apache Airflow
Data Science platform
For managing Spark
clusters and notebooks
For scheduling complex
workflows
For production grade
algorithms and R&D
For machine learning on
distributed data

#EUds10 #ToonTalks
Concept testing
(3 employees)
Initial tests
(10-50 users)
Development
(600 users)
Commercial
launch
(300,000+users)
Manual
Our journey

#EUds10 #ToonTalks
Energy Waste Checker
We don’t always notice how much energy we’re
wasting.
Toon can now expose the energy guzzlers in
your home.

#EUds10 #ToonTalks
Toon detects inefficient appliances and behaviours

#EUds10 #ToonTalks
Electricity data Appliance signals
An underlying concept: Load disaggregation

#EUds10 #ToonTalks
Run on 10s electricity meter data
1
Disaggregation algorithms
Dryer Dishwasher
Washing machine x2
Power
(Wh)

#EUds10 #ToonTalks
Determine appliance ‘fingerprint’

#EUds10 #ToonTalks
Efficiency classification

#EUds10 #ToonTalks
Personalised advice for the end user
“It looks like your
washing machine uses
more energy than an
efficient one”

#EUds10 #ToonTalks
Another example: Heating and Insulation advice
1. Analyse how much gas is used for
heating the home
2. Analyse the cooling down rate of
the home overnight from temperature
data
3. Benchmark with comparable
homes
4. Personalised insulation
advice for the customer
Insulation
It looks like your home losesenergy more
quickly than efficient homes

#EUds10 #ToonTalks
Scale of Toon data
35,000,000 washing machine cycles
detected per year
100,000 washing
machine cycles
detected per day

#EUds10 #ToonTalks
What else is on its way?
Boiler Monitoring
Toon will keep an eye on your boiler 24 hours per day.
It’ll tell you if it’s still working correctly and when the next
maintenance is due.
Monthly Energy Insights
A personalised overview of energy usage for all Toon users.

#EUds10 #ToonTalks
Stephen Galsworthy
Head of Data Science
+31 (0) 20 462 1680  
stephen.galsworthy@quby.com

Mais conteúdo relacionado

Semelhante a Saving Energy with Apache Spark and Toon with Stephen Galsworthy

TOON Stephen Galsworthy

BigDataExpo

Today, most newly built or renovated private houses are still equipped with a rather simple electricity installation, providing not much more than a series of light points, switches and power outlets throughout the building. While this so-called base case electricity installation is very robust and safe, it is being challenged by the ever increasing expectations of 21st century homeowners and occupants. Examples of increased expectations include flexible space usage, networking capabilities, automation, remote control of HVAC and other electricity powered equipment, and adaptability of the system to meet changing needs. In this whitepaper we outline the challenges presented to the base case electricity installation and provide a guideline for installers to help them meet the increased expectations. Aspects discussed include a gradual shift towards automation-enabled systems and an activity-centred approach of installation design. We encourage installers to grasp this opportunity and become advocates of a more flexible home electricity installation. This is also in their own interest, as it is likely to make their work more interesting, satisfying and rewarding. The whitepaper is based on interviews with a number of home electricity system installers and representatives of electric systems manufacturers ABB, Abitana, Legrand Group, Niko and Siemens. In addition, we organized five workshops with these manufacturers and with organizations such as Tecnolec, Enter vzw and CeDuBo[i]. In this whitepaper, the participants at the interviews and workshops are referred to as ‘our expert panel’.

Advocating a more flexible home electricity installation

Leonardo ENERGY

EnOcean Alliance: Energy Harvesting Wireless Sensors to Cloud - Graham Martin

WithTheBest

Internet of cooling and heating things

CAREL Industries S.p.A

Data streams relevant for energy use in buildings are increasing. Many households have “smart” meters in place or are installing smart thermostats or smart plugs; and the first IoT applications are entering households. We see similar developments in commercial buildings. This wealth of data provides new opportunities to better understand and control energy use in buildings. Significant energy saving potentials are claimed. Nevertheless, there are also serious challenges, like privacy and cybersecurity issues, and the variety of platforms. In this webinar an overview will be given of the main developments in this new field. The challenge is how big data can empower users of homes and office buildings to create a comfortable and efficient living and working place.

Big data for greater energy efficiency

Leonardo ENERGY

Residential Non-Intrusive Load Monitoring Overview

E Source Companies, LLC

LAS - Tech, Society and Culture Our team had to identify and explore an emerging technology. In order to be considered for use in the project it had dislpay new applications, anticipated impacts, or potential controversies as of 2015. Examples included •nanotechnology in manufacturing; •genetically modified organisms; •remote or robotic surgery; or •wireless electricity. Our team had to research and explain the technical, social, cultural, moral, and ethical issues presented by the technology.

Smart Home Security Project

Stephan Pacheco

Smart homes

Brad Fitzpatrick

A Technical Introduction to The Internet of Things

APNIC

Applications of IoT

APNIC

O2 Smarthome

O2 Business UK

This presentation was made at the Future of Work Seminar held as part of the 2015 Vivid Ideas Festival. http://www.vividsydney.com/futureofwork The topic is the City as a Campus and explores the role that technology is playing in blurring the boundaries between the home, workplace and cities in which we live and work. Technology has become ubiquitous and the expectations for it's use are everywhere. There is however, a distinct difference between people and companies and not all technology is applicable in every environment. What is important, is to take the time to understand your stakeholders requirements and to identify what technologies are meaningful and will add value to your workplace environment.

Future of Work - The City as a Campus

Bruce Duyshart

TSensorSummit-Emerging IOT Usages & Apps for Trillion+ Sensors-Bhide-Oct25-20...

sandhibhide

T sensor summit-emerging iot usages & apps for trillion+ sensors-sandhi b...

sandhibhide

Smart homes

Brad Fitzpatrick

Technical Challenges - How To Design IoT Systems

Emil Kirilov

Residential and small commercial electricity consumers will provide most of the flexibility needed in the power system to achieve the energy transition at lower cost. This flexibility will allow the electricity grid to take on more renewables and need less grid reinforcements. Consumers are happy to participate (>75% opt-in rate) via an aggregator and be rewarded with energy savings reducing their electricity. Because it reduces the overall costs of running the energy system, it means that all consumers benefit from lower energy prices. The business case is now proven as more and more countries worldwide ensure demand side response can participate in wholesale markets as an alternative to generation and the EU has provided support through its recent Clean Energy Package. Pierre Bivas from Voltalis will share his experience of operating flexibility from more than 100,000 households in France, and now installing another 150,000, backed by the European Investment Bank, while also developing in other countries with local partners.

Households are happy to join together and provide power flexibility for clima...

Leonardo ENERGY

Smarthomes boboceanu

Boboceanu Cristian

Ecoisme pitch-deck-8.12

Eduard Rubin

A Survey on Smart Metering Systems using Blockchain for E-mobility

Juan Carlos Olivares Rojas

Semelhante a Saving Energy with Apache Spark and Toon with Stephen Galsworthy (20)

TOON Stephen Galsworthy

Advocating a more flexible home electricity installation

EnOcean Alliance: Energy Harvesting Wireless Sensors to Cloud - Graham Martin

Internet of cooling and heating things

Big data for greater energy efficiency

Residential Non-Intrusive Load Monitoring Overview

Smart Home Security Project

Smart homes

A Technical Introduction to The Internet of Things

Applications of IoT

O2 Smarthome

Future of Work - The City as a Campus

TSensorSummit-Emerging IOT Usages & Apps for Trillion+ Sensors-Bhide-Oct25-20...

T sensor summit-emerging iot usages & apps for trillion+ sensors-sandhi b...

Smart homes

Technical Challenges - How To Design IoT Systems

Households are happy to join together and provide power flexibility for clima...

Smarthomes boboceanu

Ecoisme pitch-deck-8.12

A Survey on Smart Metering Systems using Blockchain for E-mobility

Mais de Spark Summit

In this session we will present a Configurable FPGA-Based Spark SQL Acceleration Architecture. It is target to leverage FPGA highly parallel computing capability to accelerate Spark SQL Query and for FPGA’s higher power efficiency than CPU we can lower the power consumption at the same time. The Architecture consists of SQL query decomposition algorithms, fine-grained FPGA based Engine Units which perform basic computation of sub string, arithmetic and logic operations. Using SQL query decomposition algorithm, we are able to decompose a complex SQL query into basic operations and according to their patterns each is fed into an Engine Unit. SQL Engine Units are highly configurable and can be chained together to perform complex Spark SQL queries, finally one SQL query is transformed into a Hardware Pipeline. We will present the performance benchmark results comparing the queries with FGPA-Based Spark SQL Acceleration Architecture on XEON E5 and FPGA to the ones with Spark SQL Query on XEON E5 with 10X ~ 100X improvement and we will demonstrate one SQL query workload from a real customer.

FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang

Spark Summit

In this talk, we’ll present techniques for visualizing large scale machine learning systems in Spark. These are techniques that are employed by Netflix to understand and refine the machine learning models behind Netflix’s famous recommender systems that are used to personalize the Netflix experience for their 99 millions members around the world. Essential to these techniques is Vegas, a new OSS Scala library that aims to be the “missing MatPlotLib” for Spark/Scala. We’ll talk about the design of Vegas and its usage in Scala notebooks to visualize Machine Learning Models.

VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...

Spark Summit

This presentation introduces how we design and implement a real-time processing platform using latest Spark Structured Streaming framework to intelligently transform the production lines in the manufacturing industry. In the traditional production line there are a variety of isolated structured, semi-structured and unstructured data, such as sensor data, machine screen output, log output, database records etc. There are two main data scenarios: 1) Picture and video data with low frequency but a large amount; 2) Continuous data with high frequency. They are not a large amount of data per unit. However the total amount of them is very large, such as vibration data used to detect the quality of the equipment. These data have the characteristics of streaming data: real-time, volatile, burst, disorder and infinity. Making effective real-time decisions to retrieve values from these data is critical to smart manufacturing. The latest Spark Structured Streaming framework greatly lowers the bar for building highly scalable and fault-tolerant streaming applications. Thanks to the Spark we are able to build a low-latency, high-throughput and reliable operation system involving data acquisition, transmission, analysis and storage. The actual user case proved that the system meets the needs of real-time decision-making. The system greatly enhance the production process of predictive fault repair and production line material tracking efficiency, and can reduce about half of the labor force for the production lines.

Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang Wu

Spark Summit

As common sense would suggest, weather has a definite impact on traffic. But how much? And under what circumstances? Can we improve traffic (congestion) prediction given weather data? Predictive traffic is envisioned to significantly impact how driver’s plan their day by alerting users before they travel, find the best times to travel, and over time, learn from new IoT data such as road conditions, incidents, etc. This talk will cover the traffic prediction work conducted jointly by IBM and the traffic data provider. As a part of this work, we conducted a case study over five large metropolitans in the US, 2.58 billion traffic records and 262 million weather records, to quantify the boost in accuracy of traffic prediction using weather data. We will provide an overview of our lambda architecture with Apache Spark being used to build prediction models with weather and traffic data, and Spark Streaming used to score the model and provide real-time traffic predictions. This talk will also cover a suite of extensions to Spark to analyze geospatial and temporal patterns in traffic and weather data, as well as the suite of machine learning algorithms that were used with Spark framework. Initial results of this work were presented at the National Association of Broadcasters meeting in Las Vegas in April 2017, and there is work to scale the system to provide predictions in over a 100 cities. Audience will learn about our experience scaling using Spark in offline and streaming mode, building statistical and deep-learning pipelines with Spark, and techniques to work with geospatial and time-series data.

Improving Traffic Prediction Using Weather Data with Ramya Raghavendra

Spark Summit

Graph is on the rise and it’s time to start learning about scalable graph analytics! In this session we will go over two Spark-based Graph Analytics frameworks: Tinkerpop and GraphFrames. While both frameworks can express very similar traversals, they have different performance characteristics and APIs. In this Deep-Dive by example presentation, we will demonstrate some common traversals and explain how, at a Spark level, each traversal is actually computed under the hood! Learn both the fluent Gremlin API as well as the powerful GraphFrame Motif api as we show examples of both simultaneously. No need to be familiar with Graphs or Spark for this presentation as we’ll be explaining everything from the ground up!

A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...

Spark Summit

Building accurate machine learning models has been an art of data scientists, i.e., algorithm selection, hyper parameter tuning, feature selection and so on. Recently, challenges to breakthrough this “black-arts” have got started. In cooperation with our partner, NEC Laboratories America, we have developed a Spark-based automatic predictive modeling system. The system automatically searches the best algorithm, parameters and features without any manual work. In this talk, we will share how the automation system is designed to exploit attractive advantages of Spark. The evaluation with real open data demonstrates that our system can explore hundreds of predictive models and discovers the most accurate ones in minutes on a Ultra High Density Server, which employs 272 CPU cores, 2TB memory and 17TB SSD in 3U chassis. We will also share open challenges to learn such a massive amount of models on Spark, particularly from reliability and stability standpoints. This talk will cover the presentation already shown on Spark Summit SF’17 (#SFds5) but from more technical perspective.

No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...

Spark Summit

In Sweden, from the Rise ICE Data Center at www.hops.site, we are providing to reseachers both Spark-as-a-Service and, more recently, Tensorflow-as-a-Service as part of the Hops platform. In this talk, we examine the different ways in which Tensorflow can be included in Spark workflows, from batch to streaming to structured streaming applications. We will analyse the different frameworks for integrating Spark with Tensorflow, from Tensorframes to TensorflowOnSpark to Databrick’s Deep Learning Pipelines. We introduce the different programming models supported and highlight the importance of cluster support for managing different versions of python libraries on behalf of users. We will also present cluster management support for sharing GPUs, including Mesos and YARN (in Hops Hadoop). Finally, we will perform a live demonstration of training and inference for a TensorflowOnSpark application written on Jupyter that can read data from either HDFS or Kafka, transform the data in Spark, and train a deep neural network on Tensorflow. We will show how to debug the application using both Spark UI and Tensorboard, and how to examine logs and monitor training.

Apache Spark and Tensorflow as a Service with Jim Dowling

Spark Summit

Apache Spark and Tensorflow as a Service with Jim Dowling

Spark Summit

With the rapid growth of available datasets, it is imperative to have good tools for extracting insight from big data. The Spark ML library has excellent support for performing at-scale data processing and machine learning experiments, but more often than not, Data Scientists find themselves struggling with issues such as: low level data manipulation, lack of support for image processing, text analytics and deep learning, as well as the inability to use Spark alongside other popular machine learning libraries. To address these pain points, Microsoft recently released The Microsoft Machine Learning Library for Apache Spark (MMLSpark), an open-source machine learning library built on top of SparkML that seeks to simplify the data science process and integrate SparkML Pipelines with deep learning and computer vision libraries such as the Microsoft Cognitive Toolkit (CNTK) and OpenCV. With MMLSpark, Data Scientists can build models with 1/10th of the code through Pipeline objects that compose seamlessly with other parts of the SparkML ecosystem. In this session, we explore some of the main lessons learned from building MMLSpark. Join us if you would like to know how to extend Pipelines to ensure seamless integration with SparkML, how to auto-generate Python and R wrappers from Scala Transformers and Estimators, how to integrate and use previously non-distributed libraries in a distributed manner and how to efficiently deploy a Spark library across multiple platforms.

MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...

Spark Summit

The Next Accelerator Logging Service (NXCALS) is a new Big Data project at CERN aiming to replace the existing Oracle-based service. The main purpose of the system is to store and present Controls/Infrastructure related data gathered from thousands of devices in the whole accelerator complex. The data is used to operate the machines, improve their performance and conduct studies for new beam types or future experiments. During this talk, Jakub will speak about NXCALS requirements and design choices that lead to the selected architecture based on Hadoop and Spark. He will present the Ingestion API, the abstractions behind the Meta-data Service and the Spark-based Extraction API where simple changes to the schema handling greatly improved the overall usability of the system. The system itself is not CERN specific and can be of interest to other companies or institutes confronted with similar Big Data problems.

Next CERN Accelerator Logging Service with Jakub Wozniak

Spark Summit

In Between (A mobile App for couples, downloaded 20M in Global), from daily batch for extracting metrics, analysis and dashboard. Spark is widely used by engineers and data analysts in Between, thanks to the performance and expendability of Spark, data operating has become extremely efficient. Entire team including Biz Dev, Global Operation, Designers are enjoying data results so Spark is empowering entire company for data driven operation and thinking. Kevin, Co-founder and Data Team leader of Between will be presenting how things are going in Between. Listeners will know how small and agile team is living with data (how we build organization, culture and technical base) after this presentation.

Powering a Startup with Apache Spark with Kevin Kim

Spark Summit

Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra

Spark Summit

In many cases, Big Data becomes just another buzzword because of the lack of tools that can support both the technological requirements for developing and deploying of the projects and/or the fluency of communication between the different profiles of people involved in the projects. In this talk, we will present Moriarty, a set of tools for fast prototyping of Big Data applications that can be deployed in an Apache Spark environment. These tools support the creation of Big Data workflows using the already existing functional blocks or supporting the creation of new functional blocks. The created workflow can then be deployed in a Spark infrastructure and used through a REST API. For better understanding of Moriarty, the prototyping process and the way it hides the Spark environment to the Big Data users and developers, we will present it together with a couple of examples based on a Industry 4.0 success cases and other on a logistic success case.

Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...

Spark Summit

Large-scale testing of new data products or enhancements to existing products in a research and development environment can be a technical challenge for data scientists. In some cases, tools available to data scientists lack production-level capacity, whereas other tools do not provide the algorithms needed to run the methodology. At Nielsen, the Databricks platform provided a solution to both of these challenges. This breakout session will cover a specific Nielsen business case where two methodology enhancements were developed and tested at large-scale using the Databricks platform. Development and large-scale testing of these enhancements would not have been possible using standard database tools.

How Nielsen Utilized Databricks for Large-Scale Research and Development with...

Spark Summit

Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...

Spark Summit

Since the invention of SQL and relational databases, data production has been about specifying how data is transformed through queries. While Apache Spark can certainly be used as a general distributed query engine, the power and granularity of Spark’s APIs enables a revolutionary increase in data engineering productivity: goal-based data production. Goal-based data production concerns itself with specifying WHAT the desired result is, leaving the details of HOW the result is achieved to a smart data warehouse running on top of Spark. That not only substantially increases productivity, but also significantly expands the audience that can work directly with Spark: from developers and data scientists to technical business users. With specific data and architecture patterns spanning the range from ETL to machine learning data prep and with live demos, this session will demonstrate how Spark users can gain the benefits of goal-based data production.

Goal Based Data Production with Sim Simeonov

Spark Summit

Have you imagined a simple machine learning solution able to prevent revenue leakage and monitor your distributed application? To answer this question, we offer a practical and a simple machine learning solution to create an intelligent monitoring application based on simple data analysis using Apache Spark MLlib. Our application uses linear regression models to make predictions and check if the platform is experiencing any operational problems that can impact in revenue losses. The application monitor distributed systems and provides notifications stating the problem detected, that way users can operate quickly to avoid serious problems which directly impact the company’s revenue and reduce the time for action. We will present an architecture for not only a monitoring system, but also an active actor for our outages recoveries. At the end of the presentation you will have access to our training program source code and you will be able to adapt and implement in your company. This solution already helped to prevent about US$3mi in losses last year.

Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...

Spark Summit

Getting Ready to use Redis with Apache Spark is a technical tutorial designed to address integrating Redis with an Apache Spark deployment to increase the performance of serving complex decision models. To set the context for the session, we start with a quick introduction to Redis and the capabilities Redis provides. We cover the basic data types provided by Redis and cover the module system. Using an ad serving use-case, we look at how Redis can improve the performance and reduce the cost of using complex ML-models in production. Attendees will be guided through the key steps of setting up and integrating Redis with Spark, including how to train a model using Spark then load and serve it using Redis, as well as how to work with the Spark Redis module. The capabilities of the Redis Machine Learning Module (redis-ml) will be discussed focusing primarily on decision trees and regression (linear and logistic) with code examples to demonstrate how to use these feature. At the end of the session, developers should feel confident building a prototype/proof-of-concept application using Redis and Spark. Attendees will understand how Redis complements Spark and how to use Redis to serve complex, ML-models with high performance.

Getting Ready to Use Redis with Apache Spark with Dvir Volk

Spark Summit

Here we present a general supervised framework for record deduplication and author-disambiguation via Spark. This work differentiates itself by – Application of Databricks and AWS makes this a scalable implementation. Compute resources are comparably lower than traditional legacy technology using big boxes 24/7. Scalability is crucial as Elsevier’s Scopus data, the biggest scientific abstract repository, covers roughly 250 million authorships from 70 million abstracts covering a few hundred years. – We create a fingerprint for each content by deep learning and/or word2vec algorithms to expedite pairwise similarity calculation. These encoders substantially reduce compute time while maintaining semantic similarity (unlike traditional TFIDF or predefined taxonomies). We will briefly discuss how to optimize word2vec training with high parallelization. Moreover, we show how these encoders can be used to derive a standard representation for all our entities namely such as documents, authors, users, journals, etc. This standard representation can simplify the recommendation problem into a pairwise similarity search and hence it can offer a basic recommender for cross-product applications where we may not have a dedicate recommender engine designed. – Traditional author-disambiguation or record deduplication algorithms are batch-processing with small to no training data. However, we have roughly 25 million authorships that are manually curated or corrected upon user feedback. Hence, it is crucial to maintain historical profiles and hence we have developed a machine learning implementation to deal with data streams and process them in mini batches or one document at a time. We will discuss how to measure the accuracy of such a system, how to tune it and how to process the raw data of pairwise similarity function into final clusters. Lessons learned from this talk can help all sort of companies where they want to integrate their data or deduplicate their user/customer/product databases.

Deduplication and Author-Disambiguation of Streaming Records via Supervised M...

Spark Summit

The use of large-scale machine learning and data mining methods is becoming ubiquitous in many application domains ranging from business intelligence and bioinformatics to self-driving cars. These methods heavily rely on matrix computations, and it is hence critical to make these computations scalable and efficient. These matrix computations are often complex and involve multiple steps that need to be optimized and sequenced properly for efficient execution. This work presents new efficient and scalable matrix processing and optimization techniques based on Spark. The proposed techniques estimate the sparsity of intermediate matrix-computation results and optimize communication costs. An evaluation plan generator for complex matrix computations is introduced as well as a distributed plan optimizer that exploits dynamic cost-based analysis and rule-based heuristics The result of a matrix operation will often serve as an input to another matrix operation, thus defining the matrix data dependencies within a matrix program. The matrix query plan generator produces query execution plans that minimize memory usage and communication overhead by partitioning the matrix based on the data dependencies in the execution plan. We implemented the proposed matrix techniques inside the Spark SQL, and optimize the matrix execution plan based on Spark SQL Catalyst. We conduct case studies on a series of ML models and matrix computations with special features on different datasets. These are PageRank, GNMF, BFGS, sparse matrix chain multiplications, and a biological data analysis. The open-source library ScaLAPACK and the array-based database SciDB are used for performance evaluation. Our experiments are performed on six real-world datasets are: social network data ( e.g., soc-pokec, cit-Patents, LiveJournal), Twitter2010, Netflix recommendation data, and 1000 Genomes Project sample. Experiments demonstrate that our proposed techniques achieve up to an order-of-magnitude performance.

MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...

Spark Summit

Mais de Spark Summit (20)

FPGA-Based Acceleration Architecture for Spark SQL Qi Xie and Quanfu Wang

VEGAS: The Missing Matplotlib for Scala/Apache Spark with DB Tsai and Roger M...

Apache Spark Structured Streaming Helps Smart Manufacturing with Xiaochang Wu

Improving Traffic Prediction Using Weather Data with Ramya Raghavendra

A Tale of Two Graph Frameworks on Spark: GraphFrames and Tinkerpop OLAP Artem...

No More Cumbersomeness: Automatic Predictive Modeling on Apache Spark Marcin ...

Apache Spark and Tensorflow as a Service with Jim Dowling

MMLSpark: Lessons from Building a SparkML-Compatible Machine Learning Library...

Next CERN Accelerator Logging Service with Jakub Wozniak

Powering a Startup with Apache Spark with Kevin Kim

Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra

Hiding Apache Spark Complexity for Fast Prototyping of Big Data Applications—...

How Nielsen Utilized Databricks for Large-Scale Research and Development with...

Spline: Apache Spark Lineage not Only for the Banking Industry with Marek Nov...

Goal Based Data Production with Sim Simeonov

Preventing Revenue Leakage and Monitoring Distributed Systems with Machine Le...

Getting Ready to Use Redis with Apache Spark with Dvir Volk

Deduplication and Author-Disambiguation of Streaming Records via Supervised M...

MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...

Último

Probability Grade 10 Third Quarter Lessons

JoseMangaJr1

Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore Escorts Service Booking Contact Details :- WhatsApp Chat :- +91-7737669865 4-May-2024(SMW) Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Mg Road Call Girls Service: 🍓 7737669865 🍓 High Profile Model Escorts | Banga...

amitlee9823

April 2024 - Crypto Market Report's Analysis

manisha194592

Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage with sex Booking Contact Details :- WhatsApp Chat :- +91-9920725232 4-May-2024(SMW) Call Girls In Model Towh +91-9920725232 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-9920725232 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —9920725232 We are available 24*7 all days of the year. Call us — 9920725232 Thank you for Visiting.

Vip Mumbai Call Girls Thane West Call On 9920725232 With Body to body massage...

amitlee9823

Discover Why Less is More in B2B Research

michael115558

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

BDSM⚡Call Girls in Mandawali Delhi >༒8448380779 Escort Service

Delhi Call girls

Klinik_ Apotek Onlin 085657271886 Solusi Menggugurkan Masalah Kehamilan Anda Jual Obat Aborsi Asli KLINIK ABORSI TERPEECAYA _ Jual Obat Aborsi Cytotec Misoprostol Asli 100% Ampuh Hanya 3 Jam Langsung Gugur || OBAT PENGGUGUR KANDUNGAN AMPUH MANJUR OBAT ABORSI OLINE" APOTIK Jual Obat Cytotec, Gastrul, Gynecoside Asli Ampuh. JUAL ” Obat Aborsi Tuntas | Obat Aborsi Manjur | Obat Aborsi Ampuh | Obat Penggugur Janin | Obat Pencegah Kehamilan | Obat Pelancar Haid | Obat terlambat Bulan | Ciri Obat Aborsi Asli | Obat Telat Bulan | Pil Aborsi Asli | Cara Menggugurkan Konten | Cara Aborsi Tuntas | Harga Obat Aborsi Asli | Pil Aborsi | Jual Obat Aborsi Cytotec | Cara Aborsi Sendiri | Cara Aborsi Usia 1 Bulan | Cara Aborsi Usia 2 Tahun | Cara Aborsi Usia 3 Bulan | Obat Aborsi Usia 4 Bulan | Cara Abrasi Usia 5 Bulan | Cara Menggugurkan Konten | Kandungan Obat Penggugur | Cara Menghitung Usia Konten | Cara Mengatasi Terlambat Bulan | Penjual Obat Aborsi Asli | Obat Aborsi Garansi | Kandungan Obat Peluntur | Obat Telat Datang Bulan | Obat Telat Haid | Obat Aborsi Paling Murah | Klinik Jual Obat Aborsi | Jual Pil Cytotec | Apotik Jual Obat Aborsi | Kandungan Dokter Abrasi | Cara Aborsi Cepat | Jual Obat Aborsi Bergaransi | Jual Obat Cytotec Asli | Obat Aborsi Aman Manjur | Obat Misoprostol Cytotec Asli. "APA ITU ABORSI" “Aborsi Adalah dengan membendung hormon yang di perlukan untuk mempertahankan kehamilan yaitu hormon progesteron, karena hormon ini dibendung, maka jalur kehamilan mulai membuka dan leher rahim menjadi melunak,sehingga mengeluarkan darah yang merupakan tanda bahwa obat telah bekerja || maksimal 1 jam obat diminum || PENJELASAN OBAT ABORSI USIA 1 _7 BULAN Pada usia kandungan ini, pasien akan merasakan sakit yang sedikit tidak berlebihan || sekitar 1 jam ||. namun hanya akan terjadi pada saatdarah keluar merupakan pertanda menstruasi. Hal ini dikarenakan pada usiakandungan 3 bulan,janin sudah terbentuk sebesar kepalan tangan orang dewasa. Cara kerja obat aborsi : JUAL OBAT ABORSI AMPUH dosis 3 bulan secara umum sama dengan cara kerja || DOSIS OBAT ABORSI 2 bulan”, hanya berbedanya selain mengisolasijanin juga menghancurkan janin dengan formula methotrexate dikandungdidalamnya. Formula methotrexate ini sangat ampuh untuk menghancurkan janinmenjadi serpihan-serpihan kecil akan sangat berguna pada saat dikeluarkan nanti. APA ALASAN WANITA MELAKUKAN ABORSI? Aborsi di lakukan wanita hamil baik yang sudah menikah maupun belum menikah dengan berbagai alasan , akan tetapi alasan yang utama adalah alasan-alasan non medis (termasuk aborsi sendiri / di sengaja/ buatan] MELAYANI PEMESANAN OBAT ABORSI SETIAP HARI, SIAP KIRIM KESELURUH KOTA BESAR DI INDONESIA DAN LUAR NEGERI. HUBUNGI PEMESANAN LEBIH NYAMAN VIA WA/: 085657271886

Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...

ZurliaSoop

Call Girl In Dwarka ☎92055#41914 ¶¶ Indian,Russian Best Quality full Educated And Full Cooperative Independent Call Girls Escort Services In New Delhi- I Have Extremely Beautiful Broad Minded Cute Sexy & Hot Call Girls and Escorts, We Are Located in 3* 4* 5* Hotels in Delhi. Safe & Secure High Class Services Affordable Rate 100% Satisfaction, Unlimited Enjoyment. Any Time for Model/Teens Escort in Delhi High class luxury and premium escorts agency Indian Russian Call Girls In Delhi Booking Good High Profile Escorts (Call Girls) In Delhi 5 Star Hotel ,Incall Service,OutCall Service, We provide services by Call Girls,College Girls,Modals Get High Profile queens,Well Educated,Good Looking,Full Cooperative Model, Russian Models,Punjabi Girls Kashmeri Girls Services etc… We Provide Hottest Female With Safe And Consensual With Most Limits Respected Complete Satisfaction Guaranteed…Service. Call Me Spacial For Including Incall//outcall Service In New Delhi Indian Russian Escorts Service,

Cheap Rate Call girls Sarita Vihar Delhi 9205541914 shot 1500 night

Delhi Call girls

Week-01-2.ppt BBB human Computer interaction

fulawalesam

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...

amitlee9823

FESE Capital Markets Fact Sheet 2024 Q1.pdf

MarinCaroMartnezBerg

ELKO dropshipping via API with DroFx.pptx

olyaivanovalion

Sampling (random) method and Non random.ppt

Dr. Soumendra Kumar Patra

Anomaly detection and data imputation within time series

Paris Women in Machine Learning and Data Science

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Bangalore Booking Contact Details :- WhatsApp Chat :- +91-7737669865 2-May-2024(SMW) Call Girls In Model Towh Bangalore +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in Bangalore NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in Bangalore, Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service Bangalore NCRWelcome To Bangalore Escorts Service – An All Over New Bangalore Very Sexy Hot Call Girls Agency Service Escorts In South BangaloreNCRBangalore’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Bangalore Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...

amitlee9823

Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage with sex Booking Contact Details :- WhatsApp Chat :- +91-9920725232 4-May-2024(SMW) Call Girls In Model Towh +91-9920725232 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-9920725232 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —9920725232 We are available 24*7 all days of the year. Call us — 9920725232 Thank you for Visiting.

Vip Mumbai Call Girls Marol Naka Call On 9920725232 With Body to body massage...

amitlee9823

Carero dropshipping via API with DroFx.pptx

olyaivanovalion

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha Call girls in dubai Call girls at dubai Dubai Call girl mistaken. Just Call girl dubai Call girl in dubai Indian Call girls dubai Indian Call girl dubai Pakistan Call girls in dubai the Pakistani Call girl dubai https://services.tochat.be/whatsapp-business-directory/b22b3d16-5b26-4e59-b6c1-db53f14dfea0?utm_medium=social&utm_source=heylink.me Dubai Call girls service Dubai Call girl services Call girl service in dubai Dubai Call girl agency Dubai Call girls agency Verified Call girls dubai correct motivation is required. Her smile enlarges as if she were Young Call girls in dubai Marina Call girls Dubai marina Call girls Jumeirah Call girls Dubai Jumeirah Call girls Bur dubai Call girls Indian Call girls in bur dubai Call girls bur dubai hiding a tremendous secret. Al qusais Call girls Al nahda dubai Call girls Independent Call girls dubai Independent Call girl dubai Russian Call girls in dubai Dubai russian Call girls Young Call girls in dubai Dubai young Call girls Call girls numbers in dubai How about leaving your father's home, being wealthy, and being able to help your sister? Even though I know what she is going to say won't be good, my ears are ringing. To have this chat, I waited until Dubai Call girls number Call girls near me dubai Call girls near my hotel Cute Call girls in dubai Model Call girl in dubai Rent a girlfriend dubai you were eighteen years old. Do you understand what I do, Eden? Since I have no idea, I shake my head and my mind races. She must be some kind of successful businesswoman, I suppose. "I own a business. Do you recognize that? Knowing my best. She left. She said that Dad told her that Dubai Call girls Call girls dubai Call girls in dubai Call girls at dubai we didn’t need her anymore when he came home. I was sad.Dubai Call girl Call girl dubai Call girl in dubai Indian Call girls dubai Indian Call girl dubai Can you tell her to come back? I like her.” Her little face is Pakistan Call girls in dubai Pakistani Call girl dubai Dubai Call girls service Dubai Call girl services all pinched. So sweet. Call girl service in dubai Dubai Call girl agency Dubai Call girls agency Verified Call girls dubai But I'm pissed off. How can he Young Call girls in dubai Marina Call girls Dubai marina Call girls Jumeirah Call girls Dubai Jumeirah Call girls Bur dubai Call girls Indian Call girls in bur dubai Call girls bur dubai turn down someone I'm paying for? “So, who's here with you?” I ask her,Al qusais Call girls Al nahda dubai Call girls Independent Call girls dubai Independent Call girl dubai Russian Call girls in dubai Dubai russian Call girls fervently hoping she wasn’t here alone. “Dad's downstairs, I think Young Call girls in dubai Dubai young Call girls Call girls numbers in dubai Dubai Call girls number Call girls near me dubai Call girls near my hotel Cute Call girls in dubai Model Call girl in dubai Rent a girlfriend

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha

AroojKhan71

Smarteg dropshipping via API with DroFx.pptx

olyaivanovalion

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand Booking Contact Details :- WhatsApp Chat :- +91-7737669865 Call Girls In Model Towh +91-7737669865 !! Best Woman Seeking Man Call Girls Service, Escorts Service in Home Hotel in NCR 24 Hours Available Service Call Girls, Contact Us +91-7737669865 (Any Time. Any Where) Call Girls in , Noida, Gurgaon, Ghaziabad,Sexy Indian Female Escorts Service NCRWelcome To Escorts Service – An All Over New Very Sexy Hot Call Girls Agency Service Escorts In South NCR’s No. 1 High Profile Independent Female Escorts Service. We Provide Good Quality Educated Profile At #K09 Very Regnebal Price 100% Safe And Original.We Are Provide Escorts Service All OYO Hotels ,3*,4*,5* Star Hotel And Home Flat, Apartment. Guest-House. Services In -Call And Out – Call Both Are Services Available. 24Hrs. Any Time Any Where. In All Over Noida Gurgaon Ghaziabad Faridabad.More Information And Contact Profile Real Pic Visit Our Website City Wise Escorts Service Agency.Good Looking Cheap And Best Models Girls U Can Get Best Click On Link……Night Call Girls Now In Hotel Le Meridien Gurgaon Near Female Escort One Shot — 5000/in call (time 1 hour), 6000/out call Two shot with one girl — 8000/in call (time 2 hour), 10000/out call Body to body massage with sex- 8000/in call (time 1 hour) Full night Service for one person– 12000/in call, 13000/out call (shot limit 3-4 shots) Full night Service for more than 1 person — please contact Us —7737669865 We are available 24*7 all days of the year. Call us — 7737669865 Thank you for Visiting.

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand

amitlee9823

Saving Energy with Apache Spark and Toon with Stephen Galsworthy

1. Stephen Galsworthy, Toon Saving energy with Apache Spark and Toon #EUds10 #ToonTalks

2. 31 October 2017

5. Data management Solar solution Network Mobility Control by demand Heat pump Services Storage The beating heart of your smart life Smart Platforms & Energy Services

6. #EUds10 #ToonTalks Boiler adapter Z-Wave Meter adaptor Gas sensor Electricity sensor Philips Hue Z-Wave Toon data sources Central heating system Solar panels

7. #EUds10 #ToonTalks Scale of Toon data 100 MB data per user collected per month 350,000 thermostats installed 300 types of sensor and user interaction data

8. #EUds10 #ToonTalks Data Privacy Data from all Toons is collected and stored anonymously. Data is tagged with an anonymous ID, generated locally on Toon. The end user maintains control over this key and we only use data in a non-anonymous way when a customer explicitly opts-in.

9. #EUds10 #ToonTalks Toon Innovators Innovators agree to share their data for research and product development. In return they they are first in line to try out our new services!

10. #EUds10 #ToonTalks Data Collection setup data collector data storage Processed data Data science platform

11. #EUds10 #ToonTalks Apache Airflow Data Science platform For managing Spark clusters and notebooks For scheduling complex workflows For production grade algorithms and R&D For machine learning on distributed data

12. #EUds10 #ToonTalks Concept testing (3 employees) Initial tests (10-50 users) Development (600 users) Commercial launch (300,000+users) Manual Our journey

13. #EUds10 #ToonTalks Data driven services

14. #EUds10 #ToonTalks Energy Waste Checker We don’t always notice how much energy we’re wasting. Toon can now expose the energy guzzlers in your home.

15. #EUds10 #ToonTalks Toon detects inefficient appliances and behaviours

16. #EUds10 #ToonTalks Electricity data Appliance signals An underlying concept: Load disaggregation

17. #EUds10 #ToonTalks Run on 10s electricity meter data 1 Disaggregation algorithms Dryer Dishwasher Washing machine x2 Power (Wh)

18. #EUds10 #ToonTalks Determine appliance ‘fingerprint’

19. #EUds10 #ToonTalks Efficiency classification

20. #EUds10 #ToonTalks Personalised advice for the end user “It looks like your washing machine uses more energy than an efficient one”

21. #EUds10 #ToonTalks Another example: Heating and Insulation advice 1. Analyse how much gas is used for heating the home 2. Analyse the cooling down rate of the home overnight from temperature data 3. Benchmark with comparable homes 4. Personalised insulation advice for the customer Insulation It looks like your home losesenergy more quickly than efficient homes

22. #EUds10 #ToonTalks Scale of Toon data 35,000,000 washing machine cycles detected per year 100,000 washing machine cycles detected per day

23. #EUds10 #ToonTalks What else is on its way? Boiler Monitoring Toon will keep an eye on your boiler 24 hours per day. It’ll tell you if it’s still working correctly and when the next maintenance is due. Monthly Energy Insights A personalised overview of energy usage for all Toon users.

24. #EUds10 #ToonTalks Stephen Galsworthy Head of Data Science +31 (0) 20 462 1680   stephen.galsworthy@quby.com

Saving Energy with Apache Spark and Toon with Stephen Galsworthy

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Saving Energy with Apache Spark and Toon with Stephen Galsworthy

Semelhante a Saving Energy with Apache Spark and Toon with Stephen Galsworthy (20)

Mais de Spark Summit

Mais de Spark Summit (20)

Último

Último (20)

Saving Energy with Apache Spark and Toon with Stephen Galsworthy