SlideShare uma empresa Scribd logo
1 de 19
Baixar para ler offline
Fineo
Turn-key Big Data
for Enterprise IoT
Fineo - Data Management Platform for IoT
NoSQL time-series database meets analytics warehouse
with native Business Intelligence integration and one-click ETL
Insights
10x lower cost
100x more data
Scale
Grow easily
Iterate faster
Faster adoption
Lower complexity
Integration
“Simple” Big Data Deployment
Web scale architecture is hard. Supporting data scientists is even harder.
Security/compliance is terrible. ETL is the worst. Integration is horrendous.
Webservertier
User
Caching
ETL
Analyst
Data Scientist
Security/AuditingStreaming
Data Warehouse
Hadoop
NoSQL SparkHive
Buffer
Realtime
● Growing time-series (event) data, but RDBMS does not scale
○ Need low-latency access to recent data
● Data warehousing overhead (real-time database to analytics store)
○ Increasingly complex with new/evolving data sources
○ Many places to find “answers”
● Big Data implementation
○ Hard to find people to manage & implement
○ Fragmented, weak security story
● NoSQL integration
○ NoSQL != no schema, but missing tools to leverage flexibility
○ Does not play nice with rest of the enterprise
Enterprise Data Challenges
Solution
Painless
Petabytes
BI IntegrationOne-Click
ETL Tools
Instant Data
Science
Built in Security &
Compliance
A data platform that actively helps you
So you can focus on turning data into information
Using Fineo
● Simple integration
○ Dashboard for viz. management
○ Intuitive REST + SDKs
○ Handle any shape of data
○ SQL + BI integration (JDBC/ODBC)
● Seamless Scalability
○ Transparently handle traffic spikes
○ Dynamic query optimization
● Built-in Security
○ SSL
○ Strong Authentication + Authorization
○ Audit in-platform with SQL
How does it work?
Fineo Internals
● Service Layer
○ SSL + Authentication + Authorization
○ Native native BI integration
● Next-Generation Compute Layer
○ Infinitely scalable writes to cloud-scale buffer
○ Staged pipelines for scale, flexibility and safety
○ Dynamic query planning, optimization & execution
● Smart Storage Layer
○ Flexible schema evolution
○ Row-based database for real-time data access
○ Columnar storage for analytics/data science
Built for the cloud,
by Big Data experts
Fineo Internals
Web Servers
Execution
Servers
Execution
Tree
NoSQL
Next-Gen
Compute
Smart
Storage
Secure
Service
S3
ArchiveErrors Query
Stage Backup
Batch
Process
Schema
Store
Stream Processing
Cloud-Scale Buffer
“What I would want to build, given the time - it’s the ‘right thing’”
- Sean-Michael Lewis, CTO Cortex Intel
Architecture Advantages
● Seamless scalability
● Native BI connectors make integration easy
● Best resources selected for each query
● Integrated schema for flexibility
● Backups for every stage - at least 10 copies of data!
Fineo Internals - Stream Processing
Cloud Buffer
Stream
Processors
Raw
Staged
Device
Events
Schema
Application
Schema
Store
Storage
NoSQL
S3
ArchiveErrors Query
Smart
Storage
Stream Architecture Advantages
● Highly modular
● Shadow validation on real data
● NoSQL store ready for immediate query
● Multi-tenant aware, single tenant capable
● No-Ops => low downtime
● Leveraged for auditing
Flexible Schema Evolution
● Store data before creating schema
○ Define the data model once you know what you need
● One-click mapping of fields to a single type
○ New fields do not impact existing queries or require source systems to change
● Flexible event mapping
○ Users only need to specify:
■ Target table
■ Timestamp (human or machine-readable)
Temperature Temp tmp
temp
SELECT temp FROM devices
Flexible Schema Advantages
● Makes NoSQL viable for average user
● Existing queries don’t change when adding schema
● Resilient to ‘fat finger’ mistakes
● Easy evolution of existing data
● Only add schema when you need it
● ETL is a click, not a job
Integrated Security
● Supports single tenant deployment
● Natively multi-tenant
○ Segregated columnar storage
○ Push-down filters per-tenant
○ Tenant validation at all entry points
● All usage tracked in platform
○ User events generate data points
○ Use SQL to audit your usage
● API Keys, User/Device ACLs
○ Strong authentication & authorization
What is Fineo?
● Simple, easy to use APIs
● Scalable read/write
● Integrated schema management
● Multi-tenant, time-series data organization
● Transparent, varied types of data stores
● Logical recombination of data + schema
● Distributed query optimization & execution
Why Fineo?
Rapid deployment, integration and evolution
Big Data expertise at startup prices
Integrated security & audit tools
10x faster on 100x more data
Want to hear more?
Let’s chat!
jesse@fineo.io

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

The New Basics of Business Intelligence Lesson 3: Multi Source Analysis
The New Basics of Business Intelligence Lesson 3: Multi Source AnalysisThe New Basics of Business Intelligence Lesson 3: Multi Source Analysis
The New Basics of Business Intelligence Lesson 3: Multi Source Analysis
 
Infrastructure monitoring made easy, from ingest to insight
Infrastructure monitoring made easy, from ingest to insightInfrastructure monitoring made easy, from ingest to insight
Infrastructure monitoring made easy, from ingest to insight
 
Elastic at KPN
Elastic at KPNElastic at KPN
Elastic at KPN
 
Building the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for FluviusBuilding the Next-gen Digital Meter Platform for Fluvius
Building the Next-gen Digital Meter Platform for Fluvius
 
HP Discover: Real Time Insights from Big Data
HP Discover: Real Time Insights from Big DataHP Discover: Real Time Insights from Big Data
HP Discover: Real Time Insights from Big Data
 
Big Data at a Gaming Company: Spil Games
Big Data at a Gaming Company: Spil GamesBig Data at a Gaming Company: Spil Games
Big Data at a Gaming Company: Spil Games
 
Five ways database modernization simplifies your data life
Five ways database modernization simplifies your data lifeFive ways database modernization simplifies your data life
Five ways database modernization simplifies your data life
 
MemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics PlatformMemSQL - The Real-time Analytics Platform
MemSQL - The Real-time Analytics Platform
 
O monitoramento da infraestrutura facilitado, da ingestão ao insight
O monitoramento da infraestrutura facilitado, da ingestão ao insightO monitoramento da infraestrutura facilitado, da ingestão ao insight
O monitoramento da infraestrutura facilitado, da ingestão ao insight
 
Last Conference 2017: Big Data in a Production Environment: Lessons Learnt
Last Conference 2017: Big Data in a Production Environment: Lessons LearntLast Conference 2017: Big Data in a Production Environment: Lessons Learnt
Last Conference 2017: Big Data in a Production Environment: Lessons Learnt
 
Building the Foundation for a Latency-Free Life
Building the Foundation for a Latency-Free LifeBuilding the Foundation for a Latency-Free Life
Building the Foundation for a Latency-Free Life
 
Building an IoT Kafka Pipeline in Under 5 Minutes
Building an IoT Kafka Pipeline in Under 5 MinutesBuilding an IoT Kafka Pipeline in Under 5 Minutes
Building an IoT Kafka Pipeline in Under 5 Minutes
 
Artik cloud deview 2016
Artik cloud   deview 2016Artik cloud   deview 2016
Artik cloud deview 2016
 
Дмитрий Лавриненко "Big & Fast Data for Identity & Telemetry services"
Дмитрий Лавриненко "Big & Fast Data for Identity & Telemetry services"Дмитрий Лавриненко "Big & Fast Data for Identity & Telemetry services"
Дмитрий Лавриненко "Big & Fast Data for Identity & Telemetry services"
 
How Yellowbrick Data Integrates to Existing Environments Webcast
How Yellowbrick Data Integrates to Existing Environments WebcastHow Yellowbrick Data Integrates to Existing Environments Webcast
How Yellowbrick Data Integrates to Existing Environments Webcast
 
Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...
Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...
Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...
 
Architecting for Real-Time Big Data Analytics
Architecting for Real-Time Big Data AnalyticsArchitecting for Real-Time Big Data Analytics
Architecting for Real-Time Big Data Analytics
 
Yellowbrick Webcast with DBTA for Real-Time Analytics
Yellowbrick Webcast with DBTA for Real-Time AnalyticsYellowbrick Webcast with DBTA for Real-Time Analytics
Yellowbrick Webcast with DBTA for Real-Time Analytics
 
Дмитрий Попович "How to build a data warehouse?"
Дмитрий Попович "How to build a data warehouse?"Дмитрий Попович "How to build a data warehouse?"
Дмитрий Попович "How to build a data warehouse?"
 
Billions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right NowBillions of Rows, Millions of Insights, Right Now
Billions of Rows, Millions of Insights, Right Now
 

Destaque

Authorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework CalvinAuthorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
Tomas Nilsson
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
Aaron Shilo
 

Destaque (19)

Many Bundles of Things - M Rulli
Many Bundles of Things - M RulliMany Bundles of Things - M Rulli
Many Bundles of Things - M Rulli
 
MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...
MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...
MongoDB IoT City Tour LONDON: Managing the Database Complexity, by Arthur Vie...
 
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework CalvinAuthorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
Authorization Aspects of the Distributed Dataflow-oriented IoT Framework Calvin
 
Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…Getting to know oracle database objects iot, mviews, clusters and more…
Getting to know oracle database objects iot, mviews, clusters and more…
 
Informix - The Ideal Database for IoT
Informix - The Ideal Database for IoTInformix - The Ideal Database for IoT
Informix - The Ideal Database for IoT
 
Developing io t applications in the fog a distributed dataflow approach
Developing io t applications in the fog  a distributed dataflow approachDeveloping io t applications in the fog  a distributed dataflow approach
Developing io t applications in the fog a distributed dataflow approach
 
Understanding the Operational Database Infrastructure for IoT and Fast Data
Understanding the Operational Database Infrastructure for IoT and Fast DataUnderstanding the Operational Database Infrastructure for IoT and Fast Data
Understanding the Operational Database Infrastructure for IoT and Fast Data
 
IOT Paris Seminar 2015 - Storage Challenges in IOT
IOT Paris Seminar 2015 - Storage Challenges in IOTIOT Paris Seminar 2015 - Storage Challenges in IOT
IOT Paris Seminar 2015 - Storage Challenges in IOT
 
Cassandra & Spark for IoT
Cassandra & Spark for IoTCassandra & Spark for IoT
Cassandra & Spark for IoT
 
Why IoT needs Fog Computing ?
Why IoT needs Fog Computing ?Why IoT needs Fog Computing ?
Why IoT needs Fog Computing ?
 
Understanding the Internet of Things Protocols
Understanding the Internet of Things ProtocolsUnderstanding the Internet of Things Protocols
Understanding the Internet of Things Protocols
 
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
How to build a Distributed Serverless Polyglot Microservices IoT Platform us...
 
Reactive Data Centric Architectures with Vortex, Spark and ReactiveX
Reactive Data Centric Architectures with Vortex, Spark and ReactiveXReactive Data Centric Architectures with Vortex, Spark and ReactiveX
Reactive Data Centric Architectures with Vortex, Spark and ReactiveX
 
The Data Distribution Service Tutorial
The Data Distribution Service TutorialThe Data Distribution Service Tutorial
The Data Distribution Service Tutorial
 
Fog Computing with Vortex
Fog Computing with VortexFog Computing with Vortex
Fog Computing with Vortex
 
Building IoT Applications with Vortex and the Intel Edison Starter Kit
Building IoT Applications with Vortex and the Intel Edison Starter KitBuilding IoT Applications with Vortex and the Intel Edison Starter Kit
Building IoT Applications with Vortex and the Intel Edison Starter Kit
 
DDS In Action Part II
DDS In Action Part IIDDS In Action Part II
DDS In Action Part II
 
DDS and OPC UA Explained
DDS and OPC UA ExplainedDDS and OPC UA Explained
DDS and OPC UA Explained
 
DDS in Action -- Part I
DDS in Action -- Part IDDS in Action -- Part I
DDS in Action -- Part I
 

Semelhante a Fineo Technical Overview - NextSQL for IoT

Lessons learned from embedding Cassandra in xPatterns
Lessons learned from embedding Cassandra in xPatternsLessons learned from embedding Cassandra in xPatterns
Lessons learned from embedding Cassandra in xPatterns
Claudiu Barbura
 
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
Splunk
 

Semelhante a Fineo Technical Overview - NextSQL for IoT (20)

Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Data Platform in the Cloud
Data Platform in the CloudData Platform in the Cloud
Data Platform in the Cloud
 
How to Develop and Operate Cloud Native Data Platforms and Applications
How to Develop and Operate Cloud Native Data Platforms and ApplicationsHow to Develop and Operate Cloud Native Data Platforms and Applications
How to Develop and Operate Cloud Native Data Platforms and Applications
 
Datenvirtualisierung: Wie Sie Ihre Datenarchitektur agiler machen (German)
Datenvirtualisierung: Wie Sie Ihre Datenarchitektur agiler machen (German)Datenvirtualisierung: Wie Sie Ihre Datenarchitektur agiler machen (German)
Datenvirtualisierung: Wie Sie Ihre Datenarchitektur agiler machen (German)
 
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
Sudhir Rawat, Sr Techonology Evangelist at Microsoft SQL Business Intelligenc...
 
Designing for operability and managability
Designing for operability and managabilityDesigning for operability and managability
Designing for operability and managability
 
How to Develop and Operate Cloud First Data Platforms
How to Develop and Operate Cloud First Data PlatformsHow to Develop and Operate Cloud First Data Platforms
How to Develop and Operate Cloud First Data Platforms
 
Ibm_IoT_Architecture_and_Capabilities
Ibm_IoT_Architecture_and_CapabilitiesIbm_IoT_Architecture_and_Capabilities
Ibm_IoT_Architecture_and_Capabilities
 
Google's Infrastructure and Specific IoT Services
Google's Infrastructure and Specific IoT ServicesGoogle's Infrastructure and Specific IoT Services
Google's Infrastructure and Specific IoT Services
 
1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release1 Introduction to Microsoft data platform analytics for release
1 Introduction to Microsoft data platform analytics for release
 
Webinar: Improve Splunk Analytics and Automate Processes with SnapLogic
Webinar: Improve Splunk Analytics and Automate Processes with SnapLogicWebinar: Improve Splunk Analytics and Automate Processes with SnapLogic
Webinar: Improve Splunk Analytics and Automate Processes with SnapLogic
 
Lessons learned from embedding Cassandra in xPatterns
Lessons learned from embedding Cassandra in xPatternsLessons learned from embedding Cassandra in xPatterns
Lessons learned from embedding Cassandra in xPatterns
 
Webinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the EnterpriseWebinar: Sizing Up Object Storage for the Enterprise
Webinar: Sizing Up Object Storage for the Enterprise
 
Gs08 modernize your data platform with sql technologies wash dc
Gs08 modernize your data platform with sql technologies   wash dcGs08 modernize your data platform with sql technologies   wash dc
Gs08 modernize your data platform with sql technologies wash dc
 
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
ADV Slides: The Evolution of the Data Platform and What It Means to Enterpris...
 
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise ConsciousnessData Engineer's Lunch #60: Series - Developing Enterprise Consciousness
Data Engineer's Lunch #60: Series - Developing Enterprise Consciousness
 
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
 
inmation Presentation
inmation Presentationinmation Presentation
inmation Presentation
 
Transforms Document Management at Scale with Distributed Database Solution wi...
Transforms Document Management at Scale with Distributed Database Solution wi...Transforms Document Management at Scale with Distributed Database Solution wi...
Transforms Document Management at Scale with Distributed Database Solution wi...
 

Último

Último (20)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 

Fineo Technical Overview - NextSQL for IoT

  • 2. Fineo - Data Management Platform for IoT NoSQL time-series database meets analytics warehouse with native Business Intelligence integration and one-click ETL Insights 10x lower cost 100x more data Scale Grow easily Iterate faster Faster adoption Lower complexity Integration
  • 3. “Simple” Big Data Deployment Web scale architecture is hard. Supporting data scientists is even harder. Security/compliance is terrible. ETL is the worst. Integration is horrendous. Webservertier User Caching ETL Analyst Data Scientist Security/AuditingStreaming Data Warehouse Hadoop NoSQL SparkHive Buffer Realtime
  • 4.
  • 5. ● Growing time-series (event) data, but RDBMS does not scale ○ Need low-latency access to recent data ● Data warehousing overhead (real-time database to analytics store) ○ Increasingly complex with new/evolving data sources ○ Many places to find “answers” ● Big Data implementation ○ Hard to find people to manage & implement ○ Fragmented, weak security story ● NoSQL integration ○ NoSQL != no schema, but missing tools to leverage flexibility ○ Does not play nice with rest of the enterprise Enterprise Data Challenges
  • 6. Solution Painless Petabytes BI IntegrationOne-Click ETL Tools Instant Data Science Built in Security & Compliance A data platform that actively helps you So you can focus on turning data into information
  • 7. Using Fineo ● Simple integration ○ Dashboard for viz. management ○ Intuitive REST + SDKs ○ Handle any shape of data ○ SQL + BI integration (JDBC/ODBC) ● Seamless Scalability ○ Transparently handle traffic spikes ○ Dynamic query optimization ● Built-in Security ○ SSL ○ Strong Authentication + Authorization ○ Audit in-platform with SQL
  • 8. How does it work?
  • 9. Fineo Internals ● Service Layer ○ SSL + Authentication + Authorization ○ Native native BI integration ● Next-Generation Compute Layer ○ Infinitely scalable writes to cloud-scale buffer ○ Staged pipelines for scale, flexibility and safety ○ Dynamic query planning, optimization & execution ● Smart Storage Layer ○ Flexible schema evolution ○ Row-based database for real-time data access ○ Columnar storage for analytics/data science Built for the cloud, by Big Data experts
  • 10. Fineo Internals Web Servers Execution Servers Execution Tree NoSQL Next-Gen Compute Smart Storage Secure Service S3 ArchiveErrors Query Stage Backup Batch Process Schema Store Stream Processing Cloud-Scale Buffer
  • 11. “What I would want to build, given the time - it’s the ‘right thing’” - Sean-Michael Lewis, CTO Cortex Intel Architecture Advantages ● Seamless scalability ● Native BI connectors make integration easy ● Best resources selected for each query ● Integrated schema for flexibility ● Backups for every stage - at least 10 copies of data!
  • 12. Fineo Internals - Stream Processing Cloud Buffer Stream Processors Raw Staged Device Events Schema Application Schema Store Storage NoSQL S3 ArchiveErrors Query Smart Storage
  • 13. Stream Architecture Advantages ● Highly modular ● Shadow validation on real data ● NoSQL store ready for immediate query ● Multi-tenant aware, single tenant capable ● No-Ops => low downtime ● Leveraged for auditing
  • 14. Flexible Schema Evolution ● Store data before creating schema ○ Define the data model once you know what you need ● One-click mapping of fields to a single type ○ New fields do not impact existing queries or require source systems to change ● Flexible event mapping ○ Users only need to specify: ■ Target table ■ Timestamp (human or machine-readable) Temperature Temp tmp temp SELECT temp FROM devices
  • 15. Flexible Schema Advantages ● Makes NoSQL viable for average user ● Existing queries don’t change when adding schema ● Resilient to ‘fat finger’ mistakes ● Easy evolution of existing data ● Only add schema when you need it ● ETL is a click, not a job
  • 16. Integrated Security ● Supports single tenant deployment ● Natively multi-tenant ○ Segregated columnar storage ○ Push-down filters per-tenant ○ Tenant validation at all entry points ● All usage tracked in platform ○ User events generate data points ○ Use SQL to audit your usage ● API Keys, User/Device ACLs ○ Strong authentication & authorization
  • 17. What is Fineo? ● Simple, easy to use APIs ● Scalable read/write ● Integrated schema management ● Multi-tenant, time-series data organization ● Transparent, varied types of data stores ● Logical recombination of data + schema ● Distributed query optimization & execution
  • 18. Why Fineo? Rapid deployment, integration and evolution Big Data expertise at startup prices Integrated security & audit tools 10x faster on 100x more data
  • 19. Want to hear more? Let’s chat! jesse@fineo.io