The Internet of Things (IoT) is beginning to impact every aspect of our lives. We now do almost everything using our mobile devices – from turning on a coffee pot to counting our daily steps, even turning off the lights and locking the doors of our homes. The digital and physical universes are merging, and creating massive amounts of data. It isn't just IoT, IDC predicts that by 2020, we’ll create 44 trillion gigabytes of data, much of which will be unstructured. In order to successfully manage the big data deluge, companies must adopt strategic approaches to ensure they can not only manage, but benefit from and even monetize big data. Adam Wray, CEO and president of Basho Technologies, will discuss how big data will affect enterprises, including its benefits and challenges, as well as steps organizations can take to prepare for the big data deluge.
7. SENSORS IN AIRPLANE ENGINES
“The Internet of Things is all about catching things in
progress rather than waiting until after the fact to
analyze the data,” says Paul Maritz, CEO of Pivotal
8. *Source IDC
THE DATA EXPLOSION
2013
4.4 ZB
2020
44 ZB
If the Digital Universe were
represented by the memory in a
stack of tablets…
9. WHAT IS DRIVING THE
Gaining insights from previously
unconnected data
Connecting people for work and play
Innovation of connected devices
DATA EXPLOSION?
10. 80 MILLION PEOPLE
1300 MESSAGES A SECOND
24 Hours a Day / 365 Days a Year
Information is requested and amended more
than 2.6 BILLION times a year
Has enabled over 42 MILLION Summary
Care Records to be created and stored
Has transmitted over 1.3 BILLION
prescription messages
NHS Data Spine
11. Smart Thermostat Market is expected to grow
from $146.9 million in revenue in 2014 to $2.3
billion by 2023.
-Navigant
THE CONNECTED HOME
Store historical date for analytics
Store User Schedules
Sessions Storage for Connected
Users
12. 1.5 TRILLION RECORDS PER DAY
400% Year Over Year Increase
Ability to Monitor entire IT
environment from Single Portal
Array of Real-Time Statistics
and Insight
Centralize & Correlate Events,
Alerts, and Notifications
13. Cloud-based management tools and data
analytics usually only accessible to larger
companies.
Millions of transactions each day.
POS FOR OVER 20K SMALL
BUSINESSES
Quick Service
Retail
Restaurant & Bar
14. 20 TERABYTES OF DATA PER DAY
BILLIONS OF MOBILE DEVICES
10 BILLION data transactions a
day – 150,000 a second – Apple
Forecasting 2.8 BILLION locations
around the world
Generates 4GB OF DATA every
second
We’re focusing on
helping people make
better decisions with
the weather.
15. WEATHER FORECAST
PREDICTS WALMART’S SALES
Ideal BERRY weather turns
out to be low wind with
temperatures below 80
degrees.
People are more likely to
eat STEAK when it's warm
out with higher winds but
no rain, but not if it gets too
hot.
16. WHAT ARE THE IMPLICATIONS?
Enterprises must choose: Modernize or Sink!
• Adopt technologies to successfully manage data generated
by new data sources and consumed by users accessing
complex data types from around the world, 24 hours a day.
• Organizations must make plans to handle data ingestion,
data storage, and analytics in order to bring value to the
business.
• Increasingly look to distributed systems to help manage the
influx of traffic, and ease current operational challenges.
17. DISTRIBUTED WORKLOADS
App App App App
Virtualization
Server
App
Aggregation
Server Server Server Server
Client-Server Era:
SMALL APPS
BIG SERVERS
ONE LOCATION
Cloud Era:
BIG APPS
SMALL SERVERS
MANY LOCATIONS
18. Everything works
at small scale
What happens when
something goes wrong
The customer
experience matters
WHY DISTRIBUTED SYSTEMS?
Scale out, up and down predictably
and linearly
Survive server, network or data
center failures
Data locality enables data operations
close to end-users
DISTRIBUTED SYSTEMS
DEVELOPERS OPERATIONS CUSTOMERS SALES
19. Challenge 1 – Isolation of Data
The growing hype
surrounding data
lakes is causing
substantial confusion
in the information
management space
Gartner
20. Challenge 2 – Consistency of Data
To Be or Not To Be
Consistent?
Understand the
question…
C A
P
X
The CAP Theorem
Dr Eric Brewer
Describes the trade-offs involved in
distributed systems
21. Challenge 3 – Data Gravity
Apps
Services
Lower Latency
&
Higher Bandwidth
Growth
Over Time Data
22. “Perhaps the biggest challenge is that the IoT has
the potential to generate orders of magnitude
more data than any other source in existence
today. So, in the world of the IoT we will test the
limits of ‘big.’”
Bill Franks, Chief Analytics Officer for Teradata
Putting Challenges In Perspective
23.
24. DISTRIBUTED SYSTEMS AND NoSQL
• Most powerful tool in distributed systems is NoSQL
• Efficient with active workloads
• More cost effective and scalable than RDBMS
• Key Value most flexible and preferred
Usage in web applications for blogs
Session management
Chat applications
Social network management
• Others – Document, Graph, Big Table
• Riak most operationally efficient Market is Expected to Reach
$4.2 Billion, Globally,
by 2020
- Allied Market Research
25. Two Sigma's vision has been to develop
technological innovations that intelligently
analyze the world's data to consistently
deliver value for their clients.
MANAGING $18B WITH TRADE DATA
Modeling teams work
with vast data sets –
from corporate
earnings to news and
weather reports.
Seek to synthesize
quantitative and
qualitative information
sources to generate
new scientific insights
with systematic
precision.
Wall Street Journal Front Page April 1, 2015
26. 5TB per Day in Real-Time Growing > 50% per Year
Over 3K Customers, 5K Appliances, 100K+ Data Sources
on Customer Networks
SECURITY AS A SERVICE