More Related Content Similar to NoSQL Application Development with JSON and MapR-DB (20) More from MapR Technologies (17) NoSQL Application Development with JSON and MapR-DB1. © 2016 MapR Technologies 1© 2016 MapR Technologies
NoSQL Application Development with JSON
and MapR-DB
2. © 2016 MapR Technologies 2
Our Speakers
Dale Kim
Sr. Director, Industry Solutions
Tugdual Grall
Technical Evangelist
3. © 2016 MapR Technologies 3© 2016 MapR Technologies
Quick NoSQL Overview
4. © 2016 MapR Technologies 4
Relational Databases Were Not Designed for Big Data
• RDBMSs are the default
choice for applications
– But large, rapidly changing,
and/or diverse data sets add
cost/time pressures
• This forces trade-offs with
your data
• Or significant costs
RDBMS
$$$
Throwing extra money
at the problem?
Throwing away data to
preserve performance?
5. © 2016 MapR Technologies 5
The Common NoSQL Data Models
{
“fname”: “John”,
“lname”: “Doe”
}
Key-Value Wide Column Document (JSON) Graph
K V …K
JSON documents are popular because they easily model:
• Hierarchical/nested data
• Evolving data
• Varying data
6. © 2016 MapR Technologies 6© 2016 MapR Technologies
Let’s Dive into JSON
7. © 2016 MapR Technologies 7
JSON is a Powerful Construct
function Thing()
{
// public
this.get_name = function() {
return name;
}
this.set_name = function(tname) {
name = tname;
}
this.increment = (function () {
var count = 0;
return function () {return ++count;}
})();
// private
var name = null;
}
var thing = new Thing();
thing.set_name("John");
// alert(thing.name); //error, private
variable
var counter = thing.counter;
8. © 2016 MapR Technologies 8
JSON Allows Easy Variation across Records
{
"_id" : "rp-prod132546",
"name" : "Marvel T2 Athena”,
"brand" : "Pinarello",
"category" : "bike",
"type" : "Road Bike”,
"price" : 2949.99,
"size" : "55cm",
"wheel_size" : "700c",
"frameset" : {
"frame" : "Carbon Toryaca",
"fork" : "Onda 2V C"
},
"groupset" : {
"chainset" : "Camp. Athena 50/34",
"brake" : "Camp."
},
"wheelset" : {
"wheels" : "Camp. Zonda",
"tyres" : "Vittoria Pro"
}
}
{
"_id" : "rp-prod106702",
"name" : " Ultegra SPD-SL 6800”,
"brand" : "Shimano",
"category" : "pedals",
"type" : "Components,
"price" : 112.99,
"features" : [
"Low profile design increases ...",
"Supplied with floating SH11 cleats",
"Weight: 260g (pair)"
]
}
{
"_id" : "rp-prod113104",
"name" : "Bianchi Pride Jersey SS15”,
"brand" : "Nalini",
"category" : "Jersey",
"type" : "Clothing,
"price" : 76.99,
"features" : [
"100% Polyester",
"3/4 hidden zip",
"3 rear pocket"
],
"color" : "black"
}
jerseypedalbike
9. © 2016 MapR Technologies 9
How Variability Is Handled in an RDBMS
To get a single product“Entity Value Attribute” pattern
SELECT * FROM (
SELECT
ce.sku,
ea.attribute_id,
ea.attribute_code,
CASE ea.backend_type
WHEN 'varchar' THEN ce_varchar.value
WHEN 'int' THEN ce_int.value
WHEN 'text' THEN ce_text.value
WHEN 'decimal' THEN ce_decimal.value
WHEN 'datetime' THEN ce_datetime.value
ELSE ea.backend_type
END AS value,
ea.is_required AS required
FROM catalog_product_entity AS ce
LEFT JOIN eav_attribute AS ea
ON ce.entity_type_id = ea.entity_type_id
LEFT JOIN catalog_product_entity_varchar AS ce_varchar
ON ce.entity_id = ce_varchar.entity_id
AND ea.attribute_id = ce_varchar.attribute_id
AND ea.backend_type = 'varchar'
LEFT JOIN catalog_product_entity_text AS ce_text
ON ce.entity_id = ce_text.entity_id
AND ea.attribute_id = ce_text.attribute_id
AND ea.backend_type = 'text'
LEFT JOIN catalog_product_entity_decimal AS ce_decimal
ON ce.entity_id = ce_decimal.entity_id
AND ea.attribute_id = ce_decimal.attribute_id
AND ea.backend_type = 'decimal'
LEFT JOIN catalog_product_entity_datetime AS ce_datetime
ON ce.entity_id = ce_datetime.entity_id
AND ea.attribute_id = ce_datetime.attribute_id
AND ea.backend_type = 'datetime'
WHERE ce.sku = ‘rp-prod132546’
) AS tab
WHERE tab.value != ’’;
10. © 2016 MapR Technologies 10
Store the product “as a business object” To get a single product
{
"_id" : "rp-prod132546",
"name" : "Marvel T2 Athena”,
"brand" : "Pinarello",
"category" : "bike",
"type" : "Road Bike”,
"price" : 2949.99,
"size" : "55cm",
"wheel_size" : "700c",
"frameset" : {
"frame" : "Carbon Toryaca",
"fork" : "Onda 2V C"
},
"groupset" : {
"chainset" : "Camp. Athena 50/34",
"brake" : "Camp."
},
"wheelset" : {
"wheels" : "Camp. Zonda",
"tyres" : "Vittoria Pro"
}
}
products
.findById(“rp-prod132546”)
Product Catalog - NoSQL/Document
11. © 2016 MapR Technologies 11© 2016 MapR Technologies
Example Use Cases for JSON/NoSQL
12. © 2016 MapR Technologies 12
Example Use Case 1: Data Lake
Sources
RELATIONAL,
SAAS,
MAINFRAME
DOCUMENTS,
EMAILS
LOG FILES,
CLICKSTREAM
SENSORS
BLOGS,
TWEETS,
LINK DATA
DATA
WAREHOUSES,
DATA MARTS
NoSQL
Database File System
SQL
Interactive
Analytics
13. © 2016 MapR Technologies 13
Example Use Case 2: IoT and Predictive Analytics
Sources
SENSOR DATA
NoSQL Database
Interactive
Analytics
Stream Delivery
Stream
Processing/Analytics
Retrospective Analysis
14. © 2016 MapR Technologies 14© 2016 MapR Technologies
What Should Native JSON Support Include?
15. © 2016 MapR Technologies 15
Recognize JSON Elements at Server-Side
{
order_num: 5555,
products: [
{ product_id: 348752,
quantity: 1,
unit_price: 149.99,
total_price: 149.99
},
{ product_id: 439322,
quantity: 1,
unit_price: 99.99,
total_price: 99.99
},
{ product_id: 953923,
quantity: 1,
unit_price: 49.99,
total_price: 49.99
},
]
}
Reads/writes at element level
• Granular disk reads/writes
• Less network traffic
• Higher concurrency
Any new elements added on demand
• No predefined schemas
• Easy to store evolving data
Entire
JSON
doc
1. Request products[0].quantity
2. Get only products[0].quantity
versus
16. © 2016 MapR Technologies 16
Organize Elements for Different Policies
/
{a:
{a1:
{b1: "v1",
b2: [
{c1: "v1",
c2: "v2"}
]
},
a2:
{
e1: "v1",
e2: <inline jpg>
}
}
}
Column Family 1
Column Family 2
• Control layout for faster data access
• Different TTL requirements
• Separate replication settings
• Efficient access controls
17. © 2016 MapR Technologies 17
Fine Grained Security within JSON Document
{
“fname”: “John”,
“lname”: “Doe”,
“address”: “111 Main St.”,
“city”: “San Jose”,
“state”: “CA”,
“zip”: “95134”,
“credit_cards”: [
{“issuer”: “Visa”,
“number”: “4444555566667777”},
{“issuer”: “MasterCard”,
“number”: “5555666677778888”}
]
}
Entire document
Element: “fname”
Array: “credit_cards”
Sub-element in array element:
“credit_cards[*].number”
Specify different permissions levels within the document.
18. © 2016 MapR Technologies 18
Extensions for Comprehensive Data Type Support
• NULL
• Boolean
• String
• Map
• Array
• Float, Double
• Binary
• Byte, Short, Int, Long
• Date
• Decimal
• Interval
• Time
• Timestamp
Examples:
{
“sample_int”: {"$numberLong”: 2147483647},
“sample_date”: {“$dateDay”: “2016-02-22”},
“sample_decimal”:{“$decimal”: “1234567890.23456789”},
“sample_time”: {“$time”: “10:26:12.487”},
“sample_timestamp”: {“$date”: “2016-02-22T10:26:12.487+Z”}
}
19. © 2016 MapR Technologies 19© 2016 MapR Technologies
How Does This Relate to MapR?
21. © 2016 MapR Technologies 21
A Single Application Development Platform
• Business applications and
analytics on the same platform
– No data movement
– Real-time access to data
• Architectural simplicity
• Optimized stack for greater
efficiency
• “Hadoop-scale” for multi-petabyte
deployments
22. © 2016 MapR Technologies 22
Open Source OJAI API for JSON-Based Applications
Open JSON Application Interface (OJAI)
Databases Streams
MapR-Client
File Systems
{JSON}
MapR-Client
23. © 2016 MapR Technologies 23
Familiar JSON Paradigm – Similar API Constructs
MapR-DB
Document record = Json.newDocument()
.set("firstName", "John")
.set("lastName", "Doe")
.set("age", 50);
table.insert("jdoe", record);
Other Document Database
BasicDBObject doc = new BasicDBObject
("firstName", "John")
.append("lastName", "Doe")
.append("age", 50);
coll.insert(doc);
24. © 2016 MapR Technologies 24
Multi-Master Global Deployments
Active-active replication
• Faster data access – minimize network
latency on global data with local clusters
• Reduced risk of data loss – real-time,
bi-directional replication for synchronized
data across active clusters
• Application failover – upon any cluster
failure, applications continue via
redirection to another cluster
Active Read/Write
End Users
Real-Time Replication
25. © 2016 MapR Technologies 25
Apache Drill for SQL Querying on MapR-DB
Access to any data type, any data source
• Relational
• Nested data
• Schema-less
Rapid time to insights
• Query data in-situ
• No Schemas required
• Easy to get started
Integration with existing tools
• ANSI SQL
• BI tool integration
Scale in all dimensions
• TB-PB of scale
• 1000’s of users
• 1000’s of nodes
Granular security
• Authentication
• Row/column level controls
• De-centralized
26. © 2016 MapR Technologies 26
Flash/SSD/NVMe Optimization
3-4X boost in throughput
solid state drives
Enhanced I/O
parallelization
27. © 2016 MapR Technologies 27
Fast, Automatic Optimizations
• No manual cleanup tasks
• Automatic sharding
• No compaction delays
• No anti-entropy task
needed to sync replicas
(strong consistency)
Red plots show MapR-DB response times.
Blue plots show other NoSQL response times.
28. © 2016 MapR Technologies 28
Independent Third-Party Evaluation Results
The Forrester Wave™: Big Data NoSQL, Q3 2016
The Forrester Wave™ is copyrighted by Forrester Research, Inc.
Forrester and Forrester Wave™ are trademarks of Forrester
Research, Inc. The Forrester Wave™ is a graphical
representation of Forrester's call on a market and is plotted
using a detailed spreadsheet with exposed scores, weightings,
and comments. Forrester does not endorse any vendor, product,
or service depicted in the Forrester Wave. Information is based
on best available resources. Opinions reflect judgment at the
time and are subject to change.
29. © 2016 MapR Technologies 29© 2016 MapR Technologies
Demo
30. © 2016 MapR Technologies 30
Q&AEngage with us!
1. MapR-DB
Learn more and get started using MapR-DB
https://www.mapr.com/products/mapr-db-in-hadoop-nosql
Free on-demand training:
https://www.mapr.com/ODT
2. Ask Questions:
– Ask Us Anything about MapR-DB in the MapR Community now through Fri (Aug 26)
– https://community.mapr.com/