I Love APIs 2015: Apache Usergrid Web Scale Mobile APIs with Elastic Search and Cassandra

1
Apache Usergrid: Web Scale Mobile APIs
with ElasticSearch and Cassandra!
Todd Nine
Principal Architect
Jeffrey West
BaaS Engineering
Coach

Agenda
2
1.
Why do we need Usergrid?
2.
What is Usergrid & How does it work?
3.
How do we achieve web scale?
4.
Learnings
5.
Roadmap & Questions
©2015 Apigee. All Rights Reserved.

3
(m) obile
B ackend
a s
a
S ervice
Datastore
Push
Notifications
Connections /
Social
Location
Users
…and more
Application

Building Systems of Engagement with Edge API BaaS
5
Systems of "
Record
Apigee Edge: Intelligent API Platform
Systems of
Engagement
App Developers
Database
ERP
CRM
Data "
Warehouse
Data Lake
Partner Apps
Consumer Apps
Employee Apps
Insights

6
Web Scale"
10’s of thousands requests/sec"
…across multiple data centers

7
Web Scale / MongoDB

• Apache Open Source Mobile Backend as a Service
• REST API for easy interaction from any platform
• SDKs available for iOS / Android / Javascript

9

Organization
Organization
Organization
Application
Devices
Users
Groups
Security
Roles
Collection
Collection
Entity
{…}
Entity
{…}
Entity
{…}
Application
Devices
Users
Groups
Security
Roles
Entity
{…}
Entity
{…}
Entity
{…}
Collection
Collection
Entity
{…}
Entity
{…}
Entity
{…}

But I just want persistence!
12
REST API
GET / PUT / POST / DELETE
Scalable, Multi-tenant
Persistence (Single Cluster)

What people do seeking ‘web scale’?
13

•  Canonical System of Record
•  Collections
–  Unique Values / Primary Keys
•  Entities
–  Schemaless design
–  O(1) entity retrieval
•  Connections / Graph
–  Scalable Relationships
Horizontally Scalable Persistence

Built on Horizontally Scalable Technologies
• Filter / Limit / Cursor:
– GET http://api.usergrid.com/{org}/{app}/{collection}?
ql=select * where title = 'A Tale of Two
Cities’&limit=50&cursor=LTIxNDg0NDUxNDpnR2tBQVFFQWdITUFDW
FJ2YlM1emJXbDBhQUNBZFFBUUQyMVZneExfRWVLRlV3TG9Hc1doZXdDQW
RRQVFIYVdjb0JwREVlS1VCd0xvR3NWT0JRQQ
– GET http://api.usergrid.com/{org}/{app}/{collection}?
ql=select * where location within {d} of {x, y}
17
( )

Usergrid Application
18

Architecture: Multi-Region Indexing & Consistency
SQS
SQS
SNS
6) INDEX
6) INDEX
Region B
Region A
C* Replication
2)WRITE
5)READ
5)READ

Bottlenecks
21
Tomcat
20% CPU
!?? WHY ??!
Cassandra
10% CPU

Iterative Process
22
10,266/s
9,700/s
7,233/s
3,750

Hurdles
•  Gatling
–  Can’t get 10k/s with one node
•  Cassandra
–  CPU
•  Compression?
•  C3.2xlarge -> c3.4xlarge
•  Row caching?
•  Tomcat
–  HTTP Connector
•  acceptCount
•  maxConnections
•  maxThreads
•  Non-blocking IO?
•  Usergrid
–  # of reads to Cassandra
23

Scaling Usergrid to 10,000 Req/s w/ Gatling
•  10k test run:
–  35 c3.xlarge Tomcat Nodes
–  9 c3.4xlarge C* nodes
–  6 m3.large ES nodes
•  Closing test ~7,800/s:
–  15 c3.xlarge Tomcat Nodes
–  9 c3.4xlarge C* nodes
–  6 m3.large ES nodes
24

Elasticsearch
(working… not so well…)
25

Risks of Dynamic Mapping
27
D
D
D
D
M

28
DD
D
DD
DD
D
DD
D D
DD
DD
DD
DD
Inbound message
requires a mapping
update
Mapping updated &
broadcast to the
cluster
M

29
M
DD
D
DD
DD
D
DD
D D
DD
DD
DD
DD
At scale (full loaded)
only one mapping
update can
processed at a time

30
DD
D
DD
DD
D
DD
D D
DD
DD
DD
DD
M

31
M
DD
D
DD
DD
D
DD
D D
DD
DD
DD
DD

30 ES Data Nodes
X
100MB mapping ﬁle
=
3,000MB

For each schema/
mapping update
3GB data transfer
X
5M entities with
different schemas
=
a LOT of data
transfer and a lot
of latency
32
DD
D
DD
DD
D
DD
D D
DD
DD
DD
DD
M

Elasticsearch
(working well)
33

{ 
"hello": { 
"world":{ 
"message": "Hello!" 
}, 
"items": [ 
{ 
"name": "apples", 
"weight": 100 
}  
] 
} 
}
{ 
"uuid": "48e262d6-b922-11e4-8a11", 
"ud_entity_id": "ad2b33c2e", 
"size": 100, 
"ug_app": "1a67aed7-bbb3-11e4-8a11", 
"fields": [ 
{ 
”name": "hello.world.message", 
”string": "Hello!" 
}, 
{ 
”name": "items[0].name", 
”string": "apples" 
}, 
{ 
”name": "items[0].weight", 
"integer": 100 
}  
] 
} 
Flattened Documents
34
Indexed as

Data Node A
Data Node B
Data Node C
Index Allocation
•  Multiple Indexes spread across the cluster
•  Indexes have shards
•  Read/Write aliases point to physical
indexes
•  Aliases assigned based on hashing of the
App UUID
35
Index_1
1
1
2
3
2
3
app_1_read_alias
app_1_write_alias

Data Node A
Data Node B
Data Node C
Tenant Management
36
Index_1
1
1
2
3
2
3
app_1_read_alias
app_1_write_alias
app_2_read_alias
app_2_write_alias

Data Node A
Data Node B
Data Node C
Tenant Management
•  Assign Write alias to new Index
•  Add new index to read alias
•  Stream data from old index
•  Delete data from old index
37
Index_1
1
1
2
3
2
3
Index_2
1
1
2
3
2
3
app_2_read_alias
app_2_write_alias
stream2es
app_1_read_alias
app_1_write_alias

Data Node D
Data Node A
Data Node B
Data Node C
Tenant Management
•  Indexes targeted to separate machines
38
Index_1
1
1
2
3
2
3
Index_2
1
1
2
3
2
3
app_1_read_alias
app_1_write_alias
app_2_read_alias
app_2_write_alias

Data Node A
Data Node B
Data Node C
Data Node A
Data Node B
Data Node C
Multiple Scaling Vectors
39
Index_1
1
1
2
3
2
3
Index_2
1
1
2
3
2
3
Index_3
1
1
2
3
2
3
Index_4
1
1
2
3
2
3
Index_5
1
1
2
3
2
3

Working with Graph & NoSQL Data
40

Data De-normalization per interaction
41
ERP
Product
CatalogsCRM Data Lake
Data
Warehouse

Graph Data Persistence: Entities & Connections
•  Entities
–  Objects / Documents representing an Entity
•  Collections
–  Logical grouping of Entities
•  Connections / Entity Relationships
–  Graph Data Structure
–  /{collection}/{entityA}/{verb}/{collection}/
{entityB}
–  /Users/me/likes/Products/iPhone6
–  /Recipes/Bread/includes/Ingredients/Flour
42
User
A
Product
A
User
B
User
C
Likes>>
<<Liking

What’s Next for Usergrid
•  Grow the community!
•  Blueprints Driver Release!
•  Service abstraction
•  Indexing closer to real-time
•  Larger Clusters
•  Distributed Work System
•  Encryption & Security
44

I Love APIs 2015: Apache Usergrid Web Scale Mobile APIs with Elastic Search and Cassandra

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (11)

Semelhante a I Love APIs 2015: Apache Usergrid Web Scale Mobile APIs with Elastic Search and Cassandra

Semelhante a I Love APIs 2015: Apache Usergrid Web Scale Mobile APIs with Elastic Search and Cassandra (20)

Mais de Apigee | Google Cloud

Mais de Apigee | Google Cloud (20)

Último

Último (20)

I Love APIs 2015: Apache Usergrid Web Scale Mobile APIs with Elastic Search and Cassandra