SlideShare uma empresa Scribd logo
1 de 41
Couchbase at Symantec
Gaurav Chandna
Sr Principal Software Engineer
About Me
Gaurav Chandna
At Symantec for 11 years.
Worked at AntiSpyware,
Behavioral malware, Norton and
now building platforms.
Engineering Manager previously
@xyxxy
2
Agenda
1 Introduction
2 Time series
3 Relational
4 Q&A
Copyright © 2014 Symantec Corporation
3
Introduction
Copyright © 2014 Symantec Corporation 4
Environment
• Service Oriented Architecture
• Message Oriented Middleware
• Multi-tenant
• Multiple data centers
Copyright © 2014 Symantec Corporation
5
Message Oriented Architecture
Copyright © 2014 Symantec Corporation
6
Replication between data centers
Copyright © 2014 Symantec Corporation
7
Time Series Data
Copyright © 2014 Symantec Corporation 8
Copyright © 2014 Symantec Corporation
9
Copyright © 2014 Symantec Corporation
10
Message
Broker
Service
ElasticSearch
Query &
Visualization
Copyright © 2014 Symantec Corporation
11
Quick side bar: Elasticsearch
Copyright © 2014 Symantec Corporation
12
Elasticsearch
Copyright © 2014 Symantec Corporation
13
• Document oriented search engine
• Based on Apache Lucene project
• JSON based
• Distributed
• Scale out
• Multi-tenancy
• RESTful
Elasticsearch
Copyright © 2014 Symantec Corporation
14
• Structured search
• Unstructured search
• Analytics
• Combine
All good?
Copyright © 2014 Symantec Corporation
15
Well… kind of
• Elasticsearch doesn’t recommend using it as data store
– Due to write coalescing
• Limited support for replication between data centers
– Especially across unreliable networks
Copyright © 2014 Symantec Corporation
16
New pipeline
Message
Broker
Service
Couchbase
ElasticSearch
Query &
Visualization
Copyright © 2014 Symantec Corporation
17
Cross data center replication
Copyright © 2014 Symantec Corporation
18
Used to replicate Elasticsearch between data centers
Copyright © 2014 Symantec Corporation
19
It’s all good!
Copyright © 2014 Symantec Corporation
20
Just one more thing…
• Document stored on Couchbase
After X days, the document gets
purged
• XDCR will purge in Elasticsearch
also
• Forked the Couchbase –
Elasticsearch connector to disable
deletion
• Looking to integrate those changes
to Couchbase for community use
Copyright © 2014 Symantec Corporation
21
And we are finally good!
Copyright © 2014 Symantec Corporation
22
Data with relationships
Copyright © 2014 Symantec Corporation 23
Relationships are everywhere
Copyright © 2014 Symantec Corporation
24
Engineering
Gaurav
iPhone Mac
John
PC Android
Right here…
Copyright © 2014 Symantec Corporation
25
Meetup
@Symantec
Gaurav
iPhone
Joshua
Android
In abstract terms
A
B
C D
C’
Y
Copyright © 2014 Symantec Corporation
26
You get information through queries
• Get all members of a particular group
• All groups ordered by number of people in the group
• List all members of a group, order by location
• List all groups where users have iPhone, ordered by total
number of people
• And so on..
Copyright © 2014 Symantec Corporation
27
Engineering considerations
• Ability to add new types
– Don’t make big changes when iWatch came out
• Ability to extend the types
• Add new relationships quickly
Copyright © 2014 Symantec Corporation
28
Table stakes
• Cross data center replication
• Performance
• Horizontal scaling
• Searching
• Querying capability
Copyright © 2014 Symantec Corporation
29
Initial implementation
• {
• ”type": ”person”,
• "name": ”Gaurav",
• "modified": "2015-04-
17T00:39:45.468Z",
• "_id": "79014cbb-69ce-4459-8f4a-
6edaf4b8313a",
• // other information
• }
• {
• ”type": ”department”,
• "name": ”Engineering",
• "_id": "79014cbb-69ce-4459-8f4a-
6edaf4b8313a",
• // other information
• }
Copyright © 2014 Symantec Corporation
30
Initial version
• Store different objects with types
as documents
• Create views in Couchbase
– Type
– Relationships between types
– Specialize on special types
• Ensure persistence during
operations
• Works well… to a point
Copyright © 2014 Symantec Corporation
31
So we needed a next generation
Copyright © 2014 Symantec Corporation
32
Second iteration
• {
• "_id":"Well_crafted_id”
• // Other meta information
• // Other associated information
• }
• Relationship information embedded
in the _id
• Takes advantage of key access to
document by id
• Can still build views on these
documents
• Does require more hand crafting to
get better performance
• Maintain flexibility
• Need to play around with different
APIs to see which work best for use
case
• More generic code
Copyright © 2014 Symantec Corporation
33
What about search?
Copyright © 2014 Symantec Corporation
34
Use Elasticsearch for search!
• Couchbase is primary data store
• Use Couchbase XDCR for one
direction replication
– Like time series data
– But don’t disable deletes
Copyright © 2014 Symantec Corporation
35
Message
Queue
Service
Couchbase
Elasticsearch
But…
• Elasticsearch prefers types to be
well defined for indexing
• Code is generic for extensibility
• Couchbase – Elasticsearch plugin
only supports 1 configuration
• Different services have different
needs
Copyright © 2014 Symantec Corporation
36
• More modifications to the
Elasticsearch – Couchbase
connector
• Custom code to deduce types for
compatibility with Elasticsearch
• Per bucket configurations so
deletes for time series but not for
relational data
• Keep dynamically defined types
and the code generic
Copyright © 2014 Symantec Corporation
37
Final thoughts
• Data modeling is extremely
important
• JSON Parser – experiment for your
use case
• Experiment with the different API
options for your needs
• Don’t be afraid to customize for
your needs!
• Polyglot persistence
– Matt’s presentation
Copyright © 2014 Symantec Corporation
38
Thank you!
Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its
affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or
implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.
Gaurav Chandna
Gaurav_chandna@symantec.com
Twitter: @xyxxy
Thank you!
Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its
affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.
This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or
implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.
We are hiring
www.symantec.com/careers
&Q A
Copyright © 2014 Symantec Corporation 41

Mais conteúdo relacionado

Mais procurados

Making Sense of Hypermedia APIs – Hype or Reality?
Making Sense of Hypermedia APIs – Hype or Reality?Making Sense of Hypermedia APIs – Hype or Reality?
Making Sense of Hypermedia APIs – Hype or Reality?
Akana
 
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
DirkjanMollema
 

Mais procurados (20)

Azure signalr service
Azure signalr serviceAzure signalr service
Azure signalr service
 
Building a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable FunctionsBuilding a document e-signing workflow with Azure Durable Functions
Building a document e-signing workflow with Azure Durable Functions
 
Azure governance
Azure governanceAzure governance
Azure governance
 
Performance Optimization of Cloud Based Applications by Peter Smith, ACL
Performance Optimization of Cloud Based Applications by Peter Smith, ACLPerformance Optimization of Cloud Based Applications by Peter Smith, ACL
Performance Optimization of Cloud Based Applications by Peter Smith, ACL
 
8 cloud design patterns you ought to know - Update Conference 2018
8 cloud design patterns you ought to know - Update Conference 20188 cloud design patterns you ought to know - Update Conference 2018
8 cloud design patterns you ought to know - Update Conference 2018
 
Azure Key Vault - Getting Started
Azure Key Vault - Getting StartedAzure Key Vault - Getting Started
Azure Key Vault - Getting Started
 
TechDays Finland 2020: Best practices of securing web applications running on...
TechDays Finland 2020: Best practices of securing web applications running on...TechDays Finland 2020: Best practices of securing web applications running on...
TechDays Finland 2020: Best practices of securing web applications running on...
 
Onion Architecture with S#arp
Onion Architecture with S#arpOnion Architecture with S#arp
Onion Architecture with S#arp
 
Azure Key Vault with a PaaS Architecture and ARM Template Deployment
Azure Key Vault with a PaaS Architecture and ARM Template DeploymentAzure Key Vault with a PaaS Architecture and ARM Template Deployment
Azure Key Vault with a PaaS Architecture and ARM Template Deployment
 
Making Sense of Hypermedia APIs – Hype or Reality?
Making Sense of Hypermedia APIs – Hype or Reality?Making Sense of Hypermedia APIs – Hype or Reality?
Making Sense of Hypermedia APIs – Hype or Reality?
 
API Description Languages: Which is the Right One for Me?
API Description Languages: Which is the Right One for Me?API Description Languages: Which is the Right One for Me?
API Description Languages: Which is the Right One for Me?
 
The Future of Hadoop Security - Hadoop Summit 2014
The Future of Hadoop Security - Hadoop Summit 2014The Future of Hadoop Security - Hadoop Summit 2014
The Future of Hadoop Security - Hadoop Summit 2014
 
Global azure virtual 2021 - Azure Lighthouse
Global azure virtual 2021 - Azure LighthouseGlobal azure virtual 2021 - Azure Lighthouse
Global azure virtual 2021 - Azure Lighthouse
 
Introduction to Azure Blueprints
Introduction to Azure BlueprintsIntroduction to Azure Blueprints
Introduction to Azure Blueprints
 
Programming with Azure Active Directory
Programming with Azure Active DirectoryProgramming with Azure Active Directory
Programming with Azure Active Directory
 
Azure Saturday: Security + DevOps + Azure = Awesomeness
Azure Saturday: Security + DevOps + Azure = AwesomenessAzure Saturday: Security + DevOps + Azure = Awesomeness
Azure Saturday: Security + DevOps + Azure = Awesomeness
 
Azure Networking - The First Technical Challenge
Azure Networking  - The First Technical ChallengeAzure Networking  - The First Technical Challenge
Azure Networking - The First Technical Challenge
 
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
I'm in your cloud... reading everyone's email. Hacking Azure AD via Active Di...
 
Techorama Belgium 2019 - Building an Azure Governance model for the Enterprise
Techorama Belgium 2019 - Building an Azure Governance model for the EnterpriseTechorama Belgium 2019 - Building an Azure Governance model for the Enterprise
Techorama Belgium 2019 - Building an Azure Governance model for the Enterprise
 
[DevDay 2016] OpenStack and approaches for new users - Speaker: Chi Le – Head...
[DevDay 2016] OpenStack and approaches for new users - Speaker: Chi Le – Head...[DevDay 2016] OpenStack and approaches for new users - Speaker: Chi Le – Head...
[DevDay 2016] OpenStack and approaches for new users - Speaker: Chi Le – Head...
 

Semelhante a Couchbase usage at Symantec

A Reference Architecture to Enable Visibility and Traceability across the Ent...
A Reference Architecture to Enable Visibility and Traceability across the Ent...A Reference Architecture to Enable Visibility and Traceability across the Ent...
A Reference Architecture to Enable Visibility and Traceability across the Ent...
CollabNet
 

Semelhante a Couchbase usage at Symantec (20)

Big Data Fundamentals 6.6.18
Big Data Fundamentals 6.6.18Big Data Fundamentals 6.6.18
Big Data Fundamentals 6.6.18
 
Big Data Fundamentals
Big Data FundamentalsBig Data Fundamentals
Big Data Fundamentals
 
Building A Business-Facing Mobile Developer Community
Building A Business-Facing Mobile Developer CommunityBuilding A Business-Facing Mobile Developer Community
Building A Business-Facing Mobile Developer Community
 
DevOps: Infrastructure as Code
DevOps: Infrastructure as CodeDevOps: Infrastructure as Code
DevOps: Infrastructure as Code
 
A Reference Architecture to Enable Visibility and Traceability across the Ent...
A Reference Architecture to Enable Visibility and Traceability across the Ent...A Reference Architecture to Enable Visibility and Traceability across the Ent...
A Reference Architecture to Enable Visibility and Traceability across the Ent...
 
Hybrid Analysis Mapping: Making Security and Development Tools Play Nice Toge...
Hybrid Analysis Mapping: Making Security and Development Tools Play Nice Toge...Hybrid Analysis Mapping: Making Security and Development Tools Play Nice Toge...
Hybrid Analysis Mapping: Making Security and Development Tools Play Nice Toge...
 
Caching Patterns
Caching PatternsCaching Patterns
Caching Patterns
 
Web App Security Presentation by Ryan Holland - 05-31-2017
Web App Security Presentation by Ryan Holland - 05-31-2017Web App Security Presentation by Ryan Holland - 05-31-2017
Web App Security Presentation by Ryan Holland - 05-31-2017
 
Introducing the Entity Framework
Introducing the Entity FrameworkIntroducing the Entity Framework
Introducing the Entity Framework
 
Migration from SQL to MongoDB - A Case Study at TheKnot.com
Migration from SQL to MongoDB - A Case Study at TheKnot.com Migration from SQL to MongoDB - A Case Study at TheKnot.com
Migration from SQL to MongoDB - A Case Study at TheKnot.com
 
Application patterns
Application patternsApplication patterns
Application patterns
 
A proven path for migrating from clearcase to git and or subversion
A proven path for migrating from clearcase to git and or subversionA proven path for migrating from clearcase to git and or subversion
A proven path for migrating from clearcase to git and or subversion
 
Cassandra Summit 2014: Internet of Complex Things Analytics with Apache Cassa...
Cassandra Summit 2014: Internet of Complex Things Analytics with Apache Cassa...Cassandra Summit 2014: Internet of Complex Things Analytics with Apache Cassa...
Cassandra Summit 2014: Internet of Complex Things Analytics with Apache Cassa...
 
AppSphere 15 - How AppDynamics is Shaking up the Synthetic Monitoring Product...
AppSphere 15 - How AppDynamics is Shaking up the Synthetic Monitoring Product...AppSphere 15 - How AppDynamics is Shaking up the Synthetic Monitoring Product...
AppSphere 15 - How AppDynamics is Shaking up the Synthetic Monitoring Product...
 
Cloud-native Data
Cloud-native DataCloud-native Data
Cloud-native Data
 
Cloud-Native-Data with Cornelia Davis
Cloud-Native-Data with Cornelia DavisCloud-Native-Data with Cornelia Davis
Cloud-Native-Data with Cornelia Davis
 
Creating hybrid cloud openstack + public cloud
Creating hybrid cloud   openstack + public cloudCreating hybrid cloud   openstack + public cloud
Creating hybrid cloud openstack + public cloud
 
Enterprise CI as-a-Service using Jenkins
Enterprise CI as-a-Service using JenkinsEnterprise CI as-a-Service using Jenkins
Enterprise CI as-a-Service using Jenkins
 
Single Source of Truth for Network Automation
Single Source of Truth for Network AutomationSingle Source of Truth for Network Automation
Single Source of Truth for Network Automation
 
NetIQ identity powered security
NetIQ identity powered security   NetIQ identity powered security
NetIQ identity powered security
 

Último

Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
gajnagarg
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
amitlee9823
 
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
amitlee9823
 
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men 🔝Ongole🔝 Escorts S...
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men  🔝Ongole🔝   Escorts S...➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men  🔝Ongole🔝   Escorts S...
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men 🔝Ongole🔝 Escorts S...
amitlee9823
 
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
amitlee9823
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
amitlee9823
 
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night StandCall Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
amitlee9823
 
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night StandCall Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
gajnagarg
 
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night StandCall Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
amitlee9823
 

Último (20)

DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -
 
Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
Just Call Vip call girls kakinada Escorts ☎️9352988975 Two shot with one girl...
 
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts ServiceCall Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
Call Girls In Shalimar Bagh ( Delhi) 9953330565 Escorts Service
 
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men  🔝malwa🔝   Escorts Ser...
➥🔝 7737669865 🔝▻ malwa Call-girls in Women Seeking Men 🔝malwa🔝 Escorts Ser...
 
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
Call Girls Jalahalli Just Call 👗 7737669865 👗 Top Class Call Girl Service Ban...
 
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men 🔝Ongole🔝 Escorts S...
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men  🔝Ongole🔝   Escorts S...➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men  🔝Ongole🔝   Escorts S...
➥🔝 7737669865 🔝▻ Ongole Call-girls in Women Seeking Men 🔝Ongole🔝 Escorts S...
 
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men  🔝mahisagar🔝   Esc...
➥🔝 7737669865 🔝▻ mahisagar Call-girls in Women Seeking Men 🔝mahisagar🔝 Esc...
 
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night StandCall Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Nandini Layout ☎ 7737669865 🥵 Book Your One night Stand
 
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...
 
Aspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - AlmoraAspirational Block Program Block Syaldey District - Almora
Aspirational Block Program Block Syaldey District - Almora
 
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
(NEHA) Call Girls Katra Call Now 8617697112 Katra Escorts 24x7
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night StandCall Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Doddaballapur Road ☎ 7737669865 🥵 Book Your One night Stand
 
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
 
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night StandCall Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Shivaji Nagar ☎ 7737669865 🥵 Book Your One night Stand
 
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
Just Call Vip call girls roorkee Escorts ☎️9352988975 Two shot with one girl ...
 
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night StandCall Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
Call Girls In Attibele ☎ 7737669865 🥵 Book Your One night Stand
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Predicting Loan Approval: A Data Science Project
Predicting Loan Approval: A Data Science ProjectPredicting Loan Approval: A Data Science Project
Predicting Loan Approval: A Data Science Project
 

Couchbase usage at Symantec

  • 1. Couchbase at Symantec Gaurav Chandna Sr Principal Software Engineer
  • 2. About Me Gaurav Chandna At Symantec for 11 years. Worked at AntiSpyware, Behavioral malware, Norton and now building platforms. Engineering Manager previously @xyxxy 2
  • 3. Agenda 1 Introduction 2 Time series 3 Relational 4 Q&A Copyright © 2014 Symantec Corporation 3
  • 4. Introduction Copyright © 2014 Symantec Corporation 4
  • 5. Environment • Service Oriented Architecture • Message Oriented Middleware • Multi-tenant • Multiple data centers Copyright © 2014 Symantec Corporation 5
  • 6. Message Oriented Architecture Copyright © 2014 Symantec Corporation 6
  • 7. Replication between data centers Copyright © 2014 Symantec Corporation 7
  • 8. Time Series Data Copyright © 2014 Symantec Corporation 8
  • 9. Copyright © 2014 Symantec Corporation 9
  • 10. Copyright © 2014 Symantec Corporation 10
  • 12. Quick side bar: Elasticsearch Copyright © 2014 Symantec Corporation 12
  • 13. Elasticsearch Copyright © 2014 Symantec Corporation 13 • Document oriented search engine • Based on Apache Lucene project • JSON based • Distributed • Scale out • Multi-tenancy • RESTful
  • 14. Elasticsearch Copyright © 2014 Symantec Corporation 14 • Structured search • Unstructured search • Analytics • Combine
  • 15. All good? Copyright © 2014 Symantec Corporation 15
  • 16. Well… kind of • Elasticsearch doesn’t recommend using it as data store – Due to write coalescing • Limited support for replication between data centers – Especially across unreliable networks Copyright © 2014 Symantec Corporation 16
  • 18. Cross data center replication Copyright © 2014 Symantec Corporation 18
  • 19. Used to replicate Elasticsearch between data centers Copyright © 2014 Symantec Corporation 19
  • 20. It’s all good! Copyright © 2014 Symantec Corporation 20
  • 21. Just one more thing… • Document stored on Couchbase After X days, the document gets purged • XDCR will purge in Elasticsearch also • Forked the Couchbase – Elasticsearch connector to disable deletion • Looking to integrate those changes to Couchbase for community use Copyright © 2014 Symantec Corporation 21
  • 22. And we are finally good! Copyright © 2014 Symantec Corporation 22
  • 23. Data with relationships Copyright © 2014 Symantec Corporation 23
  • 24. Relationships are everywhere Copyright © 2014 Symantec Corporation 24 Engineering Gaurav iPhone Mac John PC Android
  • 25. Right here… Copyright © 2014 Symantec Corporation 25 Meetup @Symantec Gaurav iPhone Joshua Android
  • 26. In abstract terms A B C D C’ Y Copyright © 2014 Symantec Corporation 26
  • 27. You get information through queries • Get all members of a particular group • All groups ordered by number of people in the group • List all members of a group, order by location • List all groups where users have iPhone, ordered by total number of people • And so on.. Copyright © 2014 Symantec Corporation 27
  • 28. Engineering considerations • Ability to add new types – Don’t make big changes when iWatch came out • Ability to extend the types • Add new relationships quickly Copyright © 2014 Symantec Corporation 28
  • 29. Table stakes • Cross data center replication • Performance • Horizontal scaling • Searching • Querying capability Copyright © 2014 Symantec Corporation 29
  • 30. Initial implementation • { • ”type": ”person”, • "name": ”Gaurav", • "modified": "2015-04- 17T00:39:45.468Z", • "_id": "79014cbb-69ce-4459-8f4a- 6edaf4b8313a", • // other information • } • { • ”type": ”department”, • "name": ”Engineering", • "_id": "79014cbb-69ce-4459-8f4a- 6edaf4b8313a", • // other information • } Copyright © 2014 Symantec Corporation 30
  • 31. Initial version • Store different objects with types as documents • Create views in Couchbase – Type – Relationships between types – Specialize on special types • Ensure persistence during operations • Works well… to a point Copyright © 2014 Symantec Corporation 31
  • 32. So we needed a next generation Copyright © 2014 Symantec Corporation 32
  • 33. Second iteration • { • "_id":"Well_crafted_id” • // Other meta information • // Other associated information • } • Relationship information embedded in the _id • Takes advantage of key access to document by id • Can still build views on these documents • Does require more hand crafting to get better performance • Maintain flexibility • Need to play around with different APIs to see which work best for use case • More generic code Copyright © 2014 Symantec Corporation 33
  • 34. What about search? Copyright © 2014 Symantec Corporation 34
  • 35. Use Elasticsearch for search! • Couchbase is primary data store • Use Couchbase XDCR for one direction replication – Like time series data – But don’t disable deletes Copyright © 2014 Symantec Corporation 35 Message Queue Service Couchbase Elasticsearch
  • 36. But… • Elasticsearch prefers types to be well defined for indexing • Code is generic for extensibility • Couchbase – Elasticsearch plugin only supports 1 configuration • Different services have different needs Copyright © 2014 Symantec Corporation 36
  • 37. • More modifications to the Elasticsearch – Couchbase connector • Custom code to deduce types for compatibility with Elasticsearch • Per bucket configurations so deletes for time series but not for relational data • Keep dynamically defined types and the code generic Copyright © 2014 Symantec Corporation 37
  • 38. Final thoughts • Data modeling is extremely important • JSON Parser – experiment for your use case • Experiment with the different API options for your needs • Don’t be afraid to customize for your needs! • Polyglot persistence – Matt’s presentation Copyright © 2014 Symantec Corporation 38
  • 39. Thank you! Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice. Gaurav Chandna Gaurav_chandna@symantec.com Twitter: @xyxxy
  • 40. Thank you! Copyright © 2014 Symantec Corporation. All rights reserved. Symantec and the Symantec Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice. We are hiring www.symantec.com/careers
  • 41. &Q A Copyright © 2014 Symantec Corporation 41

Notas do Editor

  1. This is a Title Slide Portrait slide ideal for including a portrait with a brief title, presenter’s name and title. To Replace the Picture on this sample slide (this applies to all slides in this template that contain replaceable pictures) Select the sample picture and press Delete on your keyboard. Click the icon inside the shape to open the Insert Picture dialog box. Navigate to the location where the picture is stored, select desired picture and click on the Insert button to fit the image proportionally within the shape. Note: Do not right-click the image to change the picture inside the picture placeholder. This will change the frame size of the picture placeholder. Instead, follow the steps outlined above. Tip: simple SEO adjustments can make your presentation more discoverable. Read this PDF for best practices:  http://seo.ges.symantec.com/seo-best-practices-for-file-optimization.pdf
  2. Slide 2 – About Me
  3. A is associated with B B is associated with C A is associated with C Code models them as objects and relationships and privileges Talk through the use cases of what we do Why relational doesn’t work Talk about the different options that we have in code JSON Parsing is expensive. Can talk about the initial implementation Take aways model your data properly