Mais conteúdo relacionado
Semelhante a Cloud Computing, Docker, Mesos, DCOS, Container, Big Data, Paas (20)
Cloud Computing, Docker, Mesos, DCOS, Container, Big Data, Paas
- 17. © 2016 Xavient Information Systems
• A mechanism to create a persistent volume from disk resources
• Persistent volumes enable stateful services such as HDFS and Cassandra to store their data
within Mesos
• Authorization is must to create persistent volumes
• Persistent volumes can only be created from reserveddisk resources, whether it be
statically reserved or dynamically reserved - Important
• Create through Framework and Operator API (Same as Dynamic Reservation)
https://mesosphere.github.io/marathon/docs/persistent-volumes.html
https://mesosphere.github.io/marathon/docs/external-volumes.html
An external storage service enables your apps to be more fault-tolerant. If a host fails, Marathon
reschedules your app on another host, along with its associated data, without user intervention.
https://github.com/emccode/rexray
Persistent Volumes
- 34. © 2016 Xavient Information Systems
Company Snapshot
Who We Are
• Founded in US in 2002 | Global Scale |
ExponentialGrowth
• Fortune 200 clientele | Partner of choice
for 4 out of Top 5 CSPs inUS
• Comprehensive Technology Coverage &
Personalized Service
What We Do
• Enable Enterprise Class Solutions on
Web, Mobility, Open Source, Cloud,
BigData
• Focus on Telecom, Retail, Healthcare
& Banking
• Optimize BAU SDLC Practices
How We Deliver
• Customer First Approach - Long term
ongoing engagements are testimony to
our success
• Talent Farming – Thought leadership |
Bootcamps | Embedded Factory
• Maturity & Agility – Technology
Innovation | Reusable Best Practices |
Early Adopters
- 35. © 2016 Xavient Information Systems
Global Presence
Tampa, FL
Atlanta, GA
Herndon, VA
Sterling, VA
Bethpage, NY
Frisco, TX
Englewood, CO
Denver, CO
St. Louis, MO
Bellevue, WA
Simi Valley, CA
(Headquarters)
Noida
Bengaluru
OnshoreDelivery Center /
Client Location
OffshoreDelivery Center
Mississauga, Canada
- 42. © 2016 Xavient Information Systems
Xavient’s Big Data Team
Developer and Data Scientist
10+ years industry Experience
3 years Big Data experience
Data Architect/Engineer
10+ years industry Experience
3 years Big Data experience
Sr. Engagement Manager
8+ years industry Experience
2+ years Big Data experience
Sr. Consultant
6+ years industry Experience
2 years Big Data experience
• Develop data ingestion flow in batch
and real time
• Design and develop statistical
procedures and algorithms around
data sources
• Recommend and build models for
various data studies, data discovery
and hypothesis
• Work with developers to integrate
and pre-process data for inputs into
models
• Data cleanup and transformation as
preparation for analysis
• Focus areas include: statistics,
applied math, machine learning,
model design and development
• Configure and setup various
development and data science tools.
• Platform architecture analysis and
design, including: data center layout,
hardware and network
considerations
• Security model design and
implementation for authentication,
authorization, administration and
audit
• Define recommended processes for
data ingestion, management,
transformation and egress.
• Perform installation for enterprise-
scale cluster environments.
• Resource management at cluster
and infrastructure level.
• Facilitate the selection of
appropriate Big Data technology
and tools
• Conduct technical workshops
• Define the overall architecture for
the solutions/business applications
and prepare the environment
• Initial day-to-day cluster operations
• Overall responsibility for project
success
• Primary management contact for
the customer.
• Coordinate all the Professional
Services resources
• Ensure the project is delivered
according to schedule and budget
• Report status and manage
escalations.
• Lead the implementation of
business use case(s).
• Provide expertise and guidance
during the project lifecycle.
• Provide design and development
expertise for large-scale, clustered
data processing systems
• Develop ingest, transformation and
egress capabilities leveraging latest
tools such as Kafka, Storm, Pig, Hive,
etc.
• Resolve technical issues in the
environment
• Assist with the preparation of
technical deliverables, and review
and demonstrate the system and
applications
• Provide technical support either
onsite or remote.
- 44. © 2016 Xavient Information Systems
Big Data Case 1 : Enterprise Social Networking and
Collaboration Platform
• Real time data streaming
• Search capability
• Front end UI
• Build recommendationmodel based on user search
Xavient’s SolutionXavient’s Solution
• End to end design and implementation of enterprise
ready solution/app to provide social networking
platform
• Mobile app solution
• Chat, Message exchange capabilities by leveraging
NoSQL databases
• Resource management at the enterpriselevel
Technologies: Kafka, Storm, HDFS, Hive, HBase, Elastic
Search, Java, ApacheMesos/Mesosphere, Apache Myraid
Xavient’s Solution
• Real time reports& analyticswith granular data
• Health checkand monitoringofkeyareas
• Informed and quickdecisions
• Both online & offline mode access
• Multiple Currencyselection (USD& local currency)
Key Benefits
Business Requirement
An Enterprise Social
Networking and
Collaboration Platform
- 46. © 2016 Xavient Information Systems
Big Data Case 2 : Implementing a True Enterprise Data Lake
Business Requirement
• Store all the data at one place
• Eliminate datasilos
• Build COE
Xavient’s SolutionXavient’s Solution
• Developed overall roadmap and design for enterprise
‘data lake.’
• POC and Hadoop knowledge sharing sessions
• Security setup for strong authentication and
authorization
• Data ingestion from RDBMS to HDFS/Hive. H
• Defined plan and helped implement a CoE
Technologies used: HDP, Hive, Sqoop, AWS, Oozie
Xavient’s Solution
• Nomore silosof data
• Fast response time delivered for long running
queries
• Cost per TB dropped
• Internal Hadoop asa Service cloud
• Store data for longer period oftime
• New technologystackincreasesemployeeretention
Key Benefits
About Client
Client is a leading Satellite TV
Service Provider in US
- 47. © 2016 Xavient Information Systems
• Operational dashboards
• Customer scorecards
• CDR analysis
• Proactive Maintenance
• Infrastructure investment
• Bandwidth allocation
• Product development
EDW
MPP
CRM
ERP
DATA SOURCES
DATA REPOS ANALYSIS
Billing Data
Subscriber
Data
Product
Catalog
Network Data
Clickstream
Online Chat
Sensor Data
Server Logs
Call Detail
Records
Merchant
Listing
DMP
Social Media
Ingest
Sqoop
Flume
HDFS Put
WEB HDFS
Hadoop
Cluster
Batch
Map
Reduce
Script
Pig
SQL
Hive
Online
Hbase
Accumulo
Real-Time
Storm
In-Memory
Spark
Others
Metadata Management HCatalog
Multitenant Processing: YARN
Hadoop Operating System
Linear Scale
Compute & HDFS Storage
Governance
tag, filter &
process
Operations (Ambari)
Security (Knox, Kerberos, Apache Ranger)
1
N
. . . . . . .
.
. .
.
. .
. . . . . . . . . .
. . .. . . .. . .
Big Data Case 2 : Enterprise Data Lake (Contd..)
- 51. © 2016 Xavient Information Systems
BI Case 1 : Executive Dashboard For Strategic Insights
Business Challenges Business Requirement
Xavient’s Solution Xavient’s SolutionXavient’s Solution Key Benefits
• Multi-channel dashboard
• Real time analytics on Cross-functional data with visual representation
• Quicker decision making for Senior Executives
• Cross platform dashboard reporting application
• Configurable background sync for offline data access
• Customized KPIs providing bird’s eye view
• Dynamic reporting from multiple digital marketing
channels viz. revenue, billings, costs and number of
campaigns
Technologies used: Informatica, Oracle, HTML5, Jquery,
Java, Objective C
• Real time reports & analytics with granular data
• Health check and monitoring of key areas
• Informed and quick decisions
• Both online & offline mode access
• Multiple Currency selection (USD & local currency)
About Client
The client is a Digital Marketing
& Advertising ServiceProvider