O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Upgrade Webinar
Best Practices for Upgrading Hadoop with Cloudera Manager
Vala Dormiani | Product Manager
2© Cloudera, Inc. All rights reserved.
Cloudera Enterprise powered by Apache Hadoop
A new kind of data platform.
• One pla...
3© Cloudera, Inc. All rights reserved.
Cloudera Enterprise
End-to-End Administration
4© Cloudera, Inc. All rights reserved.
Hadoop Administration Made Easy
Cloudera Manager
Focus on the solution, not the
clu...
5© Cloudera, Inc. All rights reserved.
Why You Need Cloudera Manager
Complexity
Context
Efficiency
Hadoop is more than a d...
6© Cloudera, Inc. All rights reserved.
End-to-End Administration for the EDH
Manage
Easily deploy, configure, & optimize c...
7© Cloudera, Inc. All rights reserved.
One Tool For Everything
Managing Complexity
+
DEPLOYMENT &
CONFIGURATION
MONITORING...
8© Cloudera, Inc. All rights reserved.
Raw Data vs. Hadoop Intelligence
Providing Context
? VS
Smart Configuration
Auto-se...
9© Cloudera, Inc. All rights reserved.
Simple Diagnostic Workflow
Maximizing Efficiency
NOTICE JOB IS NOT COMPLETING
IDENT...
10© Cloudera, Inc. All rights reserved.
Why Cloudera Manager
One Holistic View of Everything
Best-in-Class
• Only enterpri...
11© Cloudera, Inc. All rights reserved.
Cloudera Manager Features
12© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Install a Cluster in Three Simple Steps
1 2 3Find No...
13© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
View Service Health and Performance
14© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Gather, View, and Search Hadoop Logs
15© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Manage Resources
16© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Customizable Landing Page
17© Cloudera, Inc. All rights reserved.
Open API for Extensibility
Integration with Leading ISVs Alternative Storage Optio...
18© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
“We always know the health of our cluster and
its no...
19© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Manage Backup and Disaster Recovery
20© Cloudera, Inc. All rights reserved.
Cloudera Manager Key Features
Upgrades
21© Cloudera, Inc. All rights reserved.
The Upgrade Wizard
22© Cloudera, Inc. All rights reserved.
Why Upgrade to CDH 5
• Software platform improvements
• New features
• Security an...
23© Cloudera, Inc. All rights reserved.
Motivation for the Wizard
• Upgrades are hard and unpredictable
• Downtime to miss...
24© Cloudera, Inc. All rights reserved.
The CDH Upgrade Wizard in Cloudera Manager
• Cloudera Manager has a built-in
Upgra...
25© Cloudera, Inc. All rights reserved.
What is Included
• Confirmation of applicable manual steps
• Verification that pro...
26© Cloudera, Inc. All rights reserved.
What is not Included
• Some steps are still manual
• Backing up existing databases...
27© Cloudera, Inc. All rights reserved.
Types of Upgrades Supported by the Wizard
• Major upgrades from CDH 4 to CDH 5
• C...
28© Cloudera, Inc. All rights reserved.
Parcels vs. Packages
• Using Parcels is preferred as packages must be manually ins...
29© Cloudera, Inc. All rights reserved.
Zero-Downtime Rolling Upgrade
• New Rolling Restart if
1. Enabled HDFS HA
2. Using...
30© Cloudera, Inc. All rights reserved.
How to Start an Upgrade
• Trigger Points
1. Parcel Page:
Download Distribute Activ...
31© Cloudera, Inc. All rights reserved.
Wizard Steps
1. Log in to Cloudera Manager & Trigger the upgrade
2. Select the pac...
32© Cloudera, Inc. All rights reserved.
33© Cloudera, Inc. All rights reserved.
Useful things to have in place
• Have an automated way to backup your NameNode Met...
34© Cloudera, Inc. All rights reserved.
Other Best Practices for CDH 5 Upgrade
• In critical upgrades, create a fine-grain...
35© Cloudera, Inc. All rights reserved.
CDH 4 to CDH 5 Upgrade Steps
• Documentation: Upgrading from CDH 4 to CDH 5 Parcel...
36© Cloudera, Inc. All rights reserved.
Guided Upgrades Prevent Failed Jobs
Upgrading
Synopsis
• Customer manually upgrade...
37© Cloudera, Inc. All rights reserved.
Upgrading Recommendations and Resources
• Start planning now
• Review Upgrade Guid...
38© Cloudera, Inc. All rights reserved.
Why Professional Services
• Minimize risk to production environment
• Assist your ...
39© Cloudera, Inc. All rights reserved.
Backup & Disaster Recovery
40© Cloudera, Inc. All rights reserved.
Why You Need Backup & Disaster Recovery
Your EDH is a Mission-Critical Part of the...
41© Cloudera, Inc. All rights reserved.
Simplified Management of Backup & DR Policies
BDR in Cloudera Enterprise
HDFS
HIVE...
42© Cloudera, Inc. All rights reserved.
Benefits of Cloudera Manager’s BDR
Reduce Complexity
• Centrally manage backup and...
43© Cloudera, Inc. All rights reserved.
Data Threat Models and Solutions
Disk/Node/Rack
Hardware Failure
• HDFS replica
ar...
44© Cloudera, Inc. All rights reserved.
CDH 5 Backup and Disaster Recovery
HDFS Snapshots
• Minimal impact to
production w...
45© Cloudera, Inc. All rights reserved.
Cloudera Enterprise
Industry-Leading Support
46© Cloudera, Inc. All rights reserved.
Direct Integration with Cloudera Support in CM
47© Cloudera, Inc. All rights reserved.
Cloudera Manager + Support
Industry’s Best Hadoop Platform Support
• Leverages Clo...
48© Cloudera, Inc. All rights reserved.
Differentiated Approach to Success
Technical guidance based on insights into perfo...
49© Cloudera, Inc. All rights reserved.
World-Class Support
Customers Love Cloudera Support
8.9/10
91%
Overall satisfactio...
50© Cloudera, Inc. All rights reserved.
Cloudera Enterprise 5
51© Cloudera, Inc. All rights reserved.
Built for Production Success
Hadoop delivers:
• One place for unlimited data
• Uni...
52© Cloudera, Inc. All rights reserved.
Industrial Multi-Workload Performance
Batch, Interactive,
and Real-Time.
Leading p...
53© Cloudera, Inc. All rights reserved.
The Only Comprehensively Secure Hadoop Platform
Cloudera is the leader in
Hadoop s...
54© Cloudera, Inc. All rights reserved.
The Most Complete Partner Ecosystem
Data
Systems
Enterprise Data Hub
Security and ...
55© Cloudera, Inc. All rights reserved.
New In Cloudera Manager 5
Workload/Resource
Management
Pool, resource group &
queu...
56© Cloudera, Inc. All rights reserved.
New In Cloudera Manager 5
Monitoring Improvements
Advanced Impala query monitoring...
57© Cloudera, Inc. All rights reserved.
New in CDH 5
• Impala and Search are now part of CDH
• HDFS has caching and snapsh...
58© Cloudera, Inc. All rights reserved.
Webinar to Learn More
More Value from More Data: Production-Ready Hadoop
with Clou...
Thank You
Próximos SlideShares
Carregando em…5
×

Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production

6.065 visualizações

Publicada em

Walk through some of the best practices to keep in mind when it comes to upgrading your cluster, and learn how to leverage new Upgrade Wizard features in Cloudera Enterprise 5.3.

For most mission critical workloads, downtime is never an option. Any downtime can have a direct impact on revenue and lead to frantic calls in the middle of the night. For this reason, upgrading the software that powers these workloads can often be a daunting task. It can cause unpredictable issues without access to support. That’s why an enterprise-grade administration tool is crucial for running Hadoop in production. Hadoop consists of dozens of components, running across multiple machines, all with their own configurations. That can lead to a lot of complexity and uncertainty - especially when taking the upgrade plunge.

Cloudera Manager makes it easy and is the only production-ready administration tool for Hadoop. Not only does Cloudera Manager feature zero-downtime rolling upgrades, but it also has a built in Upgrade Wizard to make upgrades simple and predictable.

Publicada em: Software
  • Seja o primeiro a comentar

Upgrade Without the Headache: Best Practices for Upgrading Hadoop in Production

  1. 1. Upgrade Webinar Best Practices for Upgrading Hadoop with Cloudera Manager Vala Dormiani | Product Manager
  2. 2. 2© Cloudera, Inc. All rights reserved. Cloudera Enterprise powered by Apache Hadoop A new kind of data platform. • One place for unlimited data • Unified, multi-framework data access Only with Cloudera: • Leading performance • Enterprise system and data management • Fundamentally secure • Open source, open standards Security and Administration Unlimited Storage Process Discover Model Serve Deployment Flexibility On-Premises Appliances Engineered Systems Public Cloud Private Cloud Hybrid Cloud
  3. 3. 3© Cloudera, Inc. All rights reserved. Cloudera Enterprise End-to-End Administration
  4. 4. 4© Cloudera, Inc. All rights reserved. Hadoop Administration Made Easy Cloudera Manager Focus on the solution, not the cluster, with the only complete, zero-downtime administration tool for Apache Hadoop. Unique Capabilities: • Unified configuration, management and monitoring across all services • Online installation and upgrades • Direct connection to Cloudera Support • 3rd Party Extensibility
  5. 5. 5© Cloudera, Inc. All rights reserved. Why You Need Cloudera Manager Complexity Context Efficiency Hadoop is more than a dozen services running across many machines Hadoop is a system, not just a collection of parts Managing Hadoop with multiple tools & manual process takes longer • Hundreds of hardware components • Thousands of settings • Limitless permutations • Everything is interrelated • Raw data about individual pieces is not enough • Must extract what’s important • Complicated, error-prone workflows • Longer issue resolution • Lack of consistent and repeatable processes
  6. 6. 6© Cloudera, Inc. All rights reserved. End-to-End Administration for the EDH Manage Easily deploy, configure, & optimize clusters1 Monitor Maintain a central view of all activity2 Diagnose Easily identify and resolve issues3 Integrate Use with existing tools4
  7. 7. 7© Cloudera, Inc. All rights reserved. One Tool For Everything Managing Complexity + DEPLOYMENT & CONFIGURATION MONITORING WORKFLOWS EVENTS & ALERTS LOG SEARCH DIAGNOSTICS REPORTING ACTIVITY MONITORING DO-IT-YOURSELF VERSUS WITH CLOUDERA
  8. 8. 8© Cloudera, Inc. All rights reserved. Raw Data vs. Hadoop Intelligence Providing Context ? VS Smart Configuration Auto-sets configurations and guards against user error 1 Workflows Ensures that multi-step tasks are accomplished completely and in the correct sequence2 Dependencies Aware of how a particular action affects the rest of the cluster and manages the impact 3 Events & Alerts Makes you aware of what’s important at a Hadoop system level 4
  9. 9. 9© Cloudera, Inc. All rights reserved. Simple Diagnostic Workflow Maximizing Efficiency NOTICE JOB IS NOT COMPLETING IDENTIFY PROBLEM TASK IN TASK TRACKER WEB UI GANGLIA: STUDY SERVICE, HOST & NETWORK METRICS FOR ROOT CAUSE DETERMINE REQUIRED HEAP SIZE UPDATE HEAP SIZE & RESTART TASK TRACKER WITH CHEF ROOT CAUSE: LOW HEAP FOR TASK TRACKER 1 HR 2 HRS 1 HR 30 MIN RECEIVE ALERT: JOB RUNNING LONGER THAN EXPECTED VISUALLY LOCATE PROBLEM TASK IN TASK DISTRIBUTION VIEW DRILL DOWN TO TASK TRACKER HEALTH, SEE ‘LOW HEAP’ UPDATE HEAP SIZE W/RECOMMENDED VALUE RESTART TASK TRACKER ROOT CAUSE: LOW HEAP FOR TASK TRACKER 5 MIN 3 MIN 2 MIN 5 MIN WITH CLOUDERA MANAGER 4.5 HOURS 15 MIN DO-IT-YOURSELF
  10. 10. 10© Cloudera, Inc. All rights reserved. Why Cloudera Manager One Holistic View of Everything Best-in-Class • Only enterprise-grade Hadoop management application • Zero downtime rolling upgrades & BDR • Integrated with Support Simple • Manage the complexity of dozens of tools through one interface Intelligent • Extract context from your data and Hadoop system Efficient • Simplify complex workflows and create consistent, repeatable processes 3rd Party Integration • Broadest network of partners with complete integration
  11. 11. 11© Cloudera, Inc. All rights reserved. Cloudera Manager Features
  12. 12. 12© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Install a Cluster in Three Simple Steps 1 2 3Find Nodes Install Components Assign Roles Enter the names of the hosts which will be included in the Hadoop cluster. Click Continue. Cloudera Manager automatically installs the CDH components on the hosts you specified. Verify the roles of the nodes within your cluster. Make changes as necessary.
  13. 13. 13© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features View Service Health and Performance
  14. 14. 14© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Gather, View, and Search Hadoop Logs
  15. 15. 15© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Manage Resources
  16. 16. 16© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Customizable Landing Page
  17. 17. 17© Cloudera, Inc. All rights reserved. Open API for Extensibility Integration with Leading ISVs Alternative Storage Options Hundreds of Partners Certified to Run In and On Cloudera
  18. 18. 18© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features “We always know the health of our cluster and its nodes. We really can stay in touch with what's happening on the system, and we can deploy and manage things really easily” Kathleen deValk Senior Architect, Omneo
  19. 19. 19© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Manage Backup and Disaster Recovery
  20. 20. 20© Cloudera, Inc. All rights reserved. Cloudera Manager Key Features Upgrades
  21. 21. 21© Cloudera, Inc. All rights reserved. The Upgrade Wizard
  22. 22. 22© Cloudera, Inc. All rights reserved. Why Upgrade to CDH 5 • Software platform improvements • New features • Security and governance • Bug fixes • Technology Enablement • Evolution of infrastructure • Expanded application stack Security and Administration Unlimited Storage Process Discover Model Serve
  23. 23. 23© Cloudera, Inc. All rights reserved. Motivation for the Wizard • Upgrades are hard and unpredictable • Downtime to mission-critical workloads impacts your revenue • Hadoop can be especially complex • Upgrading Hadoop can have many steps that can depend on • Services Installed • Start & End Versions • Packages or Parcels…
  24. 24. 24© Cloudera, Inc. All rights reserved. The CDH Upgrade Wizard in Cloudera Manager • Cloudera Manager has a built-in Upgrade Wizard • Major Upgrades (CDH4 to CDH5) supported from CM 5.0 • CM 5.3 supports upgrades for Minor (CDH 5.x to CDH 5.y) & Maintenance Releases (CDH 5.b.x to CDH 5.b.y) • Zero-downtime for non-major upgrades • Wizard automatically performs upgrade steps that were manual in the past
  25. 25. 25© Cloudera, Inc. All rights reserved. What is Included • Confirmation of applicable manual steps • Verification that proper binaries are installed and hosts are healthy • Applicable automated commands for the upgrade • Post-upgrade messages applicable to this upgrade
  26. 26. 26© Cloudera, Inc. All rights reserved. What is not Included • Some steps are still manual • Backing up existing databases & NameNode metadata • Installing & removing packages • Wizard doesn’t capture all upgrade caveats • Does not include Minor CDH 4 upgrades, ie CDH4.2 to 4.3
  27. 27. 27© Cloudera, Inc. All rights reserved. Types of Upgrades Supported by the Wizard • Major upgrades from CDH 4 to CDH 5 • CDH upgrade wizard extended to minor CDH 5 upgrades & maintenance upgrades / downgrades in CM 5.3 Note: Can’t upgrade to a CDH version higher than CM version
  28. 28. 28© Cloudera, Inc. All rights reserved. Parcels vs. Packages • Using Parcels is preferred as packages must be manually installed • See the FAQ to learn more about Parcels • Supported: • Package Package • Package Parcel • Parcel Parcel
  29. 29. 29© Cloudera, Inc. All rights reserved. Zero-Downtime Rolling Upgrade • New Rolling Restart if 1. Enabled HDFS HA 2. Using Parcels 3. Have an Enterprise License 4. Performing a Non-Major Upgrade • Supported services will be upgraded and restarted without cluster downtime
  30. 30. 30© Cloudera, Inc. All rights reserved. How to Start an Upgrade • Trigger Points 1. Parcel Page: Download Distribute Activate Upgrade • After downloading and distributing a parcel, “Activate” is replaced by “Upgrade” if the version change is supported by the wizard • Note: Just “activating” is rarely a good idea 2. Cluster Actions Dropdown Menu: Click “Upgrade Cluster”
  31. 31. 31© Cloudera, Inc. All rights reserved. Wizard Steps 1. Log in to Cloudera Manager & Trigger the upgrade 2. Select the package/parcel upgrade version 3. Pre-upgrade warnings 4. Perform the required actions before continuing (e.g. Backing up databases) 5. Host health validation 6. Parcel is downloaded and distributed to all hosts 7. Restart selection: o Regular vs Rolling 8. Commands Progress Screen: o Activation of new parcel, Upgrading Services, Deploying client config files, Other CDH component steps… 9. Host Inspector 10. Post-upgrade warnings 11. Yarn Migration
  32. 32. 32© Cloudera, Inc. All rights reserved.
  33. 33. 33© Cloudera, Inc. All rights reserved. Useful things to have in place • Have an automated way to backup your NameNode Metadata. • You will need to backup your NN metadata prior to the update so you should have scripts ready in advance • All databases should be backed up regularly, including CM, HDFS, Hive, HBase, Oozie • You will need to take backups prior to the upgrade, but you should have automated backup procedures for these databases already • You cannot revert back to CDH 4 unless you restore a backup • Maintain your own OS, CM and CDH package/parcel repos to protect against external repositories being unavailable
  34. 34. 34© Cloudera, Inc. All rights reserved. Other Best Practices for CDH 5 Upgrade • In critical upgrades, create a fine-grained step-by-step production plan • Document the existing cluster environment and dependencies • Test the production upgrade plan in non-prod environment(s) • Test the step by step upgrade plan in sandbox, test and other non-prod environments and update the plan if anything unexpected happens • Test all compatibility with the new version. If desired, run performance tests in a performance cluster • Reserve a maintenance window with enough time allotted to perform all steps • Note that rolling upgrade from CDH 4 to CDH 5 is unsupported • Enable maintenance mode on your cluster to avoid lots of alerts during the upgrade
  35. 35. 35© Cloudera, Inc. All rights reserved. CDH 4 to CDH 5 Upgrade Steps • Documentation: Upgrading from CDH 4 to CDH 5 Parcels - Read “Before You Begin” 1. Download/distribute parcel 2. Reduce the upgrade time by reducing the amount of history that Oozie retains 3. Put the NameNode into safe mode and backup HDFS metadata 4. Stop the cluster & stop the CM service 5. Remove CDH Packages (if in-use) 6. Deactivate and Remove the GPL Extras Parcel (if using LZO) 7. Run the Upgrade Wizard • Recover from any failed steps before proceeding 8. Upgrade the GPL Extras Parcel (if using LZO) 9. Restart the Reports Manager Role 10. Finalize the HDFS Metadata Upgrade
  36. 36. 36© Cloudera, Inc. All rights reserved. Guided Upgrades Prevent Failed Jobs Upgrading Synopsis • Customer manually upgraded CDH • Misconfigured a MapReduce setting • Resulted in failure of long-running jobs With Cloudera Manager • The upgrade process is managed • Default configuration settings would have prevented job failures Cloudera Manager Benefits • Streamlined upgrades • Issue prevention
  37. 37. 37© Cloudera, Inc. All rights reserved. Upgrading Recommendations and Resources • Start planning now • Review Upgrade Guide Documentation • Talk to your Account Team about a Professional Services Engagement
  38. 38. 38© Cloudera, Inc. All rights reserved. Why Professional Services • Minimize risk to production environment • Assist your Hadoop Admin • Minimize impact on resources (i.e. development) • Educate the team on a release • Provide additional guidance on best practices
  39. 39. 39© Cloudera, Inc. All rights reserved. Backup & Disaster Recovery
  40. 40. 40© Cloudera, Inc. All rights reserved. Why You Need Backup & Disaster Recovery Your EDH is a Mission-Critical Part of the Data Management Infrastructure • Stores valuable data and runs important workloads • Business continuity is a MUST HAVE 1 Managing Business Continuity for Hadoop is Complex • Different services that store data – HDFS, HBase, Hive • Backup and disaster recovery is configured separately for each • Processes are manual 2
  41. 41. 41© Cloudera, Inc. All rights reserved. Simplified Management of Backup & DR Policies BDR in Cloudera Enterprise HDFS HIVE NODES SITE A SITE B HDFS HIVE NODES Central Configuration • HDFS - Select files & directories to replicate • Hive - Select tables to replicate • Schedule replication jobs for optimal times Monitoring & Alerting • Track progress of replication jobs • Get notified when data is out of sync Performance & Reliability • High performance replication using MapReduce • CDH-optimized version of DistCP
  42. 42. 42© Cloudera, Inc. All rights reserved. Benefits of Cloudera Manager’s BDR Reduce Complexity • Centrally manage backup and DR workflows • Simple setup via an intuitive user interface Maximize Efficiency • Simplify processes to meet or exceed SLAs and Recovery Time Objectives (RTOs) • Optimize system performance and network impact through scheduling Reduce Risk & Exposure • Eliminate error-prone manual processes • Get notified when issues occur • The only solution for metadata replication (Hive)
  43. 43. 43© Cloudera, Inc. All rights reserved. Data Threat Models and Solutions Disk/Node/Rack Hardware Failure • HDFS replica architecture • Configure rack information and number of replicas Application/User Error • Snapshots of HDFS and HBase • Optionally save HBase to S3 Datacenter Failure • Off-site datacenter replication of HDFS and Hive • Includes metadata
  44. 44. 44© Cloudera, Inc. All rights reserved. CDH 5 Backup and Disaster Recovery HDFS Snapshots • Minimal impact to production workload • No unnecessary data copy • Multiple versions maintained by HDFS • Fast local restores • HDFS consistency HBase Snapshots • Minimal impact to production workload • No unnecessary data copy • Multiple versions maintained by HBase • HBase region consistency • Optionally store snapshot to Amazon S3 HDFS Distributed Replication • Snapshot-based replication ensures consistency across replicas Hive Metastore Replication • SQL import/export between two different metastores • Fixes file paths and other cluster-specific information Cloudera Manager Select Configure Synchronize Monitor Backup and Disaster Recovery Module
  45. 45. 45© Cloudera, Inc. All rights reserved. Cloudera Enterprise Industry-Leading Support
  46. 46. 46© Cloudera, Inc. All rights reserved. Direct Integration with Cloudera Support in CM
  47. 47. 47© Cloudera, Inc. All rights reserved. Cloudera Manager + Support Industry’s Best Hadoop Platform Support • Leverages Cloudera to reduce time-to-resolution by 35% • Comprehensive view of customers for Proactive and Predictive Support • Prevent issues before they occur • Provide guidance on tools and best practices
  48. 48. 48© Cloudera, Inc. All rights reserved. Differentiated Approach to Success Technical guidance based on insights into performance patterns and the state-of-the-art Proactive Support Sophisticated analytics across multiple clusters to prevent issues before they occur Predictive Support Input into product roadmaps and projects supported by the Apache community Voice of the Customer
  49. 49. 49© Cloudera, Inc. All rights reserved. World-Class Support Customers Love Cloudera Support 8.9/10 91% Overall satisfaction score makes Cloudera the industry benchmark for support Customers agree they benefit from proactive support outreach #1 Ability to solve technical issues is the top reason to recommend
  50. 50. 50© Cloudera, Inc. All rights reserved. Cloudera Enterprise 5
  51. 51. 51© Cloudera, Inc. All rights reserved. Built for Production Success Hadoop delivers: • One place for unlimited data • Unified, multi-framework data access Cloudera delivers: • Enterprise Security • Data Governance • Complete Management • Open Source, Open Standards Security and Administration Unlimited Storage Process Discover Model Serve Deployment Flexibility On-Premises Appliances Engineered Systems Public Cloud Private Cloud Hybrid Cloud A modern data platform plus what the enterprise requires.
  52. 52. 52© Cloudera, Inc. All rights reserved. Industrial Multi-Workload Performance Batch, Interactive, and Real-Time. Leading performance and usability in one platform. • End-to-end analytic workflows • Access more data • Work with data in new ways • Enable new users Security and Administration Process Ingest Sqoop, Flume Transform MapReduce, Hive, Pig, Spark Discover Analytic Database Impala Search Solr Model Machine Learning SAS, R, Spark, Mahout Serve NoSQL Database HBase Streaming Spark Streaming Unlimited Storage HDFS, HBase YARN, Cloudera Manager, Cloudera Navigator Multiple big data opportunities in one optimized, high-performance, multi-tenant platform.
  53. 53. 53© Cloudera, Inc. All rights reserved. The Only Comprehensively Secure Hadoop Platform Cloudera is the leader in Hadoop security. Unique Capabilities: • Comprehensive and Unified • Secure at the core • No Performance Impact • Jointly engineered with Intel • Compliance-Ready • Only distribution to pass PCI audit 1. Perimeter Standards-based Authentication Security and Administration Unlimited Storage Process Discover Model Serve 2. Access Unified Role-based Authorization 4. Data Encryption & Key Management 3. Visibility Auditing & Governance Meet compliance requirements and reduce risk exposure from storing sensitive data.
  54. 54. 54© Cloudera, Inc. All rights reserved. The Most Complete Partner Ecosystem Data Systems Enterprise Data Hub Security and Administration Unlimited Storage Process Discover Model Serve Applications System Integration Infrastructure More than 1,300 partners ensure compatibility with existing investments, lower skill barriers, and help maximize value from your data.Operational Tools
  55. 55. 55© Cloudera, Inc. All rights reserved. New In Cloudera Manager 5 Workload/Resource Management Pool, resource group & queue administration Static & dynamic partitioning of resources Usage monitoring & trending Extensibility and Partner Product Integration Integration with ISVs • SAS • Syncsort • Revolution • and others Accumulo support Spark support Platform Coverage CDH5 compatibility support Install & Upgrade wizards for CDH5
  56. 56. 56© Cloudera, Inc. All rights reserved. New In Cloudera Manager 5 Monitoring Improvements Advanced Impala query monitoring YARN service monitoring YARN/MR2 activity monitoring User defined triggers Updates to ‘tsquery’ language for custom charts Scalable back-end datastore for monitoring metrics Enhanced Operational Reports Oozie HA and YARN/RM HA setup MR1->MR2 config upgrade wizard Updates to Parcel management workflows Several usability improvements including new visualizations, charting enhancements CM search box Java7 support Other Improvements Security Improvements Direct AD Kerberos Integration Kerberos wizard for easy securing of non-secure clusters Manage & deploy Kerberos client configs Added Hadoop SSL related configs New user roles for fine-grained separation of duties
  57. 57. 57© Cloudera, Inc. All rights reserved. New in CDH 5 • Impala and Search are now part of CDH • HDFS has caching and snapshots • YARN is production-ready • HBase has faster RegionServer failover, online merge, and batch indexing • Impala has dynamic resource management through Llama and YARN • Impala supports UDFs and UDAFs, leverages HDFS caching, and has improved metadata refresh • Sentry offers fine-grained role-based authorization for Search, Impala, and Hive More details about the Cloudera 5 can be found in the Release Notes
  58. 58. 58© Cloudera, Inc. All rights reserved. Webinar to Learn More More Value from More Data: Production-Ready Hadoop with Cloudera 5 • Feb. 17, 2015 at 10am PT • More details on Security, Governance, Cloud, Apache Spark, and Impala 2.0 Register at bit.ly/ProductionReady
  59. 59. Thank You

×