O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Project Rhino: Enhancing Data Protection for Hadoop

2.682 visualizações

Publicada em

Learn the history of Project Rhino and its importance, the progress that’s been made so far (including a deep dive into the new security features announced with CDH 5.3), and what’s next for Hadoop security.

Publicada em: Software
  • Seja o primeiro a comentar

Project Rhino: Enhancing Data Protection for Hadoop

  1. 1. 1© Cloudera, Inc. All rights reserved. Project Rhino: Enhancing Data Protection for Hadoop Sam Heywood – Director of Product Management, Cloudera Ritu Kama – Director of Product Management, Intel
  2. 2. 2© Cloudera, Inc. All rights reserved. Agenda • Big Data Security Challenges • Project Rhino & Security for Hadoop • Unified Authorization • HDFS Encryption • Cloudera’s Compliance-Ready Security
  3. 3. 3© Cloudera, Inc. All rights reserved. How is Big Data Different Why It’s Different Architecturally • Shared data • Highly distributed system and inter-node communication • All data is online Why It’s Different Operationally • Operate in internal network • Insider data access • No native security deployed, depends on traditional security perimeter
  4. 4. 4© Cloudera, Inc. All rights reserved. Two Reasons for Security for Hadoop Hadoop Contains Sensitive Data • All data is security relevant • Improper usage or breaches of data will cause huge damage to the business • Hadoop is governed by the same security requirements as any data center platform Hadoop is Subject to Compliance Adherence • Organization are often subject to comply with regulations such as HIPPA, PCI-DSS that require protection of personal information • Adhere to other corporate security policies
  5. 5. 5© Cloudera, Inc. All rights reserved. A Brief History of Hadoop Security Originally developed without security in mind Yahoo! focused on adding authentication Project Rhino works to add security to Hadoop 2008 2009 2013 • No authentication of users or services • Anyone could submit arbitrary code to be executed • Any user could impersonate other users • Resulting security model was complex • Security configurations were complex and error-prone • No data-at-rest encryption • Limited authorization capabilities Project aims to add: • Data Protection • Authorization • Authentication
  6. 6. 6© Cloudera, Inc. All rights reserved. Project Rhino Initiatives Authentication • Token Based Authentication • Token Preauth Authorization • Sentry Role-Based Authorization • HBase Cell Security Data Protection • Cryptographic File System and Data Encryption at Rest • Data Encryption with AES-NI & Diceros • HBase Transparent Encryption • HDFS Extended Attribute • Zookeeper, Hive and Pig Data Encryption
  7. 7. 7© Cloudera, Inc. All rights reserved. Blueprint for enterprise-grade security Cloudera and Intel Project Rhino Rhino Goal: Unified Authorization Engineers at Intel and Cloudera (together with Oracle and IBM) are now jointly contributing to Apache Sentry Rhino Goal: Encryption and Key Management Framework Cloudera and Intel engineers are now contributing HDFS encryption capabilities that can plug into enterprise key managers
  8. 8. 8© Cloudera, Inc. All rights reserved. Unified Authorization Apache Sentry
  9. 9. 9© Cloudera, Inc. All rights reserved. Sentry – The Open Standard Broad Contributions • Cloudera • IBM • Intel • Oracle Multi-Vendor Support • Cloudera • IBM • MapR • Oracle Wide Industry Adoption • Banking • Healthcare • Insurance • Pharma • Telco Third-Party Integrations • Oracle Endeca • Platfora
  10. 10. 10© Cloudera, Inc. All rights reserved. Sentry provides unified authorization via fine-grained RBAC for Impala, Hive, HDFS, and Search Goal: Unified authorization for all Hadoop services and applications Unified Authorization with Apache Sentry Sentry Perm. Read Access to ALL Transaction Data Sentry Role Fraud Analyst Role Group Fraud Analysts Sam Smith
  11. 11. 11© Cloudera, Inc. All rights reserved. • Sentry can be configured to use AD to determine a user’s group assignments • Group assignment changes in AD are automatically picked up, resulting in updated Sentry role assignments Sentry and Active Directory Groups Sentry Perm. Read Access to ALL Transaction Data Sentry Role Fraud Analyst Role AD Group Fraud Analysts Sam Smith
  12. 12. 12© Cloudera, Inc. All rights reserved. Sentry Enforcement with CDH 5.3 Hive Server 2 Enforcement code Impala MR, Pig, HDFS Apps: Datameer, Platfora, etc Permissions Rules Common enforcement code for consistency Permissions specified by administrators (top-level and delegated) Enforcement code Enforcement code Enforcement code Rule 1: Allow fraud analysts read access to the transaction table
  13. 13. 13© Cloudera, Inc. All rights reserved. Encryption & Key Management HDFS Encryption
  14. 14. 14© Cloudera, Inc. All rights reserved. HDFS Encryption Available with CDH 5.3 • Supports specification of HDFS directories as “Encryption Zones • All subsequent directory contents encrypted • Multi-tenant encryption with tenant specific keys • Separation of duties via key access restrictions • Key management via Navigator Key Trustee
  15. 15. 15© Cloudera, Inc. All rights reserved. • Encryption for HDFS, HBase • No encryption for metadata, log files, ingest paths • No key management • Complicated, manual command line configuration • Disjointed audit trail Open Source HDFS Encryption Manager Navigator Impala Hive HDFS HBase Sentry Log Files Ingest Paths Metadata Store Encrypted Data Encryption Key Legend
  16. 16. 16© Cloudera, Inc. All rights reserved. Cloudera’s Solution: • ALL data encrypted: HDFS, HBase, metadata, log files, ingest paths • Enterprise Key Management via Navigator Key Trustee • Configuration support via Cloudera Manager • Audit integration to Cloudera Navigator • Optional root-of-trust integration with HSMs Compliance-Ready Encryption & Key Management Manager Navigator Impala Hive HDFS HBase Sentry Navigator Key Trustee Log Files Metadata Store Encrypted Data Encryption Key Legend Ingest Paths
  17. 17. 17© Cloudera, Inc. All rights reserved. Cloudera Enterprise Open Source HDFS Data Encryption ✔ ✔ HBase Encryption ✔ ✔ Log File Encryption ✔ ✖ Metadata Encryption ✔ ✖ Ingest Path Encryption ✔ ✖ Key Management ✔ ✖ HSM Integration ✔ ✖ Configuration ✔ ✖ Integrated Auditing ✔ ✖ Comparison: Encryption and Key Management
  18. 18. 18© Cloudera, Inc. All rights reserved. Encryption & Key Management Navigator Encrypt & Navigator Key Trustee
  19. 19. 19© Cloudera, Inc. All rights reserved. Transparent layer between application and file system • Compliance-Ready • Massively Scalable • High Performance: Optimized for Intel • Separation of Duties via process based access controls • Key Management with Navigator Key Trustee Navigator Encrypt
  20. 20. 20© Cloudera, Inc. All rights reserved. “Virtual safe-deposit box” for managing encryption keys or other Hadoop security artifact Navigator Key Trustee • Separates keys from encrypted data • Hot/Hot-Tandem dual key manager configuration • Integration with HSMs from Thales, RSA, and SafeNet • Roadmap: Management of SSL certificates, SSH keys, tokens, passwords, Kerberos Keytab Files, and more
  21. 21. 21© Cloudera, Inc. All rights reserved. • Using views, Sentry provides column restricted access to data • Combined with UDF’s, the resulting data will be dynamically masked before displaying to the user Dynamic Data Masking with Apache Sentry Sentry Perm. Masked Access to subset of Patient Data Sentry Role Clinical Analyst Role Group Clinical Analysts Sam Smith
  22. 22. 22© Cloudera, Inc. All rights reserved. What’s Next? • Log Redaction • Highly Available Authorization • Unified Credential Management • Simplified Wire Encryption • Attribute-Based Access Controls & “Follow the Data” Security • Continued Cloudera & Intel Efforts
  23. 23. 23© Cloudera, Inc. All rights reserved. Balance Security and Privacy with Business Agility Cloudera is the leader in Hadoop security. Unique Capabilities: • Comprehensive and Unified • Secure at the core • No Performance Impact • Jointly engineered with Intel • Compliance-Ready • Only distribution to pass PCI audit 1. Perimeter Standards-based Authentication Security and Administration Unlimited Storage Process Discover Model Serve 2. Access Unified Role-based Authorization 4. Data Encryption & Key Management 3. Visibility Auditing & Governance
  24. 24. 24© Cloudera, Inc. All rights reserved. Thank You

×